Why does Jimmy dislike the church?

The church bells are a constant presence in their little apartment, drowning out life. By his violent reaction to them every time they begin ringing, it seems as though he regards them as yet another invasion by the establishment.