Why did Pat attempt to keep Caroline away from Mary in the novel, Unbecoming?

Pat and Lionel raised Caroline (Mary's baby) as their own. Pat tried to keep Mary from having any contact with Caroline because she feared Mary would take Caroline away.