Why does Marguerite stop to investigate a cave while searching for a way out of the woods?

Caleb is angry when he finds Marguerite and the children wondering lost in the woods. Caleb tells the children that you cannot trust a French person to listen to you and that Marguerite is nothing but a bound-out girl without any sense.