Under what conditions does Martin agree to go with Rufe, Seb, and Eben?

Rufe threatens to come back with armed men and make Martin go with them. Martin decides to go with them if they allow Silas to stay home, and they allow him to come back in a week.