What leads Duke Frederick to believe that the girls are with Orlando?

One of the attendants, Hisperia, overheard the girls talking about Orlando the previous day. She tells this to Duke Frederick, who immediately suspects that the girls left with him.