Why does Char think Ella is avoiding him when he comes to her home to visit?

When Char comes to visit Dame Olga's house, Ella is confined to her room by Hattie's orders. When Char leaves the house he sees Ella through a window and so he believes Ella is deliberately avoiding him.