Why is Maddie often uncomfortable while interacting with her stepbrother, James, in the novel, Alone?

James is one of Maddie's younger stepbrothers. He is deaf and Maddie often feels uncomfortable when she interacts with him because of this.