Who is the antagonist in Wolf by the Ears by Ann Rinaldi?

Charles Bankhead is the character most likely to be considered the antagonist in the novel, but the true antagonist is slavery.