Who is the antagonist in the novel, Tom’s Midnight Garden?

The novel does not include a traditional antagonist, though Hatty's aunt can be considered an antagonistic character.