In Part 2 of "Heart of Darkness" why are the manager and his uncle concerned about Kurtz?

The Manager is worried because Kurtz is alone. He hasn't returned to the station with the ivory, and he has sent everyone who worked for him back to the station. They believe Kurtz to be up to no good and believe he must be made an example of.