How does Raju justify staying on at the temple?

Raju justifies his presence at the temple by admitting that he couldn't leave without great embarrassment or disturbance. He also says he doesn't want to hurt Velan, but in reality, he is still motivated by having his needs met by the villagers.