How does Matt's garden experience prove to be frustrating?

When he thinks Indians might be watching him, Matt remembers that his father says the Indians in these parts are friendly.