The story's three main characters (the Young Man, the Second Man, and the Friend) can be seen as symbols of various aspects of the French relationship with language. In this context, the Young Man's coat button can be seen as representing small variations from expected propriety.
Exercises in Style