What leads the fishermen to their conclusions about the group of companions?

There are several things that lead the fishermen to jump to their conclusions, which are far more believable than the truth. One of these is that all six of the companions appear well dressed and waterlogged. The second of these is that they have wreckage strewn about them. The third reason is that they seem to have no recollection of what has spread far and wide over the land, nor do they seem to know where they are.