If we take other medias, and even official arts, into consideration the towns are far bigger then depicted. Just like everything else, we only get basic depictions of them. We don't need to see anything anyway.
However I've always seen the home town as being exactly what it looks like. Four - six houses and your friends house being a few feet away.