From Bulbapedia, the community-driven Pokémon encyclopedia.
Diglett Village (Japanese: ディグタのむら Digda Village) is a small village on Red Rock Isle. Years ago, the villagers were trying to cultivate the infertile land of the island when they befriended the Diglett. The Diglett helped them turn the desert to lush, green farmland. This tale was told all throughout the Whirl Islands.
Over time, they have remained in Diglett Village but their children have migrated to a nearby town that was also built by the Diglett. Those original villagers have now become elderly but still wage battle against their adult children, who they believe to be a Band of Diglett thieves, because of their great pride.
According to James, the population of Diglett Village (not including Rita and Sue) is 20, while the Band of Diglett thieves numbers more than 50.
Pokémon seen in Diglett Village