From Bulbapedia, the community-driven Pokémon encyclopedia.
A battle judge in partially standard apparel (Olivier
A battle judge, also known as a referee, is a person in the anime that presides over the judgment of Pokémon battles between Trainers in official, designated challenges, identifiable by the Poké Ball symbol on their shirts. They are found at tournaments, championships, and even some Gym Leader matches when a Gym Leader does not designate a personal hand to referee the match. There is no game counterpart to a battle judge.
Most Gym Leaders, however, do not have official league judges, and instead instate a capable student, intern, assistant, or even a family member to aid in calling matches between Leader and Trainer. Their calls are official, and appear to have the same judgmental powers as learned battle judges. This is not the case in most Kanto Gyms, where no official or unofficial referee was present to rule, except in the case of Agatha in the Viridian Gym, who allowed Scott to judge the match against her and Ash; the Gym Leaders primarily self-judged the match. Robots can also referee battles between Trainers, as seen in Ash's battles against Spenser, whose robot is named Jamero, and Volkner.
Gym Leaders themselves are qualified judges. Ex-Gym Leader Brock has provided judgment for many Trainer battles over the course of the series, Roark presided over an official Gym match between Byron and Ash, and Flannery's grandfather—the former Lavaridge Gym Leader—took up the position as the judge for her match against Ash.
The anime episode Judgment Day! showcases the training a person must go through to become a judge. In Hoenn, the Pokémon Battle Judge Training Institute is located on Bomba Island, where Professor Serena teaches and ex-student Jimmy studied. Jimmy mentions that for battle judge training, this school is the "only one there is," either indicating that all judges are trained out of Hoenn, or it's the top school for such training (if not one of the top schools).
According to Brock, there aren't many judges capable of evaluating a battle at the level necessary for battles sanctioned by the Pokémon League, so only so many exist under the profession of "official." Max hints at the profession's prestige and esteem, that given so few are capable of judging a League battle, simply being able to is an honor.
All battle judges wear a long-sleeved shirt with a standard black, turn-down collar and a Poké Ball emblem on the left side, over the heart. The colors of the shirt vary, but the more common shades are orange and blue. Aside from this, anything else goes when it comes to leg-wear, but the standard appears to be shorts. The shorts are usually lined, at the hem of the legs, with the matching color of the shirt.
Robotic judges, on the other hand, vary in size in shape, but they are normally equipped with red and green flags, the flags designating opponents (and not fouls).
Judges are trained to critically evaluate the battle field, which includes the Pokémon, the Trainers, and the field itself, if the terrain isn't ordinary. They have to be able to recognize when a Pokémon has fainted (temporarily knocked unconscious, as in some cases such as Pikachu's) and announce a winner through defeat, by determining all aspects of a Pokémon, including its type-cast and what a species as a whole is capable of (attacks and abilities).
They are responsible for officially beginning a match and upholding the rules of the battle, such as how many Pokémon a Trainer can use and what are the clauses, such as substitution limitations, making it clear to all Trainers involved what the parameters of the match are. The terms for a win are also in their ruling, as in what's required for a Trainer to lose the match. In general practice, the default appears to be until all the available Pokémon on one side are "unable to battle."
Their ruling is usually absolute, so making the correct call is imperative to the eventual announcement of a match. A wrong call might lead to the questioning of a judge's abilities to referee, especially in a professional setting. According to Serena, a battle judge must have confidence that the decision and ruling they're making is correct, otherwise leaving room for error.