This tool works by selecting from a pre-generated list of positions. This means that if you keep hitting "New Position", you will eventually see some repeated positions. (Currently there are 100 positions for each setting.)
The script that generates the positions simply plays random moves and then queries KataGo to see if the score evaluation is within ±0.3 points of equal (using Chinese rules and 7.0 komi).
The "Joseki" positions are from the OGS joseki dictionary (thanks to lightvector for providing these!), and they have also been filtered to only include fair positions.
If you select "Custom" from the dropdown menu, you can set your own parameters. These positions are generated directly in your browser, and there is no KataGo query to ensure fairness.
To play a handicap game with random placements, I recommend setting spacing to 1 and to generate one more stone than with normal handicap. For example: if you are 3 ranks stronger than your opponent, they get 4 randomly placed stones on the 19x19 board.
(Randomly placed stones are on average worth slightly less than starpoints, but not by much, so this handicap is generally worth more than the normal handicap, if you ask KataGo. However, the extra complexity of random placements probably favors the stronger player, so I believe this "normal handicap + 1" rule to be relatively fair in practice.)
Please leave feedback or bug reports on GitHub or in this forum thread.