I've been looking for a decent late Windows 98/Early XP era gaming build for a while, and it's been a bit of an issue finding decent hardware.
The primary issue is that all of the motherboards from this era are prone to the capacitor plague and thus may or may not work, and a huge number of boards will need to be recapped at least, assuming no other damage.
This is repairable, but I'm lazy and much prefer to start from a known-good base rather than attempt to fix random un-special pieces of hardware with an unknown fault and hoping it's just one of the dozen or two caps that are causing it.
I ran across a Dell Dimension 4100 on eBay for $90, shipped, that's turned out to be both incredibly reliable and perfect for the late 90s retro gaming I was after.
It's a Pentium III 1ghz box with 512mb of ram, the original 20gb HDD which I added a 16mb Voodoo 3 and Vortex 2 for, and everything is working brilliantly.
This doesn't sound like anything special, but I've had a heck of a time finding a working motherboard from the era that was happy with both a Voodoo 3 and Vortex 2 at the same time - AGP Voodoo 3 cards are 3.3v and aren't usable in AGP 4 or 8x slots, which are 1.5v which means a huge, huge portion of later motherboards are not usable, and that results in the pool of available motherboard that land in the era I'm after is dramatically reduced.
I was rather surprised when I was able to take the system out of the box, plug it in, and it immediately posted and dropped me into the BIOS; a system with no faults in this build was something new. Added in the Voodoo 3 and the Vortex 2, hooked up the hard drive (which was just missing a cable and was listed as 'no HDD' in the listing) and stuck in a Windows 98 SE CD, and the install completed with no issue.
Added some drivers, and I was ready to go.
The Voodoo 3 works just as well for older Glide games (Tomb Raider, Carmageddon 2, Quake 2) as it does on newer Glide games (Diablo 2, Mechwarrior 2) as it does on DirectX/OpenGL games (Anachronox, Quake 3).
Quite happy with how this build has come out, and quite happy with how reliable it's been so far - no crashing, no weird performance issues, no hardware conflicts, or anything that's required any amount of troubleshooting.
There are, of course, some Dell-isms in the system - most notably the proprietary power supply - but there are plenty of workarounds for the limitations imposed by Dell's weird desire to be semi-special in the late 90s and early 2000s.
All in all, it does exactly what I want, without having cost an arm and a leg (the Vortex 2 cost more than the Dell!), which is really the most one could ask for old computers like this.