I setup a server with up to 20 concurrent Terminal Server (RDP) users. It runs on an HP ProLiant with a single XEON 5140, 4 GB of memory and a RAID 5 configuration. The CPU is reasonably busy and each session requires approximately 150 MB of memory (VB application that accesses an Oracle DB, etc.)
You'll need to determine how much memory and CPU is required for each session to determine what server is required.