If I want to put it simply, I could say that the new architecture of Xeon is not that much of a big deal. The CPU core is still the well known 'Deschutes' core, used in the Pentium II as well as in the Celeron processor. The big trick is the Xeon's L2 cache. Whilst Celeron has to live completely without L2 cache until September and whilst the Pentium II offers only 512 kB L2 cache running at half the core clock speed, the Xeon is supplied with 512 kB, 1 MB and soon 2 MB L2 cache running at the same clock as the Deschutes core. It sounds simple, but it took a lot of work designing L2 cache chips which would run at 400 MHz and above. Intel is producing these chips by itself, different to the L2 cache chips found in the Pentium II. The Cache chips are called 'CSRAM' for 'custom' static RAM, where custom only means that Intel cannot buy it from a SRAM manufacturer. The chips come in sizes of 512 kB each, so that a Xeon with 512 kB includes one, the 1 MB version two and the 2 MB version will include 4 of those chips. These L2 cache chips are in a package that looks identical to the Pentium II core package, which should show you how thermally sensitive those chips are. Cooling is required for the L2 cache chips, which is why the cartridge of the Pentium II Xeon got extremely huge.
a comparison between Pentium II and Pentium II Xeon shows how huge the Xeon cartridge really is
So the faster L2 cache is the main reason why under certain conditions the Pentium II Xeon will perform faster than the Pentium II. However, there are quite a few more special things about Xeon. First of all it enables all the features of the good old Pentium Pro, including support for quad CPU systems and even 8 CPUs in one system in combination with 450NX and a special cluster controller. The cacheable memory limit lies not only by 4 GB, but it can address and cache up to 64 GB memory by using 36-bit memory address bus and the new PSE36 mode. The Xeon is of course running at 100 MHz front side bus, enabling a higher memory bandwidth of up to 800 MB/s peak. ECC, the error checking and correction is definitely required in servers and often in workstations too, so the Xeon offers ECC for main memory via the chipsets 440GX and 450NX as well as using an ECC L2 cache RAM. For manageability the Xeon has a thermal sensor, responsible for keeping it at a save temperature, as known from the Pentium Pro already. Completely new and very pleasing is the new PIROM in the Xeon, standing for 'Processor Information ROM', which includes "robust addressing headers to allow for flexible programming and forward compatibility, core and L2 cache electrical specifications, processor part and S-spec numbers, and a unique electronic signature", so that counterfeiting of the Xeon is hopefully impossible now, but possibly making overclocking impossible too. However, overclocking is light years from common practice in servers or workstations anyhow. System vendors can program a special EEPROM chip within the Xeon cartridge as well, called 'scratch EEPROM. This EEPROM is supposed to host information like "system specifications, inventory and service tracking, installation defaults, environment monitoring, and usage data. Its contents can be write-protected by the system, as well."
Having a look inside the Xeon w/512 kB L2 cache can show you the different components:
At the top you can see the 'CSRAM' L2 cache chip, the bottom shows the well known Deschutes CPU core. The two chips on the right are the thermal sensor and the 'scratch EEPROM' underneath.
The little chip on the left is the 'PIROM' chip, containing information about the Xeon chip.