"Load spikes" under transient heavy CPU load

freeMaN88

Reputable
Oct 16, 2016
3
0
4,510
Hi I'm running a 500W XFX XT500 psu to power 2 x 130W e5 12-core xeons hooked off a eaton 9130 2000VA rackmount UPS with double conversion. Other than the CPUs, no other major sources of power consumption beside small 30W GPU card, 32GB ECC ram and SSD. Load varies from 50-100W (low cpu usage) to 300W+ (max CPU with all 22 cores 100%) periodically as my workstation undertakes heavy CPU calculations for a few seconds and then CPU usage goes back down again. My UPS power management software is showing occasional big "load spikes" (150% of 1800W rated??) during periods of heavy CPU usage and/or transient load, when the load plot should usually be around 15-20% (which includes 3 monitors). During these spikes (coincidently or around similar time?) my software breaks down for whatever reason. In the absence of these spikes, all seems to go well.

Could it be that the PSU just isnt up to the transient CPU load? I can't imagine it to be a power issue from the wall since its running on a quality UPS.
 

freeMaN88

Reputable
Oct 16, 2016
3
0
4,510


Thanks. The power supply is actually not that bad technically with Jap capacitors and ripple between 33-45mV (well its not horrible), but I think it doesnt handle temps 40dC+ very well. I wont be able to replace the PSU until another week or so, so I have disabled HT and tuned down CPU turbo to keep the wattage variance and temp down as low as possible in the meantime. If it runs clean, then I would have to agree with you that its likely the PSU regulation.

I'm not sure what metrics to focus on for my kind of usage scenario in choosing an alternate PSU: a PSU with ultra-low ripple (the current CPU I think has decent ripple), fast transient response time during the initial voltage drop-off, or a smooth "turn-on" transient load curve on the 12V rail.

UPDATE:

Well I dont get spikes as often turning off HT and turning down boost.

I tested the software on a separate single socket Xeon machine with an even cheaper 300w inwin power supply, and it has the same glitches, but much less frequently (maybe once a day or less) as expected from the load of a single CPU. I hadn't expected the Xeons to be so sensitive to power supply - it even needs a little warming up first before cold boot. Have hooked up a new EVGA 850W T2 to the single socket Xeon first to try to isolate the problem further.

The EVGA is a solid unit. Has simply massive lumpy cables and strangely am a little worried about something blowing up. But it looks like it's the best $$ can buy right now.

UPDATE2: well unfortunately I am still getting system instability despite using the new powersupply. I have narrowed it down to the design of the Haswell CPU and its voltage regulation. I have been monitoring the CPU cores maximum VID which becomes higher over time as heat increases, typically after 3-5 hours of continuous transient load. Instability results when 2 or more cores start exceeding their normal VID range.

I have also been using the High Performance power setting with minimum and maximum processor state set at 100% which probably didn't help as I guess they use more aggressive P states with higher voltages and frequency. VID seems to correlate more with individual core processor frequency since under full load, all core frequency is lower. Setting the minimum to 5% helps stabilize the VID voltage ranges a lot, and increases stability up to 5 hours continuous runtime. I suppose setting the maximum to 99% would cut down the turbo boost and resulting maximum voltages, which gave the increased stability. Using balanced power mode seems to keep VID range even tighter as it seems to keep CPU frequency closer to its all core frequency, the most stable frequency. I think under balanced I should get my required 6-7hrs+ runtime and stability.

I was going to buy a 5960X to play with overclocking but now that I see how stability can be affected by even a small increase in VID for high core counts I'm not willing to mess around with any sort of voltage play in Haswells.