Sign in with
Sign up | Sign in
Your question

Extreme6 DRR3 RAM Timings

Last response: in Motherboards
Share
September 17, 2012 4:58:44 PM

I am experiencing system lockups approximately every 3 - 5 days. Anyone out there know what the timings should be for the following build? Nothing is overclocked, and RAM timing is currently set to Auto.

Note: If I reboot the system, say every 4 days, I experience no lockups at all...


Build:
ASrock X79 Extreme6 LGA 2011 Mobo (Bios Version 1.50 Current)
http://www.asrock.com/mb/overview.asp?cat=Download&os=B...

64GB G.Skill Ripjaws Z Series (8 x8GB) Quad Channel DDR3 1333 10666 Memory
http://www.newegg.com/Product/Product.aspx?Item=N82E168...

Intel Core i7-3820 Sandy Bridge-E 3.6GHz
http://www.newegg.com/Product/Product.aspx?Item=N82E168...


Does anyone have any recommendations, or seen anything like this before? Both ASrock and G.Skill claim that no adjustments are necessary for RAM on this board, but the issue continues to happen.

Currently running Memtest without any errors thus far, but its taking quite some time for 64GB of ram.





September 17, 2012 8:26:27 PM

Memtest86 finally completed. It ran for 4.5 Hours, with zero errors. If anyone has ideas, please let me know!
m
0
l
a b V Motherboard
September 17, 2012 8:29:23 PM

Right on the link for the memory you provided is the speeds Newegg posted: Timing 9-9-9-24

System lockups can be from the processor not having enough voltage. You can test the CPU with prime95.
m
0
l
Related resources
September 17, 2012 8:33:30 PM

sounds like you have all of your Ram slots full right? I have that same board, it's just a hunch but try taking out half the RAM and try just leaving slots A1 and A2 (the one closet to the cpu and the third one out) and see if that helps? Honestly i wouldn't think it was ram issue at all but if it is that would be the only thing I could think of. you could also try setting the speed from T1 to T2, T2 is a bit looser speed if I remember right from some overclocking guides.
m
0
l
September 17, 2012 8:40:09 PM

Prime95 ran successfully on all 8 cores without issue.

I had the RAM settings to auto at 1333, but I like your idea of changing it manually to the ram specifications, ahnilated.

mcopinger, yeah, all 8 ram banks are full. I can't really remove sticks, because its a HyperV host, and uses ~55% of the 64GB nearly all the time.

I'll set the timings manually, and see if it locks up again. Appreciate the feedback and will update if resolved or not.
m
0
l
September 17, 2012 8:48:43 PM

Looked into the BIOS further...

Ram timings were on "auto" and the voltage was 1.527


Changed it manually to the correct settings:

9-9-9-24
1.5V

I'll update on any progress.
m
0
l
a c 716 V Motherboard
September 17, 2012 8:51:33 PM

On any 8GB/stick density kits I recommend that you modify the both the VCCSA -> 1.20v and VTT Voltage -> 1.10v~1.20v.

Q - Are you running any form of RAID?
Q - What [+] Critical errors are listed in the Event Manager if any? - http://i1013.photobucket.com/albums/af254/Jaquith/Event...
Q - What are your full specs? (SSD, HDD, PSU, GPU(s), etc)

* Verify that on both your SSD or HDD that you have the latest firmware installed.
* Verify that you are using the latest drivers from ASRock - http://www.asrock.com/mb/overview.asp?cat=Download&Mode...
* Verify that the latest Intel drivers are installed - http://www.intel.com/p/en_US/support/detect?iid=dc_iduu
m
0
l
September 17, 2012 9:14:52 PM

jaquith,

A. Yes, LSI MegaRAID 888ELP 5 1.0TB Seagate's in a Raid 5, one 160GB WD System
A. Critical Error's (3 total) Kernel-Power Event ID 41, Task Category 63
"The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly."

A. Full Specs:

ASrock X79 Extreme6 LGA 2011 Mobo (Bios Version 1.50 Current)
64GB G.Skill Ripjaws Z Series (8 x8GB) Quad Channel DDR3 1333 10666 Memory
Intel Core i7-3820 Sandy Bridge-E 3.6GHz
Athena Power AP-RRP4ATX6508 80 PLUS 500W Redundant Server PSU
GeForce GT210 Low Profile Graphics Card
All housed in an Antec 300 Case

5x 1.0 TB Seagate Raid 5 Disks
1x 160GB System Disk

Operating Temps:
CPU 28 degrees Celsius w/ arctic silver 5
Motherboard 24 degrees Celsius
Graphics card 29 degrees Celsius

Latest Drivers from ASrock have been installed.
Intel Driver Utility indicates all my drivers are current.

m
0
l
a c 716 V Motherboard
September 17, 2012 9:57:25 PM

The RAID Card itself can be the culprit, and I assume the RAID drives were not connected when you installed the OS, and at minimum that the BIOS didn't change from it's original state (e.g. AHCI -> RAID after the fact); normally this would require the RAID Drivers being loaded @ OS install. Any RAID I've done on the X79 I've injected (~F6/RAID drivers) at OS install. Also, if the BIOS resources are limited or borderline then sure the LSI can cause the system to drop-out; verify the LSI is compatible with your ASRock. In a RAID I assume you verified that ALL of the HDD's had the SAME firmware before migrating to a RAID 5.

The PSU is barely large enough under full load and if it's older (1 year +) then the PSU can be causing the system to drop-out (41/63). In your system I recommend a 650W minimum PSU.

Check for yourself - http://www.thermaltake.outervision.com/Power
TDP 100%/System Load 100%/25% Aging = 610W
TDP 100%/System Load 90%/25% Aging = 554W
TDP 90%/System Load 90%/25% Aging = 539W
MOBO: Highend, NO CPU OC, 5 high RPM HDD, 1 SSD, 8xDDR3, Geforce 210, 1 DVD, 1 PCIe card, 6x120mm fans, and see the TDP/Load/Aging above.

What I'd recommend is running AIDA64 Extreme, use all stress test options and let her rip - http://www.aida64.com/downloads If I'm building a 'Workstation' besides the fact that I use Xeon I run AIDA64 Extreme for 48 hours straight before I install any applications.

(41/63) is generally a hardware issue and looking at what you've got here the PSU jumped out at me. I get it's a redundant PSU so the next culprit is the RAID Card. So you can remove the RAID Card and stress test -- process of elimination.
m
0
l
September 18, 2012 2:08:36 PM

jaquith,

At this point, anything is suspect so I appreciate the time you have taken to help me with this. You're correct, the RAID array was not connected during OS installation. It is running server 2008 Enterprise R2 btw.

I'll shoot off an email off to ASrock again, inquiring about the LSI card compatibility. How do I go about locating the firmware versions of each hard drive? In the LSI MegaRAID software, all drives are listed but I'm not seeing anything stating the actual firmware. I will say that I have updated the firmware on the LSI RAID card after I started seeing these issues.

I understand your assessment of the PSU, however I also used http://extreme.outervision.com/psucalculatorlite.jsp to calculate before I built the machine.

Also, I did another check with thermaltake calculator, and I get almost the same results, 410~459W. The capacitor aging seems to really tip the wattage requirements higher.

The system is barely 6 months old, so I'm not sure how much that factors into the equation. If it helps at all, I rarely see that i7 get pushed to 80%. It averages, even with VMs running, 1 -15% load.

The aida64 stress test seems to be the next logical thing for me. I'd like to get something concrete. If the PSU / RAID cards seem to be the culprit to you, I'll keep investigating until I have a definite answer. I would be okay with replacing parts, as long as there is some definitive answer for the cause of the system locking up.

Again, thanks for your time.



m
0
l
a c 716 V Motherboard
September 18, 2012 6:01:49 PM

Well I used the same exact calculator and the numbers were what I posted above. Capacitor Aging is very important because the PSU looses capacity soon as it's first switched-on, some PSU's exceed their rated and others don't, so at minimum I use 35% and not the 25%. Also, a little known fact of PSU's is 'ripple' and as the PSU reaches Rated capacity can be very bad (depends on the PSU); example here's a 1200W Rated but running 1000W (~17% under Rated) see - http://www.youtube.com/watch?v=MCuKM6Ux4B4&feature=play...

Anyway, I hate PSU discussions...Ideally 'I' prefer @ load the PSU run between 40% to 60% of Rated (300W = 600W PSU). Less noise, Less strain, Less Ripple, and Less Problems.

Let me know how the AIDA64 testing goes.
m
0
l
a b V Motherboard
September 19, 2012 2:04:22 PM

jaquith said:
Well I used the same exact calculator and the numbers were what I posted above. Capacitor Aging is very important because the PSU looses capacity soon as it's first switched-on, some PSU's exceed their rated and others don't, so at minimum I use 35% and not the 25%. Also, a little known fact of PSU's is 'ripple' and as the PSU reaches Rated capacity can be very bad (depends on the PSU); example here's a 1200W Rated but running 1000W (~17% under Rated) see - http://www.youtube.com/watch?v=MCuKM6Ux4B4&feature=play...

Anyway, I hate PSU discussions...Ideally 'I' prefer @ load the PSU run between 40% to 60% of Rated (300W = 600W PSU). Less noise, Less strain, Less Ripple, and Less Problems.

Let me know how the AIDA64 testing goes.


+1 totally agree. The PSU can be a major source of headaches if it isn't big enough. I always overspec the PSU on my systems just because I would like it to work for 10 yrs.
m
0
l
September 19, 2012 2:21:28 PM

Thanks for the input, all his information is definitely useful. I'm with you guys... been using a larger PSU than necessary in almost every build I've created. Only reason I have the one listed is due to it's redundant nature. I created a pseudo-server/dev. environment which required it had dual PSU.

Here is a link to it... the power supply itself was well over $300.00:
http://www.newegg.com/Product/Product.aspx?Item=N82E168...

Definitely didn't anticipate running into issues such as this.

After adjusting the RAM timings, server has an up-time of nearly two days. Waiting until approx 5 days of up-time to see if it crashes. At that point I am going to run AIDA64. Will post results, otherwise, time to invest in a new power supply I think.
m
0
l
a c 716 V Motherboard
September 19, 2012 3:48:53 PM

If the PSU is the culprit then the Athena Power AP-RRP4ATX6708.

So you have a wattage load meter? Some UPS have them integrated. I assume here that you have an UPS - right??!! Assuming so I would look at the events to see if there's any correlation and that your PSU and UPS are compatible (i.e. no Sine wave issues).

I would invest in a meter e.g. P3 - http://www.newegg.com/Product/Product.aspx?Item=N82E168...

The thing here is the fault period is rather long (days), and nailing down the reason isn't easy. Days are quite often 'power' related but too source power related. Any build I do I assume everything is potentially bad until long stress tests prove otherwise. Even when we purchase a new server it doesn't become integrated until it proves is can hold up and is free of defects - stress testing for days is the only proof.
m
0
l
September 19, 2012 4:06:06 PM

I was just using a calculator online. Turns out we have the Kill A Watt EZ in the office. Now I can test the wattage draw effectively.

http://www.p3international.com/products/p4460.html


To answer your question, yea the machine is on a UPS battery backup system, along side a couple other servers on that battery unit specifically. Unfortunately, not seeing anything weird with the other servers on the UPS, but this is the only server that has a different type of PSU installed. This wattage meter is a step in the right direction at least.
m
0
l
a c 716 V Motherboard
September 19, 2012 4:11:52 PM

If you unplug the UPS and the PC (server) fails then I'd say you might have found the problem not to mention another if it's not the root issue. My best guess is 250W~350W with some marginal load @ the wall then factor in the PSU's efficiency rating and you've got your actual draw.
m
0
l
September 25, 2012 3:28:34 PM

With the new RAM timings, the server made it 8 days. Finally locked, although this time the lockup was much harder than usual. The BIOS completely reset itself, and it wouldn't post at first.

The RAM timing settings had obviously been cleared out. Now I have my opportunity to hookup the KillaWatt and see what the actual draw is. I have taken the server out of the data center, and moved it to the bench (eliminating any UPS issues, and allowing me to keep an eye on it).

I'll be monitoring the power consumption, and running AIDA64. Also turns out that I have a spare RAID card that matches this one to swap if needed. Will post results of power / AIDA as soon as I can.
m
0
l
a c 716 V Motherboard
September 25, 2012 4:15:35 PM

Well now is also the time to check all of the firmware and drivers are up to date, and test.

Often on failures you'll get a 'Press F1' which can mean a variety of things including a generalized H/W failure i.e. without a post code it's a guessing game and even with one 'cause & effect.'

All I can do from here is to observe what you've written and come up with the most logical explanations. RAM is also a likely candidate, which is why on anything server related:
* Xeon
* ECC including RDIMM RAM
* All drives in RAID with error recovery control
* Supported peripherals e.g. RAID Card
* Rated (tested) components Server vs Consumer class
* Correct UPS e.g. with 'sine wave' ; example APC Smart-UPS not Back-UPS
m
0
l
September 25, 2012 6:40:47 PM

jaquith,

All of your points are well taken. Here are the results. I have run the AIDA64 Software stress test, for 3 hours at 100% without any issues. The machine wattage draw average is 235 Watts. There are also VM's running while this test is going, adding additional stress to the system. Unfortunately, I am unable to reproduce the lock-up.

We have ordered a new power supply:
http://www.newegg.com/Product/Product.aspx?Item=N82E168...

Albeit not a redundant server grade product, but we will use as a test that can rule out the power supply. If it turns out to be the PSU, we will then order new, higher wattage redundant unit.

Again, I appreciate you taking the time to work on this. I've been very frustrated with this issue, mainly because I cannot find anything concrete. Typically by this point, with all tests and things considered, SOMETHING would have indicated some sort of hardware failure or issue.

As if the frustration isn't enough, I also broke a blade on one of the 140MM fans when I opened up the tower. Good Times :) 

It'll be a while before I post an update, but just wanted to keep you, and anyone else interested in the loop.
m
0
l
September 25, 2012 8:01:28 PM

Quick Update: LSI has updated firmware that I apparently missed. Nothing relating to power consumption, but some things related to memory on the RAID card itself. At the end of the work day, going to attempt to swap the RAID card with our spare, and update the firmware/drivers for the raid card.
m
0
l
a c 716 V Motherboard
September 25, 2012 8:36:15 PM

Yep the firmware checking and updating is a good idea.

You need to run AIDA64 for more like 48 hours, thermally 4-6 hours things are at peak and level out. Then run Memtest for about the same amount of time. Yeah, that might seem like a long time, 4 days, but join the club and most of the Workstations I build go through very rigorous testing before they go out the door.

We've all got our oopsy stories, thankfully your's is cheap.

Here's the thing, RAM will throw 1-2 errors a month for the typical home user with 4GB densities, but 8GB exacerbates those errors. I've found 8GB/stick non-ECC to be very unreliable and quite often problematic. Top that off with the guessing game of the RAID Card and twist in some potential power issues and it's like throwing dice. If your 'peak' not average load is no more than 60%~70% of the PSU's capacity then you can probably take that off the list.

RAM:
Use either the XMP setting and or manually tweak the following voltages, none of them are high; they're about +0.05v above defaults but with 8GB/sticks might reduce errors.
VCCSA -> 1.20v
VTT Voltage -> 1.10v~1.20v
PLL Voltage -> 1.85v
m
0
l
September 26, 2012 3:47:28 PM

Swapped the RAID card, and updated the firmware. One more item to cross off the list. New PSU Arrives tomorrow, will install that and see what happens.

I have to look for the voltage settings you are referring to, I did't see them on my quick glance yesterday, but the BIOS is pretty extensive, I'm sure I'll have to find it.

Will let you know when PSU is installed, and the VCCSA/VTT/PLL voltages are set per your specification.

m
0
l
a c 716 V Motherboard
September 26, 2012 4:33:16 PM

The BIOS locations of voltages and settings for the following:

OC Tweaker Screen / DRAM Timing Control ; page 76~77 of the manual:
Load XMP Setting -> Profile 1

Voltage Control / Voltage Configuration ; page 90 of manual:
VCCSA -> 1.20v
VTT Voltage -> 1.10v~1.20v
PLL Voltage -> 1.85v

The XMP/Profile 1 may load: VCCSA, and PLL Voltage, so I'd compare before and after voltages. Again, every voltage I posted is within safe limits.
m
0
l
September 27, 2012 8:50:29 PM

Update:
This has been installed.
http://www.newegg.com/Product/Product.aspx?Item=N82E168...

Couldn't locate the XMP settings in BIOS. However was able to set the RAM timings, and VCCSA, VTT, and PLL voltage settings within the ranges you specified. We must have different manuals... pages 66-67 are in German :) 

Replaced the RAID card with a brand new one. Odd occurrence here.... the wattage draw dropped by nearly 100watts when I swapped in the new RAID card. Your initial hunch may have been right on key.


Now we wait. I'll keep the information flowing.


m
0
l
a c 716 V Motherboard
September 27, 2012 10:27:28 PM

rjkunde said:
Update:
Replaced the RAID card with a brand new one. Odd occurrence here.... the wattage draw dropped by nearly 100watts when I swapped in the new RAID card.

Hmm, interesting. Yeah, please do keep me up to date; I'm here till the damn thing works...hopefully now it will. BTW - RAID cards can get toasty, so if there's a monitor keep on eye on the temps.
m
0
l
a c 716 V Motherboard
September 30, 2012 2:05:50 PM

Just an observation here, but it seems too coincidental:
(freeze period 3-5 days) - http://www.tomshardware.com/forum/320565-30-extreme6-dr...

(freeze period 3-4 days) - http://www.tomshardware.com/forum/320548-30-asrock-extr...

Common elements:
ASRock Extreme X79 series MOBO's
G.SKILL RAM ; though I am not certain that's the problem

What I think is the root cause is the MOBO and or it's BIOS. After running straight for days at a time a MOBO can have heat issues in that if there's something weak e.g. VRM or other components then you can have a dropout (failure).

If you search (Google) 'asrock extreme x79 days freezing' you begin to see a pattern.

So if the problems continue, ASRock's Tier 2 level support may have better insight into the problem(s).

--

Now I'm not trying to bash any MOBO, but there have been so many issues with the X79 platform from Gigabyte, MSI and others that since I adopted this platform even for myself -- the only X79 that has been immune from many issues including OC, Temps and for the most part Stability has been ASUS in the consumer level MOBO's. Most of the key components in any series of MOBO are essentially the same, and in being the same if there's similar root issues (design/component) flaws then the problem can be systemic including bad batches.

Therefore, if the problems continue then my next step would be to contact ASRock and discuss the problems and RMA options.
m
0
l
October 11, 2012 3:20:12 PM

Jaquith,

Nice detective work. Funny you mentioned that the RAID cards get hot, because I actually custom fit a fan to lower the temps on the last one. Anyway, long story short:

RMA'd the old RAID card. Waiting for a replacement any day now.
Wattage draw is far more consistent (125 ~ -140) Typically
Server up-time is 14 days now. fingers crossed!

Since I changed two variables on this build (PSU, and RAID card), I am waiting until the magic 30 day mark. If it makes it to this, I will switch back to the old (still new but lower watt) redundant PSU and let it run again. Given the results so far, I am assuming it was in fact the RAID card, but just to be certain..... will do my due diligence.

Thank again for all the help! I'll update with any new info.

m
0
l
a c 716 V Motherboard
October 11, 2012 7:33:49 PM

My first thought was RAID or SATA related, and I've also seen a trend with ASRock and third-party SATA.

Yep keep me updated and so far it seems good. :) 
m
0
l
October 15, 2012 2:09:59 PM

:-(

Sad to report, on the 15th day, the server eventually locked up. It would seem that all indications point towards an issue with the motherboard. I'll be contacting the ASrock Tier-2 support... maybe I can finally get this issue resolved.
m
0
l
a c 716 V Motherboard
October 15, 2012 2:25:20 PM

If I were Tier 2 the first question is use and then I'd give you the 'proper tool' lecture:
Quote:
* Xeon
* ECC including RDIMM RAM
* All drives in RAID with error recovery control
* Supported peripherals e.g. RAID Card
* Rated (tested) components Server vs Consumer class
* Correct UPS e.g. with 'sine wave' ; example APC Smart-UPS not Back-UPS


However, it will be interesting what they say and two full weeks on a consumer MOBO and equipment isn't totally unexpected. In light use then I'd expect longer, but heavy use especially with 8GB/stick density RAM that's non-ECC I guarantee you had one or more errors in that period of time even with minimal loads.

Even my Home Windows Server uses Xeon, ECC, RAID 6 (plus hot spare), Adaptec RAID, and the only place I 'cheated' is non-enterprise HDD's but they all have ERC.

--

Let me know what they say and good luck. I'm very interested and thank you for the update! :) 
m
0
l
October 15, 2012 3:06:07 PM

You're totally right about the correct tool argument. If this were anything more than a test-bed environment, money would have been invested, certainly for enterprise grade equipment. Seems a hard lesson had to be learned here.

I can move the VM's while the board is being replaced, and might consider a different motherboard manufacturer too. If I have to drop in a new CPU, RAM and MOBO to make it truly stable, it might be worth considering.

Thanks for all the sage advice.
m
0
l
a c 716 V Motherboard
October 15, 2012 4:32:03 PM

On the LGA 2011, trust me on this, ASUS. My ASUS Sabertooth X79 has been on for a long time plus I use it for heavy SQL testing (but 8x4GB - 8GB RAM Drive) and knock on wood...

Keep me updated.
m
0
l
!