Difficulty running 3 video cards for IRAY render

jblackmd

Honorable
Mar 20, 2018
88
3
10,545
Hello all!

I use DAZ3d Studio and I render scenes in Iray. My system:

i7 4790/32 gig RAM/Gigabyte z97X Gaming 5//1 Tb SSDx2/GTX 1080 Ti x 2, GTX 980 Ti/Coolermaster Haf X/1200W PSU

On certain complex scenes that take more than around 20 minutes to render, I've been having problems getting the render to finish. I tested each card on the same scene, and when used alone each card will finish the render. Usually I could get both 1080 Ti's to finish, but sometimes not. I am using the newest driver that doesn't crash my Windows 10 to BSOD (391.35)

I have been able to get all 3 cards to run if I limit the power to each card to 80% using EVGA Precision XOC. So my question is why am I having problems running at 100% power? I tried just limiting the temp ceiling to 78 degrees but that doesn't cut it. My case is well ventilated, and I never run any of the cards above 78 degrees. In the bios I've tried setting the 4x PCIex slot to 4x rather than auto. I don't have any issues at all on simple scenes that don't use a ton of textures. Any help would be appreciated.
 

XxDarkMarioxX

Respectable
Dec 25, 2016
417
0
1,960
Have you tried just doing it with two off those 1080ti you have and remove the gtx980 and test it out because maybe the gtx980ti is holding back your performance do not know about doing SLI but with two 1080ti am sure that be enough power and graphics to do your IRAY render!
 

jblackmd

Honorable
Mar 20, 2018
88
3
10,545
Iray doesn't like SLI, it just sees the 2 GPU's as one, and doesn't utilize all the CUDA cores. I've tried both 1080 ti's and they usually both run at full power but not on all scenes, just the simplest ones. I've tried each card individually on a given scene and they all render to completion.

I found a power supply calculator by Coolermaster, which supports my power theory. By that calculator, at full power I would need 1270 W and I only have 1200. The system has been very stable and fast through multiple renders with my adjusted power settings to 80%.
 

jrlaudio

Prominent
Feb 11, 2019
1
0
510
Iray doesn't like SLI, it just sees the 2 GPU's as one, and doesn't utilize all the CUDA cores. I've tried both 1080 ti's and they usually both run at full power but not on all scenes, just the simplest ones. I've tried each card individually on a given scene and they all render to completion.

I found a power supply calculator by Coolermaster, which supports my power theory. By that calculator, at full power I would need 1270 W and I only have 1200. The system has been very stable and fast through multiple renders with my adjusted power settings to 80%.

The power setting in the GPU control panel is the point at which the turbo boost (and eventually the base clock) on the card begins to throttle back. It is not a setting of how much power is used, it's the point where turbo boost is limited or eventually base clock throttling is initiated. Setting it lower means the card begins to throttle back GPU & memory clock speeds as well as voltage limits earlier, significantly slowing the rendering process in Iray. It also can put limits on the number of CUDA cores being utilized in order to manage the requirement of the lower power-limit setting.

Power differences at 1200W and 1270W are not a significant difference. They are for all intents and purposes equal, although you are equally under-powered with zero headroom in both cases. Having a power supply with at least 20% more current capacity than what is calculated to be your maximum draw, is always preferred. There is no workaround for this reality, without severely impacting overall system performance negatively. It's a simple matter of physics. Limiting power consumption (another phrase for lowering system performance) is two steps backwards for one step forward, so to speak.

That being said, the real problem most likely is due to the fact that the chip set (z97) on that gaming MOBO only supports a maximum of 8 PCI lanes. A big problem in Iray using multiple cards. The number of PCI lanes is not often shown on MOBO spec sheets made for gaming, since gaming is not generally a bandwidth critical application comparatively. For DAZ/Iray I run three Titan Xp's across 80 PCI lanes on a dual Xeon workstation for that reason; @100% power BTW. Remember, those PCI lanes are shared not only by the PCI slots but also the SATA ports, M2 slots and other peripherals, depending on MOBO configuration. For three GPU's running Iray you do need at least 20 PCI lanes on the MOBO to fully accommodate the bandwidth required when rendering in Iray. Rendering is a vastly different thing than gaming rasterization. Also in Iray it is best to have each card in a 16X PCI slot.

And as you eluded to, remove the SLI bridge when using Iray. Even if you defeat SLI in control panels, the physical presence of the bridge will still effect Iray negatively. Also ... never OC GPU's when using Iray.

Also be aware that scene file size is limited to the card with the smallest VRAM, in your case the GTX-980ti at 6Gb. If the scene file exceeds 6Gb, Iray defaults to using CPU only to render ignoring all the cards. The memory is not shared across the cards, so the smallest card is the limiting factor. As far as Iray is concerned when running all three cards, all of them have only 6Gb of RAM each.

Additionally, in some specific cases and computer system configurations, DAZ's version of Iray will start acting strange, ignoring some cards while using others in an inconsistent manner. The parameters that effect this are power limits settings, driver version, OS, MOBO configuration, and a host of other factors. If this starts happening you basically have the wrong computer for the application.
 
Last edited:

jblackmd

Honorable
Mar 20, 2018
88
3
10,545
I built a new rig to accommodate all 3 cards:

Win 10
AMD Ryzen Threadripper2950 x
ASRock x399
64 gig RAM
Thermaltake Floriing
1300W
GTX 1080 Ti x3

I was hoping it would run all 3 cards to the completion of a render, and unless it is a very simple render, it wouldn't. The other issue that I don't think is fixable is that although the board has 4 slots, 2 of the cards have to be placed essentially on top of one another, making one of the cards run much hotter, into the 80's. I found that if I limit the temp on the hot running card, it basically finishes using primarily 2 of the cards. I think if the hot card was a hybrid and cooled better, the performance would be much better, but I'm not sure.

So I pulled one of the cards and put it back in my i7 rig. I figured that now with just 2 cards running I shouldn't have any issues. I did. For a render to finish, I had to set the power on one of the cards to 80%. Temp wise both cards are running in the mid 70's. I just don't get why I am having so much trouble running multiple cards, unless the ASRock RIG really just isn't up to it despite the specs. With just 2 cards I figure that 1300W should be enough power.

If anyone has any suggestions what I need to do to have at least both cards run at 100%, I'm all ears.