Win 7 64-bit Photography Processing System Help

[dwightmccann]

APPROXIMATE PURCHASE DATE: By end of 1Q10. BUDGET RANGE: $3000+

SYSTEM USAGE FROM MOST TO LEAST IMPORTANT: General use is in support of digital photography business: Adobe Photoshop, Lightroom. Also normal business processes such as Firefox and Thunderbird, MS Office. Dreamweaver. Some very lightweight gaming ... Starcraft Broodwar. Considering doing some small video editing.

PARTS NOT REQUIRED: Monitors. I have Dell 3008WFP, Wacom Cintiq 21ux, Eizo CG241W.

PREFERRED WEBSITE(S) FOR PARTS: Newegg COUNTRY OF ORIGIN: USA

PARTS PREFERENCES: After reading the Parts V2 I am leaning toward:
CPU: i7 920
Mobo: ASUS P6TD
RAM: OCZ 3P1600LV6GK [Would two kits work for 12GB? Is that sensible?]
PSU: Corsair 1000HX
Case: Antec 1200
Cooler: Scythe MUGEN -2
HDD: I need about 3TB data [RAID 5], 3TB scratch [RAID 0] for several large caches, system disk around 500GB [RAID 1] for numerous large software packages.
Video: Two matching cards. [I have had a terrible time with two Gigabyte 8800 GT cards] to drive 30" displays (I think 2560 x 1600) as noted above.

I am open to suggestions on any component for reason but not just to have alternatives ... there are too many alternatives already. LOL.

OVERCLOCKING: Don't plan on it but am open. SLI OR CROSSFIRE: No.

MONITOR RESOLUTION: Two or more at 2560 x 1600 (?).

ADDITIONAL COMMENTS: I can increase the budget if necessary. Reliability is most important, particularly making a quick backup (to the scratch RAID 0), and to external drives. I'd like to use eSata if possible for offsite backups.

I would be interested in using an add-on RAID controller such as Adaptec or LSI and to use smaller SAS drives for an additional scratch/page disk if anyone is knowledgeable.

I have built two or three machines in the past with much help from workmates. There are always more issues than I am prepared for and unexpected problems. I hope I have included everything per the guidelines.

 

[dwightmccann]

Reading through some other posts I came across the Lian Li PC-P80 [http://www.newegg.com/Product/Product.aspx?Item=N82E16811112159&Tpk=lian%20li%20pc%20p80] at Newegg which seems more attractive than the Antec 1200 as there is more drive room.

 

[dwightmccann]

I guess I'm not making any huge mistakes or someone would surely correct me!

 

[specialk90]

I just saw this so I will start. As a former photographer and a current videographer in addition to loving fast computers, I can offer some insight.

I recently upgraded from a Q6600 with 8GB of ram and 4 10k rpm Raptors in Raid 10 for the OS and Raid 0 for the scratch disk to an i7 920, ASUS P6T and Intel X25 80GB SSD. The Intel SSD makes those Raptors feel 10yrs old, and 12GB of ram for Photoshop CS4 is very very nice.

For the video card, the new ATI 5xxx series have 3 outputs and Tomshardware has used multiple 30" Dells to test it (Ati calls it Eyefinity)

Can you explain your workflow and how you use the large cache?

When I had 8GB of ram, I used a Sony A900 (24.6MP) and shot a wedding for a cousin. The PSDs with the most work were 800-900MB in size and I noticed the drives were the limiting factor (in addition to poorly threaded Photoshop filters). What part of your current editing workflow seems to slow you down?

I will add more about LSI raid controllers and hard drives once I know a bit more.

PS I'm jealous of your Cintiq. I only have the Large Intuos4 (in addition to a CG243W)

 

[dwightmccann]

I have both the Cintiq and the Intuous 4 Large, as I don't have to justify whatever I want ... I am the boss. LOL. I actually much prefer the Intuous 4 and the Cintiq is essentially just a monitor.

I currently run WinXP with 3.33GHz CoreDuo (I don't recall the model ... I'm terrible with names ... or anything), 4GB RAM (although one OCZ chip died so I have ordered another kit), RAID 1 system disk, RAID 5 work disk (about 1.8TB) and a RAID 0 scratch disk with three drives and about 1.4TB ... I don't remember right off hand which drives and I'm not near my machine. I keep program files on system. I download images to work and post process them there. I put paging and Adobe cache and catalog on scratch. I also use scratch as first level backup of the work disk after each significant set of mods. I have a second box where I do weekly backups. I have two 2TB RAID 0 externals for "offsite" backups one of which goes with me to my day job and swaps about once a week. I use the onboard RAID for system and LSI X-800 (or something like that, it's a PCI-X board) for the other two arrays.

My workflow is: CF->work disk via drag and drop -> PhotoMechanic for first cull, rename, IPTC data -> Lightroom for second cull and catalog and conversion/post processing/export. Exports are usually jpegs with selected TIFFs for printing. I only use Photoshop CS4 when I need to do serious retouching or other operations unavailable in LR. I shoot Canon: two 1DMIII's, 1DsMIII and 5DMII and a 1DMIV on pre-order. I'd like to do a little video as the 5DMII and 1DMIV are capable but it's not a big interest.

Lightroom is unbearably slow ... I can only imagine it is written in Java! LOL. Photoshop isn't too bad but can get laggy. I can only imagine that video is an order of magnitude worse.

I currently have a pair of Gigabyte 8800 GT cards. For the year I've had this machine these cards have been a terrible problem, whether I have one or both cards in the box. After a few days odd things happen such as explorer going away or strange flashing or strange slowness. I wind up having to reboot. When I do, the video drives get 'Error 10 (Device can't start)' and the only cure is deinstalling/reinstalling the drivers which is painful as my LSI RAID controller takes about five minutes to load firmware ... but other than that it seems pretty good. Then I have to reprofile the monitors with my ColorMunki because it doesn't have a utility to simply respecify the old profiles. I have searched in vain for a cure. They don't seem to heat up as I don't play any games other than the old, old Starcraft Broodwar and I used the nVidia monitor and the temp never budged. Gigabyte was no help and nVidia was no help and I have only found a couple of other people with this problem and they didn't have a solution either.

So, do you use the SSD for system? That would worry me since I assume it is not RAID 1. I can see putting paging on it. I want avoid reinstalling the system like the plague as there is just too much stuff.

This will be my first foray into 64 bits. I'd like to have about 3TB work and scratch so I can continue to have quick first level backups disk to disk. I guess I am looking at two 6GB RAM kits?

So, what do you recommend for a RAID controller. I've looked at the Adaptecs that do both SATA and SAS, thinking of getting SAS for scratch disk and putting first level backup on yet another array. I'm starting to look at a lot of drives in this box I guess.

Guess that's enough for now. Love to have any feedback. I haven't ordered a component yet but can pull the trigger on anything any time.

I just saw this so I will start. As a former photographer and a current videographer in addition to loving fast computers, I can offer some insight.

I recently upgraded from a Q6600 with 8GB of ram and 4 10k rpm Raptors in Raid 10 for the OS and Raid 0 for the scratch disk to an i7 920, ASUS P6T and Intel X25 80GB SSD. The Intel SSD makes those Raptors feel 10yrs old, and 12GB of ram for Photoshop CS4 is very very nice.

For the video card, the new ATI 5xxx series have 3 outputs and Tomshardware has used multiple 30" Dells to test it (Ati calls it Eyefinity)

Can you explain your workflow and how you use the large cache?

When I had 8GB of ram, I used a Sony A900 (24.6MP) and shot a wedding for a cousin. The PSDs with the most work were 800-900MB in size and I noticed the drives were the limiting factor (in addition to poorly threaded Photoshop filters). What part of your current editing workflow seems to slow you down?

I will add more about LSI raid controllers and hard drives once I know a bit more.

PS I'm jealous of your Cintiq. I only have the Large Intuos4 (in addition to a CG243W)

 

[specialk90]

For your external Raid 0 backups, you are taking an additional risk by using Raid 0. Besides the inherent risk of Raid 0, should the external box die, you will need to use the same box in order to read that Raid 0 array. Maybe a newer version of that external box might work but why take that risk.

What part of Lightroom is slow - the editing of images, sorting or looking thru images...?

With a 64bit OS, Lightroom x64, Photoshop CS4 (x64), lots of ram and an Intel SSD, Lightroom and Photoshop will be much faster.

I have mainly used Bridge in addition to Photoshop. I can tell you that Bridge is MUCH faster with my current setup vs my prior setup. I now have the OS + apps on the Intel SSD and photos and video on a Raid 10 (4 1TB 7200.12s) on an Areca 1680ix controller. Before, I had 4 10k rpm Raptors for the OS/Apps(onboard Intel raid) and a Raid 5 (4 500GB 7200.11s) on a 3ware 9650SE controller. I am not very experienced with Lightroom, but in general, its a more advanced Bridge.

For the OS/Apps on SSD: I also didn't want to reinstall anything and needed max uptime (this is why I used Raid 10) but SSDs in general are far more reliable than mechanical hard disks. However, when I installed Windows 7 on the SSD, I made a backup immediately after; then again after Production CS4; then again after Cinema 4D, Nuke and all plugins. I was having major problems similar to you but it ended up being a bad ram stick. So, just make backup images of your OS drive and you will be good to go. (I use Acronis Backup & Recovery 10 Workstation)

With a SSD and 12GB of ram, its better to put the Page File on a separate drive from the SSD due to fragmentation. Plus, with 12GB of ram, you will rarely need the Page File and Scratch disk.

Here is my overall recommendation:

CPU: i7 920
Ram: 12GB DDR3-1333
Mobo: ASUS P6T
PSU: your current might be powerful enough - you want at least 750w (otherwise, Corsair, SeaSonic or PC Power & Cooling are the best)
Case: I don't know what you currently have and how many drives are in it but Lian-Li makes top quality cases that can easily fit 10+ drives (my Lian-Li can hold 12)
OS: Win 7 Pro x64

Raid & Drives:
LSI 9260 http://www.newegg.com/Product/Product.aspx?Item=N82E16816118105&cm_re=9260-_-16-118-105-_-Product
I like the LSI for a few reasons: 1) its PCI Express 2.0 and x8 so there is plenty of bandwidth 2) its SAS/SATA 6Gb so you are future-proofed 3) its very fast, very affordable and NOT an Areca. (I hate Areca but love my 3ware which is now LSI)

I don't like Raid 5 at all for one simple reason: if a drive fails, you are totally exposed until the new drive arrives and the array is rebuilt, which takes a VERY long time with your amount of data(12hrs to 24hrs or more for the rebuild).

For 3TB, you could do 4 1.5TB drives in Raid 10 or 4 1TB drives in Raid 5. I have had great success with Seagates and have 8 1TB 7200.12s.

For the Raid 0 cache, try either the onboard Intel Raid or the LSI 9260. With Raid 5 for the 3TB, having the Raid 0 on the same LSI will degrade performance a little when reading from R5 to R0 and even more so when reading from R0 to R5. But with R10, there is very little performance degradation. (this statement is based upon my experience and testing)

With the ASUS P6T, I would get one of the new ATI 5xxx series that support 3 displays. This leaves you with the LSI 9260 in the 2nd PCI E x16 slot. (1-PCI E x4 and 2 PCI slots open)

 

[dwightmccann]

First, I really appreciate your time and sharing of your experience!

WRT my RAID 0 offsite, I have two arrays and swap them. They are the backup of last resort. I am very aware of the fragility of RAID 0 and try to account for it by never putting anything on RAID 0 that I expect to rely on. But if my photography studio/office burns down I will still have 99% of my catalog.

Lightroom is extremely slow loading and slow bringing in large images in Develop. Oddly, in Develop, two or three images may load fairly well and the next may take 10 to 20 seconds ... I sometimes think the application has hung. Most of the adjustments seem OK but occasionally there will be a hang. I can't figure out what happens when this occurs. I only use the first two modules. Most of my workflow is the simple result of happenstance ... when I got a piece of software and learned it rather than some cohesive plan.

Long ago and far away I did System Disk images (and multiple boot, etc.) Then had a terrible experience and have avoided it like the plague, which is my reticence to go with the SSD system disk. I think you've convinced me to bite the bullet and become familiar with Acronis and go with SSD, although I believe that RAID 1 (and RAID 5) offer as much reliability. So the SSD is driven from an onboard controller?

BTW, I am not reusing any of my current components for this new box. The current box will become the backup box and the new box will be my primary. The current backup box will be given to someone in need.

I will have to seriously consider your RAID 10 recommendation for my work disk to get a bit more speed. I will likely stick with RAID 0 for cache and quick backup.

I'm also going to take your advice on the video board. I assume this is the solution to get three monitors plus the LSI 9260 which is PCIe x8?

Do you have a specific RAM recommendation?

Finally I can see that the pagefile performance becomes irrelevant but I am sure MS has figured out a way to pervert it into use even when everything could live in RAM! I was a mainframe operating system programmer (assembler on the iron) for many years and cannot believe how MS has screwed up multitasking and memory management so badly!

I will now start assembling my complete tentative parts list and bring it back here.

Thanks again!

 

[specialk90]

I was the same way about system disk images. I had used Symantec Ghost and Symantec Save and Restore(I think thats the name) and had bad experiences restoring images. One time Windows needed to be activated and two times Adobe CS2 needed to be activated again. I tested these apps before I relied on them because I also had a problem when I tried using Western Digital's software to clone a 7200rpm drive to a Raptor(10k rpm). I have since tried and used Acronis xx Workstation without a problem.

Once I spent 10hrs trying to take an OS on a Raid 1 and move it to a Velociraptor because I didn't trust backup software. I then tried Acronis and it worked perfectly. This was for my business partner's Dell, and I didn't want to take any chance at all that he would experience even the slightest problem.

The current Acronis software is "Backup and Recovery 10 Workstation".

So, you won't be reusing any hard drives?

Yes, the LSI 9260 is PCIe x8.

Yes, I am using the SSD with the onboard controller. Tomshardware did a test comparing an Intel SSD on the Intel onboard, AMD onboard, LSI 9260 and another one I can't recall. The Intel SSD and Intel onboard was the fastest in just about every test.

It appears that G.Skill makes extremely reliable ram according to their newegg reviews. I also looked at my OCZ Obsidian reviews and found that many others have had bad ram including me.
$140 for 6GB http://www.newegg.com/Product/Product.aspx?Item=N82E16820231223
You want at least 1333MHz ram to give you the option of overclocking.

About your problems and slow Lightroom: have you downloaded and run MemTest86? I had soooo many odd problems all thanks to 1 stick of ram being slightly bad. Either that or your video cards are bad and causing some or all of these problems.

If you like, I can make a newegg Wishlist. If so, do you have a preference for hard drive maker?(Seagate, WD, Hitachi, Samsung)

 

[dwightmccann]

My current system and the spare hard drives I keep on the shelf will simply move to backup. The drives in current backup are older and will move into someone else's hands after I erase them ... I'm not concerned about the content.

I will get the Acronis Workstation.

I will get the Intel 160GB and use the Intel controller per your suggestion.

I just got a replacement OCZ memory kit today. One of my two sticks was bad. I did use Memtest86+ to determine the problem. That is not the problem I have now ... I think it is some sort of I/O conflict issue.

I think the video cards are OK but there is some sort of motherboard chipset issue.

I'm still sorting the parts list. I have very competent co-workers who have built dozens of machines but none are photographers and they have odd predilections and I wanted advice from "outside."

Thanks again. I'll be back with a more final parts list. This is not a rush project and I have a few other things going on. You can see what I do at DwightMcCann.com.

 

[specialk90]

Just a pointer about the Acronis software: I don't know if it was there in prior versions, but Backup & Recovery 10 has very good options for backup. One is called "Grandfather-Father-Son" - it makes daily backups and keeps those for x days, makes weekly backups and keeps x weeks and makes monthly backups and keeps x months. I can see this not working so well for .CR2 but the .xmp, PSDs and any other files that constantly change should work well. (you can add exceptions for file types you don't want backed up)

A tip for the SSD: change your browser's cache location to another drive. Its a very long explanation on why SSDs don't do well with thousands of very small files. Either you can believe me or read Anandtech's 30+ page essay on SSDs. (I had read it and their prior 30+ page paper before I considered an SSD)

I guess I am just adding some more pointers here.

I don't think we discussed a CPU cooler. I suggest installing a good cooler at the very beginning because they require a backplate, and removing the motherboard is a pain. However, many cases now provide a square hole in the motherboard tray to make it easier to replace/install backplates for cpu coolers.
The Noctua NH-U12P SE2 ($75) is the best for low temps and also the quietest.

Luckily, the i7's are fairly easy to overclock. Someone I know has theirs at 3.6GHz with the Noctua and its rock solid stable (he's a fellow video editor so it must be stable).

Just something to think about.

And you are very welcome.

 

[dwightmccann]

Thanks for the tips on Acronis. Thanks for the tip on browser cache. I will get a CPU cooler ... I'll check out the Noctua. I may look into overclocking but if it's not trivial I likely won't ... these machines are pretty fast.

Again, much thanks and I'll be back with my tentative parts list.

 

[snowgoer1998]

For what it's worth, I recently installed a 920 (D0) with the Noctua...

I've got it overclocked to 3.6 on stock voltages and haven't had any issues. Temps remained relatively low while stress testing...mid 60's

I'm not doing serious video/photo editing, but I am using the Adobe Premier/Elements for home use.

 

[dwightmccann]

So, how does one share a Newegg Wishlist here? I think I've pretty much got what I'm after although I'm still very open to suggestion. I will likely wait until after Christmas as the prices on various components seem to be increasing daily as though they expect that people are more and more willing to spend as it gets closer and with about $4500 in the list I don't want to take the extra hit.

 

[specialk90]

So you have your Wishlist together?

I log into my newegg acct and save the Wishlist. Then there is a box to check to "Share" it. This lets others see your Wishlist. Then just copy its web address and paste here but there might be something to alter. I can't recall at the moment but if you compare its address to other "Public" lists, then you should see what is different besides the very end.

If Microcenter is near you, they have great prices and some better than newegg - i7 920 is only $200. I bought my i7 and Intel SSD from them and they said that they will match newegg's prices.

 

[dwightmccann]

My Newegg Wishlist

 

[specialk90]

http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=8485549

I fixed your Wishlist link.

 

[specialk90]

What have you decided to do with 10 1TB drives?

Why the RE3?

I forgot to ask, but do you have a UPS?

EDIT: I checked out the Samsung F3 1TB since everyone here is always suggesting them. I have never seen such good reviews for a hard drive. Of 102 reviews, only 2 were 1 star which usually means a dead drive. They are very much faster than the RE3 - F3 starts at 140MB/s and the RE3 is 90ish (if I recall correctly). The funny thing is I am a big Seagate fan and these Sammys are looking real good.

 

[dwightmccann]

That's a lot of space, eh? Work: 6 drives in RAID 10 (rather than RAID 5, per your suggestion); Scratch: 3 Drives in RAID 0; 1 hot spare.

Several reviewers noted that the RE3 was superior in a RAID configuration. One coworker said he believed that the RE3 were RAID aware.

I will go look at the F3's ... I'm not stuck in any of these parts.

I do have a UPS ... can't imagine running without one ... 1500.

I will skulk around for better prices when I am ready to pull the trigger although I will likely stick with retailers I know like Amazon.

This machine has gotten a little more expensive than I had originally budgeted ... so I have rebudgeted ... one advantage of success.

 

[specialk90]

The "1500" for the UPS means watts? That is one heavy and expensive UPS.

I have an APC 865w 1500va, rack mountable with 2nd battery.

I can explain the "Raid Aware" comments. In some of WD's drives, they have WDTLER - WD Time Limited Error Recovery. This was mainly designed for Raid controllers(mainly software and onboard/built in). Its purpose is to prevent drives from being dropped from the Raid array if a drive doesn't respond within 7 seconds (timed out). All good hardware Raid controllers can handle this on their own. For example, my 3ware card listed in its errors and warnings section that a drive had timed out but it didn't drop it from the array.

The only place where it helps is using the onboard Raid. I lost data once when a drive timed out and the Intel raid couldn't handle it causing Windows to freeze. I then did a hard reset and my raid array was degraded. The odd thing is that this was with WD's Raptors which have TLER.

Other manufacturers have something similar but I don't know which models and what they call it.

I have used the last 3 generations of Seagate drives in Raid 1, 5 and 10 (on 3ware and Areca cards) without a problem.

For Raid, you could use Raid 6 which is like R5 but can sustain 2 drive failures. However, since you will have a hot spare, R5 isn't such a bad idea.

 

[dwightmccann]

I got the UPS on some sort of deal a year or so ago. My previous APS died and it was cheaper to replace the whole thing than the batteries ... kinda' sad.

So, you're saying that the LSI intelligence obviates the Raid-aware advantage of the more expensive drive? What about heat ... there were a few heat comments on the Samsungs.

I was considering R10 over R5 for Work because of increased performance. If R5/R6 approaches R10 in performance then it would save me a drive or two but I'm interested in having a bit more Work disk speed.

Is there a comparison chart on TH for the 1TB drives? I didn't find it.

 

[specialk90]

http://www.tomshardware.com/charts/2009-3.5-desktop-hard-drive-charts/h2benchw-3.12-Avg-Read-Throughput,1010.html

TH has tested the 500GB F3 but not the 1TB yet. There is no reason why the 1TB F3 would be any slower than its 500GB sibling since both use 500GB platters. If anything, the 1TB has 32mb of cache vs only 16mb on the 500GB model which could help slightly.

This chart doesn't include the RE3; however, the WD Black 1TB is essentially the same drive.

For heat, the fewer platters in a drive almost always means less heat. The F3 is a 2 platter and the 1TB RE3 is a 3 platter.

If you get 10 drives, you can always test R5, R6 and R10.

Just remember that a new PC without the problems from the Gigabyte video cards, should be and feel much faster. Having seen serious performance problems due to a single ram stick, I can understand how some of your applications have slowed down.

But, it seems you and I have something in common: we love to have fast computers for our work; so, I can't argue with the drive performance you are seeking.

 

[dwightmccann]

I'm going to skulk around for more info on the disk drives but other than that it seems that everyone is happy with these parts.

 

[dwightmccann]

So, what is the difference between the SSDSA2M160G2R5 for $599 offered at Newegg and the SSDSA2MH160G2R5 that lists for $499 that is offered at Amazon? I can't seem to find specs at Intel for both.