Sign in with
Sign up | Sign in

PCMark 7: Storage Suite

Second-Gen SandForce: Seven 120 GB SSDs Rounded Up
By

PCMark 7 is the latest synthetic from Futuremark. The older Vantage test gives a moderate approximation of storage performance, but there are situations where the scores vary too much from one run to another to make it a great tool for evaluating SSDs. Both metrics use the same underlying trace-based technology from Intel's IPEAK. However, Futuremark's programmers made a few changes in the latest version that it claims improve its accuracy, though it's still hard to make sense of this benchmark's results. Fortunately, at least the variability is gone.

The difference between the latest SSDs from Intel, Crucial, OCZ, Mushkin, Corsair, Patriot, and Adata all fall within a tight spectrum in PCMark 7, but the overall storage score is based on a geometric mean of the subtests. That's why it's important to take a deeper look into the individual tests themselves.

The Windows Defender test is based on a trace of Windows' Defender utility performing a Quick Scan of the system. This is very read-heavy scenario comprised of lots of random access. In fact, read operations make up 97.9% of the trace. And of the read operations, 93% are random accesses. Naturally, anti-virus scanning and file searches are the two more applicable real-world scenarios for a test like this.

Overall, there's really no difference between the SSDs. The 120 GB Vertex 3 and S511 perform marginally better than the other new SandForce-based drives. However, compared to a desktop hard drive, the crowd of SSDs offers about 2x the performance.

The second test in PCMark 7's storage suite is a trace based on importing 68 images (434 MB total) from a USB thumb drive into Windows Live Photo Gallery. This doesn't actually include copying the image files. The trace only includes the I/O activity pertaining to indexing. This type of scenario involves writing more data than you read, and most of the writes are random.

This is another scenario where we see a difference between the SF-2200-based drives employing synchronous and asynchronous NAND. The gap varies from 15% to 30%, depending on the drive.

The Video Editing test is based on the I/O activity of publishing a 1080p video in Windows Live Movie Maker. We're dealing specifically with a scenario where multiple high-def sources are combined and written to a single output file. Overall, the split is 30/70 between random and sequential reads, and reads are more prevalent than writes. In fact, the ratio is about 1:9 in favor of reads.

The problem here is the writing portion of the workload, which consists almost completely of incompressible video data. This puts SandForce-based SSDs at a disadvantage.

The Windows Media Center test is a based on a trace of an HTPC recording two simultaneous TV shows in Windows Media Center, while playing a separate prerecorded show. We're basically reading one file and writing two others.

This type of scenario involves a lot of random writes (94%), because Windows Media Center incrementally adds data to the video file as the TV show progresses. Reads are another story, as they're almost all sequential (84%). Playing a video file is different than recording one; when you play a video file, you're loading it up and playing it back as a continuous stream.

In this scenario, there's practically very little difference between SSDs.

The Adding Music test in PCMark 7 is a not exactly what it sounds like. Futuremark hooked a drive filled with 68 GiB of music files (lossless WMA) to a computer and recorded the I/O activity while Windows Media Player added the audio tracks to the music library. The important point is that this doesn't actually involve copying files to the disk. Rather, it's all about scanning and indexing music files. You'd think that means more random reads and almost no writes, but indexing involves adding to a database of information. That's why we're dealing with more sequential writes (75% of all writes) and a situation where reads are outnumbered by writes 2:1.

In this benchmark, we're presumably restricted by the low transfer rate of the external disk with all of the music files.

The Starting Application trace is extremely brief in that it's only made up of loading the PCMark 7 Whitepaper v1.0 PDF and opening Internet Explorer from the taskbar (19.236 seconds). That adds up to reading a 717 KB PDF file and loading executables, along with related file dependencies from the system drive. The amount of data read outnumbers the amount written 63:1, and most of the read accesses are random in nature (86%).

There's a clear delineation here between the drives complemented by synchronous NAND and the ones that use asynchronous memory to help cut costs.

The Gaming test involves starting and loading World of Warcaft, which is why we're dealing almost exclusively with reads. Most of the read operations are random in nature, but in terms of the total amount of data read, there's a fairly even split between sequential and random accesses. Even though there are 3002 random reads and 575 sequential reads with block sizes up to 4 KB, this cumulatively only accounts for less than 14 MB of the total 123 MB read. At block sizes between 1 and 2 MB, there are more sequential reads than there are random reads.

The margin of difference is much smaller than the Starting Applications test, so it's difficult to make any comment on performance. Of course, there's still a clear line in the sand between SSD and HDD performance.

Ask a Category Expert

Create a new thread in the Reviews comments forum about this subject

Example: Notebook, Android, SSD hard drive

Display all 46 comments.
This thread is closed for comments
  • 6 Hide
    dauthus , July 25, 2011 4:30 AM
    The Corsair force series 3 drives should be instantly disqualified due to BSoDs etc. Go look at their reviews on newegg, it is horrifying.
  • 0 Hide
    garage1217 , July 25, 2011 4:33 AM
    Nice review. You left out the corsair Force GT 120gb however which would have compared equally to the vertex as other sites have scored it. Also I own one, it ROCKS.

    On the force 3, it got horrible reviews because of a production issue. Corsair issued a full recall and now the issues with that particular drive have been cleared up which is why it was not disqualified. Very old news.
  • 5 Hide
    dauthus , July 25, 2011 4:44 AM
    Quote:
    On the force 3, it got horrible reviews because of a production issue. Corsair issued a full recall and now the issues with that particular drive have been cleared up which is why it was not disqualified. Very old news.


    You are wrong sir.
  • 2 Hide
    gregzeng , July 25, 2011 5:56 AM
    Googling told me that SSDs are almost impossible to use with Linux (EXT4). My netbook & notebook drives are in MS NTFS-COMPRESSED partitions (not Linux NTFS-4G, 'cos no compression). MS claims compressions has 'negligible' speed costs. Is that true, for about twice then storage space?
  • 0 Hide
    mayankleoboy1 , July 25, 2011 7:21 AM
    why not include the max iops editions?
    anands benchies showed that 120gb vertex3 max iops ~= 256gb vertex 3 for quite a less price
  • 0 Hide
    Hellbound , July 25, 2011 7:26 AM
    This article mentions installing the OS and applications to SSD, and the rest (movies, music) to conventional hdd's. But I'm not sure how to do that. I've google'd it and there are many suggestions how to do it. I would like to know the best way to go about this.
  • 3 Hide
    whysobluepandabear , July 25, 2011 7:47 AM
    HellboundThis article mentions installing the OS and applications to SSD, and the rest (movies, music) to conventional hdd's. But I'm not sure how to do that. I've google'd it and there are many suggestions how to do it. I would like to know the best way to go about this.

    WTF?

    Step 1.) Install SSD.

    Step 2.) Install OS on SSD and everything you want to access and run quickly.

    Step 3.) Install HDD.


    Step 4.) Send files to E, F, G, H, I, J or whatever drive the HDD is. Performance orientated apps go to the C, or whatever drive your SSD is.


    It's literally no different than if you were to plug in an external HDD via USB. You direct files and applications as accordingly.

    We'll dismiss the Z68 - which allows you to use a small SSD to boost your normal HDD - otherwise if your SSD is large enough, it's actually a worse route, and just instead use the SSD.
  • 1 Hide
    flong , July 25, 2011 8:00 AM
    This is a superb review because it deals with real-world performance. I commend Tom's for providing a thorough review - one of the most thorough that I have read on any computer site. Tom's is right, the 120 GB size SSD is the sweet spot in SSD drive performance Vs cost.

    If you read similar reviews on other sites, the Patriot Wildfire, The Corsair Force 3 GT and possibly the OCZ Vertex 3 are the top performers in the 120 GB drive performance. The Wildfire uses 32 NM Toshiba toggle flash memory which is the best. The Force 3 GT uses 25 NM memory but somehow manages to keep up with the Wildfire. Note this is not the Corsair Force 3 listed in this review, it is the Corsair Force 3 GT - emphasize the GT. The GT and the wildfire are the two fastest 120 GB drives available right now based on real-world performance benchmarks.

    The real important benchmarks to watch for are the real-world benchmarks at the end of each review. These really are the only ones that count. The other benchmarks are synthetic and they are not very accurate. The OCZ drives win all of the synthetic benchmarks but their real-world performance falls behind the Force GT and the Wildfire.

    Another critical factor is that "fill-rate" performance of the drives. This is the performance of the drives as they fill. Again, the Wildfire and the Force GT rise to the top with the Vertex 3 coming in third place.

    This review lists the Mushkin as a top performer, but it is not listed in many reviews (none that I have read) and so I have not included it in my comments. It is possible that this is a top performer also but I would like to read other reviews about it to confirm.
  • 6 Hide
    Anonymous , July 25, 2011 9:03 AM
    Same thing with OCZ, to be honest. They got an error rate of 33% over at Newegg. Honestly I won't buy a single drive from them, no matter how fast, until they've fixed their issues that have lasted for two bloody generations.

    Crucial m4 for performance and Intel 320 for value is the best.
  • 2 Hide
    compton , July 25, 2011 10:12 AM
    HellboundThis article mentions installing the OS and applications to SSD, and the rest (movies, music) to conventional hdd's. But I'm not sure how to do that. I've google'd it and there are many suggestions how to do it. I would like to know the best way to go about this.


    Besides just manually managing your files on the HDD, there is another method you can use. It's more complicated to set up, but if you can google and follow directions, you'll find it may be easier.

    With Windows 7 you can basically take your "My Documents" folder (the \Users\ stuff) and symbolically link the folders to the mechanical HDD. Everytime an application wants to save to one of your document folders, which would otherwise be on your system drive (in this case a SSD) will just end up on the HDD. From a file management perspective, you may find it easier.

    I do it manually -- just install Windows, Office Pro 2010, Pantone, Google Chrome, iTunes, ect. to the SSD. All of my music, movies, backups of my SSD (I'm only using about 22GB of my Intel 510's 111GB) end up on the HDD. My Steam folder is about 200GB as well, so it goes on the HDD.

    You just have to do stuff like change iTunes folder in advanced options to the folder on the HDD. It's really easy to do. That way, when I want to use another SSD, I have all the Steam games and media on the HDD. Fresh installs are really easy this way.

    I tried installing some of my games on a few of the SSDs I own. Some games can really benefit, but mostly the increase in speed over a fast HDD isn't worth it.

    I bought an original WD Raptor 36GB drive in 2003 that I used for many years, so I was completely comfortable trying to manage the stuff that ends up on my HDD. I ended up moving from a 60GB SSD to a 120GB SSD that is faster but I just can't bring myself to put much on it.
  • 0 Hide
    chovav , July 25, 2011 11:10 AM
    nice work guys :) 

    I do have a question though: for my own use, I need to encrypt the system drive using TrueCrypt (128bit AES). When I did that on a Vertex 2 60GB, the performance went down a hundred-fold! This apparently had to do with the fact that the data was in-compressible (bad for SandForce controllers) and the drive was completely "filled" by TrueCrypt's encryption (so you won't be able to see how much data is on it).

    This meant that the performance was actually LOWER than an encrypted mechanical hard drive, and actually almost unusable. Is there any way for you to devise a test that looks at this effect on SSD performance? This situation is not very rare, a lot of business users must encrypt their drives in order to comply with Company Policy..

    I've used AS-SSD for testing, with read/write results at around 3.5MB/s :S

    Thank you,

    Chovav
  • 0 Hide
    brenro , July 25, 2011 1:52 PM
    Tiger Direct's selling the Solid 3 for $165. That's only about 50 bucks more than a Velociraptor. I think it might be time to pull the trigger. Are these things reliable enough now to use them as the only drives in a RAID?
  • 1 Hide
    aznguy0028 , July 25, 2011 2:12 PM
    garage1217On the force 3, it got horrible reviews because of a production issue. Corsair issued a full recall and now the issues with that particular drive have been cleared up which is why it was not disqualified. Very old news.

    Nope, I had a new force 3 120gb, model 1124 (which was the one that is supposedly fixed), and I was still getting BSODs so I returned it. I waited for the whole early fiasco to be over but still had problems. Go check the Corsair forums, tons of people are still having issue. I'm staying away from any ssd that is using the SF 2281 but I'm keeping my eye on the Wildfire to see how reliable that one is.
  • -2 Hide
    garage1217 , July 25, 2011 3:13 PM
    Rocking mine just fine with no bsods or issues of any kind and tons of others are rocking their 120gb force drives with no issues. The editors of this review did not find or have issues as well, nor have other reviewers since the recall. So does that tell you anything at all?

    Sort of comes down to that old story about the girl that just cannot keep a man. She complains and moans about how it is always the guys fault or she cannot find the right guy and they all suck. She never thought it was her retarded self that was the real issue things never worked out lol So along those lines, have you ever considered it is YOUR equipment and setup, not the drive that is causing the issue? I hear crap like this every day from people with oc'ed systems that they claim is stable when it really is not causing issues that really show themselves with ssd drives. Or people that really have no clue what they are doing in the bios mucking things up and causing problems that may be tolerable to one piece of equipment and not another so when they switch, issues arise. Or a ton of other combos of crap that just do not workout at all together like a junk power supply causing instability issues, or trying to install the latest equipment on quite old hardware.

    Bottom line, I am not seeing any issues at all, tons of others are not seeing any issues and if you care to look at other reviews on say newegg of other brands, you will find the same thing across all ssd manufacturers, not just one in particular. You also find the same thing with mobos, standard drives, optical drives, power supplies, video cards. Really throws up red flags when the person claims in a review "i have tried 5 of these and not a one works!!!" Duh morons, wake up and realize what the real issue is, a lack of troubleshooting skills and the knowledge to properly use a device.
  • -1 Hide
    acku , July 25, 2011 3:29 PM
    SSDs are more problematic than just about every other piece of hardware. It's like a Jenga puzzle, you remove one block and something else goes wrong. Just look at the Solid 3 and Agility 3. Identical hardware, different performance due to firmware.

    That said, I believe that each SSD manufacturer is experiencing their own unique issues. I personally haven't experienced any BSOD issues in the lab. As an example, our workstations are allowed to sleep, which has been a cited issue with the new OCZ SSDs. Anand has the same experience - no BSOD errors in his lab. That's a similar sentiment shared by other reviewers.

    Overall, SSDs are kind of like cars. You may experience something on the road, but once you bring it to your mechanic, nothing seems wrong. Problems with SSDs are highly correlated with configuration and how they are used (what is written, how fast it is written, how much is written, etc...). And I'd go so far as to chalk 99% of the "soft errors" to be related to the flash translational layer (FTL).

    If you're looking for rock solid reliability, you're better off with a hard drive. It doesn't matter if you own a hard drive or a SSD, everyone should practice good backup practices.

    Cheers,
    Andrew Ku
    Tomshardware.com
  • 0 Hide
    banthracis , July 25, 2011 3:51 PM
    Current Cost per GB on newegg for the reviewed drives

    OCZ agility 3 120gb $1.50/gb (includes rebate. $1.75/gb without )
    OCZ solid 3 120gb $1.62/gb (includes rebate. $1.79/gb without)
    Crucial M4 128gb/256gb $1.67/gb
    Corsair Force 3 120 $1.75/gb
    Intel 320 120gb $1.83/gb
    AData s511 120 $1.91/gb
    OCZ Vertex 3 120 $1.91/gb (includes rebate. $2.08 without)
    Mushkin Chronos Deluxe 120gb $2.16/gb
    Intel 510 120gb $2.30/gb
    Patriot wildfire 120gb $2.50/gb

    Going through your benchmarks, it actually looks like the Crucial M4 drives are the best choice with consistent good performance and very low cost/gb.
    In fact, the 256 gb M4 particularly stands out for having the same $/gb as the 120gb M4, yet top of the pack performance throughout the benchmarks.

    Edit: Forgot the Deluxe in Mushkin Chronos. Price/gb was correct though.
  • 0 Hide
    acku , July 25, 2011 3:55 PM
    That's because we test with 4kb transfer sizes in random accesses. The 256 GB and 512 m4s have a native 8kb page. We're going to cover all of that in another article dedicated to m4s.

    If you want nothing to do with compression and still want a performance SSD, m4s are probably the best choice. That's why they're often included in the best SSDs for the money.

    FYI, we used a Chronos Deluxe ~= Wildfire. The regular Chronos is the async stuff, same as Force 3 and regular Vertex 3.

    Cheers,
    Andrew Ku
    TomsHardware.com
  • -1 Hide
    garage1217 , July 25, 2011 3:56 PM
    Quote:
    SSDs are more problematic than just about every other piece of hardware. It's like a Jenga puzzle, you remove one block and something else goes wrong. Just look at the Solid 3 and Agility 3. Identical hardware, different performance due to firmware.

    That said, I believe that each SSD manufacturer is experiencing their own unique issues. I personally haven't experienced any BSOD issues in the lab. As an example, our workstations are allowed to sleep, which has been a cited issue with the new OCZ SSDs. Anand has the same experience - no BSOD errors in his lab. That's a similar sentiment shared by other reviewers.

    Overall, SSDs are kind of like cars. You may experience something on the road, but once you bring it to your mechanic, nothing seems wrong. Problems with SSDs are highly correlated with configuration and how they are used (what is written, how fast it is written, how much is written, etc...). And I'd go so far as to chalk 99% of the "soft errors" to be related to the flash translational layer (FTL).

    If you're looking for rock solid reliability, you're better off with a hard drive. It doesn't matter if you own a hard drive or a SSD, everyone should practice good backup practices.

    Cheers,
    Andrew Ku
    Tomshardware.com


    I will agree to that :)  SSD drives in general are not perfect, but they are not the horrid plague that some make them out to be.
  • 0 Hide
    acku , July 25, 2011 4:01 PM
    Quote:
    nice work guys :) 

    I do have a question though: for my own use, I need to encrypt the system drive using TrueCrypt (128bit AES). When I did that on a Vertex 2 60GB, the performance went down a hundred-fold! This apparently had to do with the fact that the data was in-compressible (bad for SandForce controllers) and the drive was completely "filled" by TrueCrypt's encryption (so you won't be able to see how much data is on it).

    This meant that the performance was actually LOWER than an encrypted mechanical hard drive, and actually almost unusable. Is there any way for you to devise a test that looks at this effect on SSD performance? This situation is not very rare, a lot of business users must encrypt their drives in order to comply with Company Policy..

    I've used AS-SSD for testing, with read/write results at around 3.5MB/s :S

    Thank you,

    Chovav


    I actually hadn't thought of using TruCrypt with a SSD. But your experience with a SandForce drive makes a lot of sense. Near to no compression being done, which is horrible for SF's garbage collection which relies on "extra" space made available by compression. Basically, you're completely throttling the drive and every write leaves little to no time for idle garbage collection because everything written is encrypted. This would be the same as running Iometer with completely random data over the course of weeks at a time. That's why performance is going to be better on a hard drive. I'd recommend trying an m4 if TruCrypt is a must.

  • 0 Hide
    mariush , July 25, 2011 4:40 PM
    gregzengGoogling told me that SSDs are almost impossible to use with Linux (EXT4). My netbook & notebook drives are in MS NTFS-COMPRESSED partitions (not Linux NTFS-4G, 'cos no compression). MS claims compressions has 'negligible' speed costs. Is that true, for about twice then storage space?


    The compression algorithm used by the NTFS system can at best compress text and various uncompressed stuff down to about 50% of the original size, with little CPU usage.

    It will not compress at all the most often found files such as mp3, avi, mkv, docx, which are already compressed. Also, the majority of games' files are already compressed so enabling NTFS compression really won't save a lot of disk space.
Display more comments