Analysis: The Impact Of Shifting From 512 Byte To 4 KB Sectors
Table of contents
- 1. ‘Advanced Format’ Will Be Standard By 2011
- 2. 512 Byte Emulation: Issues
- 3. 512 Byte Sectors: Toshiba MK6465GSX (640 GB)
- 4. 4 KB Sectors: Toshiba MK7559GSXP (750 GB)
- 5. Test Setup And Transfer Diagrams
- 6. Misaligned Sector Results: I/O Performance
Slowly but surely, hard drives with 4 KB sectors are replacing the "legacy" 512 byte sector size. By January 2011, all drive vendors will have made this transition. Buyers of new PCs are safe, but there are still a few performance pitfalls to note.
The introduction of hard drives that work with a sector capacity of 4 KB instead of the conventional 512 bytes started about a year ago, but it happened without much ado. Using larger sector capacities introduces certain benefits for the hard drive makers, and is typically transparent for the end-user--but only as long as that user installs a modern operating system like Windows 7 or Vista with SP1 and up. Older Windows versions can suffer from performance troubles on 4 KB sector drives.
Therefore, we decided to look at a possible worst-case scenario. This article provides insight, analysis, and some recommendations regarding hard drives with 4 KB sector size. Effectively, this applies to the majority of new hard drives available starting January 2011.
From 512 Bytes to 4 KB
A sector is the smallest storage unit that can be found on a hard drive and it has, historically, been composed of 512 bytes. This made sense in times when hard drives stored megabytes or a few gigabytes because the sector size represents the minimum amount of capacity that will be consumed, even if the file being written is smaller. As a result, it made sense to work with a relatively small unit, just to avoid wasting capacity and to gain usable space.
However, the 512 byte sector size has turned into a limiting factor. We have to acknowledge that the average file size today is way more than 512 bytes, which turns the issue of wasted capacity upside down. This is because an Error Correcting Code (ECC) is calculated for each 512 byte chunk, and as you can imagine, ECC data also requires storage space. It goes without saying that one 4096 byte sector requires less ECC information than eight 512 byte chunks if the ECC algorithms remain unchanged. In the end, the total storage capacity of a hard drive increases as a result of less ECC data overhead.
Increasing Capacity Through 4 KB Sectors
Obviously, the hard drive vendors have identified this modification as a compelling way to increase storage capacities. The beauty of this tweak is that it actually makes sense from an architectural standpoint, as other key figures (like x86 memory pages and many file system clusters) also employ the 4 KB size. In addition, the sector size adjustment represents a relatively minor change, and it doesn’t require a lot of hardware modification (as it is the case with an areal density increase). Finally, the so-called Advanced Format allows for more robust ECC algorithms, which is important in light of ever-increasing capacities.
We took a pair of 2.5” SATA hard drives to compare performance between 512 byte and 4 KB sector size, and to look at what happens in a worst-case scenario, as older system environments may deliver decreased performance on 4 KB sector drives. This would apply to any 4 KB drive, not just the ones on our bench here today.
I am wondering what will happen when a disk fails in an array and one member is replaced by it's 4KB counterpart ? This might mean that loads of us have to stock up on "old" disks before this transition is finalized !
And vot is Linux, chopped liver?
http://www.osnews.com/story/22872/ [...] ard_Drives
MaXX99, AIU if you replace a drive the array *may* get slow on writes to that drive, depending on the established alignment of that drive. It may be possible to use a different, proper align on the new drive when it is added.
And vot is Linux, chopped liver?http://www.osnews.com/story/22872/ [...] ard_Drives[citation][nom]d0923oijs2qe[/nom]And vot is Linux, chopped liver?http://www.osnews.com/story/22872/ [...]
[/citation]
That article is 8 months old...Its now a non-issue. The current version of Gparted is smart enough to tell you if the drive is in alignment or not. I've tested it with 1TB and 2TB HDDs.
Its more of a concern for WinXP users; as there is still a major user-base that would encounter this situation.
I wonder how much of a boost SSDs will get from the new 4KB blocks?
@LuckyDucky7: since LBA in SSD is emulated anyway (they need to write by 512 _K_b increments), the only boost you'd get would be in capacity.
As for XP users, the only problem is when they create a partition from inside the installer - creating a NTFS partition with, say, a Ubuntu LiveCD, and then installing XP would solve the problem.
So in the applications performance test 512byte sectors are superior then 4kb sectors, but 4kb sectors are slightly better then 512byte sectors in the gaming performance test. So it's basically becomes pro or con depending upon what software you use your hard drive for the most
I too was curious about the PS3 use of 4k sectors. Since all new HDDs going forward are going to be 4k, if I wanted to upgrade my PS3 HDD, how would that work? If I pop the drive in and have the PS3 set it up and install the system software will it be unaligned? Will I have to put it in my PC first and partition it / align it, then put it in the PS3? Will the PS3 recognize it after this is done?
So many questions. Hey Toms, can u do some test with the PS3 and these 4K drives and let us know what you find? Maybe even the Xbox too.
The issue with these drives are that they do not seem to work well in lots of NAS-drives aswell they have huge performance problems in RAID-arrays. I have had immense problems with getting them to work in RAID5.
The larger issue is that we need a BIOS and OS that can handle different sector size aswell as handle large drives (
This will have abig impact on people migrating/exchanging older 512 byte sector drives to new 4096 byte ones.
You need to align not just to sector size but also to RAID stripe size. None of the free alignment tools enables you to do custom alignment on drives in raid-arrays. (Yes I have tried Paragon Alignment Tool and Acronis aswell). THG should write an article about that
I think SSDs have a different size sector aswell so they need to be aligned for good performance? THG should write an article dealing with the implications of the storage limitations of the BIOS and OS and the implcations it has for customers. This to raise the awareness and also pressure manufactures to deal with it
The issues I refer to are the BIOS limitations of fixed sector size and a maximum drive size of 2.1TB.
I've not seen any information what cluster size was used when formatting with NTFS. The default is 512 byte, which is obviously sub-optimal if the hardware sector size is 4kb.
Using a 4kb cluster size in NTFS should produce much better results.
beign a gamer that is stil too borke to afford win 7 (im stilon xp) i dont think it would, given the conclusion , i guessing it wont afect my gaming much really, but as a student majoring in game art design i have to wonder how it wil affect programs such as , 3ds amx ,maya, photoshop ,UDK, or source hamer editor. ? might you guys at toms consider testing these apps in xp with the differnt hard drives ?
wow this is a good article.
The issue with these drives are that they do not seem to work well in lots of NAS-drives aswell they have huge performance problems in RAID-arrays. I have had immense problems with getting them to work in RAID5.This will have abig impact on people migrating/exchanging older 512 byte sector drives to new 4096 byte ones.You need to align not just to sector size but also to RAID stripe size. None of the free alignment tools enables you to do custom alignment on drives in raid-arrays. (Yes I have tried Paragon Alignment Tool and Acronis aswell). THG should write an article about that The larger issue is that we need a BIOS and OS that can handle different sector size aswell as handle large drives (
The reason you have problems with these drives under RAID is down to different reasons, not all of which apply to every scenario. Specifically:
1) Some older controllers are not 4k sector aware and format the drives incorrectly, misaligning the partition. Theoretically this can be fixed vie new firmware but older controllers might not be upgraded by their manufacturers.
2) Some 4k drives (e.g. the WD Green Power ones) do not correctly implement features necessary for RAID e.g. TLER. This makes the controllers drop the drive when it takes too long to wake up from sleep. Some WD drives also have issues with their load cycle count killing the drives (the head keeps parking and un-parking under RAID, excessively wearing out the drive). This is because WD wants you to buy their RE (Raid Edition) drives at 80% higher cost. Switch to another brand e.g. Samsung and everything's fine.
3) High capacity drives (1.5TB+), have significantly higher failure rates because the manufacturing technology is still new - yields can be half those of lower capacity drives. This is not RAID specific, but it explains many failures.
I think SSDs have a different size sector aswell so they need to be aligned for good performance? THG should write an article dealing with the implications of the storage limitations of the BIOS and OS and the implcations it has for customers. This to raise the awareness and also pressure manufactures to deal with it The issues I refer to are the BIOS limitations of fixed sector size and a maximum drive size of 2.1TB.
SSD's have 4k sectors and need the same alignment as HDDs with 4k sectors, but in their case its much more important because mis-alignment not only leads to a small performance loss, but it significantly shortens their life time. It increases the number of read-erase-write cycles needed for each I/O, which given the limted number of writes that nand memory has, is a major problem.
Does anybody know how I can tell the difference between drives with 512 byte sectors and 4 KB sector? I didn't see markings indicating either on the older and newer drive pictures.
That article is 8 months old...Its now a non-issue. The current version of Gparted is smart enough to tell you if the drive is in alignment or not. I've tested it with 1TB and 2TB HDDs.Its more of a concern for WinXP users; as there is still a major user-base that would encounter this situation.[/citation]
So Windows is lagging behind when it comes to OS technology. Ok, business as usual then.
beign a gamer that is stil too borke to afford win 7 (im stilon xp) i dont think it would, given the conclusion , i guessing it wont afect my gaming much really, but as a student majoring in game art design i have to wonder how it wil affect programs such as , 3ds amx ,maya, photoshop ,UDK, or source hamer editor. ? might you guys at toms consider testing these apps in xp with the differnt hard drives ?
You're a student, eh? You must be a freshman. Apps have nothing to do with it. Go read a book on how PC's work.
@LuckyDucky7
From what I have read the G Skill SSD already uses 4k and it is a well rounded SSD as far SSD's go lil both worlds even though it is a SATA II drive it holds it own. Here is a review enjoy
http://www.guru3d.com/article/gski [...] sd-review/
You're a student, eh? You must be a freshman. Apps have nothing to do with it. Go read a book on how PC's work.
He's majoring in game art design. The finer workings of a PC storage system are probably not his forte. He is wondering how it will affect his work in a tangible way.
On page 6 there is a typo at the bottom. "Workloads that do not involve read operations, such as the Web server test pattern, don’t show any disadvantage at all in I/O testing" should say "write operations". There was another typo earlier in the article which I will now try to locate.