Access Time and I/O Performance
The access time clearly differs a lot between quiet and fast mode. This is the price you pay for performance or silence, depending on your viewpoint.
The quick access time in fast mode translates into up to 20% better I/O performance, but this applies mainly to small queue depths. Depending on the workload, long command queues decrease the importance of quick access times.