Introduction
Here at Myce.wiki the Hardware Review Team has recently been
considering how best to review and test Enterprise level Solid State Storage
solutions. As part of our deliberations, we naturally paid careful attention
to the Storage Networking Industry Association’s (SNIA) Solid State Storage
(SSS) Performance Test Specification (PTS) for Enterprise Solutions (now that’s
a mouthful J), which is widely
accepted throughout the storage industry as the professional and consistent way
to test and benchmark enterprise solutions. For further information on this,
please see SNIA SSS
PTS.
Myce is delighted to confirm that it will be working in
partnership with Oakgate Technology Inc to implement our Enterprise Test
Bench. Please see Press
Release
It is important to note that the testing required to review
Enterprise level Solid State Storage solution is radically different to that
for consumer solutions.
Given our track record in the review and testing of Consumer
SSDs we soon found ourselves wondering how Consumer SSDs would perform in tests
aimed at Enterprise solutions. We also noticed that Enterprise solutions
typically have a much larger amount of NAND storage dedicated for
use by their controller (the controller is the heart of an SSD and controls the
reading, writing and placement of data in the NAND storage together with the
mapping of the storage to a user’s host environment and file system). So we
also wondered if consumer SSDs would perform better in enterprise tests if we
supplemented the relatively small amount of NAND they have set aside for their
controllers by leaving space unassigned for use by the host and its file
system. In Windows this is an easy thing to do and can be achieved by leaving
space on a drive unassigned for use by partitions (by reducing the default
maximum size when creating a new volume, or by shrinking an existing partition,
in Drive Management).
The process of setting aside additional space that may
supplement the effectiveness of an SSD’s performance is commonly known as
making an additional Over Provision (‘OP’) – hence our title. An additional OP
isn’t quite the same as permanently setting aside space to be dedicated to a
controller; it does though ensure that the pool of free blocks (that do not
contain valid data) is relatively large which in turn may have the effect of allowing
a controller’s ‘Garbage Collection’ functionality to more easily stay on top of
cleaning blocks, in order to accommodate the writing of new data, especially
when faced with intense write demands. The amount of NAND permanently
dedicated to the controller is hereafter referred to as ‘normal OP’.
In the SNIA SSS PTS one of the many demanding tests
specified is known as the Write Saturation Test; this test calls for bombarding
the drive relentlessly with randomly written 4KiB blocks of random (high
entropy) data, for a protracted period of time, preferably across the full
space available for use by a host system. Part of the requirement for
enterprise testing is to record results when a drive reaches what is known as a
‘Steady State’, put simply a steady state is when the drive reaches a
performance level that does not vary significantly over time (this is important
as a drive may temporarily perform at a higher level, whilst for example, it
still has empty blocks available (for example following a secure erase, which
puts an SATA drive into a Fresh out of the Box (‘FOB’) state).
To test how Consumer SSDs perform in a Write Saturation Test
(with and without an additional OP) we used a simplified version of the test
specified in the SNIA SSS PTS. We have chosen to present the IOPS (Input/Output
Operations per Second) performance over time, in our testing.
We present our findings, which we feel you may find
surprising as well as interesting, in this article.
Let's move to the next page to look at the Testing
Approach...
Testing Approach
Hardware
Testing was performed by three members of the Myce team (due
to the multiple locations of the SSDs tested).
As you will see in due course, it is really the shape and
pattern of the scores recorded for the SSD's tested that matters and not the
absolute scores, so minor variances in test platforms and configuration are not
important to the conclusions drawn by this article.
A Secure Erase was performed at the start of every test
cycle.
Each drive was tested with and without an additional OP.
When an additional OP was added the total space set aside (i.e. the normal OP
dedicated to the controller plus an additional OP) was 96GB.
Software
IOmeter was used to
generate the test data and IO traffic.
The test run was configured in IOmeter, as follows:
Access Specification: Block Size - 4KB, 100% Write, 100% Random
IO, aligned to 4K boundaries
Disk Targets: Max Disk Size - 0 (so the test file covers the
full range of blocks available), number of Outstanding I/Os (Queue Depth) - 32,
Write IO Data Pattern - Full Random
Test Setup: Run Time was 2 hours
A User Defined Windows Performance Data Collector Set
(performance log) was used to record the ‘Disk Transfers/sec’ for the Logical
Drive being tested, at a sample interval of 1 second. The log file format
property was changed to ‘comma delimited’, so the results can easily be loaded
into MS Excel to produce a scatter chart of the results.
The log file was started just as IOmeter finishes creating
its test file and then stopped after the IOmeter run completes.
Let's move to the next page to look at the results for an
Intel 520 256GB..
Intel 520 240GB
Controller Type: LSI Sandforce 2281
NAND type: IMFT 25nm MLC
What you see here is a scatter chart of the IOPS performance
level recorded every second of the two hour test run. The red plots are for
the run with the SSD's standard OP area and the blue plots are for the run with
an additional OP that takes the total OP to 96GB.
Clearly, it is obvious that the additional OP is making a
significant and positive difference to performance.
You can also see that the deviation of the plots around what
can be considered to be the norm is wide and unpredictable.
Notice also that the plots start relatively high and then
drop significantly to a Steady (though unpredictable) State.
We believe the first drop in both plots occurs when all of
the NAND has been written to once and the drive’s Garbage Collection
functionality kicks in to erase blocks in preparation for accommodating future
writes.
You will also notice that the plots for ‘with extra OP’ take
a second significant drop (at around 1500 seconds) before they reach a Steady
(though unpredictable) State. Our theory is that this second step is where the
drive’s Sandforce controller starts to throttle the write performance as part
of its ongoing duty to preserve NAND life expectancy.
Please click here
for our review of the Intel 520 240GB.
Let’s go to the next page to look at the results for a
Corsair Neutron 240GB
Corsair Neutron GTX 240GB
Controller Type: Link_A_Media Devices LM87800
(LAMD)
NAND type: Toshiba 2xnm Toggle mode MLC
A stronger performance can be seen here from the Neutron
GTX.
A second step is apparent in the fall to a Steady State in
the test with normal OP; perhaps this is evidence of its LAMD controller
applying a throttle to preserve NAND life expectancy.
Please click here
for our review of the Corsair Neutron GTX 240GB.
Let’s move on to the next page to look at the results for
an OCZ Vector 256GB...
OCZ Vector 256GB
Controller Type: Indilinx Barefoot 3
NAND type: IMFT 25nm MLC
As one of our reviewers commented - ‘What a machine’.
Clearly, the impact of the additional OP with the Vector is
impressive; not just in terms of the high IOPs but also in the way it has a
much tighter distribution around the norm.
Please click here
for our review of the OCZ Vector 256GB.
Let’s move on to look at the next page and the results
for an OCZ Vertex 4 256GB ...
OCZ Vertex 4 256GB
Controller Type: Marvel 9187 with Indilinx infused firmware
NAND type: IMFT 25nm MLC
Another very impressive result with an additional OP.
For interest, the test results are also shown for a Total OP
of 48GB.
Please click here
for our review of the OCZ Vertex 4 256GB.
Let’s move on to the next page to look at the results for
a Crucial M4 256GB
Crucial M4 256GB
Controller type: Marvell
NAND type: Micron 25nm MLC
A very significant difference can be seen with the Crucial
M4.
So much so that personally, if I had a CM4, I would be
inclined to run with an additional OP, even if it is only with a relatively
modest amount of space.
Please click here
for our review of the Crucial M4 256GB.
Now let’s move on to the next page to look at the results
for a Kingston V300 240GB -
Kingston V300 240GB
Controller Type: LSI Sandforce 2281
NAND type: 19nm MLC Toggle
A similar shape to the Intel 520 240GB as would be expected
for a drive with the same controller.
Now let’s move on to the next page to look at the results
for a Smart Optimus 400GB enterprise drive...
Smart Storage Optimus 400GB
Controller Type: Smart Storage Custom/Specific
Nand type: 24nm MLC Toggle
The first thing that will strike you is the remarkably tight
distribution of results about the norm and then secondly that only a very small
step to steady state can be observed. The Smart Optimus 400GB is a market
leading Enterprise SSD. Myce will be publishing a full review of this drive in
a few weeks time using our new Oakgate based Enterprise Test bench.
The Optimus comes with a large amount of normal OP (140GB)
and its endurance is covered under warranty for ten full drive writes per day
for five years. Testing with a further additional OP showed no improvement.
Let’s move on to the next page to look at our Conclusions...
Conclusions
An additional OP clearly makes a big and positive difference
to the performance of all the consumer SSDs, in the context of the intense
write tests we have performed.
So, should consumers always apply a large additional OP?
Well, NO – as in the overwhelming majority of
consumer use cases the intensity and duration of write activity will never get
to a level where a difference can be appreciated.
So, should consumer ‘Speed Freaks’ apply a large
additional OP?
My feeling is that if you can afford to lose the use of the
space for data and you are a Speed Freak, then YES – why not? (I
personally run 2 OCZ Vectors in Raid 0 as my boot drive and I run with a 50%
additional OP – yep a Speed Freak J
)
So, should businesses go out and buy a consumer SDD and
run it with a large additional OP?
Well, NO – as businesses most often need to benefit
from the far superior endurance, failover, and data integrity features
supported by enterprise solutions. For example, I understand an OCZ Vector
256GB is covered under warranty for 20GB of writes per day for 5 years and a
Smart Storage Optimus 400GB is covered for 10 full drive writes per day (close
to 4000GB per day).
Parting sentence – Yes - What a difference Over
Provisioning makes.
Please note that, as is shown for the OCZ Vertex 4 above, a
somewhat smaller total OP of less than 96GB is likely to show some improvement
– our choice of a total OP area of 96GB for our testing was arbitrary but
nevertheless consistent.
You may comment on this article below...
