 |
Review: Toshiba THNSNJ960GPSZ Reviewed by: J.Reynolds Provided by: Toshiba Firmware version: JZET6102 |
Introduction
Welcome to Myce’s review of the Toshiba THNSNJ960PCSZ 960GB SATA
Enterprise SSD (hereafter referred to as the THNSNJ960PCSZ).
The THNSNJ960PCSZ is designed to meet the needs of the read
intensive, mixed workload, low power market segment. In this segment its
competitors include the Samsung 845 EVO, the Intel DC S3500, and the Sandisk
Cloudspeed 1000E - tough competition indeed. Please read on to find out how
Toshiba’s offering, with its second generation 19nm NAND, fairs.
This review also includes a sneak preview of the power
testing we will soon be adding to our enterprise reviews.
Market Positioning and Specification
Market Positioning
This is how Toshiba positions the THNSNJxxxPCSZ (also known
as HK3R2) series of drives –
Specification
Here is Toshiba’s specification for the THNSNJxxxPCSZ series
–
Product Image
Here are some pictures of the THNSNJ960PCSZ –
The THNSNJxxxPCSZ series uses a proprietary Toshiba
controller.
Now let's head to the next page, to look at Myce’s
Enterprise Testing Methodology.....
Testing Methodology
Please click
here
to view or download a detailed introduction to Myce’s Enterprise Class Solid
State Storage (‘SSS’) Testing Methodology as a PDF.
Put briefly:
All testing is performed on an OakGate Technology test unit
We perform two sets of Performance Tests:
- A full set of the Storage Network Industry Association’s
(‘SNIA’) tests with mandatory parameters, as specified in their Solid
State Storage Performance Test Specification Enterprise V1.0 – SNIA
SSS PTS Version 1.0. - A set of tests, known as the ‘Myce/OakGate Full
Characterisation Test Set’, that provides readers with a fuller
characterisation of the solution.
Myce.wiki will soon add a comprehensive, world-class basis
for testing the power characteristics of drives based on the use of Quarch Technology
hardware. In this review we provide a sneak preview of one of the low level
tests that we will be running as a standard part of our power testing. Look
out for the forthcoming press release that will announce our partnership with
Quarch, which will be accompanied by an article describing our standard power
tests and establishing an initial set of results for several leading drives.
We also review other important factors such as Data
Reliability and Failover features.
A word about SNIA testing – before striking a partnership
with OakGate Technology I spent some time researching how I might implement
SNIA testing using freely available tools such as IOMeter and FIO. I arrived
at the conclusion that whilst it was theoretically possible it was
impractical. The reason for this is as without the automation offered by a
test bench, such as the OakGate Unit, the only way to meet the SSS PTS
requirements is to run the maximum number of test cycles and then to manually
look back at the results to determine when/if steady state has been achieved in
the workload specific test cycle, and then harvest the data from the qualifying
Measurement Window. This means that the test runs would always take a maximum
elapsed time, and there would be a great deal of human effort required to
review, gather, and report upon the data. I empathise with, acknowledge, and
respect the efforts of other reviewers who endeavour to meet the SNIA’s
principles in their testing - I am privileged and thankful to be able to use a
superb test bench which automates the whole process and allows me to meet the
SNIA’s specification in full.
Before we move on, let’s remind ourselves of some basics –
When reviewing the performance of an SSS solution there are
three basic metrics that we look at:
1. IOPS – the number of
Input/Output Operations per Second
2. Bandwidth – the number of
bytes transferred per second (usually measured in Megabytes per second, ‘MB/s’)
3. Latency – the amount of time
each IO request will take to complete (usually, in the context of SSS
solutions, measured in Microseconds, which are millionths of a second).
It is true to say that IOPS and Bandwidth had all been
growing rapidly before the advent of SSS solutions, but Latency can only be significantly
decreased by eliminating mechanical devices, and thus Latency is the single
most important aspect that SSS solutions deliver to enhance performance.
Latency in a technical environment is synonymous with delay.
In the context of an SSS solution it is the amount of time between an IO
request being made, and when the request is serviced.
Bandwidth, also commonly referred to as ‘Throughput’, is the
amount of data that can be transferred from a storage device to a host, in a
given amount of time. In the context of SSS solutions it is typically measured
in Megabytes per second (MB/s).
A great enterprise SSS solution
offers an effective balance of all three metrics. High IOPS and Bandwidth is simply
not enough if Latency (the delay in an IO operation) is too high. As we will
see in the test results presented below, as Latency increases IOPS will
inevitably decrease.
Queue Depth is the average amount
of IO requests outstanding. If you are running an application and the Average
Queue Depth is one or higher and CPU utilisation is low, then the application’s
performance is most probably suffering from a ‘Storage Bottleneck’.
Another important factor to
remember is that SSS performance is influenced by previous workloads, not just
the current workload, and especially by what has previously been written to the
drive. As specified in the SNIA SSS PTS the goal of all good Enterprise level
testing is to provide consistent circumstances, so that results can be compared
fairly across different SSS solutions – it is for this reason that all of our
tests start with a purge of the drive, so that it starts in a ‘Fresh Out of the
Box’ (FOB) state. Most tests then have a pre-conditioning phase where the
drive is put into a ‘Steady State’ before the test phase begins. Put briefly, a
‘Steady State’ is achieved when the performance of the drive no longer varies
over time and settles into a consistent level of performance for the workload
in hand. You can find a detailed explanation of ‘Steady State’ and how it is
determined in the SNIA tests in our Enterprise Testing Methodology paper, which
can be viewed or downloaded as a PDF by clicking here.
For interest, here are some
generally accepted assumptions that differentiate the use and therefore the
approach to testing Enterprise/Server and Consumer/Client SSS solutions:
Enterprise/Server SSS
assumptions:
- The drive is always full
- The drive is being accessed 100% of the time (i.e. the
drive gets no idle time) - Failure is catastrophic for many users
- The Enterprise market chooses SSS solutions based on their
performance in steady state, and that steady state, full, and worst case
are not the same thing
Consumer/Client SSS
assumptions:
- The drive typically has less than 50% of its user space
occupied - The drive is accessed around 8 hours per day, 5 days per
week, and typically data is written far less frequently - Failure is catastrophic for a single user
- The consumer/client market generally chooses SSS solutions
based on their performance in the FOB state
Esther
Spanjer, Director, Enterprise Business Development EMEA at Sandisk, said, 'I am
happy to commend Myce for their high level of professionalism and cooperation
during the review process', Ms. Spanjer added, 'I wish them every success in
their partnership with OakGate Technology and their initiative to provide
authoritative performance reviews for the Enterprise Solid State Storage market'
Now let's head to the next page, to look at the results
of our SNIA IOPS (Input/Output Operations per Second) Test.....
SNIA IOPS Test
IOPS performance will typically
vary greatly depending on the nature of the IO traffic, including the mixture
of Read and Write operations, and the mixture of Block Sizes (the size of the
IO operation’s data packet, also referred to as IO Size). This test is designed
to benchmark the IOPS performance profile for random IO operations for 56
different combinations of Read/Write mix % and Block Sizes when in a Steady
State, which are of interest to most users.
All of the SNIA’s test
specifications define a ‘required’ set of parameters that must be run for the
test and then allow the operator to elect to run additional tests with
different parameters of their choice. It is the mandatory test with the
required parameters that we run. Note that all of the mandatory SNIA tests must
be conducted with fully random data
As previously mentioned, a key
principle of SNIA testing is to provide a consistent basis for comparing
different solutions from different manufacturers.
Here are the results -
You can see here a visual confirmation that Steady State
Convergence was determined at the end of Round 5.
Here is a 3D and tabular presentation of the results. Users
can simply refer to the grid to obtain the R/W mix and Block Size value of
interest. For example, Online Transaction Processing applications
typically run at a Block Size of 8K and a Read/Write Mix of 65/35, and users
can quickly understand how the device might perform under Steady State for
these access characteristics.
You can see that the 4K 100% Read IOPS result is 89,069 and
that the 4K 100% Write IOPS result is 20,306. So read performance is excellent
and write performance is good. Toshiba specifies that the THNSNJ960PCSZ is
capable of 75,000 IOPs for Random 4K Reads and 14,000 IOPS for writes – this appears
to us to be underselling the THNSNJ960PCSZ significantly!
Product Comparison
For interest we present a comparison of the 4K 100% Writes
and Reads results with those of the other Enterprise SSDs we have tested -
Now let's head to the next page, where we look at the
results of the SNIA Write Saturation Test.....
SNIA Write Saturation Test
The objective of this test is
to observe the time evolution of the drive’s performance, as a function of
time, from a ‘factory fresh’, ‘fresh out of the box’ (‘FOB’) state. When a
drive is in a FOB state (e.g. after it has been purged by, for example by a
SATA Secure Erase or SCSI Format), we can expect an initial period of time when
writes can easily be accommodated by clean/empty blocks, but once all of the clean
blocks have been written to once and the drive’s controller must first clean
blocks (with erase write operations) before it can write new data, then we can
expect a slow down. The slow-down is usually quite dramatic and is commonly
referred to as the ‘write cliff’.
The Write Saturation Test is
easy to run as it requires no steady state determination – it can be easily run
in freely available software, such as IOMeter.
Here are the results -
You can see here a steep fall followed by a gradual drop in
Write IOPS performance as the Toshiba THNSNJ960PCSZ reaches a Steady
State. The fall, that begins at around Round 47, is the ‘write cliff’.
Note that the test was halted, as specified in the SNIA SSS
PTS, when 4 x the User Capacity had been written to the drive. You can see that
the THNSNJ960PCSZ is settling towards a steady state at around the 25,000 IOPS
level, which is good for its target market segment.
You can also see that the latency graph line is a mirror
image of the IOPS graph line.
Now let's head to the next page, to look at the SNIA
Throughput Test.....
SNIA Throughput Test
The test is designed to measure the sequential Read and
Write IO performance for two Block Sizes, when under Steady State conditions.
One can easily compare the results produced by this test with box-top numbers,
which are usually stated as “Up to xxx MB/S”.
Here are the results -
You can see here that Steady State was achieved for both
Write IO sizes by the end of Round 5.
-
You can see here that Steady State for both Read IO sizes
was achieved by the end of Round 6.
Here are the average values recorded in the measurement
window –
These are excellent results.
Product Comparison
For interest we present a comparison of the 1024K sequential
reads and writes (single port) performance in comparison with those of the
other Enterprise SSDs we have tested -
Now let's head to the next page, to look at the results
of the SNIA Latency Test.....
SNIA Latency Test
The Latency Test measures average and maximum response times
using random IOs at specified Block Sizes and Read/Write mixes, taken under
steady state conditions. The test runs at a Queue Depth of 1 (1 outstanding
IO), thus the results give the baseline response time for a single IO request.
The test also reports maximum latency values, which can be
helpful to see if there might be processes within the drive that may cause max
Latency values to become larger.
Here are the results -
You can see here that Steady State was achieved in Round 14
through Round 18 (the ‘Measurement Window’).
These are the Average and Maximum Latency Values observed in
the Measurement Window (measured in Milliseconds).
Here is a 3D graph showing, at a glance, the Maximum Latency
values for each combination of Read/Write Mix and IO Size.
Here is a 3D graph showing, at a glance, the Average Latency
values for each combination of Read/Write Mix and IO Size. These are excellent
results and the THNSNJ960PCSZ is particularly strong in the mixed read/write
(65/35) results.
Product Comparison
For interest we present a comparison of the 4K 65% Reads 35%
Writes latency results in comparison with those of the other Enterprise SSDs we
have tested -
Now let's head to the next page, to look at the results
for the Myce/OakGate 4K Read and Write Latency Tests......
Myce/OakGate 4K Read and Write Latency Tests
These tests steadily increase the random 4K IO demand in
terms of IOPS, and report the drive's response in terms of Average IOPS, Average
Latency and Maximum Latency. It is designed to show a drive’s maximum IOPS
capability and report the all important Latency numbers for each level of IOPS
demanded. The Maximum latency numbers give us an insight into the occurrence
of Latency peaks that could cause an unexpected response from time to time.
Here are the results –
Firstly, here is a graph showing the result for the initial
Pre-Conditioning step (4K Random Writes) –
You can see what appears as a two step fall towards a steady
state. You can also see that there is a wide variation in performance around
the norm.
4K Latency Read Test
You can see that the drive can no longer meet the increase
in IOPS demand at 88,000 IOPS, which exceeds Toshiba’s specification of 75,000.
Impressive!
You can see a small and gradual increase in read latency up
to the maximum IOPS mark. The Average Read Latency results are excellent.
You can see that the max latency spikes appear to occur at
regular intervals and are perhaps associated with a regular event occurring in
the controller’s housekeeping routines. Let’s have a closer look at what is
happening at one of these spikes.
Let’s have a close look at the distribution of the Latency
results at the 50,000 IOPS level (at one of the spikes) –
As this is the first time in this review, that we are
looking at a High Resolution Latency Histogram, here’s an explanation – The X
axis to the left is the count of the IOs in the observation period (in a Round)
that had a Latency of the value along the Y axis (please note that the X axis
is logarithmic to allow the low order counts of the huge number of IOs that
have been measured to be visible); the Y axis is the Latency value measured in
Microseconds; The X axis to the right is the % of the Total IOs observed that
have a Latency <= to a given Latency value; the rate of getting to 100% is
highlighted by the red graph line.
You can see that 99.9% of the Latency values are <= 170
Microseconds (0.17 Milliseconds) and there are relatively few outliers, so the
quality of service is excellent.
4K Latency Write Test
Perhaps we can see here why Toshiba only specifies a level
of 14,000 IOPS for 4K random writes as the THNSNJ960PCSZ starts to fall
slightly short of meeting an increase in demand at this stage. For example,
when the demand reaches 25,000 IOPS the response is actually falling short of
22,500 IOPS.
Here we can see that Average Write Latency stays below 200 microseconds
until a demand of 17,000 IOPS.
The maximum write latency results are a bit ‘spiky’ but this
is typical.
Now let’s have a look at the distribution of the Latency
Values at the 20,000 IOPS Mark –
You can see that 99.9% of the Latency Values are <= 10.83
milliseconds.
Now let's head to the next page, to look at the results
for the Myce/Oakgate Reads and Writes Tests.....
Myce/OakGate Reads and Writes Tests
The tests are designed to show the Random and Sequential,
Read and Write, performance metrics for different combinations of Queue Depth
and IO size.
Here are the results -
Random Reads
Random Writes
You can see a healthy peak for 4K random writes.
Sequential Reads
Sequential Writes
Now let's head to the next page, to look at the results
for the Myce/Oakgate 4K Mixed Reads/Writes Tests.....
Myce/OakGate 4K Mixed Reads/Writes Tests
This test is designed to show the performance metrics for
different combinations of Queue Depth and Read/Write mix (the % of Reads and
the % of Writes making up the IO traffic)
4K Mixed R/W Test
Now let's head to the next page, to look at the results
of the Myce/OakGate Entropy Tests.....
Myce/OakGate Entropy Tests
These tests are designed to show performance metrics for
different combinations of Queue Depth and Entropy % (Entropy % is the degree to
which the data that is random and therefore incompressible). Testing with
different Entropy % levels has become important with the advent of controllers,
such as those from LSI Sandforce, that compress data before writing it to NAND.
Controllers that compress data can be expected to perform better with highly
compressible data (i.e. data with low Entropy).
The first test performs 5 minutes of Random 4K writes for
each combination of Queue Depth and Entropy %.
The second test does the same thing for a mixture of Read
and Write traffic (70% Reads, 30% Writes).
4K Entropy Write Test
You can see there is little or no variance in performance to
be found in any of the Entropy tests, as the degree of random data increases
(and this comment applies to all of the test results for the Myce/OakGate
Entropy Tests). We can therefore conclude that the Toshiba THNSNJ960PCSZ does
not compress data.
4K Entropy 70% Reads 30% Writes Test
As we saw no evidence of compression in the 4K Entropy Write
Test we skip the presentation of the 70/30 entropy results.
Now let's head to the next page, to look at Power
Consumption and Data Reliability.....
Power Consumption and Data Reliability
Power Consumption
I believe most people know that data centres are already one
of the major consumers of electricity in the industrialised world; indeed it is
estimated that currently 2% of all electricity consumption goes into IT
applications. According to the European Union the energy consumption of data
centres was 46 Terawatt hours in 2006 and is set to rise to 93 TW hrs by 2020. This
is equivalent to one hundred million 100W light bulbs burning 24 hours a day,
365 days a year.
Typically 40% of the power consumed by data centres is for
the IT load and 35% is for cooling the system. Generally speaking, if a drive
consumes more power it will produce more heat – so power consumption is indeed
a double edged sword. It is no surprise then that a significant proportion of
a data centre’s power consumption goes on servers. I understand cloud based
applications, such as Facebook, are the primary cause of the growth in servers
and the demand for storage space.
If you are a Facebook user, like me and the Reynolds sibs, and
you reside in Europe – this is most probably where your data is click here. Some
interesting Facebook statistics – Facebook has more than 1 Billion monthly
active users, it generates 1 Trillion page views per month and more than 219
Billion photos have been uploaded since launch – amazing! Here is an
interesting video showing the remarkable scale of Facebook’s largest North
American data centre click
here.
My thanks to Anna of Intel for pointing me to the following
Info-graphs -
Power Testing – Sneak Preview
I could not resist the temptation to give readers a sneak
preview of the power testing that we will soon be adding to our standard tests
for enterprise drives.
Power testing will be performed by a Quarch Technology
12v/5v Programmable Power Module (QTL 1455). OakGate has added integration
with Quarch Technology products so that I will, for example, be able to graph power
consumption alongside drive performance for long durations. We will soon
publish a press release to mark the formal launch of our partnership with
Quarch and this will be accompanied by an article which will detail the
standard power tests that will appear in our reviews (it will also establish an
initial set of results for several leading drives).
In this preview we are utilising a direct connection to the
Quarch PPM (rather than via our Oakgate unit) to measure the power
characteristics of the THNSNJ960PCSZ whilst it performs a heavy workload of 4K
random writes in a steady state.
Here is a visual presentation of the result -
You can see that all the action is taking place on the 5v
rail.
Here is an extract from the raw data that was captured during
the observation period -
You can see that the PPM is taking a measurement every 4
microseconds (every 4 millionths of a second! i.e. at 250KHz) and is recording
results with a truly phenomenal level of precision!
Here is a display of the key statistics from the observation
period -
You can see that the average power consumption was 3384mW
(3.384W) and this compares favourably to Toshiba’s specification of ‘< 4.5W
active’. This is an outstandingly low level of active power consumption.
For more information regarding Quarch Technology, please see
their web site by clicking here.
Data Reliability
The 'Unrecoverable Bit Error
Rate' (UBER),as defined by JEDEC, the global leader in developing open
standards for the microelectronic industry, is a metric for data corruption
rate equal to the number of data errors per bit read after applying any
specified error correction method. UBER = number of data errors / number of
bits read. JDEC specifies that the maximum error rate allowable for an
Enterprise level SSS solution is one error in every 10^16 bits read.
Toshiba specifies an UBER of 1 in 10^17 bits read
for the THNSNJ960PCSZ.
The THNSNJ960PCSZ is warranted for 1 Drive Write per Day over
a drive life of 5 years. (For interest this is double the level of endurance
for the Samsung 845DC EVO).
The THNSNJ960PCSZ includes sophisticated power failure
support and end-to-end data protection.
Now let's head to the next page, to look at the
Conclusions of this review.....
Conclusions
The Toshiba THNSNJ960PCSZ targets a very keenly fought market
segment, which includes competing products such as the Samsung 845DC EVO, the Intel
DC S3500, and the Sandisk Cloudspeed 1000E.
The THNSNJ960PCSZ is a powerful contender, which notably
beats the Samsung 845DC EVO in the areas of endurance, write performance, and
mixed read/write performance. The THNSNJ960PCSZ has an outstandingly low level
of power consumption.
I am pleased to award the Toshiba THNSNJ960PCSZ our top
rating of ‘Outstanding’.
