This article describes the Power Testing Myce will perform
as a standard part of its testing of Enterprise Solid State Storage solutions.
As at May 2015, Myce has been publishing Enterprise SSD
reviews for 2 years and in all this time, although we have provided, arguably, the
most comprehensive reports on the performance characterisation of solutions
based on the use of a world class OakGate Technology test bench, we have not
offered anything at all in the testing of a drive’s power consumption
characteristics. I am delighted that we will now in one mighty step move from
the bottom of the class right to the very top, by offering the most
comprehensive power testing ever seen in data storage reviews.
Myce’s Power Testing will be based on the use of Quarch
Technology products. Quarch Technology is a world leader in the supply of
testing solutions for the data storage industry – please visit their website by
clicking here. More specifically we are
privileged that Quarch has enabled us to use their latest XLC Programmable
Power Module (‘XLC PPM’). The XLC PPM is specifically designed for testing low
power sleep states on modern SSDs and as such has a remarkably accurate low
level current measurement, down to 100uA (micro amps, millionths of an amp) –
please click here for
details.
The other exciting news is that OakGate Technology has added
extensive integration between its test bench and Quarch power testing solutions
– please see OakGate’s website for details by clicking here.
So our long standing partnership with OakGate together with
our new partnership with Quarch puts Myce in an enviable position with the
ability to provide world class reviews for the power consumption characteristics
of data storage solutions. Integration between Quarch and our OakGate test
bench allows us to report on power consumption inline with reporting on
performance characteristics over protracted periods of time (for example – to
report on the change in power consumption through the course of an SNIA Write
Saturation test). A more direct use of the Quarch equipment, using Quarch’s
TestMonkey software allows us to take a very close look at power
characteristics over relatively short periods of time (for example, to observe
the power consumption characteristics when a drive is first powered up and when
a drive is idling).
We take this opportunity to say a huge thank you to Quarch
and OakGate for their commitment to a partnership with Myce.
In due course we plan to introduce a similarly impressive
level of Power Testing in our Consumer SSD Reviews, where, for example, using
the Quarch XLC PPM’s ability to record power consumption during sleep (and even
deep sleep) states will become highly relevant (particularly to users of
mobile, battery powered devices).
Now let's head to the next page, to take a look at the
Quarch hardware…..
Here are some pictures of the Quarch hardware, as used by
Myce –
Firstly, the XLC PPM (QTL1824)
A SAS/SATA Drive Control Module, which sits between a drive
and where it is connected to a SAS or SATA port, which allows the XLC PPM to
control and monitor the power supplied to a SAS or SATA drive.
A PCIe Module, which sits between a PCIe SSD and a PCIe
slot, to allow the XLC PPM to control and monitor the power supplied to a PCIe
drive (such as the Intel DC P3700).
A Torridon Interface Unit (QTL1260), which is used to
connect the XLC PPM to our OakGate test unit.
Now let's head to the next page, to take a closer look at
the Power Testing reports that will appear in Myce’s Enterprise reviews…..
We have used two leading Enterprise drives, which are known
to have excellent power consumption characteristics, to produce our example
Power Testing reports: A Toshiba THNSNJ960PCSZ SATA 960GB and a Samsung DC845 EVO
SATA 480GB. This serves to create a base line which we can use to compare
other solutions we review in the future.
Power Testing Examples – Quarch XLC PPM Integrated with Oakgate Test Unit
The first set of Power Reports we present as a standard part
of Enterprise SSD reviews see the Quarch PPM placed under the control of our
OakGate test unit. These performance tests are already a standard feature of
our Enterprise reviews and we have simply added performance monitoring
SNIA Write Saturation Test
An SNIA Write Saturation test is one of the industry
standard tests we perform as a regular feature in our Enterprise SSD reviews.
The objective of this test is
to observe the time evolution of the drive’s performance, as a function of
time, from a ‘factory fresh’, ‘fresh out of the box’ (‘FOB’) state. When a
drive is in a FOB state (e.g. after it has been purged by, for example by a
SATA Secure Erase or SCSI Format), we can expect an initial period of time when
writes can easily be accommodated by clean/empty blocks, but once all of the
clean blocks have been written to once and the drive’s controller must first
clean blocks (with erase write operations) before it can write new data, then
we can expect a slow down. The slow-down is usually quite dramatic and is
commonly referred to as the ‘write cliff’.
Please click
here
to view or download a detailed introduction to Myce’s Enterprise Class Solid
State Storage (‘SSS’) Performance Testing Methodology as a PDF.
Here are two example results of Write Saturation test
results with Power consumption monitoring added.
Firstly, for the Toshiba THNSNJ960PCSZ 960GB SATA SSD -
and, secondly for a Samsung 845DC EVO 480GB SSD –
We can observe for both drives that once the write cliff has
been hit there is a gradual increase in power consumption until the dive
settles down to a steady state. From these graphs it is also possible to form a
view of the amount of power that is consumed by a drive’s housekeeping (the
cleaning of blocks prior to writing new blocks). Roughly speaking, somewhat
more than two thirds of the THNSN960PCSZ’s power consumption appears to be
taken up in housekeeping once a drive settles into a steady state (as can be
inferred by the apparent drop in productive IOPS from before the write cliff is
hit through to when a steady state is achieved) and three quarters of the
Samsung 845DC EVO.
Myce/OakGate 4K Read and Write Latency Tests
With the drive in a Steady State, these tests steadily
increase the Random 4K IO demand in terms of IOPS, and report the drive's
response in terms of Average IOPS, Average Latency and Maximum Latency. It is
designed to show a drive’s maximum IOPS capability and report the all important
Latency numbers for each level of IOPS demanded. The Maximum latency numbers
give us an insight into the occurrence of Latency peaks that could cause an
unexpected response from time to time.
We have added performance monitoring so that we can observe
the increase in power consumption as the demand in IOPS is increased.
Here is an example of test results, with Power consumption
monitoring added, for the Toshiba THNSNJ960PCSZ 960GB SATA SSD -
As might be expected, we can see power consumption increases
as the drive responds to the increase in IOPS demand. It is becoming obvious
that we need to judge a drive’s efficiency not just by the total power it
consumes but also by how much productive work it is doing in return for the
power it consumes. For example, we can judge how efficient a drive is by
looking at the IOPS it performs per mW (milliwatt), for a given IO size. We
also need to consider if the measurement of efficiency will vary for different
mixes of read and write IOs.
Myce/OakGate 4K Random, Mixed Reads/Writes Tests
This test is designed to show the performance metrics for
Random 4K IOs for different combinations of Queue Depth and Read/Write mix (the
% of Reads and the % of Writes making up the IO traffic).
Here are two examples of test results with Power monitoring
added.
Firstly, for the Toshiba THNSNJ906PCSZ 960GB SATA SSD -
and secondly, for the Samsung 845DC EVO 480GB SATA SSD -
It is then straightforward to take these results and
calculate the IOPS per mW values for each of the drives, and plot the results
as follows-
For Random 4K IOs (in a steady state) this now gives us a
basis for comparing the efficiency of the drives and it is clear that the
Toshiba is more efficient at every mix of 4K Random Reads and Writes.
Myce/OakGate Reads & Writes Tests – Sequential Writes
This test looks at Sequential Write performance for
combinations of IO Size and Queue Depth. We have added power monitoring.
Firstly for the Toshiba THNSNJ960PCSZ -
and secondly, for the Samsung 845DC EVO -
It is straightforward to take these results and calculate
the MB/s per MW, to provide a basis for comparing the efficiency of the drives
in Sequential Write IOs (We have chosen to use MB/s per mW as we are varying
the IO Size in the test and therefore cant use IOPS) -
You can see that the Toshiba is consistently more efficient
in performing sequential writes.
Myce/OakGate Reads & Writes Test – Sequential Reads
This test looks at Sequential Read performance for
combinations of IO Size and Queue Depth. We have added power monitoring.
Firstly for the Toshiba THNSNJ960PCSZ -
and secondly, for the Samsung 845DC EVO -
It is straightforward to take these results and calculate
the MB/s per MW, to provide a basis for comparing the efficiency of the drives
in Sequential Read IOs (We have chosen to use MB/s per mW as we are varying the
IO Size in the test and therefore cant use IOPS) -
You can see that the Toshiba is consistently more efficient
in performing sequential reads.
Power Testing – Quarch XLC PPM Standalone
These tests are performed using Quarch’s TestMonkey software
with a direct USB attachment to the Quarch PPM from our controlling PC.
TestMonkey allows one to observe the power characteristics
of a drive at an extremely low level for a relatively short period of time.
The TestMonkey control screen, control tab looks like this –
A brief explanation –
The graphs show the current activity state of the test drive.
The Output controls enable the power to be switched ON and
OFF and for the nominal outputs to be varied so that one can, for example, test
how a drive responds to a non-optimal (‘margined’) power supply.
The Record Controls enable the nature of the recording that
the XLC PPM can record to be varied. For example, the level of averaging to be
recorded can be varied – the PPM samples values at a frequency of 250KHz (once
every 4 millionths of a second) and one can vary the averaging of the values
that will be committed to the XLC PPM’s memory; this has the effect of varying
the elapsed time of the recording that can be taken. The way in which a
recording will be triggered (started) may also be controlled – the trigger
types include – Manual (a recording starts following a manual request from this
control screen), Power (a recording starts when the power is switched on),
Pattern (a recording starts when a pattern is initiated), Threshold (a
recording starts when a predefined threshold as set in the setup screen is
encountered, for example, a defined level of power consumption is exceeded),
External (an External command is received, for example, from the OakGate test unit).
The Pattern Controls allow a pattern of behaviour to be
programmed and tested, as outlined in the following example.
Power Up to Idle
This test observes the power consumption characteristics as
the drive is first powered up.
The test uses a test Pattern, which gradually increases the
power supply over the first 150mS of the test. This allows us to see at what
level the drive first responds to power.
The pattern was defined in the TestMonkey Output Pattern
Editor as –
The output voltage for both the
12V and 5V rails is started at 0mV and then increased to 12V and 5V
respectively over the first 150mS. The pattern editor supports the
interpolation of the output voltage between two entries, as used in this
example – this causes the output voltages to be increased linearly over the
first 150 mS (as indicated in the screen above).
Here are example test results.
Firstly, for the Toshiba THNSNJ960PCSZ -
This is the Testmonkey screen that displays the results from
the data that has been recorded. On the ‘Chart’ tab the top chart shows the
Output Voltage over time, and the two other charts show the mA (milliamps) and
mW (milliwatts) consumed overtime. In this recording an Averaging frequency of
7.81KHz has been utilised to give a recording lasting just over 11 seconds.
You can see that there are is a distinctive shape to the way in which the drive
draws power as it powers up and then settles into an idle state.
We can alter the Display Range to look more closely at the
first 150mS –
Here you can see more clearly that the output voltages are
being increased gradually over the first 150mS. You can also see that the drive
does not respond (and start drawing power) until around 130mS when the 5V rail
has reached an output of around 4300mV.
For comparison let’s have a look at the power up
characteristics of the Samsung 845DC EVO -
You can see that the shape is quite different to that of the
Toshiba.
Again, let’s look at the first 150mS for the Samsung in more
detail –
You can see that the Samsung first comes to life after
around 110mS when the output level on the 5V rail has
reached around 3600mW.
We will look at the other Recorded Data tabs in the
following example.
The above tests do not show that one drive is better than
another but it does show if a drive requires a peak of power (and from which
rails) when it powers up.
Idle
This test takes a close look at the power consumed when a
drive is idling (i.e. has no IO traffic it needs to respond to).
Firstly, for the THNSNJ960PCSZ -
This example has used no averaging, so the recording length
is just under 350 mS.
Now let’s look at the Raw Data tab -
Here you can see the raw data values that have been
recorded. The Quarch PPM has recorded entries at a remarkable frequency of
250KHz (that’s once every 4 millionths of a second) and Power Consumption has
been recorded at an, equally, remarkable level of precision.
Now let’s look at the Statistics tab -
You can see that the Minimum, Maximum and Average Power
Consumption values have been calculated and thus we can see that the Toshiba
THNSNJ960PCSZ draws an average of 1044 mW (just over 1W) from the 5V rail when
idling; this can be compared to Toshiba’s specification of ‘< 1.0W TYP’.
And secondly, for the Samsung 845DC EVO -
You can see that the Samsung 845DC EVO draws an average of
1294mW (just under 1.3W) from the 5V rail when idling; this can be compared
favourably to Samsung’s specification of 1.5W.
We trust readers will find our Power Testing to be
informative and comprehensive.
Please do not hesitate to let us know if you would like to
see any other Power Characteristics reported upon – we would be pleased to
accommodate requests.
Myce Enterprise SSD reviews powered by -
