A suspicious alliance of Disk Management and Storage Spaces
This is a guest post by Elena Pakhomova who is the co-founder of ReclaiMe data recovery software company. Read her blog at www.data-recovery-weekly.blogspot.com.
Storage Spaces is an innovation implemented in Windows 8, allowing to combine various drives into the pools, and then to create virtual layouts (mirror, simple, and parity) on these pools. Microsoft most likely did not assume that Storage Spaces virtual disks can be used for the creation of new layouts in the Disk Management utility; however, in Windows 8, it is not explicitly forbidden. It would seem that huge possibilities for creating mind-boggling storage configurations open up – from a typical RAID 10 to exotic RAID 51. However, you do not have to take a great interest in the creation of such hybrids, because:
- performance of such configurations is usually not good enough
- the more complex the configuration, the harder it is to take into account all
the requirements for fault tolerance;
- data recovery would be impossible in case of Storage Spaces failure.
In this article I will show you how to recover parameters of a typical RAID 5 array manually and discuss the complexity of RAID 5 recovery in case of the array created over Storage Spaces virtual disks.
How to recover a typical RAID5 manually
A typical RAID5 layout
In this case it does not matter what a RAID 5 you have – software or hardware. First let’s see how data and parity blocks are placed on the physical disks. Let it be a RAID 5 consisting of three member disks created using LDM (Windows dynamic striped volume with parity) that is equivalent to the Linux md-raid left symmetric layout, sometimes called synchronous. Data and parity blocks in the array are placed in the following order:
RAID configuration parameters
Generally RAID recovery boils down to the detection of the array configuration parameters. There are the following parameters for a RAID 5 array:
- start offset on the array member disks,
- block size,
- disk order,
- first disk in the array,
- parity placement pattern (left or right),
- data placement pattern (synchronous or asynchronous).
Below I will show you how to detect these parameters manually for a software RAID 5 created using LDM.
Choosing a file
Recall what big enough file is stored on the array and you have its copy somewhere else. You need a file that would occupy all the stripes at least in a couple of rows, that is for an array, say, of five disks and block size of 128 KB, you should choose a file with the size no less than 2*5*128 КB ~ 1.5 МB or better several megabytes. One more important point is that the file should preferably be unique, so that there is only a single copy of the file on the array.
Usually you can use any movie or photo for this. Further we will work with this copy of the file which is for sure stored on the array.
Choosing a disk editor
Next we need a disk editor that can search a disk for a given hexadecimal sequence and move to a certain offset on the disk.
I used WinHex which additionally allows loading all the disks simultaneously and working with them in parallel.
Choosing a unique sequence in the file
On this step you need to open the file in a hex editor and choose a sequence for further search. It is enough to take a 16-20 bytes sequence preferably from the middle of the file where most likely file data rather than non-unique metadata is stored. Once you chose the sequence, you need to check that it occurs only once on the member disks. If you find several occurrences you should continue searching for another sequence until you come across a unique one.
In my example I used an .mp4 file with the size of 11 MB in which I found a unique sequence at offset 0×3160.
The sequence is proved to be unique since it occurs only once on Disk 2 at offset 0x9150F60.
Detecting disk order and block size
For detecting these RAID parameters I start moving through the file down from the offset at which I found the unique sequence and see what data is placed there. At the same time in WinHex I move through the disk down from the corresponding offset and compare the data on the disk with the data in the file. If the data is the same that means I am still in the same stripe on the disk relative to the offset at which I started moving. Otherwise I went out of the current stripe and therefore somewhere above there was a transition to a different disk.
Based on my experience I guess that block size is 64 KB (0×10000 bytes) and try moving at offsets:
- 0x9150F60 + 0×10000 = 0x9160F60 – mismatch, meaning that I am out of the current stripe and need to go back;
- 0x9160F60 – 0×8000 = 0x9158F60 – the same story;
- 0x9158F60 – 0×4000 = 0x9154F60 – match with the file, therefore the
current stripe ends somewhere below;
- 0x9154F60 + 0×2000 = 0x9156F60 – still in the current stripe;
- 0x9156F60 + 0×1000 = 0x9157F60 – mismatch, start to look around for
match with the file and reveal that 0x9157E00 is the needed offset.
￼￼Thus, I found out that a new stripe begins at 0x9157E00 offset on Disk 2. Now I look for a continuation of file at the same offset on the remaining disks and reveal it on Disk 1.
Knowing for sure that the stripe begins at this offset I will try to detect block size. Guess that block size is 64 KB and look at the offset 0x9157E00 + 0×10000 = 0x9167E00 on Disk 1.
Look at the file:
and then on Disk 1 where it is clear that the sequence is interrupted exactly in this place.
￼￼￼This means that my initial guess about 64 KB block size is true.
Since there are only three disks in the array and we have already found out that Disk 1 follows Disk 2, we just need to check that Disk 3 follows Disk 1. So we look at offset 0x9167E00 on Disk 3 and see a sequence continuation exactly there.
Knowing that disk order is 2-1-3 it is easy to determine the first disk by analyzing the offsets at which we found a continuation of sequence. Disk 1 follows Disk 2 toward increasing offsets; therefore, the first disk is Disk 1.
Determining start offset
Start offset is determined as the remainder of division of any offset at which stripe begins by block size. Take for example 0x915E00 offset mod 0×10000 = 0x7E00. Go to WinHex, check what data is located at this offset, and see that the array indeed begins at the offset simultaneously on all the member disks.
At this point all the RAID parameters are determined: 64 KB block size, start offset is 0x7E00 = 63 sectors, the first disk is Disk 1, disk order is 1-3-2. As for the data and parity blocks placement pattern, it is a left synchronous RAID array since disks came from an LDM array.
Note that if you don’t know what the RAID layout was used in the array, much more effort should be put additionally into determining pattern of data and parity blocks placement.
RAID 5 created on Storage Spaces virtual disks
In case of RAID arrays created with the help of Storage Spaces, in addition to the parameters listed above, you need to determine how slabs are linked to each other.
As you know, Storage Spaces cuts each disk from a pool into pieces (called “slabs”) of 256 MB which are then used in creating virtual disks.
Now let’s see how data and parity blocks will be placed on a RAID5 array made on three virtual disks which are in turn created as simple layout in Storage Spaces.
Virtual disks are created in different pools each made of two physical drives.
Then go to Disk Management and create a RAID5 on these virtual disks.
For simplicity, let’s assume that a slab contains two stripes instead of 1024 as expected and they are arranged as shown below. Of course this is a somewhat exaggerated placement but things are not much better in real cases.
It is easy to see that without knowing disk-slab-stripe translation it is impossible to recover parameters of the configuration in a reasonable time.
In case of a typical RAID 5 it is enough to determine parameters based on the known data, for example by tracing the location of a single file on the array, and then apply these parameters to the entire RAID. However, this approach will not work for the RAID created on top of the Storage Spaces virtual disks
Assume that searching the disks for sequences we know that file occupies 8-9-10- 11-12-13 blocks that in terms of physical disks and slabs gives the following:
Disk 4, slab 0, stripe 2 -> Disk 5, slab 1, stripe 1 -> Disk 1, slab 1, stripe 1->
Disk 4, slab 1, stripe 1 -> Disk 6, slab 1, stripe 1 -> Disk 1, slab 1, stripe 2.
Based on this you can get some conclusions on how these particular slabs are arranged (slab 0 and slab 1) on the disks, but there is no way to know from this how the other slabs are located on the disks.
Before you succumb to the temptation to create a RAID array on top of the virtual Storage Spaces disks, think carefully the plan of action in case of failure. It is almost impossible to restore such a configuration manually and there are no software solutions at this moment and it is unlikely to be any.
Enjoy this post? You may also like Multi data center redundancy – sysadmin considerations