Home / Devices / SAN

SAN Data Recovery Norwich

The difference in these calls lies in the fact that everything depending on the shared storage falls over at the same moment when it stumbles, as opposed to the rest of the week. SANs are reconstructed here (LUNs offline, pool metadata broken, VMFS datastores that will not mount) for companies and IT service providers throughout Norfolk, from a single shelf to a whole room of them. As such, enterprise work is core bench trade, as opposed to being just an occasional favour.

▲

Every SAN job is diagnosed free. The quote follows in writing, fixed, before a screwdriver is picked up.

In most cases you only pay if we recover your data. Electronic, mechanical, chip-level and physical work is charged at 50% upfront. Ransomware, forensic, BitLocker and CCTV/DVR jobs are charged 100% upfront on acceptance of our fixed quotation. Please refer to the data recovery cost page for full details on costs.

// thirty faults this bench knows well

The thirty ways they give up

The first job on any SAN is to match the symptom to the fault, and after twenty-odd years, these thirty account for almost everything we see through our doors.

Members failing in batches

Enterprise shelves shed drives by the handful rather than one at a time. Each drive has individual repair and imaging completed prior to group arithmetic resuming again.

The LUN went dark

The block device your servers were relying upon has gone. It will be rebuilt from the bottom up: RAID groups must be rebuilt prior to pool metadata, because doing so out of order does not prove anything.

Pool metadata in shreds

The map between blocks and volumes has failed. It will be reconstructed using the raw structures on disk which it had been describing all along, and which are still there.

A firmware update fatal to one controller

The management plane died during a flash while the data remained completely still behind it. Work proceeds directly against the disks and ignores the controller entirely.

Redundant controllers dying together

Pairs fail at the same time much more readily than the datasheet suggested. Recovery goes drive-direct, which has the advantage of never having read the datasheet.

VMFS wreckage over sound LUNs

The virtualisation layer collapses first and takes everything that was in view with it. We break the datastores apart and each virtual disk is extracted and repaired individually.

A provisioning slip that deleted a LUN

Seconds to do and days to regret. Whatever survives depends on whatever has been written since; thus, locking down the platform improves the maths straight away.

A snapshot chain cut mid-link

If there is one broken link in the chain, it brings down all the volumes that depended upon it. The chain is rebuilt at image level, one link at a time.

Thin pools writing cheques

If promised capacity exceeds real capacity, then every thin volume fed by the pool becomes corrupted at the moment it runs out. Rebuilt from the images, with the accounts able to stand up to inspection afterwards.

The write hole, opened

If cache batteries expire and a single outage occurs, parity will conflict with data across an entire group. However, at the image level, resolving the conflict will be affordable instead of destructive.

Aggregates missing their maps

NetApp aggregates and EMC pools lose track of where their volumes reside. The rebuild climbs up from the RAID groups until they can locate them again.

Sectors of unusual sizes

The 520, 524 and 528 byte formats are designed to limit ordinary tooling. The lab fabric used here reads these natively rather than using improvised adapters.

MSA metadata injuries

HPE's small business staple arrives with its vDisk structures torn. A rebuild we've gone over in this workshop so many times now it has become very boring.

Storwize shedding extents

V3700 and V5000 units drop extents after a power event. The extents are remapped from the drives one at a time, until the volume is whole.

An iSCSI target unpublished

The box lost its target definition, and the volume behind it rarely dies with it. It is extracted and presented again over plumbing that can be trusted.

Deduplication pointers broken at once

All references to all shared blocks fail at once when the index is corrupted. These are rebuilt until the shared blocks resolve properly again.

A failover that forked the truth

Both controllers were writing independently, as the handover was botched; therefore, there are now two versions of the same events. These are reconciled at image level, and both are timestamped and audited.

Hardware the vendor has forgotten

Arrays running beyond end of support fail with no official route left. The lab route does not have an expiry date, because data has never read a support contract.

Firmware mismatches after drive swaps

Replacement drives with different firmware versions can misbehave in older groups, in ways which look like failed drives. Stabilise, image and then rebuild, in that order.

LUNs hidden by a zoning change

Masking or fabric edits blind all the hosts at once while the underlying volumes remain healthy. The mapping is rebuilt and the owners are given access again.

A factory batch expiring together

Drives from the same lot hitting an hours-counter firmware defect typically fail within days of one another, which is a known and embarrassing problem across the storage industry, and should not be confused with bad luck. The surviving drives are imaged quickly before they join in.

Interposers rather than disks failing

Nearline SATA drives connected through SAS interposer boards fail much more frequently at the adapter than at the drive itself. As such, we simply bypass the adapters and image the bare drives.

The fast tier taking the hot data with it

When the fast tier fails, every tiered volume loses the blocks it was using the most. The cold tiers do nothing to make sense of this on their own. Instead, the cold tiers together with the tiering metadata rebuild whatever the fast layer was holding.

Two hosts writing one volume

A zoning slip without a clustered file system creates interleaved corruption from both sides. The two write histories are separated from images, operation by operation.

Locked out of the management plane

Service processor dead, credentials gone along with a contractor. The array will not communicate, but the drives have no such reservations, so data is extracted from them without a password.

A snapshot reserve that filled and purged itself

The amount of space set aside for snapshots is limited. Most systems deal with this by automatically deleting the oldest snapshots when the space runs out. In many cases, the restore point somebody was counting on has been deleted before anyone knew about it. What is left is carved from unallocated space on the imaged member disks.

Replication that ran the wrong way

A failover test, a mislabelled site, a relationship resumed in the wrong direction, and the healthy copy is overwritten with the stale one at wire speed. The overwritten side will be imaged and the older structures recovered from underneath, and the sooner this is stopped, the better it will work.

Encryption keys held on the failed controller

Arrays that encrypt data at rest store their keys in either the controller or on a key manager. When you lose both, your drives are readable but meaningless. This is identified during the diagnostic and said openly; there's simply no honest way around this fact.

A broken SAS expansion chain

Shelves are daisy chained together, so if a cable fails, an expander fails, or a shelf is powered up in the wrong order, a whole tier of disks can disappear at once. It looks like catastrophic loss, but in many cases it is actually a cabling issue. Establishing which comes before anybody starts rebuilding anything.

A host that formatted the wrong LUN

A new server is given several volumes and a well-meaning administrator initialises the one that looked spare. A quick format leaves most of the original volume in place, as long as the host is disconnected before it starts writing to the space it thinks is now its own.

Down the stack, then back up it

A SAN is a stack of layers sitting on top of each other: disks grouped into RAID sets, sets gathered into pools, and those pools split into LUNs, above which live the file systems and hypervisor datastores everyone uses. Recovery goes down that stack and then climbs back up in the same order. Copy every disk. Then stand the RAID sets back up. Next, repair the pool metadata. After that, lift out the LUNs. Finally, open the virtual machines and databases inside them. Much of the hardware is familiar (Dell EMC, NetApp, HPE MSA and EVA, IBM, Fujitsu), and NDAs, change control and whatever paperwork your organisation runs on fit into these steps without slowing them down.

The three machines that matter, ahead of the remaining terabytes

Businesses don't need raw blocks handed back to them; what they need is the handful of systems the business really runs on, working again by the morning. The datastores are brought up on the rebuilt LUNs, then the VMDK and VHDX files are extracted and repaired, and the named SQL or Exchange workloads are moved to the front so that operations can restart while everything else continues to copy in the background. This is triage versus heroics, done at a measured rate, as haste is how the second mistake gets made in an enterprise recovery.

// what sits on the bench

Engineering kit, not download-and-hope software

SAN recovery requires enterprise plumbing as well as recovery skills. Our workshop has large amounts of both:

FC, SAS and iSCSI lab fabric

Shelves and drives connect over our own Fibre Channel, SAS and iSCSI infrastructure, including the unusual sector sizes enterprise drives arrive with.

Parallel enterprise imaging

Entire RAID groups imaged together, after any casualty drive has first been through the mechanical and firmware benches.

Pool and LUN reconstruction

All vendor metadata is rebuilt in the order it was created: groups, pools, LUNs, then whatever file system is sitting on top.

VMFS, VMDK and VHDX parsers

Once the restored LUNs present their datastores again, each virtual disk is lifted out separately and repaired independently.

Database-aware extraction

The SQL Server and Exchange workloads named at the start are checked early and moved to the front of the queue, and therefore the services that matter come back first.

Write-blocked throughout

The original drives (or shelves) remain read-only from the moment they arrive until the moment they leave. At no point is this relaxed.

// makes & models we see

SAN platforms we recover

Dell EMCHPENetAppIBMHitachi VantaraLenovoPure StorageFujitsu ETERNUSDot Hill / SeagateSynology / QNAP rackmount

Hardware handled routinely

Unusual badges rarely mean unusual hardware, as enterprise vendors rebadge from a shared parts bin, and this workshop has met most of that bin more than once. Pools that have lost track of their maps are built back up from the RAID groups; two dead controllers simply route the work drive-direct. If a disk has an unusual sector size, we read it natively on our lab fabric instead of arguing with it. Send the labelled drives and leave the shelf in its rack.

// before you post it

Before you post it — get the drive loose if you can

Tag every drive with both the shelf it is housed in and its slot number before anything comes out of the rack. Enterprise storage will punish you for making assumptions about where things were. Take a picture of everything as you found it, and send us only the tagged drives (leave the shelves and controllers on site). Send them tracked and insured, or by your own courier. We don't collect the disks ourselves. If you would like, you may drop off the disks at reception at our Cambridge location, Monday to Friday, 9am to 5:30pm.

// getting your device to us

Getting it here — no great performance

Most items arrive at this bench by insured and tracked post. We recommend getting a quote first, either by telephone or using our free online diagnostic, and once you have received your quote, simply post the item. Post is the steadiest way to move a poorly drive, and a parcel sent from Norfolk typically arrives on the bench within a working day.

Is the drive still bolted inside a laptop, desktop, MacBook, iMac, server or CCTV / DVR recorder? Before sending, remove the hard disk drive (HDD) or solid state drive (SSD), as we do not accept whole units. From a DVR or recorder, send just the disk. From a server or RAID, take out the disks and clearly label each disk by bay number. A network attached storage (NAS) device may be sent as its disks if they can be removed; otherwise please send the NAS itself. There is only one type of drive we are unable to accept: storage soldered onto a motherboard (e.g. Apple Silicon Macs, extremely thin laptops). In such cases, if the storage cannot be taken out it cannot be accepted. If you are unsure whether you have a removable drive or not, call us before sending anything.

  • Wrap the drive in bubble wrap, pack it into a sturdy box or padded envelope and make sure there is no movement in transit. Do not include any cables, caddies etc., except when sending a single external hard drive, which should come whole with its power supply and its own cable.
  • Print out the shipping & booking-in form (PDF), add your name, phone number and a few sentences explaining what happened, and send that with the drive.
  • You can use Royal Mail's Special Delivery service, which tracks and insures your package door to door. Alternatively you could find a courier of your own to do the same thing.
  • Handing it over personally? Once you have received your price quote, our reception will take your drop-off, Monday to Friday, 9am to 5:30pm. No need to book an appointment.
// where your device is headed

Cambridge Data Recovery

Compass House, Vision Park
Chivers Way
Cambridge, CB24 9AD

↓ Print the shipping & booking-in form (PDF)

Please mark the parcel for the attention of Cambridge Data Recovery. It takes about an hour and twenty minutes to travel from Norwich along the A11, then two minutes off the A14 at Junction 32. Otherwise you can post it using a tracked service and we will receive it the following working day. We will contact you as soon as we have your drive and it is booked onto our bench.

If you're unsure of what to include in the box, please call us on 0800 689 0668 before sealing it. Or, for a first view on what we can do, you can use our free online diagnostic.

// san recovery questions

Common questions

Make sure all writes are frozen. No offers to initialise should be accepted; do not change anything while thinking about the next move. Then call us with the make, model and any documentation of the layout, and we'll walk you through removing the disks and labelling each disk by its bay number.
Yes. Rebuilding VMFS on top of reconstructed LUNs is one of our steadiest enterprise lines. Tell us which machines need to come back first; those will be extracted first. As a result, services will be running long before the last byte finishes copying.
As standard, and often invisibly. Confidentiality agreements, change control procedures, written handling records and a clean handover for your own people to restore from are all part of the normal process. A fair amount of this work arrives through IT providers whose clients only ever see them, which suits everybody.
The assessment takes up to two working days after the equipment arrives, and jobs where a business has stopped trading are placed at the front of that queue. Phone us on 0800 689 0668, tell us trading has halted, and we will agree how best to pack and send the equipment on that same call, so nothing is left sitting at your end.
// related services

More work we take on

Whenever you’re ready, the bench is.

Free diagnosis, one fixed figure written down, and in most cases you only pay if we recover your data. Start online or ring us.