Disk cloning basics using dd in Linux

January 6, 2012 4:45 PM

Backing up a possibly-about-to-die hard drive and putting it on a new drive, using dd in Linux - a few questions regarding how the cloning works.

So the drive in my wife's PC (250gb SATA, Windows Vista) looks like it might be dying - making some weird noises. So I'm in the process of imaging it, and putting the image on a new drive. I am using the "dd" tool on a Linux live CD to do this. I just want to make sure I'm doing it right.

1) I copied the drive image to a file on a portable USB drive, using:

dd if=/dev/sda of=/media/PortableDrive/my.backup

It worked - no errors. It did take almost 3 days to copy the 250gb, though.

Now I have a new drive, plugged it in, and tried the reverse process:

dd if=/media/PortableDrive/my.backup of=/dev/sda

It dies within seconds, telling me there's an Input/Output error at 17mb into the disk. Does this mean that errors on the original hard drive were copied onto the image I made on the portable drive?

So, next thing I tried was plugging the old drive back in, and doing a direct copy from one to the other.

dd if=/dev/sdb of=/dev/sda

I got the same problem - IO error at 17mb into the disk.

Next thing I tried - direct copy using the dd_recover tool, which tries harder to read bad sectors. It didn't balk at the error at 17mb (although it noticed it), but the transfer rate was stupidly slow - about 3mb/s, so I gave up on that.

Alright, so now I'm using dd_recover to copy from the image on my portable drive to my new internal drive. It is going beautifully - much faster, no errors showing up. Can I assume that the bad sectors on my old internal drive will contain messed up data on my new drive? and, most of all, Am I doing this correctly at all?

It just seems strange to me that dd copied data from my old drive to my portable without an error (just slowly), but gave an error when I tried to copy from my portable to my new drive.

posted by Jimbob to computers & internet (10 answers total)

Does this mean that errors on the original hard drive were copied onto the image I made on the portable drive?

No; dd will stop as soon as it detects an I/O error, so if you managed to make an image using dd and the image is the same size as your original device, the original device contained no unreadable sectors.

My initial suspicion would have been that there's an unreadable block on your backup disk 17MB from the start of your backup image. However, given that you got the same problem when doing a direct copy from the original disk, the only plausible explanation is that there was an unaddressable block on the destination disk at 17MB.

Can I assume that the bad sectors on my old internal drive will contain messed up data on my new drive?

I don't believe your old internal drive had any unreadable sectors, so no.

Am I doing this correctly at all?

Mostly, though the tools you're using aren't the tools I'd use.

The most likely reason why dd took three days to copy a 250GB drive is that you didn't give it a bs= option (I generally use bs=1M) which means it was doing all reads and writes in chunks of its default block size, 512 bytes. This is sloooooooow.

The tool I'd use for this is Gnu ddrescue, which you will find on the Trinity Rescue Kit live CD.

To clone the disk at /dev/sdb onto the one at /dev/sda with maximal chance of error recovery, you'd use

ddrescue --direct --max-retries=10 /dev/sdb /dev/sda sdb.log

Before doing this, make sure /dev/sdb really is your source disk - ddrescue will happily copy a blank drive over the top of your data if you ask it to.

Gnu ddrescue does data transfer in large chunks to start with, coming back to re-examine the areas around unreadable blocks in smaller chunks only when the main copy pass has finished, so it's decently quick. It also has good progress and error reporting, and if you give it a log file as I've done in the example above, you can interrupt it with Ctrl-C and then resume just by re-issuing the original command. Good tool.

Given that 17MB error, I'd be inclined to check the new drive once your existing dd_recover copy session has finished, using

ddrescue /dev/sda /dev/null sda-check.log

and make sure it progresses past 17MB before Ctrl-C-ing it.
posted by flabdablet at 5:45 PM on January 6, 2012 [6 favorites]

I don't know much about dd, but you might want to try CloneZilla for this purpose. It's a little more complex, but I believe it gives a more robust outcome. With CloneZilla, you can do a drive-to-drive clone if both are installed in the computer at the same time; sample instructions are here.
posted by Simon Barclay at 5:48 PM on January 6, 2012

As far as I know, dd will simply quit and report a read error if it runs into trouble, and ddrescue doesn't have any extra rescue functionality for the disk it writes to, only the one it reads from. So, chances are there's a small error on your portable drive, which ddrescue successfully worked around. (On preview, this is wrong, flabdablet is right) If you want to be sure, you can grab the area around 17mb using dd skip=34816 count=2048 if=/dev/sda of=/media/PortableDrive/my.additional.backup and copy it over again using dd seek=34816 if=/media/PortableDrive/my.additional.backup of=/dev/sda. DO NOT do this if you've booted up the system with the new drive or even looked at the files on it.
It probably wouldn't hurt to check all three of your drives for errors, including bad sectors, after you're done.
posted by marakesh at 6:06 PM on January 6, 2012

Sorry, I'm using dd_rescue, not dd_recover - although it does appear that dd_rescue and ddrescue are two slightly different programs.

So, what I'm getting from this is - dd balked at copying onto my new drive from the portable drive, and it balked at copying directly from old drive to new drive, so it appears there may be an error on my new drive? (This would severely piss me off) It's interesting that dd picks up this error and quits, but dd_rescue has happily written over it (and it still copying things over at the moment - in about half an hour it should be finished and I'll do the "extra" backup of the possibly bad region).

All else failing (ie. if my new drive doesn't boot) I'll try the

ddrescue --direct --max-retries=10 /dev/sdb /dev/sda sdb.log

suggestion, and see where I go from there. Thanks.
posted by Jimbob at 6:50 PM on January 6, 2012

it does appear that dd_rescue and ddrescue are two slightly different programs.

Totally independent code bases. I think Gnu ddrescue works better and is less fiddly, though dd_rescue allows for use cases like piping a disk image over a ssh tunnel that Gnu ddrescue doesn't (Gnu ddrescue needs to seek inside the target file).

it appears there may be an error on my new drive? (This would severely piss me off)

Why? It hasn't cost you any data yet. Better to find these things out early, I reckon.
posted by flabdablet at 10:05 PM on January 6, 2012

If, on the odd chance, you can't or won't replace the new drive, one workaround would be to make a filesystem on the new drive using mkfs and then mounting the file on the USB drive via loopback, and recursively copying to the new drive. Unlike dd, mkfs will detect and mark bad blocks and deal with them gracefully.

something like this:

sudo mkfs -t ext3 /dev/sda1 # substitute the desired filesystem and actual device here, of course
mkdir ~/newdisk; sudo mount /dev/sda1 ~/newdisk
mkdir ~/olddisk; mount -o loop /media/PortableDrive/my.backup ~/olddisk
cp -R /home/olddisk/ /home/newdisk

of course, you should really just get the people who sold you the drive to give you a new one, and the mount points are 100% arbitrary if you don't like mounting things in your home directory, and if you just want to get at a few files, you can just use the loopback mount command from the third line
posted by idiopath at 12:51 AM on January 7, 2012

ugh, that last line of code should have been 'sudo cp -R ~/olddisk ~/newdisk'
posted by idiopath at 12:53 AM on January 7, 2012

Well, it all worked. The image copied over to my new drive using dd_recover, I scanned the new drive for bad sectors and it's all good, Windows boots, and the wife is appeased for another day. Thanks for all the help.

Why? It hasn't cost you any data yet.

Computer hardware stores are kinda thin on the ground in my part of the world, and having to drive back in there for a replacement hard drive would have ruined my Saturday.
posted by Jimbob at 3:39 AM on January 7, 2012

ugh,that last line of code should have been 'sudo cp -R ~/olddisk ~/newdisk'

I'd suggest it should rather be sudo rsync -aHS ~/olddisk/ ~/newdisk/
posted by vsync at 6:28 AM on January 7, 2012

If, on the odd chance, you can't or won't replace the new drive, one workaround would be to make a filesystem on the new drive using mkfs and then mounting the file on the USB drive via loopback, and recursively copying to the new drive.

The disk being cloned here belongs to a computer with Vista installed, so you wouldn't use Linux tools to do that; you'd end up with a disk full of files with hosed NTFS permissions.

Cloning a Windows disk via file-level tools is best done by booting a BartPE live CD, then using diskpart, format (without /Q) and xcopy. It takes a lot longer, and is a lot more fiddly and error-prone, than using ddrescue. Me, I'd just use ddrescue and a known-good destination drive.
posted by flabdablet at 6:43 PM on January 7, 2012