How do I transfer my current Ubuntu system to a new, larger hard drive?
August 21, 2007 10:20 AM Subscribe
How do I transfer my current Ubuntu system to a new, larger hard drive?
(If there's a good explanation of this anywhere, I've been unable to find it. I've seen some explanations of how to do this with older, LILO-based systems. But I haven't seen anything that explains how to do this with GRUB, UUID's in fstab, etc).
My current setup has two hard drives, one 20 gig root+swap and an 80 gig mounted at /home. I want to replace them with my new 250 gig drive. As I understand it, the following steps need to take place:
0. Reboot my system and go into recovery mode.
1. fdisk /dev/hdc
hdc1: 249 gig bootable type 83
hdc2: 1 gig type 82
2. mkfs /dev/hdc1
3. mkswap /dev/hdc2
4. mount /dev/hdc /mnt/hdc
5. cp -a /???? /mnd/hdc
6. dd if=/dev/hda of=/dev/hdc bs=1024 (?)
7. vi /mnt/etc/fstab (?)
In step 5, what should I be copying? Not /proc, presumably. /dev? What is /sys? /srv? How do I know what should and what should not be copied?
Is step 6 the right way to copy the boot loader if I'm using grub? Isn't GRUB bigger than the old LILO?
Fstab looks very strange to me these days. Instead of specifying the devices, Ubuntu seems to be using UUID's. Where did they come from? How do I find what my new HD's UUIDs are?
(If there's a good explanation of this anywhere, I've been unable to find it. I've seen some explanations of how to do this with older, LILO-based systems. But I haven't seen anything that explains how to do this with GRUB, UUID's in fstab, etc).
My current setup has two hard drives, one 20 gig root+swap and an 80 gig mounted at /home. I want to replace them with my new 250 gig drive. As I understand it, the following steps need to take place:
0. Reboot my system and go into recovery mode.
1. fdisk /dev/hdc
hdc1: 249 gig bootable type 83
hdc2: 1 gig type 82
2. mkfs /dev/hdc1
3. mkswap /dev/hdc2
4. mount /dev/hdc /mnt/hdc
5. cp -a /???? /mnd/hdc
6. dd if=/dev/hda of=/dev/hdc bs=1024 (?)
7. vi /mnt/etc/fstab (?)
In step 5, what should I be copying? Not /proc, presumably. /dev? What is /sys? /srv? How do I know what should and what should not be copied?
Is step 6 the right way to copy the boot loader if I'm using grub? Isn't GRUB bigger than the old LILO?
Fstab looks very strange to me these days. Instead of specifying the devices, Ubuntu seems to be using UUID's. Where did they come from? How do I find what my new HD's UUIDs are?
I can't speak to the method proposed above, but I stumbled upon a pretty straightforward method when I was faced with the same situation. Here's what I did:
install both hard drives into your Ubuntu system, making sure you know which /dev/ name each has been assigned.
boot up with the GParted Live disk
you can, and I am not kidding, copy and paste the entire partition from your old to your new drive. Don't forget to make the boot and swap partitions. I like to put /home on a separate partition, myself.
Restore GRUB (also available on the GParted Live)
Seriously, I was amazed when this worked, and I lost no time (other than the 2 or 3 hours of copying). Even nicer is that you have an intact copy of your original disk, until you've confirmed that everything works.
posted by zachxman at 11:08 AM on August 21, 2007
Seriously, I was amazed when this worked, and I lost no time (other than the 2 or 3 hours of copying). Even nicer is that you have an intact copy of your original disk, until you've confirmed that everything works.
posted by zachxman at 11:08 AM on August 21, 2007
Yeah, avoid "dd". You don't need a byte-for-byte copy, and it's probably a bad idea if the disk geometries are different.
Try this:
(cd source; sudo tar cf - .;) | (cd dest; sudo tar xvf -;)
Then, install the location-specific stuff like the bootloader.
posted by cmiller at 12:01 PM on August 21, 2007
Try this:
(cd source; sudo tar cf - .;) | (cd dest; sudo tar xvf -;)
Then, install the location-specific stuff like the bootloader.
posted by cmiller at 12:01 PM on August 21, 2007
For step five, change "cp -a" to "cp -ax" to limit cp to one file system. This will keep it from copying /proc, /home, and so forth. (Step 4, I assume you meant to mount hdc1)
So for step file, do
cp -ax / /mnt/hdc1
Then if you have any other partitions mounted, say /home, and you were combining them on the new drive, you'd do
cp -ax /home /mnt/hdc1/home
and so on. If you're keeping /home as a separate partition, you'll have to mount and copy that separately.
Generally, I've used tar instead of cp, but that's because I pre-date GNU "cp -a".
To do step 5 with GNU tar, do
cd /
tar clf - . | (cd /mnt/hdc1 ; tar xvf - )
posted by fings at 2:50 PM on August 21, 2007
So for step file, do
cp -ax / /mnt/hdc1
Then if you have any other partitions mounted, say /home, and you were combining them on the new drive, you'd do
cp -ax /home /mnt/hdc1/home
and so on. If you're keeping /home as a separate partition, you'll have to mount and copy that separately.
Generally, I've used tar instead of cp, but that's because I pre-date GNU "cp -a".
To do step 5 with GNU tar, do
cd /
tar clf - . | (cd /mnt/hdc1 ; tar xvf - )
posted by fings at 2:50 PM on August 21, 2007
Avoid using cp. Use tar. cp can do odd things with special files and a full unix filesystem has plenty. tar was originally meant for tape backups and will do a bang up job. cmilller has pretty much what I would suggest.
You don't need nor want the 'z' option specified by sirstan since that will just waste cycle compressing and uncompressing the data.
posted by chairface at 3:25 PM on August 21, 2007
You don't need nor want the 'z' option specified by sirstan since that will just waste cycle compressing and uncompressing the data.
posted by chairface at 3:25 PM on August 21, 2007
Best answer: Personally, I would start with a brute-force disk image.
1. Physically remove your present 80GB drive and replace it with your new 250GB drive, making sure that the 250GB drive has the same IDE jumper settings as the 80GB drive used to have.
2. Use a live CD (I'm fond of the Trinity Rescue Kit, but an Ubuntu live CD should work just as well) to image the entire 20GB drive onto the 250GB drive - boot loader, partition table, existing partitions and all, in one big steaming lump. There is no geometry issue, since everything will be using LBA addressing; the only geometry specifications you ever encounter are the ones encoded in the partition table, and you're copying that along with everything else.
In TRK, the commands I'd use for this are:
fdisk -l to identify the drives; with the current TRK kernel, I'd expect the IDE master (probably your 20GB drive) to show up as /dev/sda, and the slave (presumably your 250GB drive) to show up as /dev/sdb.
ddrescue /dev/sda /dev/sdb ddrescue.log to do the actual image copy; dd would probably work fine too, but ddrescue has useful progress indication, and won't behave badly if bad blocks are found on the source disk. Note that this command line puts the ddrescue log file into the current working directory, which will be in RAM. If ddrescue detects bad blocks, it would probably be a good idea to save a copy of that log file to a USB stick.
Doing the same thing with an Ubuntu live CD would involve installing ddrescue before using it (sudo apt-get install gddrescue) or using sudo dd if=/dev/sda of=/dev/sdb bs=1M conv=noerror,sync instead; also, Ubuntu versions before Feisty will treat IDE drives as /dev/hdX instead of /dev/sdX.
3. Physically remove the 20GB drive, and put the 250GB drive where the 20GB drive used to be, making sure that the 250GB drive now has the same IDE jumper settings as the 20GB drive used to have. Put the 80GB drive back where it used to be.
4. Make sure your computer now boots up and runs off the new 250GB drive exactly the way it did off the old 20GB drive. I'd be very surprised to find it didn't.
5. Make a new partition on the 250GB drive (you should find about 230GB of free space to do that in), format it with your filesystem of choice, mount it temporarily as /mnt, copy all of /home to it, and unmount it.
6. Edit fstab to make your new partition mount as /home on next boot instead of your old drive.
You can still use a device names like /dev/sda3 or whatever inside fstab - the UUID stuff is optional. I think the main reason that Ubuntu has gone to UUIDs in fstab is to deal with stuff like device hames changing from /dev/hdXN to /dev/sdXN with new kernel versions.
sudo vol_id /dev/sda3 will give you the UUID for /dev/sda3.
If you want to get a bit more future-proof, read up on Logical Volume Management before you do step 5. If you make your new partition available to LVM, and then create a new logical device on top of that, then after you've copied your existing /home to the new device, you should then be able to add your existing 80GB drive to LVM, extend your new logical device onto it, and end up with 230+80=310GB of space for /home and easier data migration in future.
posted by flabdablet at 6:13 PM on August 21, 2007 [1 favorite]
1. Physically remove your present 80GB drive and replace it with your new 250GB drive, making sure that the 250GB drive has the same IDE jumper settings as the 80GB drive used to have.
2. Use a live CD (I'm fond of the Trinity Rescue Kit, but an Ubuntu live CD should work just as well) to image the entire 20GB drive onto the 250GB drive - boot loader, partition table, existing partitions and all, in one big steaming lump. There is no geometry issue, since everything will be using LBA addressing; the only geometry specifications you ever encounter are the ones encoded in the partition table, and you're copying that along with everything else.
In TRK, the commands I'd use for this are:
fdisk -l to identify the drives; with the current TRK kernel, I'd expect the IDE master (probably your 20GB drive) to show up as /dev/sda, and the slave (presumably your 250GB drive) to show up as /dev/sdb.
ddrescue /dev/sda /dev/sdb ddrescue.log to do the actual image copy; dd would probably work fine too, but ddrescue has useful progress indication, and won't behave badly if bad blocks are found on the source disk. Note that this command line puts the ddrescue log file into the current working directory, which will be in RAM. If ddrescue detects bad blocks, it would probably be a good idea to save a copy of that log file to a USB stick.
Doing the same thing with an Ubuntu live CD would involve installing ddrescue before using it (sudo apt-get install gddrescue) or using sudo dd if=/dev/sda of=/dev/sdb bs=1M conv=noerror,sync instead; also, Ubuntu versions before Feisty will treat IDE drives as /dev/hdX instead of /dev/sdX.
3. Physically remove the 20GB drive, and put the 250GB drive where the 20GB drive used to be, making sure that the 250GB drive now has the same IDE jumper settings as the 20GB drive used to have. Put the 80GB drive back where it used to be.
4. Make sure your computer now boots up and runs off the new 250GB drive exactly the way it did off the old 20GB drive. I'd be very surprised to find it didn't.
5. Make a new partition on the 250GB drive (you should find about 230GB of free space to do that in), format it with your filesystem of choice, mount it temporarily as /mnt, copy all of /home to it, and unmount it.
6. Edit fstab to make your new partition mount as /home on next boot instead of your old drive.
You can still use a device names like /dev/sda3 or whatever inside fstab - the UUID stuff is optional. I think the main reason that Ubuntu has gone to UUIDs in fstab is to deal with stuff like device hames changing from /dev/hdXN to /dev/sdXN with new kernel versions.
sudo vol_id /dev/sda3 will give you the UUID for /dev/sda3.
If you want to get a bit more future-proof, read up on Logical Volume Management before you do step 5. If you make your new partition available to LVM, and then create a new logical device on top of that, then after you've copied your existing /home to the new device, you should then be able to add your existing 80GB drive to LVM, extend your new logical device onto it, and end up with 230+80=310GB of space for /home and easier data migration in future.
posted by flabdablet at 6:13 PM on August 21, 2007 [1 favorite]
You could also try g4u, a Linux-based disk cloning utility. According to the documentation, it'll work just fine in cloning to a larger hard drive, although any space above and beyond the original disk would be unallocated and you'd need to partition it manually.
Haven't tried it myself, but I'm planning it using it this weekend to clone a Windows drive before resizing a couple of partitions with GParted.
posted by Ickster at 9:32 PM on August 21, 2007
Haven't tried it myself, but I'm planning it using it this weekend to clone a Windows drive before resizing a couple of partitions with GParted.
posted by Ickster at 9:32 PM on August 21, 2007
Looks like g4u does exactly the same job as I recommended ddrescue for, only with a file on an FTP server as an intermediate stage.
And yes, copying all blocks of a small disk to a larger one does work just fine. This is because the partition table (a) is on block 0, so it ends up in the same spot on both drives (b) contains only information about the sizes of the defined partitions, not about the size of the drive as a whole.
The only difference between the old drive and the new drive after a whole-drive copy is that the new drive will have unpartitioned (free) space at the end; and provided the existing partition table contains less than four primary partition entries, you can just make another partition to use up that space.
posted by flabdablet at 11:13 PM on August 21, 2007
And yes, copying all blocks of a small disk to a larger one does work just fine. This is because the partition table (a) is on block 0, so it ends up in the same spot on both drives (b) contains only information about the sizes of the defined partitions, not about the size of the drive as a whole.
The only difference between the old drive and the new drive after a whole-drive copy is that the new drive will have unpartitioned (free) space at the end; and provided the existing partition table contains less than four primary partition entries, you can just make another partition to use up that space.
posted by flabdablet at 11:13 PM on August 21, 2007
Response by poster: Thank you all very much!
I was surprised at how difficult this ended up being. I started using the method outlined above (except using grub's shell instead of trying to copy the boot sector with dd, using cp -ax and using vol_id to update the new fstab), but I was never able to boot successfully. It's hard to say why, since there were so many things I could get wrong. Grub's documentation could use some work.
In the end, I combined flabdablet and zachxman's suggestions.
1. I booted from a GParted Live disk and dd'ed my root disk to my new disk - so I didn't have to worry about beating grub into submission. Also, the UUIDs were right in fstab (for now).
2. I then used GParted to delete my swap partition to get it out of the way, expand my root partition to take up most of the HD, and then create a new swap partition at the end of the disk. zachxman's right - it's a great tool.
3. I then mounted the new disk and edited its fstab, since by deleting and recreating the swap, it now had a new UUID (hda1's is unchanged, since I did a byte-for-byte copy of the old disk). vol_id -u /dev/hda2 gave me the UUID to put in there.
Voila! The next step was to mount my home drive and copy its contents to my new HD, which I started before leaving for work.
posted by bonecrusher at 6:54 AM on August 22, 2007
I was surprised at how difficult this ended up being. I started using the method outlined above (except using grub's shell instead of trying to copy the boot sector with dd, using cp -ax and using vol_id to update the new fstab), but I was never able to boot successfully. It's hard to say why, since there were so many things I could get wrong. Grub's documentation could use some work.
In the end, I combined flabdablet and zachxman's suggestions.
1. I booted from a GParted Live disk and dd'ed my root disk to my new disk - so I didn't have to worry about beating grub into submission. Also, the UUIDs were right in fstab (for now).
2. I then used GParted to delete my swap partition to get it out of the way, expand my root partition to take up most of the HD, and then create a new swap partition at the end of the disk. zachxman's right - it's a great tool.
3. I then mounted the new disk and edited its fstab, since by deleting and recreating the swap, it now had a new UUID (hda1's is unchanged, since I did a byte-for-byte copy of the old disk). vol_id -u /dev/hda2 gave me the UUID to put in there.
Voila! The next step was to mount my home drive and copy its contents to my new HD, which I started before leaving for work.
posted by bonecrusher at 6:54 AM on August 22, 2007
I think the end of the disk is exactly the wrong place to put a swap partition. Data transfers are slower there, because there are fewer sectors per disk revolution on the inner tracks; also, since most of your executables will probably have been allocated early on in your disk's life, they will probably be nearer the start of the root partition than the end - meaning that something close to the largest possible seek will occur every time you need to swap out a few pages.
If you'd followed the method I suggested, then assuming your swap partition was originally allocated at the end of your 20GB drive, it would have ended up about 8% in from the outer edge of your 250GB drive, giving you close to maximum data transfer rate, very little seeking distance from your executables, and a generally smaller seeking distance from any of the files you put in /home for the first year or so.
I would have thought that 20GB would be plenty for a root partition, too; I can't really see the virtue in combining it with /home.
Keeping /home in a separate partition also makes it easy to do things like put a backup image of the entire root partition into a file under /home, then blow the root partition away with a new installation without disturbing your own files.
If you really truly want to put /home in the root partition, I strongly recommend making your swap partition the first one on your drive. This will make it as fast as it can be, and give you much less seek time than you'll get by putting it at the end.
posted by flabdablet at 8:14 AM on August 22, 2007
If you'd followed the method I suggested, then assuming your swap partition was originally allocated at the end of your 20GB drive, it would have ended up about 8% in from the outer edge of your 250GB drive, giving you close to maximum data transfer rate, very little seeking distance from your executables, and a generally smaller seeking distance from any of the files you put in /home for the first year or so.
I would have thought that 20GB would be plenty for a root partition, too; I can't really see the virtue in combining it with /home.
Keeping /home in a separate partition also makes it easy to do things like put a backup image of the entire root partition into a file under /home, then blow the root partition away with a new installation without disturbing your own files.
If you really truly want to put /home in the root partition, I strongly recommend making your swap partition the first one on your drive. This will make it as fast as it can be, and give you much less seek time than you'll get by putting it at the end.
posted by flabdablet at 8:14 AM on August 22, 2007
This thread is closed to new comments.
I would recommend booting off a CD.
partition your new HD how you want it (it can differ from the old system). Mount it up as /mnt/hd-new. Mount the old HD as mnt/hd-old
(ie, /dev/hdc1 /mnt/hd-new, /dev/hdc2 /mnt/hd-new/usr, etc).
Then make a pipe (mkfifo transfer).
Then tar into the pipe (tar vfz transfer /mnt/hd-old/home /mnt/hd-old/usr /mnt/hd-old/var /mnt/hd-old/etc...)
On another terminal untar to the new drive..
tar xvf transfer /mnt/hd-new
When thats done, move hdc to hda (physically). Boot off the live CD again, mount up your partitions again, and run lilo/grup to configure the boot loader on the new drive.
Fix fstab if you changed partitions. As to what shouldn't be copied.. skip the un-bold entries (this is off my Centos system).
drwxr-xr-x 2 root root 4096 Aug 7 04:04 bin
drwxr-xr-x 4 root root 1024 Aug 6 12:23 boot
drwxr-xr-x 8 root root 5440 Aug 6 12:24 dev
drwxr-xr-x 91 root root 12288 Aug 15 04:02 etc
drwxr-xr-x 9 root root 4096 Jun 20 14:15 home
drwxr-xr-x 2 root root 4096 Feb 21 2005 initrd
drwxr-xr-x 12 root root 4096 Aug 7 04:04 lib
drwx------ 2 root root 16384 Aug 21 2006 lost+found
drwxr-xr-x 3 root root 4096 Aug 6 12:25 media
drwxr-xr-x 2 root root 4096 May 3 01:14 misc
drwxr-xr-x 2 root root 4096 Apr 26 10:47 mnt
drwxr-xr-x 2 root root 4096 Feb 21 2005 opt
dr-xr-xr-x 125 root root 0 May 29 08:51 proc
drwxr-x--- 31 root root 4096 Aug 20 10:01 root
drwxr-xr-x 2 root root 12288 Aug 7 04:04 sbin
drwxr-xr-x 2 root root 4096 Aug 21 2006 selinux
drwxr-xr-x 2 root root 4096 Feb 21 2005 srv
drwxr-xr-x 9 root root 0 May 29 08:51 sys
drwxrwxrwt 6 root root 4096 Aug 21 04:02 tmp
drwxr-xr-x 14 root root 4096 Aug 21 2006 usr
drwxr-xr-x 21 root root 4096 Mar 13 12:27 var
my SRV dir is empty, as is selinux.. i would copy them.
As for "Is step 6 the right way to copy the boot loader if I'm using grub? Isn't GRUB bigger than the old LILO?"
You dont. You copy the setup file, and the app (lilo, grub). You then run the app against the setup/conf file so it re-installs. Dont 'copy' it with dd.
dd only works well with EXACT duplicate drives. (same size, same logical setup, etc) One last thing!
I would REMOVE your second harddrive (/home) during this entire process. Once you move the system (20gb) to the 250gb, and have moved the 250gb to HDA, you can reattach the 80gb harddrive, unmount it as /home, make a new home, and copy files over with the same process. Its a step that will cost you maybe an extra 5 minutes, but protect you from somehow accidentally damaging your userfiles.
I've done this probably 20-30 times, and I would still remove the /home drive just to be 100% over-cautious.
posted by SirStan at 10:36 AM on August 21, 2007