How do I increase partition size in a RAID 1 array with minimal downtime?

May 4, 2012 6:08 AM

RAID 1 Partition Filter: Physical capacity increased. How do we increase the partition size to match on a production machine? Details within.

We have a production server running Debian with a RAID 1. Yesterday, the 3ware RAID card showed that one of the 120 gig drives had become degraded and we replaced it with a 250 gig drive. After the array rebuilt, the tech on site decided to pull the other 120 gig drive and replace it with a 500 gig drive and increase the storage on the machine to 250 gigs. The array rebuilt again and all is running fine, however, the partition size remains at 120 gigs (minus swap, etc).

What's the best way to resize the partition with the least amount of downtime for the machine? Ideally, it would be on the fly. A few restarts would be ok. A backup/restore would probably be more trouble than it's worth, but if that's the only option, we'd like to hear about it anyways. The host handles mail and MySQL slave duties.

Snowflakery: Linux 3.1.0-1-amd64, Debian wheezy/testing, 3DM 2 version 2.07.00.003, ext3

Further info upon request. Thanks in advance.

posted by even butter to computers & internet (4 answers total)

While the partition is larger, the filesystem has apparently not been resized.

Look at resize2fs to do this.

You'll need to unmount the partition to run it. This may or may not mean downtime, depending on how your server is configured.

Here's a bit more: http://tldp.org/HOWTO/LVM-HOWTO/extendlv.html (that doc is about what to do after extending a LVM partition, which should be a similar enough situation to what you've done). There are other HOWTOs lying around the Interweb. Just search for "resize filesystem linux" or something like that.
posted by chengjih at 6:26 AM on May 4, 2012 [1 favorite]

(If you want to resize the non-root partition, you can likely do it online.)
posted by devnull at 6:53 AM on May 4, 2012

A backup - restore is the "industry standard best practice".

However, I run a server at home with a linux (fedora) software raid-5 that has been expanded online all the way from 40gb component drives all the way up to 1000gb drives.

What you need to do is this: (and please double check my work before implementing it!)

1- Make sure you have a good backup.
2- Make sure the array is healthy.
3- Use the 3ware utilities to increase the size of the raw volume. This may have already been done automatically. fdisk -l (that's a lowercase L) should list the raw size of the /dev/sdX raid 1 device.
4- If that's successful, use fdisk (or your favorite partition utility) to increase the size of the partition.
5- use resize2fs /dev/sdX to grow the filesystem.

Note: I don't use a partition table on mine- the raw drives combine to a raw virtual device, which is then filled completely with an ext3 filesystem. So I don't have to do this step. You'll have to investigate whether this can be done with the filesystem mounted. It *should* be possible, since I think ext3 doesn't really pay any attention to the end of the partition table and will just ignore the extra space in the partition until the filesystem is grown. I'd probably do that part offline just to be safe. Warning: this can only work if you aren't moving the beginning of the partition! Don't move the beginning of the partition! That will wreck everything. You can only move the end of it, and anything on the disk after the end of it will be wrecked.

It would basically work like this: (using fake numbers)
If your raid disk is /dev/sda, then the current partition table would have looked like this prior to the disk swapping:

disk /dev/sda size 120,000m
                      size      start       end
/dev/sda1  boot       100m          0       100
/dev/sda2  ext3   118,900m        101   119,000
/dev/sda3  swap      1000m    119,001   120,000 

After swapping disks and changing the raid container size, it probably looks like this:

disk /dev/sda size 250,000m
                      size      start       end
/dev/sda1  boot       100m          0       100
/dev/sda2  ext3   118,900m        101   119,000
/dev/sda3  swap      1000m    119,001   120,000 

Then you do the swap thing, leaving you with this:

disk /dev/sda size 250,000m
                      size      start       end
/dev/sda1  boot       100m          0       100
/dev/sda2  ext3   118,900m        101   119,000
/dev/sda3  swap      1000m    249,001   250,000 

Finally, you edit the ext3 partition to fill in the empty space:

disk /dev/sda size 250,000m
                      size      start       end
/dev/sda1  boot       100m          0       100
/dev/sda2  ext3   118,900m        101   249,000
/dev/sda3  swap      1000m    249,001   250,000 

I have run resize2fs online with no problems. It may take forever, and slow down the server, if the server is under heavy usage. Your best bet might be to take the machine offline at the end of the day, resize the partition and then let resize2fs run overnight.

Basically, the filesystem is contained within the partition, which is contained within the raw device, which is created from the raid disks.

Caveat: if you have a swap partition rather than a swap file, you'll need to create a swap file, initialize it (mkswap, or swapon, I think), kill swap on the partition, create a new swap partition, and create swap on that partition before expanding the ext3 partition. It might be easier to just keep swap on a file. I don't think it affects performance very much. If it does, you might be better off just using a separate disk for swap in the first place.
posted by gjc at 7:04 AM on May 4, 2012

You can make an ext3 filesystem bigger without unmounting it using resize2fs, but you need to unmount before shrinking one.

Also, I totally fail to understand putting swap partitions after the filesystem partitions. Especially on modern drives, swap partitions are so small compared to the total drive size that putting them first (on the disk's outermost tracks) has virtually no impact on filesystem performance, yet can speed up swap speed quite significantly.
posted by flabdablet at 10:33 AM on May 4, 2012