Comments on: How can I preserve Unicode characters in `mysqdump`?

Question: How can I preserve Unicode characters in `mysqdump`?

cmyers — Mon, 28 Aug 2006 11:55:32 -0800

How can I make backups of a MySQL database (via mysqldump) and preserve Unicode characters (such as em dashes)?

I use mysqldump and rsync to make backups of my TextDrive account. One of the forums I run, however, often includes unencoded Unicode entities, such as em dashes and bullets.

Because these characters aren't straight ASCII, restoring from previous mysqldump backups has mangled them, producing gibberish but preserving the other, straight ASCII text in the database.

My current mysqldump call looks like this:

/usr/local/bin/mysqldump -uuser -ppassword --quote-names --complete-insert --extended-insert --quick --lock-tables=false --skip-add-locks --all-databases | gzip > db.sql

. What's necessary to preserve these Unicode characters?

Thanks.

By: pharm

pharm — Mon, 28 Aug 2006 12:03:34 -0800

<unhelpful remark>Use a real database?</unhelpful remark>

A little searching (actually, just stuffing "mysqldump unicode" into google) suggests that the argument --default-character-set=utf8 is what you want.

By: SpecialK

SpecialK — Mon, 28 Aug 2006 12:12:01 -0800

What pharm said, but you might also look into using mysqlhotcopy instead. If you're working with large datasets, re-importing the mysqldump output can take a long time (2 hours minutes for one of my customer's databases!) whereas mysqldump is almost instant.

By: SpecialK

SpecialK — Mon, 28 Aug 2006 12:12:24 -0800

err whereas mysqlhotcopy is almost instant.

(I've been writing code for far too long already today. :( )

By: devilsbrigade

devilsbrigade — Mon, 28 Aug 2006 12:26:43 -0800

Use a binary dump.

By: devilsbrigade

devilsbrigade — Mon, 28 Aug 2006 12:27:14 -0800

And yes, use a real database.

By: cmyers

cmyers — Mon, 28 Aug 2006 12:50:10 -0800

Sorry if the answer to my question was self-obviating. I did search, both here and via Google, but I didn't think to go for as simple as a string as "mysqldump google" would be adequate.

Re mysqlhotcopy: The largest of the databases is only about 2.3 MB, and the statement dumps give me greater flexibility among hosts and setups.

Re MySQL: I know that it's a blunt instrument. My only other option is PostgreSQL, which I don't know but will look into.

Thanks, and sorry for not looking much past my own nose.

By: idontlikewords

idontlikewords — Mon, 28 Aug 2006 13:51:09 -0800

oh dammit... mysqldump doesn't preserve unicode?! is idontlikewords gonna have to choke a bitch??

By: idontlikewords

idontlikewords — Mon, 28 Aug 2006 14:30:15 -0800

Hm... After a bit of monkeying around to remove some invalid comments that mysqldump inserted into my backups (which it is good that this thread prompted me to do), I was just able to restore a mysqldump backup with unicode chars, japanese chars and etc. All my tables are MyISAM and the charset is utf8.

Here's the command I used to create the backup:

mysqldump -h hostname.com -u username -ppassword --skip-comments --skip-lock-tables database-name > backup.sql

And here's what I used to restore:

mysql --user=username --password=password database < backup.sql