Ask MetaFilter questions tagged with unicode

Looking for Outlook setting.

iconjack — Tue, 17 Nov 2009 05:33:07 -0800

Is there an Outlook setting to tell it to substitute a glyph from an alternate font if the current display font doesn't have a glyph for a particular character? For example, I send a message with left- and right-ceiling brackets (U+2308 and U+2309) to a colleague who uses Outlook. No font is specified in the message. Outlook uses Arial or whatever the default is to display the message. The stock Arial that comes with Windows does not contain a glyph for U+2308 and U+2309, so he can't see the bracket characters. Most e-mail programs will do the obvious thing and substitute a glyph from another font; however, Outlook does not, at least not out of the box. Is there a setting down in the bowels somewhere that will get Outlook to do this?

Pimp my browser.

Obscure Reference — Thu, 03 Sep 2009 12:59:12 -0800

I'm trying to get extensions to work with the chrome beta browser. My first candidate is AdBlock and I looked here. However, I can't seem to get it to work. I created a manifest.json file but when I saved it as plain text, chrome complained that it had to be in utf-8. So I saved it as "unicode" (in wordpad) but I still have the same error.

Have any of you tried to do a chrome extension and succeeded? Do any of you know how to convert a text file to utf-8?

Fixed-width Arabic fonts.

nebulawindphone — Sat, 21 Mar 2009 14:52:47 -0800

There are a couple fixed-width Unicode Arabic fonts out there. The ones I've seen are pretty hard on the eyes, though, especially at small sizes. Any suggestions for a really readable one? (The goal, FWIW, is to be able to work with mixed Arabic/Latin-alphabet text in a terminal window without going blind or blowing the Latin text up to a ridiculous size.)

Pokemon font?

nímwunnan — Sat, 31 Jan 2009 10:57:14 -0800

Is there a dingbats font that uses Pokemon as characters or can this be done with Unicode? If there isn't can somebody here make it? C'mon, how awesome would that be? (answer: _pretty_ awesome)

Deeply anal-retentive font geek question about IPA fonts.

nebulawindphone — Thu, 18 Dec 2008 14:56:14 -0800

Roman IPA fonts and the non-roman characters who love them. I'm looking for compatible fonts — a single family, or fonts from different families that look nice together — that do all of the following:

Include a wide range of IPA and other Unicode Latin characters, at least in serifed roman and ideally in italic, bold and bold italic too.
Include monospace characters, bold monospace characters, and slanted monospace characters, at least for the plain ASCII character set.
Include small caps, at least for the plain ASCII character set.

Why, you ask? I'm a linguistics student (making the IPA and oddball Latin characters a necessity), I sometimes write software documentation (where I like to use monospaced fonts for things like user input and output) and I sometimes write about topics like metaphor and frame semantics (where small caps are conventionally used for names of metaphors or of frames).

At the moment, I'm doing all three — writing documentation for a bit of linguistics software having to do with frame semantics. We're self-publishing the result, and it would be really nice if it came out looking reasonably-professional.

Here's the fonts I've looked at: SIL's Charis and Doulos don't have monospaced characters or small caps. TITUS Cyberbit doesn't have anything but plain roman serifed characters. Computer Modern Roman doesn't have bold and bold italic monospaced characters, or bold italic serifed characters. Deja Vu doesn't have small caps.

(If all else fails, I will just use Deja Vu for everything and fake the small caps by shrinking down regular capital letters. But first, I want to satisfy my anal-retentive streak by finding out what it would take to do things The Right Way. We use XeTeX, so either TeX fonts or Type 1/TrueType/OpenType fonts will work. Professional fonts are probably not in our budget, but let me know if there's one or a set that'll do this anyway — I'm still curious what the real solution would look like.)

Unicode replacement characters

srboisvert — Mon, 01 Sep 2008 05:52:15 -0800

Is there a name for the domino like box characters with 4 characters inside that display when you don't have support for a language script? There a few examples on this page that show up for me. Bengal, Bhutan and Khmer for example show up as boxes with 4 characters inside. The first two characters are often in common for all the boxes within each language example.
In case it is system dependent - Windows XP, Firefox 3.0.1, English(US).

How serious are intersecting coordinates in a font?

johnofjack — Sat, 30 Aug 2008 13:45:03 -0800

How serious are intersecting coordinates in a font? I'm making a Unicode font in FontCreator, using composite glyphs for the angstroms and c cedilla. When I try to validate the font, FontCreator reports that everything is fine except for intersecting coordinates on Å, ç, and Ç. (I haven't included any Romanian glyphs, though maybe I should.)

On those three characters, the diacritics are meant to connect.

FontCreator, rather unhelpfully, says that it's up to me how many of the errors I choose to correct.

Would any certain programs, esp. common word processors, have problems with the font if I leave in the errors? e.g. I've found mention that Flash chokes on characters with incorrect contour direction, but no mention of what problems, if any, programs might have with intersecting coordinates.

Or should those characters be drawn as simple glyphs rather than composited?

Conditional search engine selection in Firefox?

qvtqht — Thu, 07 Aug 2008 13:35:13 -0800

Conditional search engine selection in Firefox search box based on contents (in particular, character set) of the search query? I would like Firefox to use a different search engine depending on the type of string I enter into the search box. I think that regex matching, if possible, would do the trick.

I like using the search box in Firefox.

I like Google for most searches, but I want to use Yandex by default for any search that contains Cyrillic (Russian) characters.

For example, if I'm searching for "keys wallet phone", Firefox should use Google, but if I'm searching for "ключи бумажник телефон", it should switch to Yandex. (Yandex should be used even if Latin characters are present, as long as at least one Cyrillic character is there.)

How do I make Firefox switch search engines depending on the charset of the text I enter?

Why does Firefox hate hearts?

elisabethjw — Tue, 13 May 2008 00:12:13 -0800

Firefox refuses to show the heart symbol (♥) correctly. It shows up as a straight, vertical line instead. Is there anything I can do to fix it? Whenever I'm browsing Facebook (or other websites that use the ♥ symbol, but it most frequently appears on Facebook), Firefox refuses to display the symbol correctly. It works fine in Safari. I have a Macbook with OSX 10.5, and Firefox 2.0.0.14 (should be the most recent version -- I just updated today). Is there anything I can do to make this character display properly? Its a minor annoyance, and I have at times debated whether I really want to see ♥s wherever one chooses to put them, but I figure its better than seeing a screen full of lines.

My friend also has a MacBook, using the same version of Firefox, and also has this problem. What gives?

How do you deal with Chinese characters that can't be represented in 16 bits?

bonecrusher — Fri, 04 Apr 2008 08:48:52 -0800

How are people dealing with >16 bit Unicode code points? Specifically, in languages like Java, C# and C++, which assume 16 bit characters (I believe), how are you supporting GB 18030? I would suspect that the various languages' methods like substring(), charAt(), operator[], etc can't be safely used in China. If your wstring, say, contains a Chinese string, then .size() doesn't tell you how many characters are in it, right? On a related note, what interesting Chinese characters require more than >16 bits? I'm thinking about making a short presentation for my co-workers on this subject and I'd like to have some interesting examples. (Oh, and I'm going to run any examples by my Chinese colleagues first, so don't bother trying to make me say "penis" or something in front of my co-workers :-))

Captcha? Каптча? カプチャ?

soma lkzx — Fri, 08 Feb 2008 20:40:12 -0800

Is using a Latin-alphabet captcha accessible across languages/character sets? I'm designing a site which will hopefully deal with people worldwide, across a ton of languages and alphabets. Is it realistic to use an out-of-the-box captcha solution, even though they're all based on the Latin alphabet? Are the characters universally recognizable? Easy to enter on everyone's keyboard?

I don't think the site is going to be spam central or anything, and I'll gladly strip it down to numbers-only to give bots a little road bump, I'd just rather have a little protection in place rather than nothing. Alternatives I've come across (math equations, 'common sense' questions) mostly seem silly or more trouble than they're worth, but I'm open to suggestions.

We are not the masters of subtitling yet

saysthis — Sun, 16 Dec 2007 18:39:20 -0800

I (well, my boss) am/is inputting Chinese-language sub-station Alpha subtitles into VirtualDub for DVD's, and what comes out are, surprise surprise, piles of unicode. The citizens of the PRC are not known to be fluent in unicode, so what could be going wrong and how do I, his designated software monkey, fix it? Chinese language packs are installed, for the record.

How to make MD5, Java and Unicode play nicely

idb — Mon, 10 Dec 2007 19:44:55 -0800

How do I make MD5, Java and Unicode play nicely together? I don't really get unicode and character sets. Now I need to. I'm trying to reproduce the same effect in java that this javascript gets when the chrsz variable is set to 16. When the variable is set to 8, the java version gives the same result as the javascript. I need the java version to give the same result as the javascript version does when chrsz equals 16.

I'm not great at java. I know enough to be dangerous. I thought I was pretty hot stuff at javascript, but it might help if I knew what << or >>> meant in javascript. As you can imagine, that's really hard to google.

Is there any way to surf Japanese web pages on a US cell phone?

Vorteks — Thu, 29 Nov 2007 07:40:58 -0800

I have a US cell phone (LG Muziq, Sprint is the provider). I would REALLY like to be able to read Japanese web pages on it, but whenever I surf to a page with Japanese text it just shows up as garbage. I've installed Opera Mini on my phone, but that didn't help, even though the demo on their website does show Japanese characters. Is there any way to get this to work? I don't need to be able to type in Japanese, or send Japanese text messages. I just want to be able to read Japanese web pages. If I can't do this on the phone directly, then maybe there's another website I could visit that would convert the Japanese text to images?

Ask METARfilter: text weather to twitter?

edwardvielmetti — Tue, 30 Oct 2007 05:59:23 -0800

I'm looking for sources of weather data (current conditions, forecasts) that are in text formats so that I can turn them into a Twitter weather bot. What I have in mind is either existing ASCII products (METAR, AVN MOS) or derivatives of image products that (e.g. a compact text encoding of a radar map through some kind of image analysis). Ideas welcomed. The current weather bot is http://twitter.com/a2weather - it's only partially a bot since I have some stuff that I log to it by hand or from scripts.

Some things that have been useful are Twitterfeed (RSS to Twitter, scheduled every so often) and a simple shell script for posting to Twitter via curl.

The radar piece is what I think would be the coolest - I suspect that throwing Imagemagick at a radar GIF and then counting pixels on a county by county basis would do most of the trick.

Desperately seeking insular G!

quercus — Wed, 22 Aug 2007 18:38:29 -0800

How do I upgrade unicode? This is an insular g

It's Unicode is U+1D79. I have Mac OS X and it's not listed in the Unicode characters on the Character Palette. How do I upgrade my unicode database to get an insular g? Thanks.

Unicode, Shunicode

jak68 — Wed, 23 May 2007 16:13:35 -0800

Fonts Filter: In MS Word 2002, I'm trying to type a "combining diaresis" (umlaut applied to the preceding character). Sometimes it actually works; sometimes i get a square box. I have no idea why it works when it does, or why it doesnt work when it doesnt. It does work consistently in Word 2007 on a different computer, so the easy answer is to upgrade Word. However, 2002 is supposed to be unicode compatible, and it DOES work on 2002 -- sometimes. I just dont know why it works when it does, or doesnt when it doesnt.

When it doesnt work, I get a square box, and the font changes to "Tahoma" (from the previous arial or arial unicode). When it does work, I get the combining diaresis correctly applied to the previous character, and the font remains what it was (I've seen it work with times new roman, tahoma, arial, etc).

To type in the diaresis (aka umlaut), i'm using a temporary custom key assignment that I created using MKLC (microsoft keyboard layout creator). Using MKLC, you can assign any unicode character (in this case, the unicode for the combining diaresis) to any key combination.

Any advice? I'd rather not shell out 300 bucks for Office 2007, especially when i dont really know why it sometimes DOES work with 2002.

I havent done anything differently as far as I can tell, to the computer, between the times when it does work and the times when it doesnt.

help!

Unicode Conversion Formatting Issue

AArtaud — Thu, 22 Feb 2007 20:19:13 -0800

Unicode Filter: Converting an Office 98 document to Unicode has caused formatting problems. I can do a find and replace ok for the characters, but when I switch the font to Unicode the spacing around the Unicode characters goes haywire with gaps. Microsoft suggests using a Word Add-In called the "broken fonts add-in" that supposedly comes with Office 2003, but I am unable to find that on my Office 2003 disk. Any suggestions would be greatly appreciated.

how to make a "registered" symbol in unicode?

drjimmy11 — Mon, 30 Oct 2006 19:41:02 -0800

how to make a "registered" (r-in-circle) symbol to go in a text file to appear in Flash we're trying to make a "registered" symbol appear in flash after being imported as part of a variable from a .txt file. We cant use "& reg;" b/c flash gets screwed up by the "&". The flash is made by a 3rd party and they made a TM symbol by using "%E2%84%A2."

I believe this is UTF-8. we've googled far and wide for an equivalent code for "registered" but nothing we can find works. We keep coming up with "A2CE" or something with only four characters that displays nothing. (we tried %A2%CE and various permutations but no luck)

any help would be greatly appreciated.

Will an ipod display international characters?

the cydonian — Thu, 19 Oct 2006 01:35:49 -0800

How does the new gen ipod handle international characters in ID3 tags of an mp3 file? So I'm thiiis close to getting myself a 30 gig video ipod, and was wondering it'll scale to internationalized characters. My mp3 collection (all legit) is mostly world music, and I have, on prior occassions, painstakingly ensured that all non-English songs have their ID3 tags in their native languages. Which means, my iTunes collection has song titles and artists information in, among other scripts, Chinese, French, Thai, Urdu, Tamil, Telugu, Hindi and Punjabi.

I asked the sales guy at the local Apple showroom about this, but I don't think he understood the question; he was polite about it, but he appeared incredulous as to why anyone would presume non-English songs wont play on an ipod.

I googled, of course, but I don't seem to be hitting the right keywords, which is why I'm asking here.

(If it's any help, I know that itunes for Windows supports unicode; heck, that's where I editted most of my entries.)

PostScript + Unicode

iconjack — Thu, 31 Aug 2006 21:04:53 -0800

Can I use Unicode characters in PostScript strings? Does PostScript understand Unicode? (in particular, the PostScript as implemented by GhostScript 4.6 or beyond)

If so, how can I insert a Unicode character, for example U+2204, into a string literal?

How can I preserve Unicode characters in `mysqdump`?

cmyers — Mon, 28 Aug 2006 11:55:32 -0800

How can I make backups of a MySQL database (via mysqldump) and preserve Unicode characters (such as em dashes)? I use mysqldump and rsync to make backups of my TextDrive account. One of the forums I run, however, often includes unencoded Unicode entities, such as em dashes and bullets.

Because these characters aren't straight ASCII, restoring from previous mysqldump backups has mangled them, producing gibberish but preserving the other, straight ASCII text in the database.

My current mysqldump call looks like this:

/usr/local/bin/mysqldump -uuser -ppassword --quote-names --complete-insert --extended-insert --quick --lock-tables=false --skip-add-locks --all-databases | gzip > db.sql

. What's necessary to preserve these Unicode characters?

Thanks.

Mysterious characters in addresses

chef_boyardee — Tue, 30 May 2006 12:20:11 -0800

Help me figure out these weird Spanish, Danish, and Portugese characters in an address given to me via the Internet. I don't think what I see is what is meant. The sample from Denmark is
ьsterbrogade
where the first character looks like this. That's a Russian letter, so that can't be right. What is the correct character?

The Spanish address is
Gracia 15, 5º 2ª
Where there is Gracia, fifteen, comma, five, then what looks like a degree symbol, then a two, then what looks like a little "a" in superscript. It's rare that I see superscripts in an address so I am worried that these must be written as something else. I actually see a lot of superscripted addresses like this come from Spain.

Similarly, the Portugese address is
Outubro nº 198 6º
where the "n" and "6" are followed by what looks like a degree symbol.

How should these be written?

Modifying php-based web calendar to allow UTF-8 (Japanese) input

planetkyoto — Wed, 29 Mar 2006 22:19:23 -0800

Probably easy question about PHP and unicode (UTF-8) and RegEx. I'm trying to modify a php webcalendar (VTcalendar) to allow Japanese text in calendar postings. I've found all the variables to get the UTF-8 headers, and so japanese text manually inserted into pages appears fine. But, there's an input validation thingy I don't know how to modify. (short snippet inside) The calendar item input form rejects any Japanese text, and I think I've traced it to the file "inputvalidation.inc.php" which starts with the code below. if I try to delete the part about allowable characters in line 7, I get an error about the '^' in the last line. can this be modified to allow UTF-8 characters?

if (!defined("ALLOWINCLUDES")) { exit; } // prohibits direct calling of include files
define("constValidTextCharWithoutSpacesRegEx",'\w~!@#\$%^&*\-+=\{\}\[\]\|\\\:";\'<>?,.\/');
define("constValidTextCharWithSpacesRegEx",'\s'.constValidTextCharWithoutSpacesRegEx);
define("constCalendaridMAXLENGTH",20);
define("constCalendaridVALIDMESSAGE", '1 to '.constCalendaridMAXLENGTH.' characters (A-Z,a-z,0-9,-,.)');
define("constCalendarnameMAXLENGTH",100);
define("constCalendarnameVALIDMESSAGE", '1 to '.constCalendarnameMAXLENGTH.' characters (A-Z,a-z,0-9,-,.,&,\',[space],[comma])');
define("constCalendarTitleMAXLENGTH",50);
define("constKeywordMaxLength",100);
define("constSpecificsponsorMaxLength",100);
define("constPasswordMaxLength",20);
define("constPasswordRegEx", '/^['.constValidTextCharWithoutSpacesRegEx.']{1,'.constPasswordMaxLength.'}$/');

Special characters in Quark

omair — Sat, 25 Feb 2006 15:24:34 -0800

Font recommendation to solve a MS Word --> QuarkXPress issue? I have a lengthy book written in MS Word, pretty simple in all respects. Basic formatting, layout, etc and the font throughout is Times New Roman.

The Unicode version of Times New Roman has lots of characters that I use to write the English transliteration of words from another language. These are characters such as the letter 'T' with a dash through it, the letters 'H', 'Z', or 'S' with a "hat" accent above them, and such...

However, since these characters are not in the first 256 characters of Times New Roman, QuarkXPress does not pick them up, since it does not have Unicode support.

Is there a font out there that contains characters similar to these (doesn't have to be exactly the same) as well the normal characters in the first 256 spots so that I can use it with Quark, or does anyone have a better idea as to how I can solve this problem?

Yes, for my next book I'll find another application, like InDesign (although I haven't researched its capabilities yet) since Quark is just so crappy...