Ubuntu text format

July 23, 2008 3:35 PM

How can I convert a text file with characters representing hexadecimal to binary in Ubuntu?

I know there are websites with conversion calculators, but I want to be able to take a text file and output another text file with binary representations of the hex from the input file. Thanks for any help.

posted by amsterdam63 to Computers & Internet (20 answers total)

Here's a Python program, hex2bin.py:

#!/usr/bin/python
import sys
import binascii

data = open(sys.argv[1], "r").read()
data = data.replace(' ','')
data = data.replace('\t','')
data = data.replace('\n','')
open(sys.argv[2], "wb").write(binascii.unhexlify(data))
posted by and for no one at 3:46 PM on July 23, 2008

Response by poster: Unfortunately, I'm not very computer saavy, so I'm not sure how to implement that.
posted by amsterdam63 at 3:47 PM on July 23, 2008

Note for people coming across this in the future: It'll work on any system that has python (not just Ubuntu, and not even just Linux.) You use it by saying something like:

python hex2bin.py ihavemuchhex.txt iwantbinaryhere.txt
posted by Tomorrowful at 3:48 PM on July 23, 2008

It should be possible to do that using twelve piped SED calls:

cat sourcefile | sed s/0/0000/ | sed s/1/0001/ | sed s/2/0010/ | ... | sed s/F/1111/ >destfile
posted by Class Goat at 3:49 PM on July 23, 2008 [2 favorites]

Ah, ok. Here's your step-by-step:

Copy that text.

Fire up a text editor (Applications--->Accessories--->Text Editor if my memory serves).

Paste in the text.

Save it as hex2bin.py.

Open a Terminal and navigate to where you saved it.

Follow the instructions I gave above on how to run it.

Come back when you realize I've mistyped something because I don't have a machine on hand to test it with.
posted by Tomorrowful at 3:50 PM on July 23, 2008

(It's been 25 years since I did much mucking with Unix; maybe there's something else that has to be in there. Something like sed -c "s/0/0000/" or something like that. My memory is growing dim.)
posted by Class Goat at 3:51 PM on July 23, 2008

Tomorrowful, thanks for helping out.

Class Goat, that sed command line is sick and twisted :-)

And just for the sake of completeness, here's the opposite program, bin2hex.py:
(it's shorter because it doesn't have to clean out the white space)

import sys
import binascii

open(sys.argv[2], "w").write(binascii.hexlify(open(sys.argv[1], "rb").read()))
posted by and for no one at 4:04 PM on July 23, 2008

Response by poster: Here is what the input text file consists of:

3000
54a0
2611
f023
62c0
187c
0408
927f
1240
927f
0a01
14a1
16e1
62c0
0ff6
2003
1002
f021
f025
0030
4000


This is what is in the output file:

0ꁔᄦ⏰쁢簘ࠄ羒䀒羒Ċꄔ쁢̠Ȑ⇰◰　@

Something is wrong and I have no idea what it is.
posted by amsterdam63 at 4:14 PM on July 23, 2008

Do you want binary as in a text file with 0s and 1s instead of hex numbers? The Python script will convert your text file into an actual binary file, not another text file.
posted by zixyer at 4:23 PM on July 23, 2008

If you want someone to write this program for you, you'll also need to specify how big your numbers are (looks like you've got 16-bit numbers in your sample data), whether they're signed or unsigned, and whether they're big or little endian.
posted by zixyer at 4:25 PM on July 23, 2008

Response by poster: Yea, I'm sorry I didn't make that more clear. I want it to be a new text file that displays the 1's and 0's that correspond to the hex numbers in the input file. So the first line in the input file is 3000, which in binary would be 0011000000000000.
posted by amsterdam63 at 4:27 PM on July 23, 2008

Sorry about the confusion. That program is a little longer... Let me think about it for a moment.
posted by and for no one at 4:29 PM on July 23, 2008

This looks like it works. Python whitespace is significant, be careful w/copy & paste.

#!/usr/bin/python

import sys
import binascii

output = open(sys.argv[2], "w")
for line in open(sys.argv[1]).readlines():
    line = line.strip()
    data = binascii.unhexlify(line);
    r = []
    for c in data:
        c = ord(c)
        i = 0x80
        while i:
            if c & i:
                r.append('1')
            else:
                r.append('0')
            i = i >> 1
    output.write(''.join(r)+'\n')

posted by and for no one at 4:46 PM on July 23, 2008

Well, that's ugly. Looks like meta stuck extra (br) inside the (pre) tag.

It looked good on preview. At least it still works.
posted by and for no one at 4:49 PM on July 23, 2008

Response by poster: Yea baby, that worked. You're the man. Thank you so much. I really, really appreciate the help.
posted by amsterdam63 at 4:51 PM on July 23, 2008

Class Goat, that sed command line is sick and twisted :-)

Remember the first rule of engineering: a stupid idea that works isn't stupid.
posted by Class Goat at 7:00 PM on July 23, 2008

Oops: ...using twelve piped SED calls...
That would be "sixteen piped SED calls". Anyway, you knew what I meant.
posted by Class Goat at 8:54 PM on July 23, 2008

This sort of thing is what perl was created for so I'll include this for completeness and for easier copying and pasting ;) Just put this in a file like hex2bin.pl:

#!/usr/bin/perl
use strict;
while(<>)
{
  $_ = unpack("B32", pack("N", hex $_));
  s/^0+//;
  print "$_\n";
}

and either make it executable:
chmod a+x hex2bin.pl
or use perl itself:
perl hex2bin.pl
It either takes files on the command line or from stdin (pipes or file redirects) and outputs the results to stdout. The second line of the loop is to get rid of the leading zeros, you can delete that line if you want the them. Example usage:

hex2bin.pl hexfile.txt > binfile.txt

posted by cftarnas at 11:35 PM on July 23, 2008

Doh, forgot perl can do the binary in the printf, makes it even easier:

#!/usr/bin/perl
use strict;
while(<>)
{
  printf("%b\n", hex $_);
}

It's actually easier just to do it as a perl command line:

perl -pe '$_ = sprintf("%b\n", hex)' < hexfile.txt > binfile.txt

posted by cftarnas at 11:50 PM on July 23, 2008

Surprised there isn't a binary that does this natively...
posted by prodevel at 12:00 AM on July 25, 2008