Is there any easy command or application that will randomise lines in a text file?
November 12, 2004 6:55 AM   Subscribe

Is there any easy command or application that will randomise lines in a text file?

Answer: yes, and as usual it turns up immediately after asking the question. For those interested:

# Program to output lines of file in random order

# Read File
while (<>) {
push(foo, $_);

# Shuffle lines
for (0..$#foo) {
$r = rand($#foo+1);
($foo[$r], $foo[$_]) = ($foo[$_], $foo[$r]);

# Print it out
for (0..$#foo) {
print $foo[$_];

perl [inputfile] > [outputfile]

My thanks to Steve who answered me seven and a half years ago.
posted by humuhumu to Technology (11 answers total)
But if you just want one random line from a file, you don't need to copy the whole file into memory:

perl -ne '$l = $_ if rand() < 1/$.; END { print $l }' your_file
posted by nicwolff at 7:19 AM on November 12, 2004

Response by poster: Nice shorthand, nicwolff. I had a file of 597,208 lines that needed randomising - only took a few seconds with the above method.
posted by humuhumu at 7:50 AM on November 12, 2004

Yup, very odd that this is missing from the standard unix toolkit. A shorter version of what humuhumu posted from my shell aliases:

perl -e'@l=<>;@l[$i,$s]=@l[$s=rand($i+1),$i]while++$i<@l;print@l'

Remember you can just assign the entire input array @foo=<>; no need to iterate over it.

Also, if you use zsh (very nice, I can recommend it (and have, at length here on Ask)) and don't want the overhead of starting perl, here it is as a zsh function, don't think it'll work under bash
randlinz () {
        RANDOM=$(date +'%s') 
        while read -r a
        while [ $i -lt $#_buf ]
        for a in $_buf
                while [ $i -gt 0 ]
                        echo $_buf[$i]
As soon as you have a significant number of lines using the perl version will be a lot faster though.
posted by fvw at 9:15 AM on November 12, 2004

Ehm, OLDIFS=$IFS and IFS=$OLDIFS ofcourse. I only added them while posting after realising ommitting that would cause problems in obscure cases (as said, I don't really use the zsh version).
posted by fvw at 9:16 AM on November 12, 2004

A Python version which I think is more readable ;-):

python -c 'import fileinput, random; x = list(fileinput.input()); random.shuffle(x); print "".join(x)'
posted by grouse at 11:58 AM on November 12, 2004


$lines = file(foo.txt);
foreach($lines as $line) {
    echo $line;

posted by sad_otter at 11:59 AM on November 12, 2004

gah. didn't quote the filename. oh well.
posted by sad_otter at 12:02 PM on November 12, 2004

grouse: That's cheating. In that case, perl -MAcme::Randomise -e 'print randomise <>;'
posted by fvw at 7:40 PM on November 12, 2004

fvw: your answer presupposes that Acme::Randomise is preinstalled, where I only used stuff from the Python standard library. The OP was not looking for a programming tutorial but something that works, so a one-liner that doesn't require installing additional stuff is probably best.
posted by grouse at 9:29 AM on November 13, 2004

I know, merely trying to start a language war. Ignore me :-)
posted by fvw at 12:06 PM on November 13, 2004

A bit late, but it turns out perl does actually come with an Acme::Randomise installed by default, although named a bit more sensibly:

perl -MList::Util -e '@a=<>; print List::Util::shuffle(@a);'

posted by fvw at 5:36 AM on November 19, 2004

« Older Ladies: how would you interpret it if a man didn't...   |   WiFi in Mobile Home Newer »
This thread is closed to new comments.