Is there any easy command or application that will randomise lines in a text file?
November 12, 2004 6:55 AM   Subscribe

Is there any easy command or application that will randomise lines in a text file?

Answer: yes, and as usual it turns up immediately after asking the question. For those interested:

#!/usr/bin/perl
# Program to output lines of file in random order

# Read File
while (<>) {
push(foo, $_);
}

# Shuffle lines
srand;
for (0..$#foo) {
$r = rand($#foo+1);
($foo[$r], $foo[$_]) = ($foo[$_], $foo[$r]);
}

# Print it out
for (0..$#foo) {
print $foo[$_];
}


perl [inputfile] > [outputfile]

My thanks to Steve who answered me seven and a half years ago.
posted by humuhumu to Technology (11 answers total)
 
But if you just want one random line from a file, you don't need to copy the whole file into memory:

perl -ne '$l = $_ if rand() < 1/$.; END { print $l }' your_file
posted by nicwolff at 7:19 AM on November 12, 2004


Response by poster: Nice shorthand, nicwolff. I had a file of 597,208 lines that needed randomising - only took a few seconds with the above method.
posted by humuhumu at 7:50 AM on November 12, 2004


Yup, very odd that this is missing from the standard unix toolkit. A shorter version of what humuhumu posted from my shell aliases:

perl -e'@l=<>;@l[$i,$s]=@l[$s=rand($i+1),$i]while++$i<@l;print@l'


Remember you can just assign the entire input array @foo=<>; no need to iterate over it.

Also, if you use zsh (very nice, I can recommend it (and have, at length here on Ask)) and don't want the overhead of starting perl, here it is as a zsh function, don't think it'll work under bash
randlinz () {
        RANDOM=$(date +'%s') 
        _buf=() 
        i=1 
        OLDIFS=IFS
        IFS=""
        while read -r a
        do
                _buf[$[$#_buf+1]]=$a 
        done
        while [ $i -lt $#_buf ]
        do
                i=$[$i+1] 
                r=$[($RANDOM%$#_buf)+1] 
                t=$_buf[r] 
                _buf[r]=$_buf[i] 
                _buf[i]=$t 
        done
        for a in $_buf
        do
                while [ $i -gt 0 ]
                do
                        echo $_buf[$i]
                        i=$[i-1] 
                done
        done
        IFS=OLDIFS
}
As soon as you have a significant number of lines using the perl version will be a lot faster though.
posted by fvw at 9:15 AM on November 12, 2004


Ehm, OLDIFS=$IFS and IFS=$OLDIFS ofcourse. I only added them while posting after realising ommitting that would cause problems in obscure cases (as said, I don't really use the zsh version).
posted by fvw at 9:16 AM on November 12, 2004


A Python version which I think is more readable ;-):

python -c 'import fileinput, random; x = list(fileinput.input()); random.shuffle(x); print "".join(x)'
posted by grouse at 11:58 AM on November 12, 2004


<?php

$lines = file(foo.txt);
shuffle($lines);
foreach($lines as $line) {
    echo $line;
}

?>
posted by sad_otter at 11:59 AM on November 12, 2004


gah. didn't quote the filename. oh well.
posted by sad_otter at 12:02 PM on November 12, 2004


grouse: That's cheating. In that case, perl -MAcme::Randomise -e 'print randomise <>;'
posted by fvw at 7:40 PM on November 12, 2004


fvw: your answer presupposes that Acme::Randomise is preinstalled, where I only used stuff from the Python standard library. The OP was not looking for a programming tutorial but something that works, so a one-liner that doesn't require installing additional stuff is probably best.
posted by grouse at 9:29 AM on November 13, 2004


I know, merely trying to start a language war. Ignore me :-)
posted by fvw at 12:06 PM on November 13, 2004


A bit late, but it turns out perl does actually come with an Acme::Randomise installed by default, although named a bit more sensibly:

perl -MList::Util -e '@a=<>; print List::Util::shuffle(@a);'

posted by fvw at 5:36 AM on November 19, 2004


« Older Ladies: how would you interpret it if a man didn't...   |   WiFi in Mobile Home Newer »
This thread is closed to new comments.