Ask MetaFilter questions tagged with regularexpression

Shell script handling CSV columns

fightoplankton — Wed, 10 Jun 2009 16:30:20 -0800

I'm using a shell script to handle some excel docs by converting to csv and reformatting it a little bit. I'm having some difficulty going on with the next part of the script. Some help would be great. :) CURRENT SCRIPT:
#!/bin/sh
ls *.xls
echo "Enter filename for excel input file: "
read filen
xls2csv -x "$filen"".xls" -b WINDOWS-1252 -c "$filen"".csv" -a UTF-8
sed '1,3d' "$filen".csv >"$filen"_temp.csv
cut -d "," -f 1,4- "$filen"_temp.csv > "$filen"_temp2.csv
awk -F, '{sub($NF, "");print}' "$filen"_temp2.csv > "$filen"_temp3.csv
sed 's/.$//' "$filen"_temp3.csv > "$filen"_fixed.csv
rm "$filen".csv
rm "$filen"_temp.csv
rm "$filen"_temp2.csv
rm "$filen"_temp3.csv
echo "$filen"".xls corrected and saved as ""$filen""_fixed.csv"

EXAMPLE OUTPUT (first three lines only):
"Device ID","1) S31 Which best describes how you answered the online reading comprehension quiz?","2) S32 Which best describes how you answered the online timed retrieval quiz?","3) B19. If you want your product to be easy to find in the supermarket then you should make its container","4) C19. So that he can shift attention between the radio and his incessantly talking girl friend when she is in the car, Joe adjusts his radio","5) B20. Early selection is most likely to occur for","6) C20. Early selection for a red target is most likely to occur when there is","7) B21. In a lexical decision task, when the target is a bird name, e.g. robin, it is usually preceded by the prime BODY but is sometimes preceded by the prime BIRD.","8) C21. In a lexical decision task, when the target is a dog name, e.g. collie, it is usually preceded by the prime CAR but is sometimes preceded by the prime DOG.","9) B23. Suppose that that you see a brief display with 12 colored letters: 4 red, 4 white, and 4 blue. At the offset of the display you hear tone. A tone instructs you to report only the letters of a particular color: high for red, medium for white, and low for blue. About how many letters do you report?","10) B22. Sperling (1960) found that partial report produced the highest estimate of the number of available letters when the tone occurred ","11) C23. According to the logic of Sperling’s (1960) partial report method, an observer who reports three letters from a row in a 4 x 4 display that was cued at the display’s offset must have seen at least","12) C22. Sperling (1960) found that the greatest difference between full and partial report in the number available of letters was when the tone occurred ____ milliseconds after the offset of the visual display"
96A39,6,4,"4 c","4 c","5 c","5 c","5 c","4 i","3 c","1 c","1 i","5 i"
1E90A4,5,3,"4 c","4 c","5 c","5 c","2 i","5 c","3 c","1 c","4 i","4 i"

First, I need to remove the numbering of the questions from the first line. So "1) S31 Which best describes how you answered the online reading comprehension quiz?" would become "S31 Which best describes how you answered the online reading comprehension quiz?"

Next I need to remove an entire column if its row header doesn't begin with either B or C or is the "Device ID" column. For example, column starting with B19 is kept, the C23 is kept, and"Device ID" column is kept. The column with the question starting S31 must be entirely removed.

Thanks.

Regular Expression Ninja

JPowers — Wed, 14 May 2008 16:26:19 -0800

I want to become a regular expression ninja (a black belt). Where should I start? I'd prefer something interactive (vs. a book or blog post), like a website where I can test them out. I found this site, and it's OK, but are there any others to choose from?

Regex Alms for the Perl-less?

ChasFile — Fri, 28 Dec 2007 02:12:28 -0800

Help me composite some regex. (That phrasing makes this question sound way less nerdy than it is) Here's a regex to find some stuff between square brackets:
/\[[^\]]+\]/

Here's one that finds something like Alt: or alt. or Alternative: or alternate or Alternate: or Alternative or Alt.: or you get the idea [note it ends with \s+; it is important for my application that the "Alt___ " I'm testing for has white space at the end, and that I test for it. In the final answer we can test for that word boundary any way we like, we just need to make sure that we do.]:
/(A|a)lt(\.|ernat(e|ive))?:?\s+/

So what I need is a regular expression for "stuff between square brackets where the first thing inside the brackets will NOT match the second regex." Or "Stuff inside square brackets that begins with anything BUT alt or Alt. or Alternate or alternative: or alt.: or etc. etc.

I feel like this should be easy, but I never bothered to totally and completely grok regex, and obviously I'm hurting now because of it. I'd very much appreciate any help anyone could give, and in exchange you'll get co-author credit for the amazing piece of software that this thing will ultimately be a part of! ;-)

I'd like to redirect an entire directory to one specific file.

Tuwa — Tue, 13 Mar 2007 20:10:18 -0800

301 redirectrs in .htaccess: how do I redirect calendar/* to /calendar.html ? I've tried modifying various examples online and they result in either "internal server error" or a 404 caused by incorrectly redirecting /calendar/* to /calendar.html/* . Two examples from the experimentation (I'm in over my head; I just want this to work):

# redirect 301 /calendar http://www.domain.com/calendar.html
# 404 due to mistake above.

# RedirectMatch 301 ^/calendar/(.*).htm$ http://www.domain.com/calendar.html [L]
# internal server error