Ask MetaFilter questions tagged with perl

Using grep to extract x characters of text after a predictable pattern

infinitejones — Fri, 15 Mar 2013 03:39:17 -0800

I'm trying to work out how to use something like grep or sed or awk (or maybe even some Perl) to extract a string of characters which appears in a predictable place in a series of text files. Help/advice/tutorials very welcome! I have a number of text files in a Google Drive folder, which gets replicated on my Mac, and a new one gets added every day. The files are plain text and they aren't structured formally (as in, they're not XML or anything) but they each contain a certain string ("Total Portfolio Value") which is always followed by a space, followed by "$xx,xxx.xx", followed by five more spaces. The value of $xx,xxx.xx changes every day, and that's what I'm trying to extract, to put into a separate file.

I can use Automator to check whenever a new file appears and run a shell script on the file, so I'm trying to work out what goes in the shell script.

As much as anything else I'm using this as a practical exercise to teach myself a bit about text processing using grep/sed/awk, Perl and regular expressions (any/all of the above!) so just a few pointers about the best approach the contents of the shell script would be great!

How to solve a complex statistics problem with a script?

j0hnpaul — Fri, 30 Nov 2012 18:26:42 -0800

In this game, you roll a number of six-sided dice to get a total. The total is either the highest single die result, or the sum of any multiples rolled, whichever is higher. For example: If I roll three dice and get a 3, 4, and 6, my total is 6. But if I roll a 4, 4, and 6, my total is 8, the sum of the two 4s. What I want to find out is the mean, median, mode, and standard deviation of the possible totals given N dice. How might I create a simple script to compute this? With two or three dice, I can easily figure this out by listing all the possible results, basically by brute force. But that's a pretty labor-intensive method when it comes to four or more dice.

I have a Macintosh, and I'm comfortable using the Unix command line and several programming languages for simple problems, but I'm not even sure where to start automating something like this. I'd be grateful for any guidance.

How to get perl to connect to mysql using DBD:mysql

sunshine37 — Fri, 23 Nov 2012 12:00:18 -0800

I'm over my head again trying to run a program for a class and you were a great help last time. I'm trying to connect to mysql from Perl and have installed DBD:mysql on my Mac (OS X 10.5.8). I have no idea what I'm doing. Can you help? I get the following error whenever I try to run my perl test script:

dyld: lazy symbol binding failed: Symbol not found: _mysql_init
Referenced from: /Library/Perl/5.8.8/darwin-thread-multi-2level/auto/DBD/mysql/mysql.bundle
Expected in: dynamic lookup

dyld: Symbol not found: _mysql_init
Referenced from: /Library/Perl/5.8.8/darwin-thread-multi-2level/auto/DBD/mysql/mysql.bundle
Expected in: dynamic lookup

Trace/BPT trap

All instructions posted for the class are for Windows and I am lost. I googled this error but all the answers I found are over-my-head computer speak. I rarely do more than the most basic commands in the terminal, so I have no idea where to even begin to know what might be wrong. Can you help me? Let me know what other information I can provide.

What IT conferences should I budget this year?

The Supreme Dominar — Tue, 10 Apr 2012 16:50:01 -0800

What IT/programming/technology type conferences should I try to go to this year (June 2012 - June 2013)? I have to come up with a list of 3 by Thursday for my manager! I'm primarily a software developer who also does some OS X and SUSE Linux Enterprise Server management, and a little bit of Oracle administration (for a vendor app we use; most of my stuff stores its data in MySQL). What language I'm using on any given day depends on what platform I'm developing for:

OS X: C/Objective-C; perl for quick tasks
Linux: PHP, Java, and some Ruby on Rails; perl and bash for quick tasks
Windows: C# and some straight C/C++ with the bare Win32 API

A lot of what I write are "glue" or "integration" programs that help tie our various vendor apps together.

So beyond the obvious (WWDC), where should I go? In previous years I've been to YAPC and ApacheCon. I tried to go to RubyConf but missed the registration deadline (miscommunications on my part). I've been doing a lot of WordPress related stuff this year so a WP internals conference would be dandy (how to write plugins well). Also, if there's some good Oracle training that would be beneficial.

Budget is ~$3000 for registration, hotel, and travel. I work in Higher Education so anything with a bent towards that is good. Other than Oracle, the training type conferences where it's "Here's how to write programs in X language" don't interest me because I'm a good enough programmer to pickup any language I need to in short time. WRT language conferences I'm more interested in "Here's a framework you should be using. Here's major changes that are upcoming. Etc."

Convert text with whitespace to csv

v-tach — Tue, 20 Mar 2012 12:32:21 -0800

perl or sed or awk help for newbie I am trying to get my tax preparation done and so I need to get info into Moneydance. One of my credit card accounts does not give ofx downloads, just pdfs of the monthly account summaries, and I have managed to extract the text from the pdfs and I now need to transform the text into csv format to import it. I have roughly a thousand transactions to process. I have MacOS 10.7 and can use the terminal but no regex skills.

The text is currently formatted as

MMM DD MMM DD PAYEE INFORMATION CITY PROVINCE
TRANSACTIONNUMBER COMMENT
$AMOUNT

and here is an example of two records

DEC 19 DEC 21 RADIO PARADISE 530¯872¯4993 CA
85450930354980007961385 Foreign Currency¯USD 20.00 Exchange rate¯1.046500
$20.93
DEC 23 DEC 24 SUNTERRA LENDRUM MA CALGARY AB
55181360357461606795546
$101.45

How can I transform this to a csv file such as

MMM DD, MMM DD, PAYEE INFORMATION CITY PROVINCE, TRANSACTIONNUMBER COMMENT, AMOUNT

which I can import into my financial software?

Perl one-liner help

odinsdream — Tue, 13 Mar 2012 19:33:14 -0800

Perl one-liner help - increment a number in a find-replace situation starting at a specific number? Here's my one-liner:

perl -p -i -e 's/msdata:rowOrder=\"[0-9]*\"/"msdata:rowOrder=\"$1".(++$i)."\""/ge' file.xml

This finds all the rows in file.xml with the text msdata:rowOrder="18" and replaces the number in quotes incrementally throughout the file with 1, 2, 3, etc.,

How can I modify this so that I can specify the start number (i.e., start at 1000, then 1001, 1002, etc.,)

How to deal with Chinese in a Perl program on a mac

stonepharisee — Fri, 14 Oct 2011 02:35:01 -0800

Working with Chinese and English text, in a Perl programme, on a Mac. I feel like an idiot. I need to write a Perl programme on my mac to shuffle some Chinese sentences. None of Aquamacs, TextEdit, Pages or Word seems to provide support. Googling unicode support provides a tsunami of information that doesn't seem to help.

Listing the various combinations I have tried would be a monument only to human doggedness and stupidity. They all result in any combination of files in which the Chinese is corrupt, files which Perl thinks are binary and illegal, files which terminal thinks are binary, or similar.

To summarize the challenge, If I could do the following, I would be happy:

/usr/bin/perl -w

$string = "两个艺术家交换了签名。";

print $string, "\n";

Any suggestions?

Find me (or create me) a crash course in Semantic Web...

KMH — Fri, 07 Oct 2011 08:55:49 -0800

Can you find me an online (?) course giving me the essentials of the following alphabet soup of Semantic Web related... things? XML, XHTML, CSS, RDF, OWL, JSON, SPARQL, one of JAVA, Perl, Python etc... I'm trying to find some sort of curriculum I can follow, preferably online, with some worked examples / projects / interactive tutorials, that will give me a grasp of the essential nuts and bolt of XML, XHTML, CSS, RDF, OWL, JSON, SPARQL, one of JAVA, Perl, Python etc...

Yes, I'm aware that these are all different kinds of things (apart from having impressive-sounding acronyms/odd names).

I'm trying to get a solid grounding in the important technologies in my (fairly obscure) field without actually taking a full-time masters course. OPen to the idea of doing *some* sort of formal course though, ideally free but could be a few hundred GBP if it leads to formal accreditation.

Timescale is about a year, with a couple of hours per week to invest, maybe 1 or 2 online, 1 or 2 offline per week.

Other salient facts: I learn quickly and get frustrated if I don't get beyond the basics fast, but learn much better through guided practice; very task-oriented.

Also: passed a basic practical course in mathematical applications of C++, and programmed e.g. binary-to-hex converters in some obscure development environment or other without any major difficulties about 10 years ago and guessing it hasn't got any harder.

Not wedded to the exact list either - if there are other cool things I should (could quickly) learn happy to hear them!

Where my army of robots?

mammary16 — Mon, 12 Sep 2011 01:18:27 -0800

I'd like to automate the creation of online exercises on my company's platform (as in: automatically generated matching, multiple choice, highlighting activities). Looking for some advice on what programming language(s) to use for a particular task.

The company I work for has an online platform for creating online courses. As such, there are many different templates that a user can use: fill-in-the-blanks, highlighting, multiple choice, etc.

The developers are flat-out busy and cannot help me with this task, so I've got to go it alone (which I'm excited about, since I'd like to learn how to do these sorts of things).

Every time you want to create a new activity, you are brought to a Create/Edit item page. On this page, you enter the activity title, introductory text, and text for interactive tokens (for matching, gap fill, reordering activities, for example).

I'm what you'd call a false beginner when it comes to programming; I took a course on Java in university, this winter I learned some Perl (mostly to familiarize myself with the Mechanize module), and currently I'm studying Python (along with the NLTK).

What I'd like to do is the following:
1. Create xml files from a raw data dump. XML files will be structured to hold data for a particular activity type (i.e., there'd be a structure for matching activities, which contains title, intro text, matching items). This will be fairly easy, I think (once I learn how to use XML!)
2. Using some sort of browser automation, input the data from those XML files onto the online platform.

The idea being, I can automate the creation of activities so that there is no human intermediary, no chance for human error (and it would be much cheaper).

The sort of features that this program would have to deal with are:
1. Plain text fields (Title, number of points a learner is assigned, matching tokens, MC questions).
2. Rich text fields (using a built-in TinyMCE WYSYWIG editor)
3. Newly created fields (i.e., you click a + button and it creates a new Multiple choice question). This was something I couldn't figure out when I was using the Mechanize module.
4. Each item type has a slightly different setup, so I'd have to create a function for each separate type.

Looking at the source code for these pages, I see lots of references to Javascript.

At this point, I'm not interested in uploading images, video, or audio (all of which are supported by the TinyMCE editor). Mostly, I'd like for text to inputted in these fields and then submitted. Eventually, this would be part of a bigger program that would automatically create activities from a transcript, or a csv of words and definitions.

I've started to play around with Selenium, which seems like it might be appropriate (although it seems to be more about regression testing) but I was wondering what some of you experts would do, when faced with a similar task. As I mentioned, I know a bit of Perl and am quickly learning Python.

Curious to hear your thoughts... thanks in advance!

Help me fix my nifty webapp

sararah — Thu, 07 Jul 2011 12:27:49 -0800

Help me repair or replace this nifty instrument availability web app on my facility's website. I am trying to update some features on the website for the instrumentation facility where I work. The instrument availability table is something I'm pretty clueless about. As far as I can tell, the script is written in Perl with a MySQL table that collects the logon-logoff information. This was written by some guy named Kyle six years ago, and even my predecessor isn't sure how it is pieced together. We can't even actually find the Perl script buried deep in the web server. So I'm kind of starting from scratch.

What I have:
1. Five (soon to be six) computers that get logged on and logged off multiple times per day. It is possible I may extend this to other machines that are used in the facility in the future, but for now I'm just focusing on the five that I manage. (Obviously there are more than 5 machines in the current table, and some of them are no longer in use, and some should read BROKEN but I don't know how to change the color.)
2. A logon-logoff script on each computer that calculates how much time the user is on the computer which we use for billing purposes (I can get more information on this script if it would be helpful to figure out a solution). This data is collected in a central program for billing purposes.
3. Computers on a variety of platforms. WinXP, a few flavors of RedHat, and some Solaris machines just for funsies. The Solaris machines are the only ones (of my machines) that are actually working with the script right now since the other machines were moved last fall and something in the database was borked when they were brought back online. (VXR300 and VXR400 are the only two of mine that work in the table at the moment. AV600 and DRX400 don't work properly and I have a new one to add.)

What I want:
1. Something similar to the current setup to this that tells users live usage information so they can know what machines are available. It could be fancier than this, or it can be just as simple.
1a. Something that is easily scalable for when we add new instrumentation.
2. Options for open-source or commercial software packages that would do the same thing that I wouldn't have to code.
3. If #2 is not an option, direction as to the best scripting language to use for this task and how to accomplish it. "Hiring a computer science student part-time" is an option we have if I can't figure it out ;)

Thanks in advance!

Early-2000s Perl Beckons with Plentiful Documentation

circular — Mon, 27 Jun 2011 20:56:52 -0800

I'd like to get my feet wet with Perl. Baby steps, just a little beyond "Hello World." I'd like to create a simple little app. I have a few questions... There's a lot of great Perl info out there that shows early-2000s CGI methodology. It seems easier than using a framework. I tried Mojolicious but I just don't have the ability to see where the framework ends and Perl begins -- that is, I don't have enough experience to pick up where the documentation leaves off.

So, question 1: Is it a big deal to just use the early-2000s stuff instead of a framework like Mojolicious? I was worried about e.g. clean URLs but it seems like some redirect work in .htaccess could fix that.

Question 2: Assuming it's not a huge mistake to forego frameworks for now, I'd like to make a simple CRUD app with an admin side. My first idea was to create a call log. Where to start with the admin-side part? For example, the Perl tutorial at Tizag.com will show me how to read from a database. How about writing to it, and password-protecting the admin area? Any good tutorials out there?

Thanks!

Decoding the question of the coder I need

Shepherd — Thu, 05 May 2011 11:21:29 -0800

What kind of Web programmer/coder do I need for this project? Where do I find such a person? I have an idea for a Web site; I'd like to invest a bit of money in getting it to the point where I can maybe take it to Kickstarter and really get it off the ground.

My Web skills begin and end with HTML and a bit of CSS. I tried learning some PHP once and got a nosebleed.

The nexus of the idea is a functionality where people can enter text in an entry page of the site; this text will then be re-presented on an aggregate page that shows lots of different texts, being entered by different people at different points on the globe, with everything updating in real time.

The "data entry" pages are simple and look like a straight-up text entry box.

So Person A in Nebraska can be typing an essay about water tables and agriculture;
Person B in Hong Kong can be typing a letter to his/her mother;
Person C in Toronto can be typing a film script;

and there's another page somewhere where you can see Window A, Window B, and Window C, and see the texts that Person A, B and C are entering in real time.

There’s a lot more to it, but that’s the core. I’m not trying to be maddeningly vague, but I also don’t want to explain the whole thing in a way that will be (a) nauseatingly long or (b) totally give it away. The engine of the concept is this "simulcasting" of text entered by multiple people at multiple points.

So... what kind of a Web programmer am I looking for? I know design is its own thing, but I really need to build the fundamentals of this so I can move on to proof-of-concept and then maybe Kickstarter. My programming knowledge is comprised of

10 PRINT "BUTT"
20 GOTO 10

... so I don't have the foggiest of what kind of subset of coder/programmer will best suit my needs.

And where's the best place to look? I need to start fleshing out a budget for this project, both for the initial phase and as a Kickstarter goal.

Help learning regex

Raichle — Fri, 08 Apr 2011 12:29:05 -0800

I have a brain-block when it comes to Regex and really need to learn it. Can anybody suggest really good tutorials with little exercises to reinforce the info? I've tried so many times to get a grasp and it just escapes me and would be invaluable for school/work since I'm doing a lot of string processing in Perl and PHP. Thanks!

How to sell EPUB books for programmers

amtho — Tue, 22 Feb 2011 13:37:34 -0800

Selling EPUB e-books, adding on to a custom shopping cart app: handle like any downloadable file? DRM? If DRM is a bad idea, good layperson-accessible article explaining clearly why not? If DRM is a good idea, how is it implemented? Assume I'm some kind of coding genius (just to skip the "it's too complex to do it yourself" answers -- I may decide it's too complex, but I'm not going to do so without understanding how it's implemented), what do I need to know to handle selling e-books?

This would be for a small, niche, independent publisher. Its clients are primarily educators and counselors. They're already using a somewhat basic, but rather customized, shopping cart web site (in Perl, FWIW) with payments handled by a separately-hosted service that basically stores credit card and purchase list information which the client retrieves and processes (which assures everything is in stock, etc.).

I'm their web developer. I don't have an e-book reader. The company that has printed their books can convert them to EPUB format easily, but that doesn't include DRM -- which makes sense, since DRM seems likely to be something that's going to be added for each purchase as it happens.

I need some basic clues. Googling turned up stuff that doesn't look like what I need: arguments about the benefits/costs of DRM; DRM info for cell phone/other devices; where to find DRM-free stuff.

I need to know:

How to implement DRM myself (in Perl, but any language) assuming I'm handed a bunch of EPUB files; and/or

Where to find an excellent, focused, short, clear article explaining how my client can decide whether they should implement DRM or not.

I already know (I think):

How to find basic information on the EPUB format;

That, for some markets, DRM makes no sense because piracy is less of a concern than obscurity. (I'm not sure whether that's true for this client's market.)

That just adding the purchaser's name into the text is one idea. It sounds good, but I'm not convinced it's enough).

parse pix/asa logs

fozzie33 — Thu, 16 Dec 2010 05:05:54 -0800

Trying to create a Perl or shell script to parse/convert Cisco PIX/ASA logs into a CSV format. Has anyone had any success with this? Since all messages aren't standard, I'm trying to think of the best way to approach this issue. Has anyone had any success, or have a good AWK, Grep, SED, or perl script that can get this done?

Renaming a whole bunch of files at once, in bash

Asparagirl — Sun, 12 Dec 2010 13:08:05 -0800

Please help me rename ~1300 image files in bash, en masse. I have a long list of files that are named like this:

buczacz_(186-1-5723 a.)_0001_(IMG_6459).jpg
buczacz_(186-1-5723 a.)_0002_(IMG_6461).jpg
buczacz_(186-1-5723 a.)_0003_(IMG_6462).jpg
etc.

As you can see, there is a space between the "5723" and the "a." in those names. I would like to replace that space with an underscore character so that the filenames end up like this:

buczacz_(186-1-5723_a.)_0001_(IMG_6459).jpg
buczacz_(186-1-5723_a.)_0002_(IMG_6461).jpg
buczacz_(186-1-5723_a.)_0003_(IMG_6462).jpg
etc.

This needs to be done in bash on a shared web hosting environment, so I can't install any fancy tools to assist me. I'm sure there is a simple regular expression to handle something like this, but I don't know how to construct it. My geek-fu is not strong enough. Please help?

Perl regex hell

caution live frogs — Fri, 17 Sep 2010 14:38:01 -0800

I have been hammering Google for about three hours now trying to find the answer to what I expected to be a simple question: WTF is up with regular expression backreferences in Perl? I cannot make this (simple) find-replace work. Trying to automate some file processing, stripping out junk in text output files from a data collection program and turning them into .csv files.

I was doing this via an AppleScript, calling TextWrangler to do the grunt work using grep regex find/replace, and it worked well - but it was really slow once I compiled the AppleScript, so I am trying to use a call to a shell script instead to speed things up.

The code in question:

sub replaceString {

  my ($search, $replace) = @_;

    if( s/$search/$replace/ig ) { 

        print; 

    } 

    else { 

        print; 

  } 

 }

When I pass it text as the search and replace arguments (ie, find "foo" and replace with "bar") it works. When I pass it arguments using backreferences, it doesn't work, even though the same argument in TextWrangler works just fine.

Specific example: I need it to strip an initial 0 off of some timestamps in the file. If I use "\ +0(\d\d:)" and "\1" for my find and replace values, the zero is replaced by "\1" - literally. "045:" is changed to "\145:". It doesn't work using dollar signs either. I've spent most of my afternoon trying to figure out how to do this, but every damn result I find in searching just tells me how to use regex, not how to pass regex group elements from one match to the next. Perl apparently just forgets all regex values the second it starts interpreting a new match?

I can't figure out why a program designed to work with text is so obstinately stupid when it comes to something as simple as a find-replace using pattern matching. There must be a really simple way to do this.

(Notes: the subroutine will be called to replace a long string of matches, sequentially, so I need to be able to keep $search and $replace as undefined args passed by the main routine. In the end I need to be able to package this as a bundled app that anyone in my lab can run on the lab Macs [the reason I compiled the script], so I can't get too crazy with how this thing works.)

Why would you write this in Perl?

yomimono — Fri, 03 Sep 2010 13:17:51 -0800

Can you explain to me a block of Perl? The tricky part is the last line, which is just a variable sitting on its own - no operation, no control structure, nothing. sub data_source { my $self = shift; my $d = $self->{data_source}; $self->{data_source} = shift if @_; $d; }

I get everything except that last line, which really only makes sense if $d is some kind of function pointer. I'm under the impression that if that were so, &$d(); would be the correct syntax. I think I must be horribly confused in some way; can you help me out?

(code is from an open-source package with a busy author whom I don't want to bother with a possibly-basic Perl question.)

Perl is making my head explode

routergirl — Fri, 21 May 2010 13:55:44 -0800

Script-filter: I've been tasked with finding out who at our company is synchronizing to their home folders on our fileservers, and how often. Although initially it was recommended I do this in perl, my head is blowing up trying to get it to work. Lots of blundering attempts to ask for help inside. I hate scripting in perl. Hate it. I don't have an alternative scripting language I prefer. They all seem to make me react the same way. My head simply locks up and refuses to learn it, which I recognize as a very real problem. I can configure routers and switches all day long, and I've created shell scripts and batch files with very little trouble. But perl seems to be a different animal entirely for me.

Let's say for this we have 6 locations, all meshed via VPN and all accessible on the domain. Each location has a Win2k3 file server. On each file server, there is a \home\department directory for each department. In each \home\department directory, there is a "Users" folder. And in each "Users" folder, there is a folder for each user in that department.

My task as it was given to me: Find out who is using their home folder, and how often. To do this it was suggested I create a script that will go through these User directories (and subdirs) and pull up the last accessed (or modified) file date. Provide easy to understand results. So if I last synchronized my home directory on 5/19/2010, I would want an email or a text file resulting from the script to show:
Routergirl - 05/19/2010.

It would be nice, even, if it came back with something like:
Fileserver - Engineering - Routergirl - 05/19/2010.

My current script has had me baffled for several days, and when I go to people in the company who know more about perl than I do, they seem to either want to run with it and create a new one on their own that still doesn't work, or they're too busy to help me. The person who has been the most help is my brother, who is currently 3k miles away and busy with his own job. Although perl was initially recommended by my boss, he is perfectly willing to accept other methods that provide the hoped for results.

Is there a better way to do this? Could it be that perl is just the wrong solution? Maybe there is some automated free program we could be using for this that would save a load of headaches. I'm not convinced that checking last modified or last accessed date on the files will even be a good check for this. People could circumvent it by browsing to their home folder on the fileserver and just modifying a file, therefore not actually synchronizing, but bringing up current results. Maybe there is something that can actually check when a user last accessed the directory itself?

Entirely open to any and all suggestions. I realize this question is fairly scattered. If it comes down to all of you saying, "Yes, perl is the best way to do this," then I know to sign myself up for some force feeding...errrr....classes. If anyone can provide resources on existing scripts that do this exact thing already (my google fu has failed me so far - they're all just enough different that they won't work), that would be much appreciated. If anyone knows of a better way - please tell me, cos maybe I'm just missing the obvious, and I've been known to overthink.

Java and Perl Refresher

blueplasticfish — Wed, 06 Jan 2010 09:26:09 -0800

Been years since I've written any Perl or Java (Core/Swing/Servlets/JSP). Have an interview soon, where can I find a quick and thorough refresher?

Too much music! I should have such a problem...

chez shoes — Tue, 08 Dec 2009 10:39:12 -0800

Perl script to manage enormous, and growing, music library? First things first - I am not a programmer by any stretch of the imagination, but I do have some SQL and Perl knowledge (more SQL, less Perl). And, I'm willing to learn more.

The problem: 1000's of MP3 files. Some are duplicates (and triplicates) from compilation albums, some have really bad metadata. What I'd like to do is run a script that will pull all the dupes into a file, so I can see how bad it is. Then, in the interest of saving space, I'd like to delete all redundancies. And further, I'd like to be able to manipulate the metadata - normalize such things as &/and, bulk-change misspellings, etc.

If a tried and true script to do this sort of thing exists, can you point me to it? Thanks in advance.

PERL before swine

bluejayway — Mon, 19 Oct 2009 11:23:53 -0800

PERL filter: I'm a PERL ignoramus and trying to run a script from Baseball Hacks to download a bunch of zipped files from retrosheet.org. I'm getting an error that says: URL must be absolute. I can't figure out how to make the url absolute, as each file's location is different and supplied by variable values. Any hints would be super appreciated. (Script is after the jump). Here's the script- copied directly from the book- I only modified the date ranges and changed the non-html-friendly characters for the purposes of posting here:
use FileHandle;

use LWP::UserAgent;
$ua = LWP::UserAgent->new;
$baseurrl = "http://www.retrosheet.org/";

for ($year = 60; $year LESSTHAN= 92; $year++) {
foreach $league ("al", "nl") {
my $filename = '19' . $year . $league . '.zip';
my $url = $baseurl . '19' . $year . '/19' . $year . $league . '.zip';
my $req = HTTP::Request->new(GET =GREATERTHAN $url);
my $res = $ua-GREATERTHANrequest($req);

print STDERR "fetching $filename\n";

if ($res-GREATERTHANis_success) {
my $fh = new FileHandle "GREATERTHAN$filename";
if (defined $fh) {
print $fh $res->content;
$fh-GREATERTHANclose;
} else {
print STDERR "could not open $filename: $!\n";
}
}
else {
print STDERR $res->status_line, "\n";
}
}
}

Batch convert Word docs to text files?

yarrow — Thu, 09 Jul 2009 12:29:40 -0800

Batch convert Word docs to text files? Does anyone know of a utility to convert a large number of Word documents (hundreds) to text files in Windows XP? They are in Word 2003 (.doc) format. The Word docs are about a 1MB each because they have images, but the text files they turn into are only about 4KB. I want to run a text processing script I wrote in perl on them, but I'm not sure how to get them in text format in the first place without opening each and converting it by hand. Looking for free solutions if possible, just because it's a pain for me to pay for software at work.

Help me learn Perl, Python, Lisp, Haskell, Ruby

jgunsch — Thu, 09 Jul 2009 09:39:29 -0800

Help me learn a few new programming languages. Python, Perl, Lisp, Haskell, Ruby. In college now, been programming for six or seven years already, all self-taught. I started with Java for a couple years, moved on and now consider myself fairly fluent in PHP and its friends (JavaScript, CSS, etc.). I've been having a lot of fun with jQuery lately! And now I'm stuck in a summer internship working on an ASP.NET project written in VB.

I'd love people's recommendations for books/resources to get started in a few high-level languages: Perl, Python, Lisp, and possibly Haskell or Ruby. I went through an O'Reilly book on Perl (I believe it was this one a couple months ago, so I have a handle on the syntax, but am not terribly capable with it.

Opinions/information on which languages people would recommend (or others) and for what purposes would be incredibly helpful. Other language suggestions are welcome, as well. I don't really have reasons to learn these specific languages beyond "well, I probably should know enough Python to know what it's good for, understand what's going on in Python code if I saw it, and hack something together for a basic task." I'm perfectly capable of searching for tutorials/books on my own, but people's personal recommendations have that knack of knowing which ones worked and didn't for them. ;)

Books/Internet resources/etc. that anyone could point me to would be awesome. Bonus points for a good crash course in Linux shell scripting.

Best language for highschool bioinformatics course?

bergeycm — Thu, 25 Jun 2009 11:22:31 -0800

I'm teaching a course on bioinformatic programming for high schoolers. Which language should I teach it in? And do you have any nifty ideas for easy projects that fall under the bioinformatics header? It's a six-week, twice-weekly internship for highschoolers with absolutely no programming experience. The catch is, it has to be a bioinformatics course. What language would be best? I think the two contenders are Perl and Python, though I'm open to other options.

Possible Perl Advantages:

- I know Perl, so I won't need to pick up another language and I'd be better at debugging it.
- If the kids go on to AP Comp Sci or a 101 course in college, the language is often Java. Both Perl and Java use C-style syntax.
- The bioinformatics online support for Perl seems better than for Python.
- Long-string manipulation and speed seem much better in Perl.
- Though complex Perl is scary-looking, well-writen basic perl doesn't seem that intimidating.

Possible Python Advantages:

- Less steep learning curve.
- Possibly more resources for beginner programmers.
- The whitespace-is-important thing and there-is-only-one-best-way-of-doing-something thing may be better for teaching good programming techniques.

Also, any basic bioinformatics projects you can think of are much appreciated. The goal is really to teach programming, but the internship requires that it be done through bioinformatic applications.