Help creating a bot
December 24, 2007 2:57 PM Subscribe
I need help creating a web bot (for non-malicious purposes). Are there any tools out there to facilitate this?
posted by chips ahoy to computers & internet (11 answers total) 4 users marked this as a favorite
I need to retrieve some datasets from a web server. To get the datasets, it involves making a form request, getting assigned a random job number, waiting for the job number to finish (usually 5-15 minutes), and then downloading the datasets from the links presented on the job webpage (i.e. http://www.blah.com/jobs/[jobnumber]).
The access to this server is unlimited (a public government server) and not against its usage policy and my planned usage is very light, so let's skip that issue. The problem is that the interface is clumsy and it will be very tedious to sit there and submit each job for the data I need. I want to automate this and let the computer worry about it. Is there a good scripting language that can help me with this? I guess Perl is the obvious solution but I am not very well versed in it and I've seen very little in the way of short, elegant Perl code.
I'm very good with DOS scripts and Delphi (which I will probably resort to if there's no simple method).