PHP -> Twitter API at specified times: what am I Doing Wrong?

July 29, 2009 6:29 PM

Please help me write a PHP script that publishes Twitter updates at pre-specified times.

I'm working on a script that will update Twitter with reminders about upcoming shows at a music festival, building on lodev's idea here.

I've currently got this script, which calls a text file structured like in this made-up example:

XXXUPDATE SponsorCo Stage: THE BEATLES in 30 minutes XXXTIME August 21 2009 21:30 +0200 XXXDONE 0 XXXENDLINE
XXXUPDATE MegaCorp Stage: ROLLING STONES in 30 minutes XXXTIME August 21 2009 22:45 +0200 XXXDONE 0 XXXENDLINE

Proprietary format, I know. :) There are six Twitter accounts right now for separate concert stages, more may follow.

All six separate scripts are called from a Ruby file via pycron, every minute. I understand that's plenty often and I don't really need that kind of temporal resolution, but it uses relatively few processor cycles so I'm not really complaining.

As you can see in the PHP code:

if (($now <= ($utime + 30)) && ($now >= ($utime - 29))) {
if ($update[3] !== 1) {

When an update is ready for publication it should, theoretically at least, satisfy the first "if" statement and be published. The second "if" checks a "done" flag in the text file which I have as of yet not used (i.e. they're all set to 0 and the current code doesn't amend them).

Now, I "stress-tested" this with some test updates on all six accounts, including many simultaneous ones (carrying the same timestamp) on the different accounts for ten minutes with six (one for each account) every minute, and out of maybe a hundred-something tweets (I ran multiple iterations of the test) only one was dropped, i.e. it was in the text file but was not published.

Even one dropped update is unacceptable to me, and I strongly suspect I am Doing It Wrong. But how? The problem basically boils down to the following question:

What better method would there be to push out the updates reliably using the server config I currently have (XP, Apache, PHP5, Ruby, pycron), very preferably without using databases?

When it comes to scripting I really only speak PHP and a little JavaScript: even the (very basic) Ruby script was kindly provided to me for a different project.

So there's just too little time before the event to learn a new language or start out with MySQL or whatever before the event, and I'm not really eager to change to different software or change my server configuration (or set up another box) just for one-off project that will last just a few days. This is just to say that it's not due to an unwillingness on my part, but rather due to practical considerations.

If anyone could offer their insights I would be ever so grateful. Thanks in advance, guys.

posted by goodnewsfortheinsane to computers & internet (9 comments total)

The problem I see with your code is that it only has a one minute window in which it will consider a given job, thus if a job takes too long to complete it can knock a pending job out of the running for consideration. You already have a 'done' flag which I presume you set to 1 when a job successfully completes, thus there is no reason to limit your time window to one minute; just scan for any jobs for which the trigger time is less than the current time and which have a done flag of 0.
posted by Rhomboid at 7:05 PM on July 29

I would suggest using the either the PHP or Ruby Twitter API package and post with it, and check the return status (if available in the API), and/or search the feed using the API to verify that it was posted.
posted by theclaw at 7:56 PM on July 29

I'm sure there are limits to the number of posts you can make in the Twitter API. I don't remember offhand how many or any such useful details, but its definitely something to look into. Try running your script every 30 mins and see if you lose any.
posted by cgg at 8:13 PM on July 29

Why not use an existing service that does this for you? I won't name names, since we're players in the area, but you can even use RSS to feed some of them (or just do it by hand, up front).
posted by kcm at 8:38 PM on July 29

As rhomboid says, using a time window rather than simply determining that the time has passed is irrelevant, THOUGH

Even one dropped update is unacceptable to me

But is an "antique" update unacceptable? Twitter has, at time, had downtime. Do you want a tweet to go out if it's 30 minutes late? Something to consider.

Either way, I would expand that time window. If not ALL past posts at least make it as wide as it's acceptable to have an old post. Maybe 30 minutes if acts are on stage for 60 min?
posted by phearlez at 9:36 PM on July 29

CoTweet let's you schedule updates, manage multiple accounts, and more. They give beta codes almost instantly.
posted by disillusioned at 3:42 AM on July 30

there is no reason to limit your time window to one minute; just scan for any jobs for which the trigger time is less than the current time and which have a done flag of 0.

This is the correct answer for a short-term fix.

As for doneness, I'd start working on it now. Modifying a text file in a particular position correctly strikes me as non-trivial (unless you're just going to rewrite the damn thing every time, which might be easiest -- although more dangerous!). Really, the easy thing here is to use a DB, but if you're not familiar with SQL I understand shying away.

Another tip: you may need to parse the $update[3] var into an int or avoid using strict equality. If you're using strict equality, the string "1" is not considered equal to the integer 1 -- and I believe you'll be getting strings back from your regex.

I'm sure there are limits to the number of posts you can make in the Twitter API. I don't remember offhand how many or any such useful details, but its definitely something to look into.

twitter rate limiting. Your GETs are limited, either by IP or by auth'd account. Updates are apparently not. Still, it could be a problem, depending on how the underlying API handles its communication with twitter.

The absolute right thing to do here is to add API error handling to your side and unit test everything so that you can be confident you've met all edge cases. Learning how to properly write tests is not the sort of thing you can do in a day, but unit testing is absolutely awesome for this sort of thing -- as you continue learning to program I'd encourage you to pick up good testing habits as soon as possible; I can't believe how many years I spent writing untested code.
posted by fishfucker at 7:32 AM on July 30

Thank you all for some great suggestions. I could widen the window and use the flags, or an API check, but for now I think I'll use CoTweet -- it does exactly what I want and it looks reliable.
posted by goodnewsfortheinsane at 8:12 AM on July 30

Cotweet worked perfectly. You can see the fruits of my labour here, here, and the rest of the stages you could work out from the Following, if you should so wish.

Thanks again, everyone.
posted by goodnewsfortheinsane at 6:38 PM on August 29