How to get immediate notification of Craiglist posts with a filter?

August 20, 2010 4:36 PM

What is the best way to monitor/notify myself of new Craigslist listings for housing?

I'm watching housing listings on Craigslist, searching for a new apartment. I have a filter list of criteria that selects out certain neighborhoods, price range, pets, etc.

However, an amazing number of people never respond to me when I email them about their listing. I'm thinking it's because they are getting 20 other emails before mine.

So what's the best way to get immediate notification of each new listing, given that I have a certain filter set up?

I've tried setting up a couple different RSS feeds using NewzCrawler, etc., but they are quite slow. I often don't get the ads until 20 mins after they're posted.

There's something called HeyCraig that sends emails (which is good) but it doesn't seem to work with the filter I've got set up.

Ideally, each time a new listing comes up that matches my filter, a window would pop up or a beep would sound at the computer I'm sitting at. Additionally, I would immediately get the listing by email so that I can see it on my iPhone while I'm away from the computer.

Any tips?

Thanks,
Mike

posted by mikeand1 to Computers & Internet (18 answers total)

When I was trying to find housing a few years back, I would obsessively put in my criteria and click search.

I found that actually doing a search would make things show up a few minutes before they showed up in the general list. And I bet (though I didn't try to automate) WAY before any notifications got sent out.

I was successful with my obsessive and time-consuming strategy twice in the past few years.

Of course, if you have a life, this may not be for you. I just tested it, and doing the search still will bring up posts before they appear in the general list.
posted by rabbitrabbit at 5:00 PM on August 20, 2010

RSS feed. Do your exact search, as you want it to be sent to you. Then click the orange RSS icon in your browsers address bar, or at the bottom of the page. This will bring up the address of the RSS feed. Then use a service like Feed My Inbox to have the listings sent to you. I've done it this way (although I just use Google Reader) and it's really handy.
posted by coupdefoudre at 5:15 PM on August 20, 2010

Response by poster: I'm trying the RSS feed just as you suggested, but the listings show up in my RSS reader 20 mins after I get them by searching on Craigslist (as the previous poster notes).

Is there a particular reader I should be using? NewzCrawler is about 20 mins behind. (And I've got it set to check every minute.)
posted by mikeand1 at 5:18 PM on August 20, 2010

Duh. I am an idiot, totally missed that line in the OP!

Well... unfortunately I don't really have a good solution for you then. The only thing I can think of is that in my experience, when I submit a listing to CL they email me and say "Thanks for your listing! It'll go up soon" etc etc. Then, when it does go up, the timestamp is closer to the time I actually submitted it initially than when it really appeared on the site. So maybe you aren't actually getting them 20 min after everyone else, the time is just reflecting when the posting was submitted, not when it began being displayed.

Other than that - I can't imagine that the 20 minutes lagtime really matters. I'm in NYC and I've never had a problem getting a response even if I don't write immediately after the posting goes up. Maybe your problem is more that you are writing to those brokers that flood CL with bogus postings and aren't super reputable.
posted by coupdefoudre at 5:33 PM on August 20, 2010

I recommend using Google Reader. Once you have your RSS feed(s) set up you can leave the page open and it will automatically update in real time (or close to it).
posted by Capa at 5:47 PM on August 20, 2010

Response by poster: It isn't just a difference in time stamps; I can see the post put up right on the Craigslist website (using my filter), but it doesn't show up in the RSS feed until later.
posted by mikeand1 at 6:05 PM on August 20, 2010

This is a VERY slight refinement on rabbitrabbit's suggestion but it's worked for me.

Bookmark the search and/or keep a tab with that bookmark open and refresh as often as you feel necessary.

(if you run a multiple monitor setup you can also keep a dedicated window open on your "spare" monitor and have an auto-refresh Greasemonkey script do the refreshing).
posted by dolface at 6:37 PM on August 20, 2010

Response by poster: "Bookmark the search and/or keep a tab with that bookmark open and refresh as often as you feel necessary. "

That's what I'm doing now; I was hoping for a solution that doesn't require this sort of attention. I do have a life, after all!
posted by mikeand1 at 6:39 PM on August 20, 2010

RSS feeds are the solution to the problem "how can I be notified of new content on a website" but they are not a real-time solution to that problem. Twenty minutes is pretty good!

The only other way to keep track of it in smaller increments than twenty minutes would be some kind of script running on your computer at shorter intervals, say once a minute.

That would be essentially trivial as a programming task, but probably violates their terms of use, which say you shouldn't:

use automated means, including spiders, robots, crawlers, data mining tools, or the like to download data from the Service - unless expressly permitted by craigslist

As you yourself admit you're only guessing that timing is the reason you don't get replies, maybe it would be useful to revisit that assumption? How about you show us the kind of email you write, so that we can check for other factors?
posted by AmbroseChapel at 7:27 PM on August 20, 2010

This is totally doable.

Watch That Page Changedetect are both web based services that will do this for you, though I suspect not as frequently as you want.

If would be easy to script though (well easy enough with bash and diff).


#!/bin/bash
while [ 1 ]
do
URL="http://www.gatheringstorms.org" # CHANGE THIS LINE
FILE="index.html" # AND THIS ONE
curl -Os "$URL/$FILE"

if [[ -e $FILE ]] ;then

diff $FILE "$FILE.1"
if [[ $? -ne 0 ]]; then
echo "file changed"
mv $FILE "$FILE.1"
else
echo "nothing doing"
fi
else
mv $FILE "$FILE.1"
fi
sleep 120
done

posted by tallus at 7:31 PM on August 20, 2010

Erm maybe I should explain what that script does. It fetches the page specified by URL/FILE (currently my home page which isn't likely to change fast) every two minutes (for ever or untill you kill the program). If it changes it spits out file changed otherwise nothing doing. Email/pop up/sounds are totally doable as well, but would take longer than the 5 minutes it took to write that as well as being a little more system specific (though adding beep on a line by itself above echo "File changed" should work anywhere I would think.

And its a bash script so Unix-y (e.g .OS X, linux etc) only.
posted by tallus at 7:37 PM on August 20, 2010

Response by poster: "As you yourself admit you're only guessing that timing is the reason you don't get replies, maybe it would be useful to revisit that assumption? How about you show us the kind of email you write, so that we can check for other factors?"

Sure. This is in SF, by the way, so the market is hot. I spoke to one landlord today who posted a listing this afternoon and already had 8 in-person applications by 7 pm. In any case, I usually write:

Hi,

My wife and I are interested in this apartment. We're both professionals, no kids, 800+ FICOs, with excellent references. Credit report sent on request. Please let me know when we might be able to see the apartment.

Thanks,
Mikeand1 Lastname, [phone]
posted by mikeand1 at 8:12 PM on August 20, 2010

Response by poster: Tallus,

Thanks for the script, but how do I incorporate the filter I'm using?
posted by mikeand1 at 8:15 PM on August 20, 2010

I spoke to one landlord today who posted a listing this afternoon and already had 8 in-person applications by 7 pm.

But if they bothered to accept 8 applications, doesn't that imply that they're not choosing tenants on a first-come, first-served basis? My guess is they'll rent to the best applicant, not the first. The 20 minutes may not matter much, or at all.

Just out of curiosity, is this in a rent-controlled area? I'm wondering why prices haven't risen with the demand. Sorry if this is a dumb question -- the rental market in my area of the country barely even has a pulse.
posted by jon1270 at 4:16 AM on August 21, 2010

If you have an Android device, the "Craigslist" app (used to be called Craigsnotifica or something) is really great for this. I used it for all my apartment and furniture hunting. Don't know if it will solve your 20-min lagtime issues though.
posted by ista at 9:21 AM on August 21, 2010

Response by poster: "But if they bothered to accept 8 applications, doesn't that imply that they're not choosing tenants on a first-come, first-served basis?"

Yes, that was an example of somebody who called me back. For each such person, there are three who don't call me back. I was just using that as evidence that these ads are getting lots of responses.

"Just out of curiosity, is this in a rent-controlled area?"

Yes, as I mentioned, this is San Francisco, where there is (moderate) rent control. Not that this has kept prices down that much. A nice 2-BR place in a good neighborhood can exceed $3k/month.
posted by mikeand1 at 10:03 AM on August 21, 2010

Thanks for the script, but how do I incorporate the filter I'm using?

Go to craiglsit and do a search with filters and it should give you a url like this (2 beds between $1500 and $3000 in this example):

http://sfbay.craigslist.org/search/apa?query=&catAbb=apa&srchType=A&minAsk=1500&maxAsk=3000&bedrooms=2

add that in the URL= line

then change

curl -Os "$URL/$FILE"

to

curl -s $URL -o $FILE

(the $FILE bit is mostly so we have a saner file name to deal with)
posted by tallus at 1:46 PM on August 21, 2010

Response by poster: Great, thanks.
posted by mikeand1 at 10:52 PM on August 21, 2010