How to scrape posts and replies for a Facebook group I belong to?
July 11, 2018 12:57 PM   Subscribe

I'm working on a research project requiring the download of member posts and replies from a closed Facebook group. Preferably, I'd like to do the work myself using some type of software program or script, but I'm not opposed to paying a small reasonable fee to have a group perform the service. I don't need to capture likes or photos, just the text. I found a script that looks like it might accomplish what I need, but it appears to be broken.
posted by Bushmiller to Computers & Internet (2 answers total) 1 user marked this as a favorite
You probably want to use Selenium with your preferred programming language since you'll need Javascript to be loaded.
posted by dilaudid at 1:16 PM on July 11, 2018 [1 favorite]

I'm sympathetic to your troubles, as I've had a hard time getting facebook group data recently as well. It turns out that in April '18, facebook changed their API when it comes to scraping group data (Cambridge Analytica may have played a role in this change).

The app you're using to download from their API now needs permission to access the data in a way it did not previously. I believe the keyword for you to search for is groups_access_member_info (enables your app to receive member-related data on group content, for secret groups it would be user_managed_groups for admins only).

The scraper you linked doesn't respect that new requirement (line 19 here is the old method). Alas, as far as I have been able to find no method exists on github that satisfies the new rules (due to the newness of the change). I have also not found good instruction on how to actually implement this new requirement in code, just that facebook has to approve your code somehow before it can access group data.

Anyway, best of luck. I have not been successful getting group data for a project this month, and I'll be watching this thread to see if anyone has useful advice!
posted by bessel functions seem unnecessarily complicated at 9:23 PM on July 11, 2018

« Older What the hell is this thing?   |   Help me write the next chapter of my cliched child... Newer »
This thread is closed to new comments.