Web Scraper/Sorter... What am i looking for? does it exist?
December 6, 2018 5:28 AM Subscribe
I'm looking to create/buy/download a piece of software that will help me automate a large portion of research i'm doing to uncover stock frauds.
I'm having a wee bit of trouble finding what i'm looking for, so i'd appreciate knowing if such a thing exists commercially, for free, or if i'd need to hire someone to build it for me.
Let's say I've identified a certain lawyer who seems shady, and who works with thinly traded public companies.
I'd like to use a software to extract all companies from whom he filed SEC paperwork.... and then look up all the phone numbers relevant to said individual... and see what other people are involved in those companies, what addresses (office suites) might come up under different companies sharing the same address or phone number or board members.
I know different websites that contain the information i'm looking for... sites like secinfo.com, which has SEC filings with hyperlinks that make it very easy to find the same person mentioned in multiple company filings, as well people search sites that give you reverse phone number lookups and the like. (i have additional sites and data pieces I want to add, and make a great big "family tree" out of it to help me track it.)
what 'm looking for is a way to have the computer do much more of the legwork instead of needing to do this manually. sort of like a custom stock screener of sorts...
Do I need to build a custom AI bot to do this for me? is there something on the shelf i can buy or download to do this?
Thank you mefites!!
I'm having a wee bit of trouble finding what i'm looking for, so i'd appreciate knowing if such a thing exists commercially, for free, or if i'd need to hire someone to build it for me.
Let's say I've identified a certain lawyer who seems shady, and who works with thinly traded public companies.
I'd like to use a software to extract all companies from whom he filed SEC paperwork.... and then look up all the phone numbers relevant to said individual... and see what other people are involved in those companies, what addresses (office suites) might come up under different companies sharing the same address or phone number or board members.
I know different websites that contain the information i'm looking for... sites like secinfo.com, which has SEC filings with hyperlinks that make it very easy to find the same person mentioned in multiple company filings, as well people search sites that give you reverse phone number lookups and the like. (i have additional sites and data pieces I want to add, and make a great big "family tree" out of it to help me track it.)
what 'm looking for is a way to have the computer do much more of the legwork instead of needing to do this manually. sort of like a custom stock screener of sorts...
Do I need to build a custom AI bot to do this for me? is there something on the shelf i can buy or download to do this?
Thank you mefites!!
If you have not already, it might be useful to look at the SEC's Accessing Edgar Data page that describes how all the filings are stored in multiple formats. The formatting might give you ideas about the data retrieval requirement.
posted by Glomar response at 7:42 AM on December 6, 2018
posted by Glomar response at 7:42 AM on December 6, 2018
I'd recommend rolling your own. The Bastard's Book of Ruby has a good intro to Web Scraping to get you started.
posted by aspersioncast at 10:02 AM on December 6, 2018 [1 favorite]
posted by aspersioncast at 10:02 AM on December 6, 2018 [1 favorite]
« Older Crafty Cross Stitch for Goofballs: Xmas 2018... | British K-Pop equivalent to Kero Kero Bonito? Newer »
This thread is closed to new comments.
posted by helloimjennsco at 5:41 AM on December 6, 2018 [1 favorite]