Dump that Stock!

December 9, 2008 8:07 PM

Stock market related questions-- help me drown in data!

I want to improve my coding skills and virtually playing the stock market seems like it might be quite a fun way to do so, but I desperately need data to enable my endeavor.

So I'm looking for two elements really,

1. A detailed historical dump of the stock market, per smallest time unit possible, current I can easily find a daily current/high/low/average but I want to play around with minute (or second) values ideally but can't seem to find that anywhere.


2. Programming: Are there any stock-brokers that let you use their API (so you can program your own automated trader/interfaces) on a play account, for free?

The first element is the main one, I can essentially just play the stock-market offline if I have that, but it'd be great fun to actually link in with a play account on a (semi) real-time basis. Who knows, in a couple of years, maybe I'll try it for real.

Extra points for good resources on introductions to the various models used to predict the markets.

Any help appreciated!

posted by Static Vagabond to work & money (5 comments total)

What you are looking for is called time and sales data in the industry. It's a tabular listing of all the trades of a stock in the order in which they occurred. It's usually for sale at the exchange where it's traded.
posted by pjern at 8:43 PM on December 9, 2008

1. You are not going to be able to get any high-quality free tick (time/sales) data. You might be able to get low quality data somewhere, but it will suck for any sort of simulation. The big difference between high quality and low quality data is that low quality data will have bucketing, wrong timestamps, or gaps.

High quality data is very expensive, and **huge**.

2. interactive brokers and tradestation both have retail-client API's. IB is way better imho, and I know they offer demo accounts for testing software.


As someone who does exactly this for work, their is plenty of programming skills and market juice at the daily timeframe. eoddata.com is a pretty amazing source of low-priced (or free if you are willing to deal with a hassle) daily data.


As for the models, honestly you are better off not reading anything. If you read about the favored existing ideas, you are losing probably your only chance in life of coming up with something totally different.
posted by H. Roark at 9:29 PM on December 9, 2008

Have you looked into Tradestation? I have never personally used it, but I have considered it. It provides a fairly sophisticated programming interface and historical data. It can be had fairly cheap if you use it to make a few trades each month.
posted by joelr at 3:09 AM on December 10, 2008

IB's data is not true time and sales, from what I've read -- perhaps its changed from 2007 though, but I suspect not. I think Tradestation might have a stronger historical data set.
posted by Big Fat Tycoon at 6:55 AM on December 10, 2008

NYSE will sell you Trade and Quote data; as others mentioned, it's not cheap and it is voluminous.

If you've got an association with a University you may be able to get this data for free. I'm associated with three in the UK and only one can provide researchers with this information.

In terms of models, you'll need mastery of elementary statistics before moving into ARCH/GARCH (and related) techniques. The basics there start with Pindyck & Rubinfeld, lead to Enders and then on to various journals, but if for any chance you'd like more current / advanced reference don't hesitate to drop me a mail.

You might want to not look so closely at the equity markets, but rather focus your efforts on daily changes in the stock markets but considering other factors that impact valuation - just off the top of my head interest rates, exchange rates, CPI, the change in oil prices, bullion prices, lots of interesting work is being done now integrating news into share price changes, etc.

This approach would have the double advantage of smaller data sets (perhaps only closing share prices) as well as a greater opportunity to arrive at a unique insight.

Look at it this way - every grad student in Econometrics has done at GARCH modeling at some point in their academic career, and with high frequency data no less - this way you'd be looking in hopefully a new or at least a lesser explored direction.

I've worked a lot equities as an asset class at an I-Bank (more current in fixed income & credit derivatives these days), and still teach Forecasting Financial Markets at the Masters level, so I might be able to give you a more helpful reply if I know more about your background.
posted by Mutant at 7:13 AM on December 10, 2008