This Dummy needs a guide to learn Bayesian statistical analysis

May 8, 2007 1:13 PM

Gather 'round, friends, and tell me how to learn Bayesian statistical analysis.

I'm doing an MSc in epidemiology and, as such, have a decent enough handle on the major (frequentist) approaches to data analysis, ie the various flavours of multivariate regression models as applied to longitudinal cohort and case-control studies. Like most of my colleagues, though, this knowledge comes from a non-mathematical education -- I know how to build and interpret a regression model in R but don't know (and couldn't understand if shown) the underlying algebraic proofs.

I've decided I need to learn (at least the rudiments of) Bayesian analysis, for two reasons: It appears that some pretty neat things are possible in the Bayesian world that cannot be done with frequentist models; and I am investigating a specific epidemiological problem that has both frequentist (ie log-linear modelling of 2k contingency tables) and Bayesian approaches.

My program does not offer any courses in Bayesianism so it's up to me. As my knowledge of math is spotty, I need a non-algebraic intro; really, the "Dummy's guide..." approach. Any suggestions?

posted by docgonzo to science & nature (8 comments total)

More dealing with Bayesian networks, Kevin Murphy's pages (here's his graphical models + bayes nets stuff) were great when I was trying to get a handle on this. His basic tutorial on the Bayes rule for example is very clear.

If you're one of those people who like to play around with things whilst you're learning, you could try using his Bayes net package. Do you have access to Matlab? the Bayes Net Toolbox (BNT) comes highly recommended.
posted by handee at 1:25 PM on May 8, 2007 [1 favorite has favorites]

An introduction to Bayesian Statistics using a disproof of ESP as an example. You can take or leave his position on various issues, but it's a useful way to grasp the subject as a generalist, as long as you have a handle on high school level algebra.
posted by ardgedee at 1:59 PM on May 8, 2007

An Intuitive Explanation of Bayesian Reasoning
posted by Zed_Lopez at 2:27 PM on May 8, 2007 [2 favorites has favorites]

Bayesian statistics can be a controversial technique, depending on how it's used.

The Bayesian approach calculates a posterior probability, or what statistician R. Fisher called the "reverse probability" from multiplying a "normalized" likelihood with the prior probability of the null hypothesis, or Bayes' theorem:

P(θ|X) = P(X|θ) P(θ) / P(X)

where θ refers to determining the parameters (mean, variance, etc.) of a null hypothesis, given the empirical, or observed data set ("X").

In English, what's the probability that what you're looking at has a certain characteristic of interest, given the data set?

As a basic example, Bayes' theorem allows us to state the probability of the fairness of a given coin pulled from a bag of coins, both before and after any tosses, if we make reliable assumptions about the fairness of the coins within the bag, prior to taking any out. That prior assumption is where P(θ) comes in.

A frequent complaint about the Bayesian approach is that it makes subjective, sometimes contentious assumptions about the prior nature of the data being observed. The source of this philosophical contention in the use of Bayesian over classical frequentist testing stems from where (or, perhaps, with whom) these prior probability distributions originate.

So-called "non-informative" priors, such as the Jeffreys' prior, express vague or general information about the parameter, to try to make as few "subjective" assumptions about the data as possible.

Informative priors use previous experience or information to set parameter values in the prior: e.g., a simple guess of the expected temperature at noon tomorrow could be calculated from today's temperature at noon, plus or minus normal, day-to-day variance in observed noon-time temperatures.

So-called "conjugate priors" are used when the prior distribution takes on the same form as the posterior distribution and when the mean and variance are usually dependent, and are therefore often used in analysis of empirical data, which is usually constrained by dependence.

The "empirical Bayes" approach was introduced by Herbert Robbins as a way to infer how accident-prone someone is, given the observed fractions of accidents already suffered by the larger population. The objectivity of this testing stems from using observed, "empirical" data to generate informative priors used in Bayesian inference.

Many empirical Bayesian techniques have been applied in various areas within the field of systems biology, which are data-rich and analysis-poor. In particular, Brad Efron at Stanford is one of the big names in this field, and wrote a fun, straightforward paper that bridges Bayesian and Frequentist modes of thinking. Other papers of his on the subject of empirical Bayesian testing can be found here.
posted by Blazecock Pileon at 2:37 PM on May 8, 2007 [1 favorite has favorites]

I'm taking a seminar on Bayesian statistics. We're working through Bayesian Methods for Ecologists [easy to follow with lots of examples] and Data analysis using regression and multi-level hierarchical models [which might work well for self-study]. Bayesian stats has a steep learning curve (especially when you're working on it by yourself). Why not get a group interested and work on it together?
good luck.
posted by special-k at 9:36 PM on May 8, 2007

The links from ardgegee and others are useful (although no offense to Zed_Lopez but I'm starting to get annoyed by the frequency with which the link he posts, since it takes so long to get to the parts of Bayesian reasoning that are most useful and most controversial, and doesn't really make it clear what the issues are).

I've drafted about three comments, all of which I gave up on because there seems to be a contradiction lurking around the corner given
'As my knowledge of math is spotty, I need a non-algebraic intro;'
and
'have a decent enough handle on the major (frequentist) approaches to data analysis, ie the various flavours of multivariate regression models'
'I am investigating a specific epidemiological problem that has both frequentist (ie log-linear modelling of 2k contingency tables) and Bayesian approaches'
since it seems there's a lot of mathematics you do know.

I'm itching to recommend a full book on it (I like this one but it's more geared to physicists and engineers perhaps), but these will invariably include lumps of mathematics. I think it's somewhat important to grasp the nettle here, since whilst Bayesian analysis is powerful it's also dangerous. As Blazecock Pileon says:
it makes subjective, sometimes contentious assumptions about the prior nature of the data being observed
and if you're trying to combine it with the sort of level of techniques you've got then a webpage primer isn't going to be enough.

If I were you, I'd start from the book I linked (on preview, perhaps better special-k's) and use Amazon's 'Customers that bought this also bought' features and dig around for a likely looking text more suited to your field, and Google up a few reviews. Get a book, read through it, and if some part of it looks too intimidating then don't stress about it and skip it. You'll probably get more out of a quarter of a book than ten webpages, and the bits you skip will still be there if you pick the book up again later.
posted by edd at 3:01 AM on May 9, 2007

The Sage publications on quantitative techniques in the social sciences are pretty nifty (little green paperbacks). There's also a book called "the Ecological Detectives" (princetown university press) which I've looked at from time to time that looks nice enough.
posted by singingfish at 1:54 PM on May 9, 2007

Been pointed out to me that 'although no offense to Zed_Lopez but I'm starting to get annoyed by the frequency with which the link he posts' makes no sense. I meant 'the frequency with which the link he posts shows up', and I mean that generally everywhere - not the frequency with which Zed posts or does anything else!
posted by edd at 8:45 AM on May 10, 2007