Historians must count

March 9, 2012 6:02 AM

I have a research project in mind. It involves the use of statistics. I am a typical humanities grad with no real background in the quantitative social sciences, and need some guidance. What I'm interested in is: tracking, tallying, and correlating some of the everyday events recorded in personal diaries from a particular historical period.

I'm an historian of reading. One of the main sources of evidence reading historians use to find evidence of historical reading patterns is the personal diary. But we know we're essentially cherry-picking: how do we balance the instances we find with the (numerically much greater) silences that exist alongside them in the archive? How might we demonstrate, statistically, just how common a practice reading was in a certain place at a certain time?

I live close to a major archive with many hundreds of personal diaries and journals from my period of historical specialization. I would like to select a sample from this archive and see how many of the personal accounts within it refer to reading, and how (if at all) this activity is correlated with other activities (e.g., going to the movies; playing cards in the evenings). I'm also interested in questions of frequency: are those who refer to reading continually demonstrably different from other diarists? I have an inkling that both diary-keeping itself, and the compulsion to record one's reading, are symptoms of an obsessive personality (a kind of outlier status), and I would like to find some way of addressing this idea statistically.

So, what I would love at this point are: (1) some suggestions of books (on, say, quantitative analysis for historians) I can go to for deciding things like sample-size, significance, and how to establish correlation; and (2) any practical suggestions from people who who might have done similar research.

Finally, if anyone can point to work going on in the field of "everyday life studies" (loosely defined) that might provide some models for the kind of study I'm envisaging here, that would also be great. Thanks!

posted by Sonny Jim to Society & Culture (12 answers total)

My background is also in the Humanities, and I think the type of research you want to do is Content Analysis.
posted by Pineapplicious at 6:17 AM on March 9, 2012

I'm a quantitative social scientist. I think that the path of least resistance would be to partner up with someone that knows how to do this. Then NEXT time you can do it alone.

Yes, this would be simplistic content analysis (really snazzy CA would have hypotheses and stuff...).

But really, CA is a ton of work, so finding someone that already knows how to do it would be much more pleasant for you.
posted by k8t at 6:30 AM on March 9, 2012

Hi there -- I'm a humanities BA getting a PhD in Library and Info. Sciences. I've recently been through the "learning stats" process in my program. I understand where you're coming from.

For this --


But we know we're essentially cherry-picking: how do we balance the instances we find with the (numerically much greater) silences that exist alongside them in the archive?

Well, you can't ;) . Personal diaries are great sources of data, but they are virtually impossible to use to draw any kind of statistical inferences. There is simply no way to know the scope and extent of those "silences" as you put it. You could maybe do a limited content analysis (as Pineapplicious recommends, within Person X's diary for example -- e.g. 15% fewer mentions of reading by X in the summer months) but that doesn't seem like what you are looking for. Is there any reason you wouldn't want to do a good qualitative study? Just because you can't put numbers on it, doesn't mean it's not great data! Having done both CA and more in-depth qualitative analyses, I would recommend that route.

....as for ths.
How might we demonstrate, statistically, just how common a practice reading was in a certain place at a certain time?

I would look outside of personal diaries and at more statistically sound proxies of reading behavior. For example, number of libraries/bookstores within a certain city or historical publishing statistics. There are many studies that do this, but I don't have a reference handy.

If you're interested, I'll look through some of my references later when I have more time and post some more links. Good luck!
posted by pantarei70 at 6:34 AM on March 9, 2012

Thanks for the lead on content analysis. Yes, I'm vaguely aware of the field, but what worries me is that none of the material I'll be looking at is digitized, so I (realistically) wouldn't be able to assemble any kind of textual corpus that I could then run CA programs on. The archive in question is very old-school, and my tools (at least in the first instance) would be a notebook and a pencil.
posted by Sonny Jim at 6:36 AM on March 9, 2012

@pantarei70: yes, I'm already planning a qualitative piece using this archive, and the idea of finding a quantitative angle on it is really an outgrowth of that. The qualitative piece involves looking for references to books by a certain author, and then describing the social and physical spaces in which his books were "consumed." So, much more manageable, I think.
posted by Sonny Jim at 6:40 AM on March 9, 2012

It is okay if they're not digital. But finding someone with expertise will be helpful.
posted by k8t at 7:26 AM on March 9, 2012

When I did my research methods class in grad school, we used Shulamith Reinharz's Feminist Methods in Social Science Research. It's well written, clear, and lays out practical as well as theoretical explanations of different research methods. Reading your description of the type of research you want to do, I think this book would be helpful to you--if I recall correctly, there's a whole chapter on content analysis. Even if you would not describe your work as explicitly feminist, Reinharz would say that the type of research you are doing is the kind that feminists would find useful--which is why it's included in the book.

It is also an older book, so there would be less reliance on digital methods and more advice for working with old-school materials.
posted by hurdy gurdy girl at 7:41 AM on March 9, 2012

I read Sidney Verba's Designing Social Inquiry: Scientific Inference in Qualitative Research in a graduate seminar on the philosophy of social science and it seems pretty close to what you are looking for.
posted by jayder at 8:30 AM on March 9, 2012

You can absolutely do content analysis with non-digital content. That's how I've done it. Just keep in mind that 1) CA doesn't really support inferential statistics, and 2) you will need a second coder (if you're not using software) to establish reliability of your method.

Finding a partner with more quantitative analysis experience is a great idea. "Digital humanities" is a very hot area right now -- and your project might be right up someone's alley.
posted by pantarei70 at 11:26 AM on March 9, 2012

Are you associated with a university? Ours has a "statistical advice service" which is exactly the sort of thing you need. It's for researchers without much stats background to consult with at the outset of a project that might involve quantitative methods, and to get advice on how to set up the project, how to gather and code the data, and what sorts of methods would be appropriate. You are then expected to try and run the stats yourself, but the advice service will give you references to read to help you learn how to do it, and will check your work afterwards to make sure you did nothing stupid. You have to give them a footnote acknowledgement if they provide minimal assistance, or co-authorship if they end up doing a lot of the stats for/with you.

It's worth checking out whether your university has something similar: maybe even informally. Ask around other humanities people to see if they have ever talked to the people in statistics and if they have suggestions for support. If you get really desperate (and have the funding), you could probably pay a stats grad student to spend a few hours conceptualising the project with you.
posted by lollusc at 5:04 PM on March 9, 2012

Finding a partner with more quantitative analysis experience is a great idea. "Digital humanities" is a very hot area right now -- and your project might be right up someone's alley.

Ha. Yes, digital humanities is a hot area right now. I kinda do it myself. And the research school in my university is very keen to facilitate collaborative partnerships across faculties; perhaps I should talk to them about the idea.

It's worth checking out whether your university has something similar: maybe even informally. Ask around other humanities people to see if they have ever talked to the people in statistics and if they have suggestions for support.

Yeah. I should probably do that. I have another piece of quant analysis I want to do with another archive, too ...

Thanks for the advice everyone. It demonstrates to me that I need to think more deeply about the idea, read some CA primers, and speak to a few colleagues and administrators. In the meantime, I'll stick to the qualitative, journal-article project I had initially envisaged. (Abstract due at end of month; full draft by mid-year. Yay!)
posted by Sonny Jim at 3:22 AM on March 10, 2012

If you want to send me your email, I can send you the assignments that I give for content analysis week when I teach methods.
posted by k8t at 11:30 AM on March 10, 2012