What are some useful ways to organize historical research materials?
April 21, 2017 4:15 PM Subscribe
I'm using primary and secondary sources to research a historical event and the people involved. Some of these sources are available online, others are documents that I've photographed myself. I have hundreds of pages of documents to transcribe and extract information from. How can I best organize it all? Shoot, even if you can recommend a book on historical research methods, that would be useful too.
The other way of phrasing this question is: how do you, the researcher, organize your research materials? Especially if you're a historian and you're dealing with the kinds of sources (and the problems inherent to those sources) that I'm working with.
I've looked at some older questions on this topic, and I'm afraid that no matter how wonderful EndNote is, it is far outside my budget. Scrivener looks great, and I may end up using it to write my thesis, but it doesn't look like it will help with what I'm trying to do right now (although feel free to correct me if I'm wrong). Zotero is useful for my secondary sources, since I can copy info directly from my library catalog, but the majority of my research material is not easily compatible with Zotero.
What I need is to have a bunch of references on people/places/things that I can quickly and easily access, tied in some way to many dozens of different offline sources, some of which give contradictory information. I'm trying to figure out what happened when, who knew whom, who came from where, and so on. I'm looking at official forms, testimonies, photographs, you name it. Almost everything is handwritten, which means sometimes I need to go back and correct my own transcription.
I can organize the documents themselves well enough, by creating a folder for each person and adding .pdfs and so on to each one. But I need a way to bring that all together in one place, and a million different folders seems like sort of the opposite of that.
I don't know any database software, and there's enough people that this is hard to organize, but not enough people that anything will be remotely uniform -- I mean, each person has their own random collection of facts gleaned from random, sometimes contradictory sources (which means I can't just make fields for "name" "age" and so on, when spellings and dates of birth weren't always accurately or consistently recorded).
So far my best option seems to be getting a bunch of 4x6 index cards and making one for each person, place, or thing involved in this event. I'm still a little hazy on how to organize that, but at least I could lay everything out in front of me (FWIW that's what my professor recommended, but she also hates computers). Part of me worries that I'll wind up with a bunch of bulletin boards with string connecting various documents, conspiracy-theorist style ("it's all so obvious!"), but I'd like to avoid that for my sake and for the sake of my patient-but-not-that-patient significant other.
I'm sure everyone has their own preferred methods, so I know I'll have to experiment a little before I find something comfortable. No matter what I do, I assume there's a better option out there than my current massive Excel file with 150 rows and 60+ columns. Most of the organization is just in my head, and that's starting to be a problem.
Please do ask for clarification if I worded this in a way that sounds really crazy and disorganized. Being crazy and disorganized is sort of my biggest problem right now. And like I said, if you think what I really need is a book that will help me better understand the process of historical research, that's OK too.
The other way of phrasing this question is: how do you, the researcher, organize your research materials? Especially if you're a historian and you're dealing with the kinds of sources (and the problems inherent to those sources) that I'm working with.
I've looked at some older questions on this topic, and I'm afraid that no matter how wonderful EndNote is, it is far outside my budget. Scrivener looks great, and I may end up using it to write my thesis, but it doesn't look like it will help with what I'm trying to do right now (although feel free to correct me if I'm wrong). Zotero is useful for my secondary sources, since I can copy info directly from my library catalog, but the majority of my research material is not easily compatible with Zotero.
What I need is to have a bunch of references on people/places/things that I can quickly and easily access, tied in some way to many dozens of different offline sources, some of which give contradictory information. I'm trying to figure out what happened when, who knew whom, who came from where, and so on. I'm looking at official forms, testimonies, photographs, you name it. Almost everything is handwritten, which means sometimes I need to go back and correct my own transcription.
I can organize the documents themselves well enough, by creating a folder for each person and adding .pdfs and so on to each one. But I need a way to bring that all together in one place, and a million different folders seems like sort of the opposite of that.
I don't know any database software, and there's enough people that this is hard to organize, but not enough people that anything will be remotely uniform -- I mean, each person has their own random collection of facts gleaned from random, sometimes contradictory sources (which means I can't just make fields for "name" "age" and so on, when spellings and dates of birth weren't always accurately or consistently recorded).
So far my best option seems to be getting a bunch of 4x6 index cards and making one for each person, place, or thing involved in this event. I'm still a little hazy on how to organize that, but at least I could lay everything out in front of me (FWIW that's what my professor recommended, but she also hates computers). Part of me worries that I'll wind up with a bunch of bulletin boards with string connecting various documents, conspiracy-theorist style ("it's all so obvious!"), but I'd like to avoid that for my sake and for the sake of my patient-but-not-that-patient significant other.
I'm sure everyone has their own preferred methods, so I know I'll have to experiment a little before I find something comfortable. No matter what I do, I assume there's a better option out there than my current massive Excel file with 150 rows and 60+ columns. Most of the organization is just in my head, and that's starting to be a problem.
Please do ask for clarification if I worded this in a way that sounds really crazy and disorganized. Being crazy and disorganized is sort of my biggest problem right now. And like I said, if you think what I really need is a book that will help me better understand the process of historical research, that's OK too.
Best answer: My wife is a historian and I helped her to organize the massive amounts of stuff she collected during her PhD (which included hundreds of pages of photographed documents). We used Zotero and it didn't have major drawbacks. The directory structure for keeping the files was basic : secondary sources in one folder with titles like AuthorName_Year_ShortTitle.pdf, primary sources organised by source (name of the archive, archive folder/number). In some cases, I turned primary sources into pseudo-secondary sources by compiling photographed documents into pdfs. I'd say that the physical organisation should be kept as simple as possible.
Now, for the logical organisation (which is actually hard to define beforehand), if it's actually impossible to trick Zotero into doing your bidding, your best bet is to learn the basics of Microsoft Access (or equivalent) and organise your data according to the principles of relational databases. From what I understand of your situation, you'd need at least separate tables for "documents" (with a field telling you where it is), for "people", for "places" and for the relations between all those entities. Once it's all in the database, extracting data is simple, but there's a price to pay for organization.
posted by elgilito at 5:08 PM on April 21, 2017 [2 favorites]
Now, for the logical organisation (which is actually hard to define beforehand), if it's actually impossible to trick Zotero into doing your bidding, your best bet is to learn the basics of Microsoft Access (or equivalent) and organise your data according to the principles of relational databases. From what I understand of your situation, you'd need at least separate tables for "documents" (with a field telling you where it is), for "people", for "places" and for the relations between all those entities. Once it's all in the database, extracting data is simple, but there's a price to pay for organization.
posted by elgilito at 5:08 PM on April 21, 2017 [2 favorites]
Best answer: I have a historical research project I'm working on that spans over 50 years. I've been using Scrivener and I LOVE it. It's billed as writing software rather than organizing software, but it has helped me organize my project in a way that I can actually progress.
I have folders for: Project management, research locations, notes on broad subject matter, notes on specific people/places/things. Each of those has either subfolders or subdocuments with notes about those various things. The really cool thing is that you can move them around in your directory structure, then turn on "Group Mode," and view them as a single document (kind of like chapters). If you don't like they order your parts are in, you can move them around again. You can also drag and drop files into a research folder (PDFs, Word docs, web pages) and access them easily from Scrivener.
I'd been using a combination of Google Docs and Evernote and it was a disorganized mess. Scrivener makes me feel like I can actually turn my project into something.
The one drawback is that the tutorial, which will take a couple hours to get through, is pretty much a must. Past that, though, you're golden.
posted by mudpuppie at 5:29 PM on April 21, 2017 [3 favorites]
I have folders for: Project management, research locations, notes on broad subject matter, notes on specific people/places/things. Each of those has either subfolders or subdocuments with notes about those various things. The really cool thing is that you can move them around in your directory structure, then turn on "Group Mode," and view them as a single document (kind of like chapters). If you don't like they order your parts are in, you can move them around again. You can also drag and drop files into a research folder (PDFs, Word docs, web pages) and access them easily from Scrivener.
I'd been using a combination of Google Docs and Evernote and it was a disorganized mess. Scrivener makes me feel like I can actually turn my project into something.
The one drawback is that the tutorial, which will take a couple hours to get through, is pretty much a must. Past that, though, you're golden.
posted by mudpuppie at 5:29 PM on April 21, 2017 [3 favorites]
Best answer: PS:
So far my best option seems to be getting a bunch of 4x6 index cards and making one for each person, place, or thing involved in this event.
Scrivener does this. It has an index card function. You can turn your notes into index cards and move them around the screen to organize them.
It also has a 30-day free trial. Those 30 days are counted as days you actually use the program -- not number of days after you download it. You could try it out for free to see if it's right for you.
Also, I swear I'm not a shill. It's just that I literally feel like Scrivener has empowered me to do the things you're talking about.
posted by mudpuppie at 5:32 PM on April 21, 2017 [1 favorite]
So far my best option seems to be getting a bunch of 4x6 index cards and making one for each person, place, or thing involved in this event.
Scrivener does this. It has an index card function. You can turn your notes into index cards and move them around the screen to organize them.
It also has a 30-day free trial. Those 30 days are counted as days you actually use the program -- not number of days after you download it. You could try it out for free to see if it's right for you.
Also, I swear I'm not a shill. It's just that I literally feel like Scrivener has empowered me to do the things you're talking about.
posted by mudpuppie at 5:32 PM on April 21, 2017 [1 favorite]
Best answer: Bibliographic software like Endnote is designed for modern, published materiel only. It can't handle the varietions of older published material (like no publishing date), let alone the sheer diversity of unpublished primary sources.
As for data organization: it depends on what source you are using and (more importantly) what research question you have.
Here's what I did when actively engaged in historical research:
For photographs of primary sources: I organized these by archive, and catalogue number, with a note of the date and other useful information in the folder title. I took notes from these photos - and often organized the notes as I was taking them.
For long texts being subjected to discourse analysis: I would take detailed notes on the text into a word processor document where I would write my commentary and analyses (often in bold or another colour of text). These notes would then be what I wrote my chapter from.
Straight-forward quantitative data (like acreages) was entered directly into a spreadsheet for calculations. For quantitative data that was more complex (records of land holding), I created a relational database where tables of people were linked to tables of land-holdings.
For data which was qualitative but not really a consistent, longer text (like a series of petitions, or hundreds of orders by a court, each one just a paragraph or so), I created a quick-and-dirty spreadsheet-database where I could have the date, full source reference (including page), notes on the item, and then could either categorize the item, or enter a series of tags. Then when I wanted to pull up all the items on a single issue (e.g. pigs on river banks), I could do a simple search for "pigs", and pull out all the pig items for when I was writing about pigs. It was like index cards, but in a spreadsheet (which was lighter, made for less typing - I used copy-paste for the reference numbers - and could be exported to a text file for other purposes).
One reason I didn't use any of the more specialized writing software like Scrivener is that I like do-it-myself solutions, but also, it didn't exist when I started (or I didn't know about it - it was 2004). Some of this software may be much better than my excessive reliance on spreadsheets, especially for longer textual notes - but I do love spreadsheets and find them still useful when creating catelogues of disparate material on the fly. I'm currently working on a project to find & analyse all sorts of web-based material, and I'm back to my old spreadsheet ways: each specific item has its own row, where I record the access date, URL, title, organization, some brief notes on the content, and then I can have standardized coding (exclusive or not). I have 200 something items catelogued this way; with sorting or with filters, I can pull them out by country or by medical condition or by medium.
PM me if you'd like to chat details - I could make suggestions, but ideally, you want to think more specifically about your sources (what's going in) and what you want out (what do you want to learn from them) to help you design your data-management system.
/was a history phd student, now work in job where organizing information is 80% of what I do
posted by jb at 7:57 PM on April 21, 2017 [1 favorite]
As for data organization: it depends on what source you are using and (more importantly) what research question you have.
Here's what I did when actively engaged in historical research:
For photographs of primary sources: I organized these by archive, and catalogue number, with a note of the date and other useful information in the folder title. I took notes from these photos - and often organized the notes as I was taking them.
For long texts being subjected to discourse analysis: I would take detailed notes on the text into a word processor document where I would write my commentary and analyses (often in bold or another colour of text). These notes would then be what I wrote my chapter from.
Straight-forward quantitative data (like acreages) was entered directly into a spreadsheet for calculations. For quantitative data that was more complex (records of land holding), I created a relational database where tables of people were linked to tables of land-holdings.
For data which was qualitative but not really a consistent, longer text (like a series of petitions, or hundreds of orders by a court, each one just a paragraph or so), I created a quick-and-dirty spreadsheet-database where I could have the date, full source reference (including page), notes on the item, and then could either categorize the item, or enter a series of tags. Then when I wanted to pull up all the items on a single issue (e.g. pigs on river banks), I could do a simple search for "pigs", and pull out all the pig items for when I was writing about pigs. It was like index cards, but in a spreadsheet (which was lighter, made for less typing - I used copy-paste for the reference numbers - and could be exported to a text file for other purposes).
One reason I didn't use any of the more specialized writing software like Scrivener is that I like do-it-myself solutions, but also, it didn't exist when I started (or I didn't know about it - it was 2004). Some of this software may be much better than my excessive reliance on spreadsheets, especially for longer textual notes - but I do love spreadsheets and find them still useful when creating catelogues of disparate material on the fly. I'm currently working on a project to find & analyse all sorts of web-based material, and I'm back to my old spreadsheet ways: each specific item has its own row, where I record the access date, URL, title, organization, some brief notes on the content, and then I can have standardized coding (exclusive or not). I have 200 something items catelogued this way; with sorting or with filters, I can pull them out by country or by medical condition or by medium.
PM me if you'd like to chat details - I could make suggestions, but ideally, you want to think more specifically about your sources (what's going in) and what you want out (what do you want to learn from them) to help you design your data-management system.
/was a history phd student, now work in job where organizing information is 80% of what I do
posted by jb at 7:57 PM on April 21, 2017 [1 favorite]
Best answer: When I was writing my thesis I used Scrivener. For each general argument/idea I created a card in Scrivener, then typed in the relevant evidence from the source + a citation and location of the original. I could move the order of the cards around when necessary, and each time I used a piece of evidence I changed the font color.
When it came to actually sketching out the components of each chapter, I used physical note cards. I had like a full pack and a half by the time I was done. Sometimes with writing you need to be the conspiracy theorist.
posted by lilac girl at 7:16 AM on April 22, 2017 [1 favorite]
When it came to actually sketching out the components of each chapter, I used physical note cards. I had like a full pack and a half by the time I was done. Sometimes with writing you need to be the conspiracy theorist.
posted by lilac girl at 7:16 AM on April 22, 2017 [1 favorite]
Best answer: I know historians who keep all their research materials from the archives on Evernote (with pro membership) and use the tagging function to keep things organized. The auto OCR helps with some of their documents.
posted by col_pogo at 10:30 AM on April 22, 2017
posted by col_pogo at 10:30 AM on April 22, 2017
« Older Do I dare go green? I’m envious of that hair! | What is this plant in this overgrown community... Newer »
This thread is closed to new comments.
posted by jgirl at 4:50 PM on April 21, 2017