What is a good software resource for test item analysis between teachers?

February 12, 2008 9:43 AM

Help! I’m a language arts teacher and they are asking me to analyze data…

Hopefully a fellow teacher or a technical wizzard out there can help me. My school is moving towards a data-driven improvement model, which in many ways is a good thing. As a new department chair, I look forward to being able to compare teaching strategies and data related to student achievement with my peers. Over the last few years, we have worked together to create meaningful shared assessments that have common rubrics. Unfortunately, this year we are being asked to complete substantial item analysis within and between classes and we are not being given much information about how to accomplish this. The school has a good amount of hardware support, but our techies don’t really help out much when it comes to using software to assist with instruction and analysis of the types of data with which we are working. Given time and a lot of trial and error, I think I will be able to whip up an excel spread sheet that will keep track of a single class, but I would like something that the whole department could use and my google skills are failing me. Any suggestions? Free would be great, cheap might be doable.

posted by Macduff to education (8 comments total)

Google skills? Google documents.
posted by fatllama at 10:16 AM on February 12, 2008

Can you be more specific about what kind of data you want to store and what you want to do with the data? Do you have specific metrics that you are being asked to compute to do the analysis, or do you have to come up with that yourself?

If you just need to store multiple spreadsheets, fatllama's advice is good. Depending on what you need to do with the data, creating a SQL database might make sense.
posted by burnmp3s at 10:27 AM on February 12, 2008

Which publisher do you use for classroom texts? Your publisher can provide you with ExamView Assessment Suite and you can use ExamView to create and store your own assessments as well as track grades by class and perform item analysis.
posted by mattbucher at 10:45 AM on February 12, 2008

Statcrunch is a really cheap web service that might work for you. More details are needed, though.
posted by desjardins at 10:47 AM on February 12, 2008

What would really be helpful is to know (a) what data you are collecting, (b) what outcomes you are charged with demonstrating with said data, (c) what institutional resources you have for completing the assessment (budget, person hours, faculty expertise/willingness), and (d) as close as you can come to a sense of what is expected...i.e. what it is supposed to "look" like. Any sense of assessment process or delivery is going to depend directly on these factors.
posted by mrmojoflying at 11:09 AM on February 12, 2008

There is a book called Datawise by Richard Murnane which explains exactly how to do what you're asking.
posted by dagnyduquette at 12:45 PM on February 12, 2008

mrmojoflying:
(a) The data is in the form of five level rubrics (1=lacking proficiency, 5= excellence) with five criteria (organization, analysis, purpose/thesis, support, language). This is my major concern because I have yet to find an example of an analysis set up that looks at much more than the final grade on an assignment. (Perhaps someone can steer me in the direction of something that works for portfolios?)
(b) Hopefully, we will be able to look at student achievement in writing over four years as well as trends within the criteria that could guide curriculum revision; i.e. "It looks like forty percent of sophomores have struggling with moving beyond the minimally proficient range for organization for the last three years. What can we change in our instruction to help them out?"
(c) I have computers and a confused and somewhat willing department, but we have really no budget at all for this year at least.
(d) The administration simply wants numbers that sound intelligent, but they do not seem to want to define them for us because that might be "too restrictive". In other words, they don't know what they want. We simply want a nice way to organize data in a way that is slightly more complex than a basic grade book and that can be sorted and evaluated in a way to show trends.
posted by Macduff at 12:51 PM on February 12, 2008

It would be fairly easy to input this data form in a spreadsheet and use pivot tables to bring up the summations that you want. This will ultimately give you the numbers your administration wants. Also, you can easily chart each students progress. The spreadsheets can then be loaded into a simple database for further analysis using virtually any DBMS. Because the data is so straightforward you really could get by with Open Office, although Microsoft Office 2003 (not 2007) has a lot of on line resources for the education market such as 1 or 2. There are literally too many to link to.
posted by ptm at 2:11 PM on February 12, 2008