How to work on a certain dataset?

March 15, 2009 3:25 PM

I have a data/software question that's been draining the life out of me for the last couple of days.

OK, I have access to some dataset at BADC (to be more specific HadCM3 dataset), it is basically monthly data in the .PP format and I can convert it to NetCDF. But I know nothing about NetCDF. What I'd like to do is to replicate something similar to this article."Simulating multiyear drought events in North America with the HadCM3 climate model"
Any insights with respect to this issue will be greatly appreciated. Oh and yes, and finally if you have any insights with regards to what kind of software to use in order to manipulate this type of data will be appreciated as well.

So the question boils down to this basically, If you were me, how would you start working on this dataset?

posted by caelumluna to science & nature (4 answers total)

Well, I'm quite familiar with HDF5, but not very much so with NetCDF, so personally I'd probably start by converting the file into a newer version of NetCDF which uses HDF5 as its data store.

There are R libraries for importing from NetCDF and HDF5, so depending on how much data we are talking about, I'd probably start there as well.
posted by grouse at 3:35 PM on March 15, 2009

I looked at the NetCDF library for the R (conveniently called ncdf package) but I couldn't figure out the way to edit the data.
posted by caelumluna at 4:23 PM on March 15, 2009

Here's a tutorial I found via Google.
posted by grouse at 9:45 PM on March 15, 2009

Well NetCDF is a product of UCAR, developed by Unidata. So you can start with their NetCDF page, which includes a NetCDF software list. Also see Princeton's list. "User" software for interacting with .nc files includes MatLab, AcrGIS, and Mathematica. I work for NOAA and a lot of what I do with NetCDF involves Ferret or GrADS, but that is mostly visualization.
Libraries are available for many of the major languages such as Perl, Python, and C. We mostly use Java, which has its own Unidata libraries. A lot of the scientists that use NetCDF are Fortran guys, although I certainly can't recommend that ;)
Finally, the NetCDF mailing list is quite active, with the current developers answering questions daily. Although if you have something specific you might want to start with [e-mail].
Good Luck!
posted by Who_Am_I at 1:21 PM on March 17, 2009