Matlab textparsing filter: So for a project, I am working with over one hundred years of hourly data collected for water levels. I have all of this collected in a giant, ~35 Mb text file. Toward the last decade or so, there are double entries for water level thanks to semidiurnal tide measurement upgrades, and so there is an irregular number of columns that isn't easy to work with the usual data import strategies. I need your help, citizens of Metafilter, so that I can successfully rig the Matlab textscan command (or something else) to read in this formatted data, and make accommodations for empty value strings! More inside.
posted by oceanjesse to Computers & Internet (13 answers total) 1 user marked this as a favorite
So, I have a bunch of lines that look like this
8656483 19000101 00:00 0.329
8656483 19000101 01:00 0.217
But then, around line
841400 or so, I have something like this:
8656483 19970904 11:00 0.622 -.603
8656483 19970904 12:00 0.241 705
I'm looking to form a matrix that looks like this below, but I can't seem to get there with the command that I'm using.
[8656483, 19000101, 00, 0.329, 0;
8656483, 19000101, 01, 0.217, 0;
8656483, 19970101, 11, 0.622, -.603;
8656483, 19970101, 12, 0.241, 705;]
Here is the command workflow that I'm trying to use, but to no avail:
>> fid = fopen('~/Desktop/values.txt');
>> C = textscan(fid,'%f %f %2f %f %f', 'delimiter', '', 'EmptyValue', 0);
When I open the variable C though, I end up with these as contents:
   [0x1 double] [0x1 double]
Is anybody a guru with the textscan command, and can tell me where I've gone astray? Also, if you have a better suggestion, that would also be great. I've tried the data import tool with excel, and I've also tried the import data option in the matlab gui, which scoffed at my file and didn't even load.
For what it's worth, I'm running Matlab R2012a on OS X Lion.