software to graph/analyse numeric data in xml file?

March 3, 2009 8:51 PM

software to graph/analyse numeric data in xml file?
I've got lots and lots of numbers and I'd like to look at a pretty graph.

Hello,

I have some data in an xml files that I want to plot and analyse.

The data consists of sample points, each sample has a value and a timestamp. The samples are all sorted by time, so I think it should be fairly straight forward to plot the points

I might end having to look at lots of similar files, so it would be good to find a really efficient way of processing the data.

Mostly I'll just need to look at the graph, occasionally I'll want to super impose two graphs on top of each other.

I have used Mathematica, so thats somewhat familiar ground, but I wouldn't have a clue how to import data into it

any suggestions?

cheers

Mat

here's a snippet of the xml:

the first number is the value
the second is the timestamp



<array>

<real>-0.47238370776176453</real>

<real>2616.4930366660001</real>

</array>

<array>

<real>-0.45421510934829712</real>

<real>2616.502291666</real>

</array>

<array>

<real>-0.43604651093482971</real>

<real>2616.512025</real>

</array>



posted by compound eye to science & nature (7 comments total)

microsoft excel would probably handle the data well... it has pretty straightforward import wizards and chart creation wizards. someone else will have to chime in with more detail on that, mathematica, or something else though...
posted by eli_d at 9:12 PM on March 3

or the openoffice.org spreadsheet program... (free and open source)
posted by eli_d at 9:13 PM on March 3

Documentation from Mathematica is often quite useful.

(Although I think I'd want to use Matlab for such a graph. Don't have a good reason for that.)
posted by nat at 9:25 PM on March 3

Here's how you can do it in a couple of lines of Mathematica code. I enclosed your snippet in <xml></xml> and saved it in "test.xml".

To import the XML file, type

z = Import["test.xml"]

You'll see that z comprises XMLElement objects and so on. The way to proceed is to specify some simple Mathematica pattern-matching transformation rules to interpret these.

Here are some suitable rules. I strip away everything except "array" and "real" tags. I treat "array" as a List. I parse "real" as a mathematical expression (in this case, a number):

rules = {
XMLElement["array", _, list_] :> list,
XMLElement["real", _, {digits_}] :> ToExpression[digits],
XMLElement["xml", _, content_] :> content,
XMLObject["Document"][_, content_, _] :> content
}

Now z //. rules gives your parsed data. For example, try

ListLinePlot[z //. rules]

I hope this helps.
posted by hAndrew at 1:59 AM on March 4

Thanks very much for the help and suggestions,

Nat thanks I'm looking through the mathematica documentation.

hAndrew thank you for the code.

While I haven't been able to get it to work, you've started me off in the right direction.
it imports just fine, but I can't get it to extract anything from the xml

But now that I know there's something called pattern matching, I've found the guide/RulesAndPatterns section of the documentation, so I guess I'll work from there

cheers
mathew
posted by compound eye at 4:00 AM on March 4

I am happy to help if you find yourself stuck. Just send me a MeFi mail. Another good place to ask Mathematica questions (and find out if similar questions have been asked before) is mathgroup.
posted by hAndrew at 11:44 AM on March 4

Thanks hAndrew,

While the intention of your code made sense to me, i just couldn't make it filter anything, and not really understanding the syntax, I found the documentation a little terse.


In the course of trying to figure out the mathematica language syntax, I stumbled across sage, which is open source and uses python as the scripting language.

and was able to make a plot using:

from xml.dom import minidom 
doc = minidom.parseString(data) 
L = [] 
for a in doc.firstChild.childNodes: 
     if a.nodeName == 'array': 
        x = float(a.childNodes[3].firstChild.data)  
        y = float(a.childNodes[1].firstChild.data)
        L.append((x,y)) 
list_plot(L) 

cheers mat
posted by compound eye at 12:05 AM on March 5