methods or products to query objects?
May 20, 2011 9:25 PM   Subscribe

What are some "end user" products (preferably open source) or methodologies that allow complex queries of objects? I want to allow both programmers and non-programmers to be able to access the data.

The data is currently organized in such a way that it requires an expert to query the data. Only some of the data is relational and other parts are not. What I envision is spending time and turning this data into objects in Ruby, Python, C# or perhaps Java. My hope is I could spend weeks to months doing this and end up with something users could deal with. I know this sounds a lot like data warehousing, but I don't think it follows the patterns I have seen this used for.

Are there any open source programs or methodologies I could read up on? I want to allow query by forms or perhaps a query language similar to SQL. LINQ for example is not a good option as it is too complex for end users to master.

Thanks in advance!
posted by santogold to Computers & Internet (10 answers total) 1 user marked this as a favorite
Crystal Reports
posted by Blazecock Pileon at 10:35 PM on May 20, 2011

Have you looked at Apache Solr? It's a search engine in a box.

Although maybe that is too high level for what you want.
posted by sbutler at 10:36 PM on May 20, 2011

I'm really not sure I get your situation, but are you looking for something like an XML database you can transform the data into and hit with XPath or XQuery? Or maybe you are a candidate for a more popular NoSQL solution like, for example, MongoDB. Its query language, like its data, is just Javascript, so it's pretty natural for most web developers to work with. Maybe also look at HTSQL as a model for allowing end users to do semi-arbitrary queries of the relational data. Insofar as it's RESTful, maybe you can figure out how to extend it to handle your other data using an XPath-like syntax.
posted by Monsieur Caution at 10:38 PM on May 20, 2011

It depends what you mean by "and other parts are not." Are they unstructured documents you need to do full-text indexes on? Are they semi-structured documents that you want to extract some structured information from? Are they structured but just not in a relational form?

I came in here to recommend HTSQL - I haven't used it, but we looked at it briefly for a similar situation. Especially if you have some dev time to dedicate, it could work well as part of a system for something like this. MeFi Mail me if you still have questions - I do very similar stuff for work :)
posted by devilsbrigade at 10:53 PM on May 20, 2011

Sounds like you want to write some database views over your data, to hide the complexities.
posted by orthogonality at 10:56 PM on May 20, 2011

Nothing is full text.

Some of the data is relational some of it is not. It is all in an SQL database but there data that needs to be extracted trough programming because the authors chose not to use the relational model (I don't know why).

I could extract this through programs and normalize the data into a different database but that would be a lot of work and I still would not have something end users could use easily.

The reason coding converting this data to objects appeals to me is that I could make it fit into an object model quite elegantly.

I have spent time with google and wikipedia and have not yet seen a way (for end users) to query objects easily and wonder if there is something that I am missing or if I am asking the wrong questions.

posted by santogold at 1:13 AM on May 21, 2011

The reason coding converting this data to objects appeals to me is that I could make it fit into an object model quite elegantly.

And the reason converting the data to objects won't work is because your data isn't well-normalized. You need to get your house in order first.

First normalize your data. Then read up on Object Relational Mapping (ORM) which is precisely what you're talking about. In the Java world we use Hibernate (NHibernate in the .NET world).
posted by Civil_Disobedient at 5:13 AM on May 21, 2011

Haven't used them myself, but you may want to take a look at some of the open source OLAP tools out there, like Mondrian or Cubulus.
posted by jenkinsEar at 8:11 AM on May 21, 2011

if you don't want to drive yourself insane, normalize your data, then the tools become a lot easier.
posted by empath at 11:41 AM on May 21, 2011

Garbage In, Garbage Out. Normalize your data.
posted by beerbajay at 2:55 AM on May 22, 2011

« Older Life inside a hyperbolic plane?   |   How can I underclock the CPU in Windows 7 on a... Newer »
This thread is closed to new comments.