Data Organization Software Anyone?

June 28, 2006 2:53 PM

DataOrganizationFilter: I am looking for software that will allow my market research team to efficiently store and access information from PDF and Excel files.

I currently work in the marketing department for a health insurance company. We are looking for ways to more efficiently store and (easily) access all of our competitive intelligence files (competitor's summary of benefit PDFs, PDFs of quoting info, and excel files we have put together to compare the information).

Ideally, we would have some sort of database and a front end GUI to access the information (also being able to pull up the original file). For example, we want to find all of the carriers' plans in Rockford, IL, which contain the subject "infertility." We type the word "infertility" in, press a button, and wa-la the plans and plan info pops right up on the screen (vs. us having to open one file at a time).

Are there are software packages out that that would be able to provide us the above functionality?

I would be more than happy to check out any open source software that might be floating around, even though we'd be willing to spend some money to get the best system for our needs.

Also, do you know of any vendors that might be able to provide us with a backend database and a front end piece of software to access the database? (I'm just curious what's out there)

Thanks in advance!

posted by drkrdglo to Computers & Internet (6 answers total)

Full text search is your friend. Available FTS engines likely have filters to process Excel and PDF.

But since your data will be in a variety of formats (not the file type, but the ordering and organization of information within the file) you're probably going to have to do some manual post-processing or tagging.

The last software package I saw similar to what you want was one I was building on contract (our goal was to create a system allowing the organizing and cataloging of disparate information from multiple sources, and allow it to be searched a variety of way) ; there may or may not be off-the-shelf or open source packages that will meet your needs.
posted by orthogonality at 3:31 PM on June 28, 2006

Sounds not unlike a small intranet portal. Or is a browser based UI not acceptable?
posted by Good Brain at 3:48 PM on June 28, 2006

And I'm not sure you really want this.

Rather than typing infertility and seeing the competitors' entire plans, wouldn't you want something more like an executive summary:
Our plan A will pay up to $10000 for up to three attempts; our plan a costs $400/mo.
Our plan B will pay up to $8000 for up to two attempts; our plan a costs $200/mo.
Competitor X's plan A will pay up to $12000 for four attempts; it costs $500/mo

And an ability to search or rank plans on these criteria?

E.g., we're in marketing, so find all competitor's plans than cost more and provide less coverage for condition Y (so we can issue a press release); find all competitor's plans that cost less and provide more for condition Y (so we can find other scary conditions for which they provide less coverage than we do, and tell potential customers, "sure, Competitor X will cover infertility a bit better than we will, but if you get a flesh-eating necrotizing fasciitis, they won't provide jack!")
posted by orthogonality at 3:49 PM on June 28, 2006

I used to sell this stuff - it's a document management system you want. It usually includes the ability to index records, built-in permissions and security, lots of easy view and search customization and presets, OCR abilities, highlighting, electronic notes and redaction. Documents can be imported (electronic) or scanned (paper).

Google 'document management' and you'll get a zillion replies.

I'm partial to Laserfiche solutions myself, as I used to sell them and found that my clients always found them to be reliable and reasonably priced.
posted by gregariousrecluse at 4:53 PM on June 28, 2006

This is a great question! Are there any open source alternatives to Laserfiche?
posted by Freen at 7:05 PM on June 28, 2006

The basic deal with document management is that it is a lot more complicated than most people ever give it credit for until they fail.

Systems like Laserfiche are built with lots of compliance issues in mind - HIPAA, Sarbanes Oxley, etc etc etc - they are very good at what they do, and make it very easy for admins and end users, while also being robust, flexible and secure. A document repository should be proven bulletproof - period.

Open source? Sure. you could accomplish everything you need functionally (finding stuff) with a copy of google desktop, or any number of applications. If a team of users uses it every day in a professional environment, however, buy is cheaper and more efficient than build the majority of the time. A big part of that also has to do with the local VAR that will install it - their expertise alone usually proves invaluable, as document management/document imaging should be their core focus.

Hope this helps.
posted by gregariousrecluse at 11:03 AM on June 29, 2006