Document mis-management?

May 20, 2009 6:42 AM

How can I convince a document management vendor to stop embracing 100 DPI / JPG as a universal format for scanned documents?

I work with a document management vendor that optimizes their entire program around 100 DPI / color JPG images.

We're talking about a whole variety of documents, most of which are written / printed text and would be handled nicely at 200/300 DPI B&W (TIFF) format. Perhaps 5% of the documents involved are pictures.

I understand why they use 100 DPI / JPG: it's a universal setting where any image is easily viewable within the application. Honestly I get the impression that they didn't want to deal with scaling early on, but for the most part it's just their "universal format of choice." If you print the images, attempt to fax them, etc, you run into major problems with quality.

They've worked around these issues by designing complex algorithms that automatically apply gamma correction, etc when printing. I'm impressed with what they've done, but I'd like to see the possibility of OCR down the road.

To add insult to injury, you can use their program to scan black and white TIFFs, but when you print them out, they have enormous margins on them that render them unreadable. If you export first, and then print (I realize the goal is paperlessness in the long run) they look beautiful (no margins).

I want to explain to them that in a sense their system is a "black hole" encouraging people to abandon any hope of OCR in the future. We'd really like to be able to fax these documents electronically, but electronic fax servers create this dithered mess out of the various shades of gray picked up from the white background.

How can I handle this? Am I out of line? I have talked to other vendors that consider this heresy, so I'm pretty sure I'm on the right side.

Parting ways is not an option at the moment.

posted by anonymous to computers & internet (13 comments total)

Parting ways is not an option at the moment.

This may be true, but if you tell them "I have spoken to other vendors who would love the business and I need you to work with me on this or we'll have to part ways in a few months," it is probably the only way to get them to reconsider. After all, if you're still paying the bills, aren't going anywhere, and this is the easiest way for them to do it, why would they change?

(Also, yes: this is an unbelievably shitty way to handle electronic documents.)
posted by Optimus Chyme at 6:58 AM on May 20

Parting ways is not an option at the moment.

Why not? This program sounds like garbage.

My guess is that they have these sizes and settings hard-coded all over the place in their code and that there's no real hope of changing without a rewrite. Why would they be doing this is they had the resources to fix it?

With the price of storage space getting so low (1TB hard drives sell for $75 on newegg these days) storing documents this way is crazy. Frankly, these guys sound like a fly-by-night operation.

Oh and by the way, I don't know if any commercial vendors use it yet but DjVu is a new open source standard for storing highly compressed images of documents really clearly here's an example
posted by delmoi at 7:18 AM on May 20

i am going to disagree with optimums chyme. unless the op accounts for a significant portion of the vendor's revenue threatening them is likely to do more harm than good. when i worked for a product company, making a polite well thought out feature request was the best way to see it added to the software.
posted by phil at 7:24 AM on May 20

If tey have/want government clients you could show them this.
posted by Pollomacho at 7:26 AM on May 20

One customer telling you that they want a higher-density format is a feature request. Multiple users telling you that, in a public forum, is a development priority.

Is there any kind of user community around this product? Evangelize your issue.
posted by mkultra at 7:33 AM on May 20

Assuming this is like most other enterprise software companies I've encountered: don't talk to support, they only can get pressure on bugs, not on feature requests. For feature requests you need to talk to their sales people. And sales people care about one thing: selling. If they can have your money without having to discuss this with product managers internally, they will. You'll need to drop some heavy hints about essential features that could prevent you from renewing the contract to get some traction.
posted by dhoe at 7:49 AM on May 20 [1 favorite]

I want to explain to them that in a sense their system is a "black hole" encouraging people to abandon any hope of OCR in the future. We'd really like to be able to fax these documents electronically, but electronic fax servers create this dithered mess out of the various shades of gray picked up from the white background.

The requirements for OCR are very different from the requirements for documents that look nice when displayed on a raster screen.

Faxing is the feature request to focus on. Don't worry about the technology under the hood - that is the responsibility of the service provider. Focus on the features you need as a customer, and fight for those.
posted by b1tr0t at 7:50 AM on May 20

If parting ways is not an option, how about not paying the bill until they do it your way? Do you need to talk about this with your lawyer?
posted by caddis at 7:55 AM on May 20

If parting ways is not an option, how about not paying the bill until they do it your way? Do you need to talk about this with your lawyer?

Do not stop paying your bill. You may be unhappy as a customer, but unless the contract specifies things you don't have - "documents shall be printable" or whatever - you're simply an unhappy customer. Your options as an unhappy customer are to get them to add/change features to make you happy, or leave. Ceasing payment is a violation of your contract, most likely, and amounts to a temper tantrum besides.
posted by Tomorrowful at 8:07 AM on May 20

Document management? With JPEGs? At 100DPI? Seriously? Yes, heresy is a polite word for what that is.

> Parting ways is not an option at the moment
> I have talked to other vendors

If you are under contract: wait them out, fire them, and get someone who knows what they're doing. There's no law that says you can't front-load another vendor in parallel and phase the 100DPIers out. Do not threaten them, just fire them.

If it's political and parting ways really isn't an option in the long term, get some attention dedicated to this problem. You have proof that 100DPI JPEGs suck. Show this evidence to someone who makes decisions and get some polite pressure put on your document management vendor and maybe they will start to work with you.

> designing complex algorithms that automatically apply gamma correction, etc
> but electronic fax servers create this dithered mess out of the various shades of gray picked up from the white background.

So...it's broken. It doesn't meet your requirements. They refuse to meet your requirements. Someone else can meet your requirements. Hire them instead.

Seconding the sloppily hardcoded numbers thing. Oh really - they are capable of 'designing complex algorithms' but they can't handle different image sizes? Programmatic image scaling is not rocket science. You could throw a rock at the internet and hit a library and/or SDK that can scale anything to anysize, applying super-cosmic magical Lanczos scaling kabuki magic with fractal sauce on the side while they're at it for good measure. (Crap, I still use StretchBlt in some of my Win32 apps for display.)
posted by ostranenie at 9:24 AM on May 20 [1 favorite]

Standard archival scanning in Canada for most universities is 300dpi. Using 100dpi should be illegal.
posted by blue_beetle at 11:33 AM on May 20

Using JPEG for text should be illegal too. JPEG is simply unsuitable for images with hard light/dark edges like text or line art: the compression introduces artefacts which tend to obscure the text.

100dpi is right out too. What compression settings are they using for the jpegs?
posted by pharm at 11:40 AM on May 20

IMO, the key is whether this would be considered a "feature request" or a "rewrite". Business may listen to feature requests, but if something would essentially require a rewrite, they'll just tell you "sorry, maybe in version 2.0." And a fundamental change like this sounds like a rewrite, sorry.
posted by smackfu at 8:02 PM on May 20