Flatbed scanners and dynamic range for sketching

January 28, 2010 5:41 PM

Why do flatbed scanners seem to have so little dynamic range for sketches? I draw for fun, usually with a 0.5mm mechanical pencil. I do not put a lot of pressure onto the pen because it allows for more details/subtle shading. When I scan the images, the "light" details are always blown out and no amount of fiddling xsane's settings (gamma, brightness, contrast) seems to reveal them. Is there a trick I'm missing, or paper+scanner just really sucks and I should just use my tablet? Example scanned drawing lacking much of the visual information that the eye can see on paper. Scanners I have used over the years: HP ScanJet 3300C, Canon lide (briefly), and now a Brother MFC-5840CN.
posted by a007r to Media & Arts (22 answers total)

So to clarify, you have tried turning the brightness of the scanner bulb down? That usually helps a lot. Also, scanning at a higher DPI can help you get more detail in your images.
posted by DMan at 6:13 PM on January 28, 2010

Response by poster: Well I have no idea if this actually lowers the intensity of the light being emitted by the scanner, but I did try changing the brightness setting in xsane to various levels with no difference that I could see. You made me curious, so I just tested by opening the lid and looking at the scanning "light" cathode thingy (which is green instead of white), and I saw no difference in the physical intensity of the light between full brightness (+100.0) and minimum (-100.0).

I'm scanning at 300 DPI, but 150 DPI for doing the quick tests regarding this issue (because it's faster).
posted by a007r at 6:30 PM on January 28, 2010

Why was the image in "indexed" color instead of rgb when I looked at it? shouldn't it be scanned in rgb? have you tried scanning it as a photo? have you tried, in photoshop, going to adjustment>levels, taking the black eyedroper and clicking on the point in the image that you want to be the darkest?
posted by cda at 6:38 PM on January 28, 2010

I've had much better luck getting detail in drawings by taking a photograph rather than scanning. You just pin or tape the paper to an evenly lit surface and use a camera mounted on a tripod (or some stable surface). If you shoot a little dark then gently adjust later in Photoshop (or GIMP, etc.), you can even get some of the paper texture if you want it.

Your sample scan actually looks pretty easily correctable though, as cda says. (Unless there's additional detail that's getting completely blown out.)
posted by Fifi Firefox at 6:44 PM on January 28, 2010

Best answer: it gets blown out because graphite has a highly reflective sheen. you can either work in charcoal or some other less reflective medium or more likely find something matte to put down over the graphite. I know there's a matte spray fixative available but no real idea who makes it. your helpful art store nerd would know pretty readily.
posted by patricking at 6:55 PM on January 28, 2010

Response by poster: > "why was the image in "indexed" color instead of rgb when I looked at it? shouldn't it be scanned in rgb?"

Oh yeah, sorry, that was the "web version". It was scanned in full 24 bit color then converted to grayscale, and then exported to the indexed color version for the web. It's visually the same though. A colorimetric analysis (by GIMP) onto the grayscale version (not indexed) says it has 165 colors, so not much was lost there.

@Fifi Firefox: The photography idea makes sense I guess, it just isn't very easy though (compared to, say flunking a paper in a scanner). You have to have a tripod set-up and near-perfect lighting. I may give it a try, but I foresee that I will fail at the "create an even lighting" part.
But yes, there's a lot of data blown out that you don't see in the image. For example, in the eyes, around the pupil, there was an iris made out of a lighter gradient of gray, and it's invisible on the scanned image.

@patricking: interesting, I had not thought of that! Graphite being reflective could be part of the problem, combined with, perhaps, scanners that have light cathodes that are too bright, and perhaps that consumer scanners are generally designed for photos and text rather than fine details...
posted by a007r at 7:16 PM on January 28, 2010

By default, my epson 3170 scanner likes to clip the lightest and darkest tones, forcing them to white and black. To get the full range I have to make sure it's not doing this.

Also, try scanning at higher resolution, perhaps even at the native resolution of the scanner if necessary. Then scale down in photoshop. It might do a better job downscaling than the scanner.

No experience, but I'm thinking that Brother might not be the best scanner around...
posted by DarkForest at 7:31 PM on January 28, 2010

Since you're using SANE already...could you write a script to take a low brightness scan A, then take a high brightness scan B, then merge them in GIMP?
posted by miyabo at 7:36 PM on January 28, 2010

Response by poster: Okay, so I gave a quick shot (pun intended ;) at the photography approach.

- Original (for posterity) as seen by the scanner
- Photographed version: as you can see, there are many, many more details in shading that were completely ignored by the scanner. However, lighting is tricky, and the result is much less clean that what a scanner produces. Advice welcome! I did not use a flash, just a tripod and a halogen lamp, then converted to grayscale and adjusted the levels a little bit so that it could be compared (without losing important details).

I'm of course curious about other ideas/hypotheses you might have!
posted by a007r at 7:36 PM on January 28, 2010

Response by poster: > "No experience, but I'm thinking that Brother might not be the best scanner around..."

Thing is, I have noticed this on all the scanners I had, I think my HP Scanjet 3300C would exhibit the same symptoms.

> "could you write a script to take a low brightness scan A, then take a high brightness scan B, then merge them in GIMP?"

Well if adjusting the brightess actually made light details appear, I wouldn't even be having this problem, I'd just scan in lower brightness :) but it doesn't. The settings don't seem to affect the hardware, they just seem to be post-processing added on top by SANE or something (at least that's my hypothesis at the moment).
posted by a007r at 7:39 PM on January 28, 2010

You are asking a lot from the scanner. It would be easier and much cheaper to change pencils than scanners. A 0.5mm pencil is actually a poor sketching tool. Use too soft of a lead and you end up breaking it every 2 seconds. Too hard of a lead and you get faint lines. Good sketching requires at least an HB and a much thicker lead.
posted by JJ86 at 7:44 PM on January 28, 2010

After a quick look at xsane, check your threshold settings to make sure you aren't clipping lighter tones, and make sure you aren't using the auto-enhance function.
posted by DarkForest at 8:03 PM on January 28, 2010

Best answer: Your photographed version shows way, way more of the paper's texture than your scanned version does. Consumer-grade scanner optics, by and large, are designed to make white paper scan as a plain white background. They do this by deliberately over-illuminating their sensors slightly, specifically so that they don't pick up the paper texture. As a result, there is actually very little that xsane or any other post-processing tool can do; the information you want enhanced is simply not there in the data sent by the scanner.

Most scanners also don't have any way for software to change their lamp's illumination level. However, you can do it optically by putting a filter sheet between your drawing and the scanner glass, darkening the whole thing enough to stop the scanner's sensors from saturating in near-white regions. That will get you a wider range of useful post-processing options. You can even play with cellophane in various colors and undo the resulting tint in post-processing, if you want to try this out on the cheap.

But you may well find that your drawings really don't have enough contrast range to let you process out the paper texture while keeping the drawing detail you're after. Human vision is very, very context-sensitive; drawings that look perfectly fine on paper might look like arse after scanning, simply because your visual system is expecting paper to appear more textured than a screen. And if that's the case, the only way you'll ever get the results you want is by changing your drawing technique and/or tools.
posted by flabdablet at 9:46 PM on January 28, 2010 [2 favorites]

As a comic book writer and friend of a few comic book artists I'll give you a hint. Transferring your work from a drawn page to a digital version best takes place with one of two things:

1. A tablet (such as a Wacom) that has an overlay. You put your drawn artwork on the tablet and draw over it with a stylus. The actual software that you use, Photoshop (expensive) or GIMP (free) has settings for various things like pressure and pen nibs and a whole rainbow of effects.

2. Or have it done at a Kinkos or the like. They will reproduce it there in what's called halftone. And the repro is usually very inexpensive. Especially for a single drawing. And the quality is better than any home scanner. They will give you a digital copy with the print if you bring a flash drive or burn it to a disk for a small fee or even for free in some cases.

Good luck with your fine art!
posted by Splunge at 11:21 PM on January 28, 2010

I get a lot more detail setting a scan to grayscale
posted by psychobum at 12:48 AM on January 29, 2010

Best answer: I scan a lot of drawings in my sketchbook. Since as they're from my sketchbook, the vast majority are done with crappy .5mm mechanical pencils, like your drawings. My typical results look much more like this. As you can see, the shading's definitely there. Your washed-out scan really looks like the result of some sort of software setting, or perhaps the scanner drivers, even, because I know from extensive personal experience that even old, crappy consumer-grade scanners are physically capable of doing better than that. The complicated suggestions for redrawing everything with a Wacom tablet, taking photos (which gets you into lighting and angle issues!), adjusting the scanner lamp and adding filter sheets should not be necessary.

If I were you, I'd start by plugging the scanner into a computer running something other than xsane, or installing a different scanning interface on your computer (if possible): with freeware, I often run into bugs (or weird settings) that just aren't there when I use a given piece of hardware in a Windows or Mac machine. Annoying, I know, but if you can get acceptable results using another program or computer, you at least know it's probably not the hardware of your particular scanner that's at fault.

So, software: I generally scan things in through GIMP or occasionally Photoshop (on XP and OS X.) I've been using Canon scanners recently, but I've used others in the past. I always treat the sketch as a color photo (a pretty standard mode on a lot of scanner software - probably "full range" in xsane?) at 300dpi. (Shift it to greyscale only after it's correctly imported into GIMP; I've run into some crappy greyscale scanner presets that deal badly with pencil drawings.) Beyond that, you want the scan to be as raw as possible, with most (perhaps even all) of the color editing features turned off: in the Canon interface I'm currently using, Auto Tone is on by default, but it can do bad things to pencil drawings. Unsharp Mask is also occasionally problematic, though its effects are generally a lot more subtle than the washout you're seeing. If you're stuck using xsane, dig through all the advanced options and try tweaking things or turning them on or off one by one. A quick look at their documentation makes me think Autoenhancement and Threshhold options have the potential to be at fault?

Good luck; I'm sure you'll be able to get this to work somehow.
posted by ubersturm at 12:59 AM on January 29, 2010

How thick is the paper? With thinner paper you can remove glare by putting something dark (I use a black t-shirt) between the reflective lid of the scanner and the paper I am scanning.

Originally this was a measure to avoid see-through when scanning newspapers. It also works to some point for removing paper glare. One negative side effect, however, is that paper is unevenly translucent, so your background might show up a bit blotchy.
posted by kandinski at 1:00 AM on January 29, 2010

It's internal gamma correction which is blowing the highlights out I think.

In Xsane, bring up the "Standard Options" menu (Window-> tick "Show Standard Options") and set the Gamma Correction entry to "User Defined". This works for me on a test page with graduated H pencial marks ranging from light to hard. With Gamma Correction set to "Default" I get the blown highlights that you complain of, with it set to "User Defined" I get a much fuller image that also shows all the paper imperfections.

Note that this is the 'Standard Options' for the scanner backend I think, not all drivers will offer this. There's also this page from the xsane manual which suggests that some drivers offer a threshold option (mine doesn't appear to) which could also be at the root of your problem.
posted by pharm at 6:19 AM on January 29, 2010

Indeed, reading the manpage for the Epson backend (my scanner is an Epson Perfection 1640SU), it says

The --gamma-correction switch controls the scanner's internal gamma correction. Valid options are "Default", "User defined", "High density printing" "Low density printing" and "High contrast printing".

so in the "Default" mode when doing a greyscale scan, the scanner is applying an internal gamma correction to the output, which looks to me like it's also applying a thresold correction as well. Setting this to "User Defined" prevents the scanner from doing this.

I imagine the default is because most people want to scan high contrast line art: graphs for businesses, cartoons, that kind of thing. They want the background paper to just go away. You want every last detail, so you should choose "User Defined" (if you're using an epson scanner) and go to town...
posted by pharm at 6:33 AM on January 29, 2010

Response by poster: Wow, lots of insightful replies. I love MeFi :) before I start with my long reply, here's my latest experiment: photograph taken in daylight, with a tripod pointed towards the floor, self-timer, ISO 50, and a "flash card" bouncing slow-sync flash (just a folded sheet of plain paper in front of the flash so that it diffuses the light and bounches some of it towards the ceiling).
- the raw photograph
- tweaked with GIMP

It's pretty damn good at picking up all the details, and a small amount of paper texture shows up at the end (ex: lower-right corner), it kinda makes for a vignette effect. Using only natural light and no flash yields pretty similar results:
- raw photograph
- tweaked with GIMP

"Consumer-grade scanner optics, by and large, are designed to make white paper scan as a plain white background. They do this by deliberately over-illuminating their sensors slightly, specifically so that they don't pick up the paper texture. As a result, there is actually very little that xsane or any other post-processing tool can do; the information you want enhanced is simply not there in the data sent by the scanner."

This is what I suspected...

"A tablet (such as a Wacom) that has an overlay."

You mean, a physical overlay onto the tablet itself? Surprising... as far as I know/have experienced, tablets' regions are contextual/do not span the whole screen, and thus you never know where your cursor is going to end up if you don't look at the screen... thus I'd tend to simply trace over the drawing by using layers in GIMP. I had that idea in mind, it's a possibility indeed, but I'm lazy :)

"Or have it done at a Kinkos or the like."

Well if I was a professional this would be interesting, but I don't think my one or two doodles a year warrant the trouble :)

"Your washed-out scan really looks like the result of some sort of software setting, or perhaps the scanner drivers, even, because I know from extensive personal experience that even old, crappy consumer-grade scanners are physically capable of doing better than that"

It may indeed be the Brother-provided "brscan2" Linux drivers (or even, all Brother drivers, not Linux-specific) that cause this, or some other kind of artificial limitation. For the sake of the experiment, I did try to use this scanner on Windows, but I could never get the stupid drivers to install (well I should blame Brother's poor installers). Who said Linux had poor hardware support? ;)

"I always treat the sketch as a color photo (a pretty standard mode on a lot of scanner software - probably "full range" in xsane?) at 300dpi. (Shift it to greyscale only after it's correctly imported into GIMP; I've run into some crappy greyscale scanner presets that deal badly with pencil drawings.)"

Absolutely, I discovered yesterday (before posting here) that scanning in 24 bit colors and converting to grayscale in GIMP revealed more detail than scanning in grayscale (but still not enough). Hypotheses regarding that is that in grayscale, a) the scanner might apply more processing, b) scanning in color just gives you more information ("oversampling" I think).

"How thick is the paper? With thinner paper you can remove glare by putting something dark (I use a black t-shirt) between the reflective lid of the scanner and the paper I am scanning."

It's "plain" paper (well, I don't know paper types much). I gave a shot at putting a dark material between the lid and the paper, no luck (did not try flabdablet's suggestion of putting a filter sheet between the drawing and the sensor/glass, however).

"In Xsane, bring up the "Standard Options" menu (Window-> tick "Show Standard Options") and set the Gamma Correction entry to "User Defined". This works for me on a test page with graduated H pencial marks ranging from light to hard. With Gamma Correction set to "Default" I get the blown highlights that you complain of, with it set to "User Defined" I get a much fuller image that also shows all the paper imperfections.

Note that this is the 'Standard Options' for the scanner backend I think, not all drivers will offer this. There's also this page from the xsane manual which suggests that some drivers offer a threshold option (mine doesn't appear to) which could also be at the root of your problem."

Sadly, I think my scanner(s) are in the "don't offer these options" category. All I get in the "Standard Options" window (and all I ever got) is the SANE logo and no options :(

I guess I should email Brother's Linux tech support to know if their hardware is actually capable of exposing such options.
posted by a007r at 7:48 AM on January 29, 2010

It does look like your scanner is applying some kind of thresholding internally then. Since the Linux drivers you're using are the official Brother ones, it seems likely that this would be the case for the Windows drivers as well.

All the scanners you've tried are cheap consumer grade scanners. It may well be that all scanners in this class apply threshold filters to the data when scanning in greyscale mode: it makes sense from the manufacturer's point of view.

Better scanners let you apply your own gamma correction to the 8-bit conversion when scanning in 8-bit/channel mode, as well as scanning up to 16 bits / channel (albeit not always with full 16bits: I doubt there's much real data in the bottom bit). Your scanner will do 16bits/channel scans: you might well find that if you can scan in higher bit depth that no conversion or thresholding is done on the assumption that the user is going to be doing the post-processing themselves.
posted by pharm at 8:57 AM on January 29, 2010

Fwiw here are a couple of scans I made myself from a few scribbles with an H pencil. User Defined gamma correction and Default gamma correction.

The latter clearly shows the kind of thresholding that you're seeing with your scans.
posted by pharm at 9:16 AM on January 29, 2010