32 bit image conversion to 8 bit image
February 13, 2010 9:58 AM Subscribe
how can i convert a 32bit grayscale image into an 8 bit image? photoshop does not do this precisely.
photoshop allows this, but it is not exact; to get a preview of a 32 bit grayscale image you must fiddle around with the gamma and exposure by hand, then you can convert that result into an 8 bit image. I need something that is procedural since a slight difference in preview settings creates a shade of gray that may differ by many shades. The gray in my case represents a specific height, which is why i need this to be an exact conversion.
I am looking for a program that can do this, or a perhaps an image library of python or java or something easy that i can script.
thanks!
photoshop allows this, but it is not exact; to get a preview of a 32 bit grayscale image you must fiddle around with the gamma and exposure by hand, then you can convert that result into an 8 bit image. I need something that is procedural since a slight difference in preview settings creates a shade of gray that may differ by many shades. The gray in my case represents a specific height, which is why i need this to be an exact conversion.
I am looking for a program that can do this, or a perhaps an image library of python or java or something easy that i can script.
thanks!
Can you just dump the data -- it's an intensity image, right? -- and convert it yourself with a bit of code? Then read it in as raw image data.
posted by whiskeyspider at 10:29 AM on February 13, 2010
posted by whiskeyspider at 10:29 AM on February 13, 2010
ImageJ converts 32 down to 8 bit in a snap, but it's a little opaque to me how it rescales it. You can set the max and minimum manually/numerically in the color menu and then convert it, probably similar to photoshop.
I'm having the exact same issue with some of my data, so if you get a work-around I'll be 256 shades of ecstatic. Namely a lookup table for imageJ that can be bespoken for 32 bit images.
My working conception of 32 bit images is that when you're looking at it on screen you're already seeing a scaled version of it with whatever lookup table the software is applying to it. Visually, you're missing a lot of information that can only be displayed numerically. So you should extract the grey value into pure number form and display it separately in a graph or such.
If this is incorrect I too, would like, nay need, to know.
posted by Cold Lurkey at 10:30 AM on February 13, 2010
I'm having the exact same issue with some of my data, so if you get a work-around I'll be 256 shades of ecstatic. Namely a lookup table for imageJ that can be bespoken for 32 bit images.
My working conception of 32 bit images is that when you're looking at it on screen you're already seeing a scaled version of it with whatever lookup table the software is applying to it. Visually, you're missing a lot of information that can only be displayed numerically. So you should extract the grey value into pure number form and display it separately in a graph or such.
If this is incorrect I too, would like, nay need, to know.
posted by Cold Lurkey at 10:30 AM on February 13, 2010
Python's PIL has somewhat limited support for reading 32-bit images, but once you have them in memory, you can use the ImageMath module to do a controlled conversion.
posted by effbot at 10:41 AM on February 13, 2010
posted by effbot at 10:41 AM on February 13, 2010
Response by poster: thanks for these replies.
@demiurge:
it is a .tif format, however it is also huge (1 gig). does imageMagick or PIL have any problems tifs or with with such huge files?
@whiskeyspider
since it is such big a file (1 gig), i dont want to be doing too many conversions into .raw. Photoshop takes several minutes to do the conversion on my machine. I could crop the image and dump that somewhere, but im not certain how that would help.
@Cold Lurkey
yes! this is what gets me, when a 32 bit image is being displayed, it already does the conversion, and i dont see why programs that can display them can't convert them properly. You are saying that ImageJ has the capability, but only when scripted? (which is what you are working on?) Personally I would rather stay away from playing around with scripting - especially since my images are large and certain functions may therefore have underlying issues with speed and memory management that may be hard to track down if I am unfamiliar with the ins and outs of a particular set of functions of ImageJ or any other scritping - unless someone can tell me straight up that large images are no problem.
posted by figTree at 10:50 AM on February 13, 2010
@demiurge:
it is a .tif format, however it is also huge (1 gig). does imageMagick or PIL have any problems tifs or with with such huge files?
@whiskeyspider
since it is such big a file (1 gig), i dont want to be doing too many conversions into .raw. Photoshop takes several minutes to do the conversion on my machine. I could crop the image and dump that somewhere, but im not certain how that would help.
@Cold Lurkey
yes! this is what gets me, when a 32 bit image is being displayed, it already does the conversion, and i dont see why programs that can display them can't convert them properly. You are saying that ImageJ has the capability, but only when scripted? (which is what you are working on?) Personally I would rather stay away from playing around with scripting - especially since my images are large and certain functions may therefore have underlying issues with speed and memory management that may be hard to track down if I am unfamiliar with the ins and outs of a particular set of functions of ImageJ or any other scritping - unless someone can tell me straight up that large images are no problem.
posted by figTree at 10:50 AM on February 13, 2010
Not free, or cheap, but I'd bet matlab could do this.
Maybe try octave?
posted by kickingtheground at 11:10 AM on February 13, 2010
Maybe try octave?
posted by kickingtheground at 11:10 AM on February 13, 2010
Can you define better what sort of control you need? If you need precise control of the bin size you're using, then this sounds like a job for Matlab (and shouldn't be processor-intensive, if you can define your bins ahead of time and don't need to procedurally derive them from the distribution of values in the entire image).
If you're just looking for the downscaled version you can see on your screen but can't "save as," can you take a TIFF screenshot and use that? There's probably some intermediate spot where you could use ImageMagick or Python or other simple scripting to do a more mindless downsample (eg, throw away the least significant 24 bits).
posted by range at 11:11 AM on February 13, 2010
If you're just looking for the downscaled version you can see on your screen but can't "save as," can you take a TIFF screenshot and use that? There's probably some intermediate spot where you could use ImageMagick or Python or other simple scripting to do a more mindless downsample (eg, throw away the least significant 24 bits).
posted by range at 11:11 AM on February 13, 2010
Response by poster: @kickingtheground alas i dont have matlab:(
@range; Printscreen would work, but i want to keep the same resolution of the image. The "control" i need is the same as a printscreen would provide. im not sure what you mean by downsample with a 24 bit intermediate. I am not looking for ultra exact conversion - in other words 40 or 41 (or whatever a script decides) shades of gray in the 32 bit image can become 1 shade or gray in the 8 bit.
posted by figTree at 11:29 AM on February 13, 2010
@range; Printscreen would work, but i want to keep the same resolution of the image. The "control" i need is the same as a printscreen would provide. im not sure what you mean by downsample with a 24 bit intermediate. I am not looking for ultra exact conversion - in other words 40 or 41 (or whatever a script decides) shades of gray in the 32 bit image can become 1 shade or gray in the 8 bit.
posted by figTree at 11:29 AM on February 13, 2010
yes! this is what gets me, when a 32 bit image is being displayed, it already does the conversion, and i dont see why programs that can display them can't convert them properly.
That doesn't mean that they are converting them in a completely linear way. I just don't think there's a way you can guarantee that a conversion you're doing off the screen is going to preserve the relationship between the binary values in the image file.
What kind of analysis do you need to do on the image? I wonder if Pixcavator would handle this file properly
posted by delmoi at 11:38 AM on February 13, 2010
That doesn't mean that they are converting them in a completely linear way. I just don't think there's a way you can guarantee that a conversion you're doing off the screen is going to preserve the relationship between the binary values in the image file.
What kind of analysis do you need to do on the image? I wonder if Pixcavator would handle this file properly
posted by delmoi at 11:38 AM on February 13, 2010
My sense is that it's basically a question of whether you're trying to preserve numerical data or perceived brightness. If you're trying to retain the integrity of the underlying data, then it's pretty simple, in that you've got more precision than you want, and you just throw away the extra significant digits, so a single pixel's data record of 0xF1 A2 00 3B would get truncated to 0xF1 -- this is not quite rounding, but more like applying a floor() function. This is really easy in most scripting languages, and "real" rounding is probably not much harder.
If you're trying to preserve something closer to human perception, which may or may not be what your viewers are doing, then you may need to apply a gamma or some other weighting so that your bins more accurately represent how we perceive brightness (this is outside my direct experience, but I could easily imagine a scenario where you'd want to take all your 32-bit values that start with FF through F0 and turn them into the 8-bit value FF, then take E9-EF and call it FE, and so on, using the compression at one end to open up a wider range at the other end). This would take more complicated scripting, but Matlab could definitely do it and I'm surprised if it's not doable in Python.
posted by range at 11:45 AM on February 13, 2010
If you're trying to preserve something closer to human perception, which may or may not be what your viewers are doing, then you may need to apply a gamma or some other weighting so that your bins more accurately represent how we perceive brightness (this is outside my direct experience, but I could easily imagine a scenario where you'd want to take all your 32-bit values that start with FF through F0 and turn them into the 8-bit value FF, then take E9-EF and call it FE, and so on, using the compression at one end to open up a wider range at the other end). This would take more complicated scripting, but Matlab could definitely do it and I'm surprised if it's not doable in Python.
posted by range at 11:45 AM on February 13, 2010
I think this sounds like a perfect job for GDAL. Using gdal_translate in combination with a VRT, you should be able to achieve what you are looking for. You can get pre-built GDAL software for Windows from OSGeo4W or Mac from KyngChaos.
posted by hobu at 12:23 PM on February 13, 2010
posted by hobu at 12:23 PM on February 13, 2010
Octave is a free Matlab clone, and is completely compatible with Matlab code with the caveat that the more complex Matlab libraries are not implemented. You could easily write a simple script to load the image, perform your processing, and save it back out as tiff.
However, I'm not sure if Octave will choke on 1G(!) of data! Honestly though, it would take you all of 10 minutes to find out.
If you told me exactly what you wanted to do (your explanation at the top is unclear to me,) then I'd be happy to write this for you.
posted by no1hatchling at 2:30 PM on February 13, 2010
However, I'm not sure if Octave will choke on 1G(!) of data! Honestly though, it would take you all of 10 minutes to find out.
If you told me exactly what you wanted to do (your explanation at the top is unclear to me,) then I'd be happy to write this for you.
posted by no1hatchling at 2:30 PM on February 13, 2010
If you had access to a truly floating-point compositing app like Nuke you could do this in 2 seconds. Alternatively you might be able to save the .tff as an hdr and use HDRshop to convert it.
posted by jnnla at 2:57 PM on February 13, 2010
posted by jnnla at 2:57 PM on February 13, 2010
OP: rom experience ImageMagick should be able to deal with 1G images, but depending on the filter settings it may need to load the entire data in virtual memory during processing; so ensure you have enough swap.
posted by knz at 5:16 PM on February 13, 2010
posted by knz at 5:16 PM on February 13, 2010
Response by poster: thanks for the replies!
well, ive had a chance to test out ImageMagick - result: it does not have the capacity to deal with 32bit images.
some of the other comments here are a bit more technical than i was expecting. to me converting a 32 bit image into 8 bit image is basically a matter of rounding taking data with more than many groups (up to whatever a 32bit image can handle), and reclassifying it into 255 such that the result visually looks the same, but contains less exact information. Im not sure how preserving numerical versus perceived value fits into this operation? Would any of you recommend any particular excellent reading (book or online) that would introduce some of these concepts that I have apparently not fully thought through?
@jnna - I know what a floating point is, but what is a floating-point composition app? are most image processing software of this category?
@hobu - GDAL looks very powerful and capable! unfortunatly it seems very complex to setup with the correct versions, and would require a lot of time for me to figure out how you would even know that you need a certain module in combination with VRT, and in addition there is no python examples (the language im most comfortable in). This is a great backup for if all else fails though, thanks for this. In fact the image i am working with is geographical information, so it is natural that GDAL would work for this purpose. Im still unsure looking at the GDAL homepage if it is included in some opensource GIS programs like Quantum of GRASS?
@no1hatchling - Octave looks promising, i have worked with Matlab before, so i think i will try out this option before looking at GDAL. Thanks for your offer, i may take you up on it if i need some help setting up a script.
Who would have ever thought that such a seemingly easy thing would be so complex!?
posted by figTree at 7:31 PM on February 13, 2010
well, ive had a chance to test out ImageMagick - result: it does not have the capacity to deal with 32bit images.
some of the other comments here are a bit more technical than i was expecting. to me converting a 32 bit image into 8 bit image is basically a matter of rounding taking data with more than many groups (up to whatever a 32bit image can handle), and reclassifying it into 255 such that the result visually looks the same, but contains less exact information. Im not sure how preserving numerical versus perceived value fits into this operation? Would any of you recommend any particular excellent reading (book or online) that would introduce some of these concepts that I have apparently not fully thought through?
@jnna - I know what a floating point is, but what is a floating-point composition app? are most image processing software of this category?
@hobu - GDAL looks very powerful and capable! unfortunatly it seems very complex to setup with the correct versions, and would require a lot of time for me to figure out how you would even know that you need a certain module in combination with VRT, and in addition there is no python examples (the language im most comfortable in). This is a great backup for if all else fails though, thanks for this. In fact the image i am working with is geographical information, so it is natural that GDAL would work for this purpose. Im still unsure looking at the GDAL homepage if it is included in some opensource GIS programs like Quantum of GRASS?
@no1hatchling - Octave looks promising, i have worked with Matlab before, so i think i will try out this option before looking at GDAL. Thanks for your offer, i may take you up on it if i need some help setting up a script.
Who would have ever thought that such a seemingly easy thing would be so complex!?
posted by figTree at 7:31 PM on February 13, 2010
It is an extremely simple operation, but the problem here is file formats. We have a saying at our lab, the most difficult things are cables and file formats. Perhaps if you cannot use PIL directly, you can find a utility that converts from tif to raw 32-bit ints and then load them into Numpy and convert them that way, like whiskeyspider said. Believe me, if know how to write a program to do it, it might be easier to just give up and write it than trying 10 different apps that don't work. Unfortunately.
posted by demiurge at 8:16 PM on February 13, 2010
posted by demiurge at 8:16 PM on February 13, 2010
Response by poster: demiurge: good to note. indeed i have the capacity to do this, and have taken this approach in the past in other problems i have run into with rasters. This time i figured i might hammer it out without converting the raster to a txt format. Im not sure how well numpy will work witha 2 gig txt file....guess ill try and see.
I do scripting, but am not a computer programmer. Can you point me to some website, or tell me a few terms i can look up that explains exactly why converting 32bit to 8 bit directly is so much more tricky than 32 bit to txt to 8bit? I feel like i am missing out.
posted by figTree at 8:51 PM on February 13, 2010
I do scripting, but am not a computer programmer. Can you point me to some website, or tell me a few terms i can look up that explains exactly why converting 32bit to 8 bit directly is so much more tricky than 32 bit to txt to 8bit? I feel like i am missing out.
posted by figTree at 8:51 PM on February 13, 2010
Don't do a text file, do a binary file. This is often called a raw file (not to be confused with the RAW format that is used in photo processing).
I'm not suggesting that going through an intermediate format is better, it just might be more convenient with the skills you have.
Anyway, it looks like PIL supports loading 32-bit tiffs. I just made one and loaded it using the 'I' color mode, which is for 32-bit integers. You can convert it to 8-bit ('L' mode). Try:
PIL doesn't have a problem with large files, but you may need a large memory machine to do it, since it doesn't do any smart memory management.
Good luck.
posted by demiurge at 10:26 AM on February 14, 2010
I'm not suggesting that going through an intermediate format is better, it just might be more convenient with the skills you have.
Anyway, it looks like PIL supports loading 32-bit tiffs. I just made one and loaded it using the 'I' color mode, which is for 32-bit integers. You can convert it to 8-bit ('L' mode). Try:
from PIL import Image
i = Image.load("your32bit.tif")
j = i.convert('L')
j.save("your8bit.tif")
PIL doesn't have a problem with large files, but you may need a large memory machine to do it, since it doesn't do any smart memory management.
Good luck.
posted by demiurge at 10:26 AM on February 14, 2010
Response by poster: well, strike 1 was Imagemagick.
strike 2 is PIL.
While the code works and all (thanks demiurge!) the result is a completely white image with a bit of a black border on the right and bottom of my image (?). Why? well, i certainly dont know:P Looks like GDAL is next
posted by figTree at 1:09 PM on February 15, 2010
strike 2 is PIL.
While the code works and all (thanks demiurge!) the result is a completely white image with a bit of a black border on the right and bottom of my image (?). Why? well, i certainly dont know:P Looks like GDAL is next
posted by figTree at 1:09 PM on February 15, 2010
Response by poster: Just reread everything, one thing i have not tryed with PIL is using the Imagemath module as effbot suggested. I guess ill be doing that and GDAL, see what works and post back later
posted by figTree at 2:01 PM on February 15, 2010
posted by figTree at 2:01 PM on February 15, 2010
This thread is closed to new comments.
posted by demiurge at 10:14 AM on February 13, 2010