What's the best way to create a 3D model from the "average" shape of real-world objects?

August 27, 2008 10:57 AM

I'm beginning a project where I need to construct a 3D model based off the "average" shape (I'm not sure that's exactly the proper term) of a number of similar real-world objects, and I'm not sure on the best workflow or process. Example: I have 25 bananas and I want to create a 3D model based off the average shape of all these bananas.

There are a few options I've explored, and I'm trying get some advice on the best route to pursue before I'm up to my neck in "data" and find that there's a more efficient way.
- One option is to make 3D models of each object, then find a way to construct an average 3D model based off the source models. I don't have much 3D modeling software experience, so I'm not sure if this is even possible using the software my university has license to (Cinema 4D and Maya.) I am planning to use photo-modeling software (where you construct 3D models from a series of calibrated photos) to create the models, so...
- Another option would be to create an average image for each angle required by the photo-modeling software, but I'm unclear on how to do this. I've seen some imaging research done that constructs an average image based off two source images, but I don't know if this research allows for multiple (maybe...20?) input sources. Also, I've only seen this research while surfing my rss feeds, and I don't know if I saved the article, so I don't have a www to refer to.
- Alternately, instead of using this imaging research, I thought of using image-morphing in After Effects (I know - cheezy 90's movie effects...) to find a mid-point between two images. I'd output that mid-point, run the morph on another two images, output that mid-point morph, and then run a morph on the two previous output images...and repeat the process.
This last option seems most plausible process, but it also seems to be the most time-consuming as well. Are there any options I might not be thinking of? Is there a process that seems most efficient/time-saving than another? Does anyone have experience doing something remotely similar to this? :) Thanks meta.mind!

posted by itchi23 to computers & internet (18 comments total)

There was a project a few years ago where digitized faces for computerized-humans were created by taking the average of 25 real faces. That sounds like what you're trying to do. (Sorry, I don't have anything else for you, such as a link.)
posted by Class Goat at 11:01 AM on August 27, 2008

This really depends on what you are using the models for. If you want to model an "average person", for example, there are a bunch of different methods based on the use. For something like stress testing an elevator, you probably only care about weight and your average person might be a sandbag that weighs the same as the statistically average person. For designing the interior of a car, you would need to know how tall they are, where their arms and legs can reach comfortably, etc.

My method would be to select specific traits that are important, measure them quantitatively, and create a composite based on those measurements. For bananas, that might mean scanning the colors seperately and averaging them together, measuring the widths and heights and averaging those, measuing the angle of the curves and averaging those, etc. It's much easier to focus on one property at a time rather than trying to do the whole thing at once. Once you have all of the measurements, creating the average model is just an exercise of working within those specs.
posted by burnmp3s at 11:08 AM on August 27, 2008

What are you going to do with the result of this? Your final bullet there about After Effects would seem to indicate that you don't really need a 3D model.
posted by XMLicious at 11:13 AM on August 27, 2008

---
My method would be to select specific traits that are important, measure them quantitatively, and create a composite based on those measurements. For bananas, that might mean scanning the colors seperately and averaging them together, measuring the widths and heights and averaging those, measuing the angle of the curves and averaging those, etc. It's much easier to focus on one property at a time rather than trying to do the whole thing at once. Once you have all of the measurements, creating the average model is just an exercise of working within those specs.

Well, the actual shape and shape relationships are the focus of this project, and it would be impractical to generate quantitative averages due to the source material (unfortunately, I can't really say what the source material is...university research and sensitive-nature kind of issues.) Thanks though!
posted by itchi23 at 11:21 AM on August 27, 2008

What are you going to do with the result of this? Your final bullet there about After Effects would seem to indicate that you don't really need a 3D model.

After Effects may be incorporated in later phases of project, but right now having the 3D models is necessary because a rapid prototype/3D print of the averaged object is part of the research scope.
posted by itchi23 at 11:23 AM on August 27, 2008

I agree with burnmp3s. The kind of 'average' you take depends on what you're going to do with the averaged data after you get it.

One other technical method you haven't mentioned is a 3D scanner. They are expensive, but if you are doing this project at a university, you may be able to get access to one. Most of those types of systems will generate a mesh or point cloud. If you had a way to do point correspondence between several scans, you could get an average by averaging each set of corresponding points. However, this would quickly fall apart with complicated objects without some serious computational geometry.
posted by demiurge at 11:24 AM on August 27, 2008

As people have said, this really depends on your definition of "average". And I don't actually know that the after-effects morph is going to please you.

This, more than likely, requires deep computational geometry. I don't think we can really point you along the right path until you've developed your definition of "average".

Keep in mind that if you go with a simple geometric mean, that your "average banana" will include remnants of any outliers--bumps, folds, size irregularity, etc.
posted by Netzapper at 11:55 AM on August 27, 2008

Another thing is, does the rapid prototype thing need to look like one of the original objects? Depending on how the shape of the original objects vary, a computed average of them might not actually look correct. For example, if you averaged the shapes of a group of bananas with widely varying curvatures I would expect that you'd get a shape with weird flanges on the ends that wouldn't really look like a banana.
posted by XMLicious at 11:58 AM on August 27, 2008

As said above, it really depends on what you are modeling. Your banana example is deceptively simple since it looks like a shape that could be described parametrically.

If your objects are equally simple, you could try fitting some sort of parametric geometric shape (like a superquadric) to each and then taking an average of the parameters.
There's a good deal of literature on such fitters out there.
posted by NormieP at 1:39 PM on August 27, 2008

I don't know if this works for you, but the approach that comes to my mind is to average the images first, and then create the model from the blended image. So load up 25* photos of bananas or whatever in to your favorite image manipulation program, and tweak away. When you have a good "average" representation, then create the 3D model.

Of course this won't work if the point of the project is to average various 3D models, but if the objective is to have a representative (if not perfect) model, it should work pretty well.

*simplified for readability, you'd probably need 4-6 photos per object, so if it was 10 objects, you'd need 40-60 calibrated photos.
posted by forforf at 4:06 PM on August 27, 2008

Hmmm - all of these are really good points. The banana was a too simple example. I suppose I can say the objects are complex shapes - let's say a specific animal part taken from a variety of samples. For example - let's take a human's right femur. Obviously there'd be a variance in sizes, structure, but overall retaining a similar shape and scale throughout all the samples. For the purposes of this project, all abnormal physicality (in the femur example, stuff like bone spurs, fracture scarring, etc.) would be eliminated from the sample pool.
A 3D scan is a possibility, but might not work for practical reasons surrounding this project.
Any additional suggestions are welcomed.
posted by itchi23 at 4:23 PM on August 27, 2008

Even with a 3D scan, you would have to register the datasets together. This is actually sounding a bit similar to a project I've talked about. Say you had 10 CT scans of different people's right femur. You would need some kind of deformable 3D registration technique to correlate the data from each of the scans before you even try to average it correctly. This kind of thing is still an open research problem. I'm not an expert in this field, but you can take a look at 3-D Deformable Registration Using a Statistical Atlas with Applications in Medicine for voxel based stuff or you could go for geometrical correspondence: search for "3D Shape correspondence" on Google. Good luck.
posted by demiurge at 6:11 PM on August 27, 2008

itchi23, here's the first hit on google for "shape interpolation", it shows three heads being "averaged".

http://www.cad.zju.edu.cn/home/xudong/Projects/mesh_morphing/main.htm

This is a huge research area (many siggraph papers etc) & there is no "correct" answer just a bunch of different answers to different questions. Fun stuff to read though and the papers all generally have neat pictures.

If you need a 3D model then averaging the images and then trying to produce a 3D model is not the first thing I'd try. Generally image-based morphing techniques are in no way designed to produce a set of different images that still describe a consistent 3D object.

Without more specificity about your problem there exists a really large range of solutions.
posted by Wood at 6:13 PM on August 27, 2008

If you have matched models, you could try using PCA? That's what we use in 2D to model variation in shape (and therefore averages, after a fashion).

So, in the 2d case, say you use 15 points to define a banana, and you can then align these points (so point1 on model1 corresponds to point2 on model2), then you can do statistics on the matched points. If you do modelising, alignment, then PCA, it's called a Point distribution model (invented by Cootes and Taylor). Another thing you might want to google is ASM (active shape model).
posted by handee at 11:46 PM on August 27, 2008

One problem you're going to run into with morphing the 3d models, is that (at least in the apps I use) both 'ends' of the morph need to have the exact same topology for the morph to work. This could be a pain using any 'generated' 3d models, as their topology would probably vary from model to model.
I'd define a standard set of feature points to mark on each 3d object (if it was a human: the tip of the nose, the left elbow, etc.), then use a 3d arm-digitizer, import the point cloud into Maya and then average the positions for each corresponding feature of points (using one of the features as the reference for the rest) using MEL or (preferably) PyMEL, then create a mesh from that.
If you don't have a digitizer, you can take a set of photographs and use that, though it's more of a pain.
posted by signal at 10:45 AM on August 28, 2008

signal, the topology problems you describe are exactly what I would expect, but did you see Wood's link? If that's completely automated it seems incredible. (So I'm wondering what the phrase "with proper boundary conditions," means exactly.)
posted by XMLicious at 11:07 AM on August 28, 2008

Wood's link is, in fact, incredible, especially if, as it says in figure 5, the models aren't aligned and can be rotated, and the topologies are different (which it doesn't say), how does it know that an ear becomes an ear and not an eye?
posted by signal at 11:29 AM on August 28, 2008

In the paper by Xu et al., they assume that a correspondence map has been constructed prior to their technique being applied. They give some references, but they are too complicated for me to understand with a cursory glance. You could try to read those or talk to an expert in surface correspondence (surely around every corner).
posted by demiurge at 9:59 AM on August 29, 2008