How does self-checkout work?

February 17, 2008 11:12 AM

When I go through self-checkout at HEB grocery store and forget the code for a vegetable, I can press identify and the machine can tell what vegetable it is. I'm curious what technology is used to do this. I don't think it is just someone nearby pressing a code, because I've watched and no one is looking my way. Thanks!
posted by meta87 to shopping (10 comments total)

Here's IBM's system for doing just that. It uses a camera and the scale.

The machine evaluates colour, texture and shape, then weighs and prices the purchase.
posted by chrisamiller at 11:21 AM on February 17, 2008

I don't know what's specifically going on in your grocery store. This article - from back in 2001 - suggests automated visual identification systems were in development, as well as that the checkouts could be remotely supervised via camera - so you might have a human assist from someone in an office you can't see. I saw one other bit about a system that analyzed factors like other purchases and average weight and selected most likely options, but I think that was still just a patent and it sounds like you're getting a definitive selection. I bet if you asked a produce manager on the floor they would know if you want the correct answer.
posted by nanojath at 11:24 AM on February 17, 2008

There's a whole field of study focusing on image processing and computer vision (the former in Electrical Engineering and the latter in Computer Science typically). It covers all of the intricacies of modeling the human eye and brain in recognizing shapes, colors, various intensities, edges, etc. in software, and they're able to do some amazing stuff. In fact, a problem such as this (where you can actually train the computer to recognize distinct fruit) is not that difficult, relatively speaking.
posted by spiderskull at 11:29 AM on February 17, 2008

I figured the camera fed the live image back to the self-checkout manager person that sits at the nearby station. They eyeball the fruit or vegetable, and insert the correct code manually. Guess not?
posted by nitsuj at 11:56 AM on February 17, 2008

I used to work at a grocery store and often worked the self checkout. At our store, the customer would hit a button that said produce, which would make our screen flash, notifying us.

There were video cameras over each station's scale, and a monitor at our workstation showed the video from all of the stations. So once my screen started flashing, I could look at the monitor to see what kind of produce it was, and punch in the code.

From the customer's point of view, it would seem that I wasn't looking at their items, but in reality I was, via the monitor. Of course, sometimes you couldn't tell from the monitor and would have to ask the customer what it was.
posted by saucy at 11:58 AM on February 17, 2008

Something like this claims to be able to do it autonomously.

The features of the object are extracted using color, texture, size, shape, and density by the camera/scales. These pictures are then run through a series of processes which separate the important bits from the background noise (see picture on page 3).

Flexible, trainable systems like this (speech, handwriting, face recognition) are manufactured to some level of competency and then often use some variation of neural nets to supplement their behavior regarding new things or presentations. These are basically a series of layered "nodes" whose connection weights can be altered according to various algorithms as they get feedback, and are good at modeling pattern recognition functions of the brain that are difficult to hard-code.

This is just my best guess though, I don't have first hand experience.
posted by sophist at 12:04 PM on February 17, 2008

Every time I pass the monitor staff for the self checkout they are looking at a monitor on the camera's watching each self check out station. They are the ones okaying the final checkout motions and what not, and I assume this is it as well.
posted by stresstwig at 1:11 PM on February 17, 2008

Does anyone actually know if an automated system like the ones described by sophist, spiderskull etc are in place in a store (not just an article or research paper)?
posted by neustile at 2:49 PM on February 17, 2008

If the store had such an automated system, why would you ever have to type in a code? That is, why wouldn't the self-checkout be set up to always recognize your produce? I think this argues for them currently using a camera and human being approach.

(It's possible the automated system is not very accurate and so it's used only as a "last resort", but I'm not sure any supermarket would pay for a system like that.)
posted by metric space at 3:18 PM on February 17, 2008

There may be an RFID in the sticker on the produce.
posted by ikkyu2 at 10:27 AM on February 18, 2008