How do I catch the eye of a self-driving car?
May 26, 2022 6:40 AM   Subscribe

It is common for pedestrians and drivers to catch each others' eyes to communicate intent and proceed safely. How is this expected to work with self-driving cars?

When I'm approaching a crosswalk on foot and see a car approaching at low speed, it is sometimes not clear whether the driver intends to stop and allow me to cross safely. When that happens, I try to make eye contact with the driver, so we can signal intent to each other. I do the corresponding when I am a driver, interacting with pedestrians and cyclists.

I don't know exactly how intent is communicated in these micro-interactions, but it seems to work very well and allow safe interactions between pedestrians and cars.

Have people figured out how this will work with driverless autonomous vehicles? How will the vehicle let you know, "you can proceed; I'll wait for my turn."

Are flashing headless (or lack of flashing headlights) sufficient? Are there standards developing among autonomous vehicle designers?
posted by Winnie the Proust to Travel & Transportation (28 answers total) 4 users marked this as a favorite
 
In theory, autonomous cars are going to actually obey the law. This means yielding to pedestrians automatically, with no need to make eye contact with anybody; you don't need to signal "I will let you go" because they'll always let you go.

In thoery.
posted by Tomorrowful at 7:12 AM on May 26, 2022 [8 favorites]


Hey! Something I can answer! So I never did anything with self-driving cars but I did do quite a bit with computer vision in advertising. Your question is valid and an interesting one but to answer it simply: don't do anything different.

A lot of effort is put into figuring out your intent, including things you're not aware of. Even with humans the eye contact thing is probably just more of a personal confirmation that you're interacting with someone more than actually signalling intent. As in there is body language and all sorts of things you're signaling that you're not aware of and you may think your eye contact is the first step but it isn't. Think of it this way, you know how a dog comes up to you and you can tell if it is generally being aggressive or not then it "signals" you that it isn't? Somehow you know the dog isn't aggressive or not, generally, and we can drill down into what "knowing" means but it isn't important at least to this question. Doing things differently will throw it off, so just act naturally.
posted by geoff. at 7:15 AM on May 26, 2022 [3 favorites]


In theory, autonomous cars are going to actually obey the law. This means yielding to pedestrians automatically, with no need to make eye contact with anybody; you don't need to signal "I will let you go" because they'll always let you go.

But how do they know you want to cross? The differences between people standing on the sidewalk because they want to cross the street vs. because they're waiting for a bus or smoking/vaping or waiting for someone or sending a text are pretty subtle. Is it going to stop every time there's a person nearby standing still and waiting -- or for that matter a person nearby walking towards the sidewalk edge (i.e. a person walking along a street will cross every road that intersects with that street. As a driver, you see a person walking down the street and not turning their head or aiming their body like they're going to turn, you know they're going to cross and you stop. Can a car read those cues?) Will a car know that children are less predictable and if there is a child you need to err on the side of slowing down or stopping even more (e.g. don't pass closely behind because kids will just randomly change direction and go back).

I had never thought about this question of self-driving cars and pedestrians before, but now that I saw this question it really worries me. I don't think computers are smart enough to read human intent reliably.
posted by If only I had a penguin... at 7:30 AM on May 26, 2022 [7 favorites]


"The differences between people standing on the sidewalk because they want to cross the street vs. because they're waiting for a bus or smoking/vaping or waiting for someone or sending a text are pretty subtle"

Well they will be programmed to obey the law, so you'd not expect them to stop for a courtesy road crossing, you'd be expected to work around the cars. At a crosswalk, however, if you are waiting the car should recognise someone waiting in that area and stop for them.

Crossing roads will have to be more overt - pedestrian crossings that get activated (which there already are a lot of) or designated painted areas for overt intent. But autonomous cars can't allow for anyone that just wants to cross the road wherever, it will have to b at designated existing infrastructure. Think of it as the same as a train/airport shuttle - they won't stop between stations, but they will stop for you and open the doors for you at the stations. The difference is that the car will (presumably/hopefully) have sensors capable of allowing for and reacting to people 'not following the rules' between crosswalks.

The only way the car can know you want to cross is if you are standing at a crosswalk and/or have pressed a button. The only way you can know if an autonomous car is stopping for the crosswalk is if it comes to a halt. Perhaps they could put 4 way blinkers/indicators on to signify they are stopping?
posted by Brockles at 7:47 AM on May 26, 2022


Well they will be programmed to obey the law, so you'd not expect them to stop for a courtesy road crossing, you'd be expected to work around the cars. At a crosswalk, however, if you are waiting the car should recognise someone waiting in that area and stop for them.

Isn't it the law that if someone is waiting to cross the street the cars have to stop? And turning cars have to yield to pedestrians which means the car has to know who is wanting to be a pedestrian. Since almost all intersections are crosswalks, I think we're back to "how is a driverless car going to figure out somebody wants to cross" (vs. somebody is standing around for some other reason, or walking towards an intersection but planning to turn).

Maybe you could program them to see the sidewalk cut-outs and know that's a place where people might cross, but A) THese are often blocked by things like leaves or snow and B) they're often visible quite late due to parked cars near the intersection blocking the view of that little piece of sidewalk and C) You still have the problem of recognizing intent: People also wait (for buses, other people, their pooping dog etc. etc.) at intersections.
posted by If only I had a penguin... at 8:13 AM on May 26, 2022


And then there are the ancient crosswalk stripes that should have been repainted 10 years ago, but never were...
posted by amtho at 8:34 AM on May 26, 2022 [1 favorite]


Response by poster: Penguin raises the important question of how a driverless car will detect and understand the intention of a pedestrian.

My question also relates to the follow-on: how will a driverless car communicate to a pedestrian the fact that "I see you and will stop for you". We'll need standard UX for that if driverless cars and pedestrians are to interact smoothly.
posted by Winnie the Proust at 8:38 AM on May 26, 2022 [1 favorite]


just dont worry about it because theyll either be vaporware forever, or their deployment will depend on totally restricting your rights as a pedestrian, in which case you wont even be able to take the risk.
posted by AlbertCalavicci at 8:48 AM on May 26, 2022 [10 favorites]


It's specifically because of this and related questions that have caused a number of autonomous-navigation researchers to conclude that "level 5 autonomy" (maybe even level 4) is an equivalent problem to building Strong AI, with a similar expected timeline (aka somewhere between "very far away" and "permanent science fiction").

A more entertaining idea, to me anyway, is laying out the full Expert System that might get you through an intersection under construction:

1. If the light is red, stop.
1.a UNLESS a human standing in the intersection stops crossing traffic while waving you through
1.a.i BUT ONLY if they are dressed like a police officer or construction worker
1.a.ii BUT NOT if it's Halloween or there's a Village People concert nearby
1.a.iii OH WAIT, also not if they're just trying to mess with us #DEV NOTE: somebody please solve this one plz

It's very hard to find non-booster sources for this kind of news but I'd recommend this talk, and if you're short on time this section, where he talks about basically this exact issue.
posted by range at 8:54 AM on May 26, 2022 [8 favorites]


AlbertCalavicci has it. (on preview: so does range)

There are so many other everyday traffic interactions that rely on the same sense of another human being behind the wheel of the other car. Was it kevinbelt who asked a question the other day about getting honked at by people who think they're leaving a gap for him to turn, when it isn't safe for him to turn? That's another one. Same thing when two cars approach a narrow gap from different directions - who goes first?

Self driving cars - which I sincerely hope will never come to exist - are a bad answer to the wrong question.
posted by rd45 at 8:58 AM on May 26, 2022 [4 favorites]


Glib but also serious, as I’ve thought about this quite a bit.

Wear a t-shirt with an actual sized print of a stop sign.
posted by vocativecase at 9:16 AM on May 26, 2022 [8 favorites]


I think about this stuff way too much. There's no way to communicate with an autonomous car the same way you communicate with a driver. The solution is going to be more rule-bound pedestrian safety, like signalized crossings, banning right on red, and left-turn arrows. Self-driving cars will also have to have a rule that the proceed incredibly slowly through crosswalks (painted or not) in case they have not detected someone who steps or rides off the curb. Based on the incredibly stupid things I see human drivers do every day *in school zones,* I actually think autonomous will be an improvement.
posted by haptic_avenger at 9:27 AM on May 26, 2022


My question also relates to the follow-on: how will a driverless car communicate to a pedestrian the fact that "I see you and will stop for you". We'll need standard UX for that if driverless cars and pedestrians are to interact smoothly.

I think technically you're supposed to wait for the cars to stop before you start crossing, so I think you know they're going to stop when the wheels stop turning.
posted by If only I had a penguin... at 9:28 AM on May 26, 2022 [1 favorite]


Will a car know that children are less predictable and if there is a child you need to err on the side of slowing down or stopping even more (e.g. don't pass closely behind because kids will just randomly change direction and go back).

It's not that hard to gauge intent programmatically. You simply collect all 'targets' and then produce possible paths that are relevant to you. Of course quite a few assumptions need to be included (like max human speed) and it needs to be done often so the computer doing it has to be pretty quick.

And then you define your reactions, which can include full stopping if someone is moving towards the road - or simply slowing if their current trajectory is not able to be accurately determined.

That's a current issue due to cost and accuracy of the machinery.


A more entertaining idea, to me anyway, is laying out the full Expert System that might get you through an intersection under construction:

More likely - at the point enough autonomous cars are cruising around to make this an issue - transponders will be set up to communicate with autonomous cars, or they will temporarily switch to 'guy in the call-center drives with video' mode.
posted by The_Vegetables at 9:31 AM on May 26, 2022


There have been videos of weapons systems designed to counteract RPG missiles fired at tanks in the Ukraine threads - so if it can catch a randomly fired missile it can catch a pedestrian.

Trophy systems cost around $900k each. When that gets down to $20k, Level 5 will be no big deal.
posted by The_Vegetables at 9:40 AM on May 26, 2022


"they will temporarily switch to 'guy in the call-center drives with video' mode."

It's far more likely there will be 'safe pass' lanes created with autonomous car-friendly markers/cones and a light system in place rather than the half assed hand waving stuff they do now.

Or they could just have all autonomous cars avoid a given area and drive around it, I guess.
posted by Brockles at 11:33 AM on May 26, 2022


Honestly, I'm looking at all of these "autonomous" vehicles in development (along with the currently non-autonomous monstrosities called "light trucks" selling like hotcakes with hoods six inches above the top of my head) and concluding that the most important thing I can be doing right now around them (as an American) is making it patently clear to the NHTSA to take into account the safety of people outside these vehicles al-goddamn-ready before the June 8 comment deadline. Make that the most important principle taken into account when calling a vehicle "safe." If there is no person to communicate with inside the vehicle, or the person inside the vehicle is not paying attention to the road, then the vehicle itself *must* become responsible for the communication with other road users, and I see zero indication that any of these car designers are even aware this is a potential issue, they're all focused on *detection* at best. (Good explanation of potential safety tech capabilities from The Verge.)

The thing is, *laws differ between jurisdictions*. We can say we provide federal standards and recommendations but that doesn't matter if we can't guarantee that *every intersection ever* always fits these standards well enough for the machine learning model the car is operating by. Oops, the paint faded a bit, or is the old standard of two horizontal lines instead of a bunch of vertical ones, or is Pride-rainbow-striped, or has been done as art to get *human* drivers to slow down, confusing the machine. Oops, the town hasn't sent somebody by to trim these overgrown trees in awhile. Oops, somebody knocked into the sign. Oops, it's dark, or rainy, or snowy, or covered by fallen leaves, so the presence of an intersection is less clear. "Statistically," says the MLM, "what are the numerical chances this is an intersection based on the information I've been fed before?"

Back to the differing *laws*: some places, if a person not in a car is within 10 feet of a crosswalk, the driver of the car is required to stop. Others, the driver of the car is not required to stop unless the person not in a car is already *in* the crosswalk. That's just *two* versions of possible intersection laws among lord knows how many slight variants. A new set for each jurisdiction is a chance for a lot more potential "unexpected behavior," because this is the opposite of simple. And nobody is choosing "prioritize the most vulnerable person" in every single one of these interactions (rather, providing aggressive driving mode (and a chill one, but I would bet strongly that people choose the mode that conveniences them most rather than the mode that would actually protect people outside of their vehicular bubble), we are still working on making these opaque choices made by machine learning models *explainable* to the average human being. Let alone *communicating them* to the person walking or rolling or biking outside of a protective steel frame whose path said autonomous vehicle is about to intersect.

(A related article you may find interesting: Ford trials geofencing tech that could automatically reduce the speed of vehicles to improve safety for all. I used to be a hell of a lot more optimistic about self-driving cars than I am now, but geofencing can take away the excess speed that is so frequently fatal to pedestrians, especially in urban areas. By using geofences to keep car speeds low around areas where pedestrians can be most frequently expected, they'll make it a lot easier for cars to stop short, regardless of whether or not a car is using LIDAR or computer vision to automatically detect pedestrians and other things that indicate a car should come to a stop. They're talking about 30 km/h zones (19 mph). I only wish anybody in the US was taking this option more seriously.)
posted by Pandora Kouti at 11:57 AM on May 26, 2022 [3 favorites]


The examples we see around us of autonomous systems working well are typically cases of "easy" classifiers. Path projection, prediction, all of that is easy (though of course was impossible until recently because even though it's easy algorithmically, it has to be unthinkably fast). Highway driving is even relatively easy -- pedestrians are scarce (nominally zero), no traffic lights, lots of space, clear sightlines, federally enforced maximum road curvatures, etc. That's what gets the Tesla Autopilot on the road even though nobody else there consented to be part of Elon's research project.

The problem is not detecting a pedestrian (actually, that's a problem too, but let's assume we solve it for now). The problem is telling the difference between two pedestrians, one of whom has the authority of overrule traffic lights. Or the difference between a slow-moving bicycle and one that's resting on its kickstand at the edge of a road. It would be very helpful, for example, to be able to flag objects that are stationary, but establishing an upper bound on velocity requires time to pass, and we're approaching at 40mph, so we can't know the difference between slow and stopped. So now we're in the business of detecting the difference between a bicycle and a bicycle with a rider. This list basically goes on forever.

The closer analogy would be a TROPHY system where incoming missiles were person-sized and arrived at a walking pace 3 feet off the ground. If you're cool shooting a million bullets at anything with (altitude > X) and (velocity > Y) then that's great -- that's an easy classifier -- now you "just" need a super fast computer and you're good to go. The reason this becomes a hard AI problem is that unless/until you outfit the world and the people with helpful tags, you need robust image classifiers and they are Very Much So a work in progress.

(On preview, I 100% agree with Pandora Kouti -- if we're not careful we will end up solving these problems by creating a world where our car-centered urban nightmares are dialed up to 11.)
posted by range at 12:16 PM on May 26, 2022 [2 favorites]


I think the problem is ultimately not whether they can work out your intent, but whether they will care about you at all. One of the struggles with autonomous cars is how to get them to constantly solve trolley problems - if the car is designed to protect the occupants it will run straight over a pedestrian rather than risk the occupants.
posted by dg at 2:05 PM on May 26, 2022 [1 favorite]


Response by poster: Reading these responses, two thoughts come to mind:

1. it's remarkable that companies have gotten away with investing so much money and generating so much hype without addressing basic questions about the viability of their technology.

2. this is an issue that can probably be solved with the blockchain, and undoubtedly a number of crypto companies will spring up to address it.
posted by Winnie the Proust at 4:30 PM on May 26, 2022


I don't know anything about how blockchains work other than I ran a program that created a hash and another program verified the hash which is essentially like how handshakes work since the beginning of time.

To address your question about how people paying to explore the viability of technology. Yes, that's how investment works. If you can solve the self-driving car problem you'll be the biggest investor since Ford.
posted by geoff. at 5:23 PM on May 26, 2022


Response by poster: My blockchain comment was an (apparently failed) attempt at humor.

Regarding investment, yes, I understand the value of speculative and high-risk investment. I'm just surprised that the issue of interaction and communication with pedestrians hasn't been discussed by any of the big players, given the visibility of these self-driving car projects. There are some interesting UX problems, and it's an area where common standards would be important.
posted by Winnie the Proust at 5:01 AM on May 27, 2022


It feels like what is needed is a set of "going to brake" lights on the front of the vehicle. Which would be possible because the AI "would know" its going to brake before it brakes.
posted by zerobyproxy at 6:43 AM on May 27, 2022 [1 favorite]


In theory, autonomous cars are going to actually obey the law. This means yielding to pedestrians automatically, with no need to make eye contact with anybody; you don't need to signal "I will let you go" because they'll always let you go.

They cannot be doing this, or you'll be able to cross an interstate wherever fancy strikes you seriously impeding traffic. In practice, I suspect they have to be programmed to enforce the right of way (possibly randomly?), so that people don't do this, regardless of "eye contact". So AlbertCalavicci is right, unfortunately.
posted by Dotty at 6:46 AM on May 27, 2022 [1 favorite]


I don't think there is any other possibility that makes sense than more rigid pedestrian crossing zones that are explicitly triggered by the pedestrian, which would then be obeyed by the cars. It's a logical approach and limiting the number of pedestrian crossings would make sense from a traffic flow aspect.

I think part of the issue here is that people are trying to imagine how autonomous cars would function in our current driving environment. I think our entire driving environment would change. The more autonomous cars are available the more sense it makes to have them communicate with the road structure and other cars and fun all that traffic in ways that makes the most sense for traffic flow. This will mean minimising their interaction with pedestrians so a very logical way of approaching this would be more pedestrian areas and separating those from the car areas. When traffic flow can be in controlled by a kind of cloud awareness of where everybody needs to go then you need less physical space for the cars involved. Obviously in an ideal world a bunch of autonomous Uber style roaming bus options would be hugely reduced the number of individual cars on the road but that's possibly fanciful sci-fi thinking.

I have mentioned this on here a long time ago but I think that the only logical future for self driving cars is for them to hook up into train-like groups that could run very close together (for traffic density and aerodynamic/energy saving purposes) and would communicate each other's routes so that these groups of, say, 10 cars can run together for a time and then split when they get nearer to their respective destinations. The only way autonomous cars will ever work in any way that is convenient as if they are all talking to each other. There is no way in hell you will get millions of entirely independent cars trying to navigate around each other without them driving into each other just like they do with humans controlling them. The focus for autonomous cars right now has seem to be on replacing the human which I think you're stupid. They should be looking at trying to replace independent traffic in a convenient way. It's much more about creating a convenient group transport system (using your own personal pod if need be) than about it making sense for your car drive to yourself if you want any hint of success.

That is possibly too much of a leap of imagination for most car owners (and even more so, legislators) so I suspect we will go through a period of autonomous cars creating merry fucking chaos and then a huge fuss that forces people to come out it differently. There needs to be at least three generations of evolution of the way we move around before autonomous transport makes sense and only the last generation would most likely work but there is absolutely no way you could sell that vision to anybody. Muh Freedums.
posted by Brockles at 7:06 AM on May 27, 2022 [2 favorites]


I don't think there is any other possibility that makes sense than more rigid pedestrian crossing zones that are explicitly triggered by the pedestrian, which would then be obeyed by the cars. It's a logical approach and limiting the number of pedestrian crossings would make sense from a traffic flow aspect.

I can't imagine how much it would cost to turn every single pedestrian crossing into some automated-signal-sending thing. You'd end up with whole neighbourhoods where people could just walk around the block (whatever block they live on) and that's it. And could no one cross the street in a blackout?
posted by If only I had a penguin... at 7:48 AM on May 27, 2022


Geofenced car-trains and pedestrianized zones are basically the only thing giving me hope about this future!

I would encourage everyone intrigued by this topic to get involved in local traffic safety initiatives.

I can't imagine how much it would cost to turn every single pedestrian crossing into some automated-signal-sending thing.

Not every intersection, but the most high-volume intersections. And you can prioritize things like school-zone intersections, and create a chain of them for safe routes to school. Identifying micro-micro level solutions like this is why everyone needs to get involved in local governance.
posted by haptic_avenger at 9:00 AM on May 27, 2022


I can't imagine how much it would cost to turn every single pedestrian crossing into some automated-signal-sending thing

I don't imagine it would be very expensive at all. You just need a solar panel, a button and a pole with batteries inside it. You'd only need a light or a small RF device inside to send the signal. Even if it was a vertically flashing LED strip down the pole it'd be relatively cheap. We are past the days of needing to dig trenches and put infrastructure in for these kind of things.

Dig a hole, drop a self contained pole with a panel on top, twist to orientate? Done. Or just drill 4 holes and bolt it to existing concrete/sidewalk.
posted by Brockles at 1:36 PM on May 27, 2022


« Older Over-exertion related sickness   |   Should I let my birth father know that I exist? Newer »
This thread is closed to new comments.