"Hello," I think… I could be mistaken.

April 27, 2010 10:50 AM

Suppose you were designing a Star Wars-esque protocol droid. How would it learn new languages it had never encountered before? I'm writing a science fiction story and I'm trying to figure out how a language-learning robot character might realistically handle a situation.

A few questions in here. Feel free to approach them from an AI programming perspective, or from a psychology of learning perspective. It's all hypothetical future sci-fi from this point on, so there are no wrong answers.

Imagine your robot is alone on a mission, when it meets some intelligent creatures speaking an unknown language. It hears them speaking first and fails to identify their words as anything in its vast library of known languages. What happens next?

Would the droid start talking in some other language, with lots of gestures and body language (à la "You, Tarzan; Me, Jane")? Or perhaps just silently observe their speech until it felt comfortable assuming the meaning of certain words, and then try to use them? What's the fastest possible way to begin basic, usable communication, with a minimal risk of being misinterpreted as hostility? And how long would it take for the robot to become rudimentarily conversant with them in that language? Minutes? Days? Weeks?

Let's assume the robot has no limitations like clock speed or memory. It could have a screen and speakers for showing A/V content, if that's helpful. The robot's top priority is establishing good will with the creatures and demonstrating a willingness to speak with them. And in this case, that the creatures are also friendly and want to talk to it too.

Also… presumably the robot would begin by sizing up the creatures, with a different approach for humanoids versus other known alien types. What assumptions could it make about humanoids that would be almost universally applicable for communicating with them?

posted by The Winsome Parker Lewis to Writing & Language (37 answers total)

I think it would silently observe.

I think of this the way people decode cryptograms, looking for the most common letters first. Likewise, I think the protocol droid would look for most common words first--articles, personal pronouns--then look for the names of objects with which those people are interacting. I don't think you'd want to droid to make gestures, because who's to say what an alien's non-verbal communication is going to be like?

Also… presumably the robot would begin by sizing up the creatures, with a different approach for humanoids versus other known alien types. What assumptions could it make about humanoids that would be almost universally applicable for communicating with them?

This would be much easier for, say, actual human languages rather than the languages of humanoids or, god forbid, other aliens. Take personal pronouns. If you have a hive-minded alien, there may be only one. Or there may be none. The landscape of an alien's home planet would likewise influence language--think of that saying about all the Eskimo words for snow.

In reality, I don't think a protocol droid would really work. But that's science fiction for you.
posted by PhoBWanKenobi at 11:00 AM on April 27, 2010

I think it learns new languages by downloading them from a server when needed. Or by installling a ROM.
posted by Chocolate Pickle at 11:34 AM on April 27, 2010

Let's assume the robot has no limitations like clock speed or memory

If technology (hardware and software) is unlimited, then the bottle neck would be simply how fast new data could be presented to the robot. If it could intercept communication, for example, then it could constantly be learning through many channels. If it was in the middle of a bustling city, it would learn faster than if in a village.
posted by whiskeyspider at 11:35 AM on April 27, 2010

Silent observation would work with concrete nouns on Earth, but is not a general solution.

Why can't you start with some of the "communicating with aliens" information already out there? Like Lincos.
posted by DU at 11:35 AM on April 27, 2010

It would start my learning obstenive definitions, I think; words that have simple referents. So it would observe from awhile, and then use a statistical analysis to determine when it was likely correct, based on successfully number of correlations. Then it would start observing patterns regarding how these things are related in more complex grammatical structures. Chanages in forms, assuming the changes are similar enough to the original, would suggest different case or tense useages that would fit the context or social situation. Enough observations that allow for nuanced differences might allow a statistical correlation again for correct usage. These would be the building blocks for more abstract ways of referring to ideas and things that aren't as tangible.

This wouldn't apply to all langauges or situations, but perhaps it would be enough to get things rolling.
posted by SpacemanStix at 11:36 AM on April 27, 2010 [1 favorite]

I do not think that "how long" is answerable beyond a flip "more than a second, less than a billion years" fashion. Your protocol droid might be very fast, but who knows what the clock speed of the aliens are? I'm guessing you could get a much idea of the behavior of a hummingbird in a day versus that of a turtle. The larger the vocabulary, the longer it would take to learn it. Dumb aliens would be, other things held equal, easier to learn than smarter ones. Aliens who were far, far more intelligent than the people designing the protocol droid would be staring at some amusing, clunky contraption that would be the equivalent of the cat learning to knock your deodorant off the dresser when it wanted some attention — oh, look, some barely sentient life form has made a tool to communicate that it wants to be fed or petted. Isn't that adorable? Meanwhile, neither you nor your protocol droid can get a handle on the thought processes of something that makes you look like a member of Protista as it silently opens a quantum entanglement communications channel and has a little giggle with its pals using machine telepathy about yet another beastie wandering into the area and isn't it predictable how they always [[UNTRANSLATABLE]]?

As to how, you'll be interested in "machine translation." All forms of it involve building a large corpus of text (or in this case clicks, whistles, hisses, and hoots). No universal non-threatening gesture could exist. That same slow stare that cats interpret as trust makes dogs nervous. Your protocol droid would want to observe without being observed and match communications (vocal? scents emitted? red baboon butts? postural? infrared? the list is endless) against the alien life cycle, as much of it as it could study. Throwing more hardware at the problem does not help; this is all down to the organism in question.
posted by adipocere at 11:45 AM on April 27, 2010 [4 favorites]

think of that saying about all the Eskimo words for snow.
'cept it's just a saying and not generally considered to be true.

On topic, consider that the species encountered may not be restricted to one language. When observing, how long would it take to determine there was more than one language being used? What if a given pair of observed speakers are fluent in both languages but each prefers to speak whichever one they're more comfortable with?

Aliens who were far, far more intelligent than the people designing the protocol droid would be staring at some amusing, clunky contraption that would be the equivalent of the cat learning to knock your deodorant off the dresser when it wanted some attention

I don't think you need to worry if the species is vastly more intelligent and, as the question stated, "want to talk to it" because it'll be simpler for the robot to just speak English (or Galactic Common or whatever) and wait for the vastly intelligent dudes to figure it out.
posted by juv3nal at 11:48 AM on April 27, 2010

It hears them speaking first and fails to identify their words as anything in its vast library of known languages. What happens next?

Parses the info core for rare/unusual matches that aren't stored locally or mentione don in myth and legend. Checks for common roots to a known language by analyzing speech patterns and inflections. Attempts to communicate by using any languages discovered above - "Hello." "Salamu alaikum." "Hola." - and check for reactions.

What's the fastest possible way to begin basic, usable communication, with a minimal risk of being misinterpreted as hostility?

Pictograms.

What assumptions could it make about humanoids that would be almost universally applicable for communicating with them?

Body language and head movements are communication aids.
posted by anti social order at 11:49 AM on April 27, 2010

Handwave handwave genetic algorithm!
posted by Artw at 11:51 AM on April 27, 2010 [1 favorite]

Response by poster: Okay great answers. I guess I should pull the reins in a little because "aliens" are so unknowable.

In my story, they're humanoids. They speak with vocal cords and are generally similar to humans in their non-verbal communication. Think of something like the panspermia idea in Star Trek (am I allowed to mention that in a thread about protocol droids?). Klingons, Vulcans, Romulans, Ferengi... all similar though they speak different native languages. A robot could probably learn how to communicate with a race like that, if it was programmed cleverly.

Also, the goal isn't full fluency; just enough to convey basic ideas and refer to people/objects by name. Like, say, a five-year-old's proficiency.
posted by The Winsome Parker Lewis at 11:53 AM on April 27, 2010

Also, the goal isn't full fluency; just enough to convey basic ideas and refer to people/objects by name. Like, say, a five-year-old's proficiency.

I think to go along with the observation notion, if your robot can manage to do it without being discovered, it might be beneficial for them to seek out a pre-school/day-care/kindergarten and observe language-acquisition aged kids actually being taught and just piggyback on their lessons.
posted by juv3nal at 12:03 PM on April 27, 2010 [1 favorite]

Honestly? Body shape will be not that important. I do not think that a humanoid would be significantly more or less difficult to communicate with versus, say, something with six limbs or a tree-dwelling snake-like creature with linked-up ganglia on every third spinal segment. If you reached for some truly exotic environments and biochemistries (such as creatures living on the surface of a neutron star, like Dragon's Egg), sure. Sophonts on an Earth-like planet, no. In what biome they emerged, their resource requirements, and their reproductive strategies would be far more important than the number of arms or having a big knobby sensory cluster balanced on top would be for translation.
posted by adipocere at 12:07 PM on April 27, 2010

The droid would first need to figure out what clusters of sounds are actually separate words. Which sounds are likely to begin words, which never occur at the beginning of a word, etc. Then it would have to try to figure out which of these words are nouns, which are verbs, etc. (For the sake of fiction, you can probably hold that all languages have at least nouns and verbs.) It may then start to determine how the language is inflected (i.e. how suffixes and prefixes change meaning), whether or not it is agglutinative, and which subject-verb orders are used. From there it might try to figure out which words are irregular. Once it has enough of a base, it may be able to start comparing vocabulary to other languages it knows to see if it can find any that might be related, even distantly, which might help form hypotheses which can be tested by speaking and seeing if it is understood.

The droid should also be on the lookout for signs and other written material. Once a correspondence can be made between written words and their pronunciation, the droid can improve its vocabulary by reading.

A perfect memory is the main requirement for this task, and having the memories hooked up to each other such that each new discovery strengthens some and severs others (neural network style) is also good.

The droid (if a mechanism) can have the ability to transmit its growing language database to peers or to a central system with a much larger database of known languages. You might send several droids out into the field and they would continually compare notes by your world's equivalent of radio. Every once in a while they could compress and squirt their data to the central repository, and get occasional updates, though this should be expensive in some way. They should be mainly autonomous with small group support, I wager.
posted by kindall at 12:11 PM on April 27, 2010 [2 favorites]

You do this with a massively parallel network of robots each of which goes into the field and follows people around until it bugs the piss out of them.

When each robot hears something, it tries to isolate the context of what it sees with the sound and sends that up to the mother ship. The mother ship starts doing pattern matching and when sufficient equivalent patterns are associated with a thing, it starts testing them by indicating an instance of that association and speaking a choice of the existing sound patterns for that object. It continues doing this for objects until the vocabulary is built to a high confidence. Then it starts looking for meta patterns to identify grammar and it starts to generate propositions to test the grammar.

Each robot doggedly follows someone around until such a time as they are no longer getting useful information, then it finds someone else.
posted by plinth at 12:39 PM on April 27, 2010

It would be good if your 'droid could present itself to the aliens in such a way so that it engendered alien conversation.
For example, maybe glowing, or falling over, or other random activities that would be unusual enough that would likely result in the alien's making direct comments about it.
An extra benefit is that the conversation antics can help liven up the story.
posted by forforf at 12:39 PM on April 27, 2010

Response by poster: Some more great ideas, thanks! I like the one about having the robot do something specifically to provoke a certain type of comment, as a way of focusing its starting point.

Unfortunately, in my story the robot must be alone. And the aliens it's trying to talk with are also alone and far away from home, so there's no background communication to analyze. There are only a couple aliens present.
posted by The Winsome Parker Lewis at 12:47 PM on April 27, 2010

This reminds me of a guy I heard on a tv program a while back... Daniel Tammet could learn languages in no time. He learned Icelandic, apparently one of the most difficult languages to learn, in a week. He's also the world record holder for reciting Pi to an extraordinary number of digits.

What brought me to this was that he "sees" numbers as pictures. Something well outside the norm of how you and I might recognize and remember a number. But it provides an extremely fast way to learn and recall information (for him), and for him it just happens, it's effortless. Perhaps your droid is programmed after a model of Mr Tammets brain, which was extracted and decoded many generations earlier.

Or, following a link from that wikipedia article to the term Synethesia, perhaps your fictional droid is a Synethete that could create a match for a term in one language and map it to another. Perhaps the particular word in each language mapped to an internal "image" or some other common firing of synapses to quickly understand a language.

I, clearly, have no idea what I'm talking about, but maybe it will spark an idea somewhere.
posted by chocolate_butch at 12:55 PM on April 27, 2010

Response by poster: That's fascinating, chocolate_butch! I saw that program too but had forgotten about it until now. Definitely some good brainstorm fodder, there. :-)
posted by The Winsome Parker Lewis at 12:57 PM on April 27, 2010

Perhaps your droid has the capacity to project pictures and video, a la R2-D2, and can use them as a conversation starter. The droid can show a picture of something in its files, or even a picture of something it's taken recently, such as something in the surrounding environment.

The droid could also start at an even more basic level, such as with audio tones and lights, just to establish the beginnings of a communication give-and-take.
posted by Cool Papa Bell at 1:02 PM on April 27, 2010

Your robot watches lots of TV. It's a fat, sloppy couch potato that always has a TV guide in one hand and a remote in the other. It is watching its own screens or wall projections -- made so it can share information with others but also a good excuse for it to be sitting and watching TV with others.

When it needs to learn about buying and selling gold and jewels, it flips to the shopping channel and starts talking about the "timeless classic elegance" of certain items. When it needs to learn about driving, it watches alien Top Gear and starts speaking (and driving) like alien Jeremy Clarkson.

Technically, it could of course watch all channels simultaneously and quietly, but it would be better for your story if it just Tivoed the other channels and concentrated on a channel or two at a time. Maybe it could be distracted sometimes because it's trying to listen to what's happening on a soap it follows.

Aliens make friends with it because it's watching their favorite shows and adopting the traits of characters on television and character watching the television with it.

It picks up body language and emotional skills by pretending to be others, maybe something like method acting.

And it can synthesize television shows after watching enough, so it can create extra episodes for aliens to watch, and it can put people into their favorite shows.

They love your robot.
posted by pracowity at 1:15 PM on April 27, 2010 [1 favorite]

I suppose one alternate angle is, you could postulate that all forms of communication follow some basic, underlying rules. If the robot knows the rules, then he has to simply deconstruct the communication. Sort of like how Fourier spectral analysis can deconstruct a signal in the time domain to individual wavelengths in the frequency domain.

In a sense this may be "true" (although not necessarily supported by any theory I'm aware of), since humans speak both (say) japanese and english. The japanese word for 'happy' and the english word for 'happy' may sound and be written differently, yet in the heads of a japanese & western person, their brains will (I imagine) process the concept similarly since they both refer to the same thing, in a fundamental brain-language.

In terms of your story I admit this is a bit of a cheat. I guess it depends on how central the learning process is to your plot.

At the same time it's more science-fictiony since you're taking the reader into an unknown concept, that of a fundamental uber-language. You can call it that too. Be sure to thank me in your book ;)
posted by thermonuclear.jive.turkey at 1:29 PM on April 27, 2010

As for how to get a communication process started: have the robot simply repeat what he just heard someone else say. This will achieve two things: it will make the robot appear as a valid communications target (and not just a "thing") and it will cause the aliens to use a limited range of next actions on it ("What did you say?" "Why are you interrupting?" "You can talk?" "And who are you?" etc.). The fact that it was able to produce a valid sentence in the language will also likely cause them to try to communicate more, even if this fails at first.
posted by PontifexPrimus at 1:36 PM on April 27, 2010 [2 favorites]

Response by poster: I like your idea, PontifexPrimus! Maybe the robot should not only repeat what the aliens say, but mirror their body language as well (a way to both learn, and make them feel more at ease).
posted by The Winsome Parker Lewis at 1:46 PM on April 27, 2010

That is a good idea (parroting, initially)...once it becomes clear to the aliens that the robot desires to communicate, the next logical place to go is mathematics (being universal), counting, the names of numbers, mathematical operations, concrete nouns (these are four 'apples', etc). then, the difference between nouns and verbs (numbers vs. operations), and branching out from there...for example 'adding' and 'eating' are very similar concepts. work from that basis into the surroundings (nouns from pointing at things, verbs from pantomiming action) and then onto more abstract concepts like 'love' or 'jealousy' ('you adding, me wanting') and etc. but definitely, mathematics is the place to start.
posted by sexyrobot at 2:00 PM on April 27, 2010 [1 favorite]

Aside from the robot aspect, the situation you describe is "radical translation" which has been extensively discussed in philosophy since W. V. O. Quine and Donald Davidson worked on the idea in the late 20th century. wikipedia on radical translation.

Radical translation is when someone attempts to translate a language starting absolutely from scratch, and one of the main philosophical points of interest is the question whether such a translation is possible to do with accuracy. One of the famous problems raised is that even if you are observing a native speaker of the language pointing at a rabbit and saying a word ("Gavagai!" is Quine's made-up example), you can't be sure if the speaker intends to refer to the rabbit itself, to one of its parts, to what it's doing right now, or something else even less obvious. So, searching for "radical translation" and the term "gavagai" may yield results you'd be interested in reading.
posted by LobsterMitten at 3:03 PM on April 27, 2010 [1 favorite]

My protocol droid would politely encourage language interaction with it via an obvious visual progress indicator until it was capable of speech in that language. It would also encourage interaction via some example if intermittent variable reward, where it would somehow reward a person for speaking with it, by playing a little tune, doing a dance, or providing useful functionality. This could be used for comic effect by having it happen during inopportune times.

Once it was fully functional, the drod would be unfailingly helpful to those that spoke with it. This could be a character flaw. It would prioritize it's relationships based on the amount of interaction it has had with a person or thing, unless it was instructed to be specifically loyal.
posted by dobie at 3:34 PM on April 27, 2010

Read about language acquisition in human children. They are protocol droids made of inefficient meat.
posted by i_am_joe's_spleen at 4:19 PM on April 27, 2010

I can't find the article now, but I remember reading once about a linguist who specialized in that sort of thing thrown into a room with one (or two, or something like that) speakers of an unknown (to him) language and challenged to figure it out, as a sort of demonstration to the audience of linguistic students. He used a bunch of procedures he had developed himself to elicit numbers, plural/singular, simple sentence structure, etc. After something like half an hour he had a pretty good sketch of how the language worked, and he had also figured out it was Hungarian (which he did not know, but knew ABOUT and thus recognized it by its characteristics).

I may be misremembering some details there, but I think the basic principle is sound: if you have or can obtain willing informants, it makes no sense to just observe. You would have a set rouine for eliciting the basics, and you would then use that framework to fill in the vocabulary and other gaps. How long it would take depends on what your robot's goals are. If it just wants to find materials or shelter or whatever, probably not long at all to get up to functional level.

Of course, all this only applies to languages recognizable as such to the robot, and similar in functionality to human language. A sentient storm overmind on a gas giant that spoke (only to itself) by modulating tornadoes or an insect-like species that spoke with smells and wing positions would pose problems unless you could teach your robot to be aware of every such change in circumstance and treat them all as possible communication channels.
posted by No-sword at 4:54 PM on April 27, 2010

Response by poster: Good points, No-sword. My aliens aren't gaseous Slylandros, so no worries there. Their mode of communication is basically human. I'm very curious who the linguist you referenced was, and what his method is. Such a thing is exactly what I'm looking for, for my story! Please post again if more details pop into your head! :-)
posted by The Winsome Parker Lewis at 5:07 PM on April 27, 2010

I may be misremembering some details there, but I think the basic principle is sound: if you have or can obtain willing informants, it makes no sense to just observe.

Observing to begin with makes plenty of sense to me because you wouldn't want to inadvertently do something right off the bat that would be interpreted as hostility. If you gather some data by observation maybe you can diminish the chances of that happening.
posted by juv3nal at 11:10 PM on April 27, 2010

Not much help here except I remember the main character in the novel The Sparrow was a linguist, amongst other things. When they arrived at the other planet, he went through this exact issue. I seem to remember some explanation of how he was going about learning how to communicate with the alien life forms he encountered. Maybe that would help some?
posted by bDiddy at 9:05 AM on April 28, 2010

Instead of just trying to learn the aliens' language, perhaps your droid also tries to teach them some English (or whatever language the protags speak, i.e. Space English) at the same time. Point to a tree, say "tree," look at them and wait pointedly for their version of the word. Then repeat their word, and prompt them for the English word (perhaps say it again). More of a meeting-halfway kind of approach.

Another possibility is that the droid is more of a big touch screen than a mobile agent. It displays various things on the screen and says the corresponding English word, and invites the natives to touch and interact with it. The droid arrives while everyone is asleep or away (either under its own power or delivered by the protags). The droid just observes as the natives touch the screen and say various things. Since it is immobile (as far as the natives know) it is much less of a threat and their curiosity may be engaged. Perhaps this would be followed by mobile droids (styled in the same way, perhaps with a screen on the front so they are obviously walking versions of the same devices the natives have already seen) that actually interact with the natives.

Needless to say the droids would have to have excellent hearing. It should be possible for them to follow multiple conversations at once in many different directions at various distances.
posted by kindall at 10:28 AM on April 28, 2010

Following up on No-sword's comment, you'll want to look up information on linguistic field methods or field work. Access to an academic library will help.
posted by nangar at 5:47 PM on April 28, 2010

I think kindall comes close to what I was imagining. The droid shows up and the aliens take notice. It uses its screen to show an image of itself and says its name. ( or just "translation droid", whatever) Then it shows a picture of each alien in turn, maybe flashing it for emphasis. There are no possibly threatening gestures with a screen. More complex communication can be built up from there using the linguistics field procedures mentioned above. This assumes that the aliens have the ability to mirror emotions as humans do. That is, the aliens will understand that the droid is something with which they can communicate and they ascribe it not only a certain level of intelligence, but also a consciousness like their own. ("Ah, the droid is showing a picture of itself and saying something. If I were it and wanted to communicate with myself, I would point to myself/ thump my paw on my torso and proclaim my name [or clan, or species, etc]. So, that's probably this robot dude's name)

"for example 'adding' and 'eating' are very similar concepts." What planet are you from, sexyrobot?

Heh,it's eponysterical on so many levels!
posted by HE Amb. T. S. L. DuVal at 12:37 AM on April 30, 2010

Then it shows a picture of each alien in turn, maybe flashing it for emphasis. There are no possibly threatening gestures with a screen.

You're making some cultural assumptions there which don't necessarily hold up. For example.
posted by juv3nal at 3:55 AM on April 30, 2010 [1 favorite]

That's an interesting perspective, juv3nal. I would think that the aliens possible fear of soul-stealing via photography/video would be trumped by the fact that they are seeing an android in front of them. But, who knows?

By no threatening gestures, I meant that the android won't physical need to move. So, there would be no body language to be misinterpreted as hostile (or even impolite). Of course, the lack of a greeting gesture might in itself be considered to be impolite or hostile.
posted by HE Amb. T. S. L. DuVal at 1:32 PM on May 1, 2010

What about approaching language learning from a musical point of view? In English, at least, much of a speaker's intent is conveyed by volume, cadence, and tone. Perhaps your droid could begin by making noises that mimic the rhythmic flow of sentence structure. * This might encourage nearby speakers to engage with it as people do when speaking with small children- exaggerating the musical qualities of speech through something like baby talk. I'm guessing if you went with this approach, the appearance of the robot would need to be very benign, even cute. As the droid gains rudimentary vocabulary, perhaps its speech will begin to sound like someone humming a tune with a few sparse lyrics thrown in.

Also, the Musical Language episode of Radiolab has some awesome food for thought on this subject.

* I thought of this because I am currently trying to learn Mandarin, a tonal language. Humming the sounds of a phrase before attempting to repeat it has been so valuable to comprehension!
posted by alight at 9:34 AM on May 2, 2010