Non-technical book about neural networks?

February 26, 2017 8:36 AM

Is there a non-technical book that explains neural networks in sort of an introductory way for general readers?

I recently read Gideon Lewis-Kraus's NYT Mag article on Google Brain research, and I want to know more. Trouble is, I'm really not a math or computer science person. Is there a readable book that introduces neural networks--either on the human brain side or the CS side--without too much technical stuff? Or is that an oxymoron?

(The resources in this prior neural networks question are too out of my league.)

posted by Dr. Send to Science & Nature (12 answers total)

the human brain side of things and the CS side are not too closely related. sometimes some inspiration goes from one to the other but for the most part they are totally separate and understanding one won't help you understand the other. if you're interested in Google Brain's research, especially on neural machine translation, studying the biology won't be that useful.

(computer vision neural networks are maybe somewhat similar to the vision systems of animals. to the best of my knowledge, neural networks for translation and other problems in language are almost totally unrelated to however language works in the brain.)

there's not really a math-free explanation of how they work that I know of. The resources at colah.github.io are good but still at least a little technical. This is a good one for understanding LSTMs, which are a key technology for processing of sentences and other sequences of symbols, and which Google certainly makes use of for their translation work.
posted by vogon_poet at 9:41 AM on February 26, 2017 [2 favorites]

The gentlest introduction I've seen is Make Your Own Neural Network by Tariq Rashid. It assumes very little mathematics understanding, like how to read a graph and approximately middle school algebra. Part 2 has you actually build a NN with Python, again from a truly basic starting point.
posted by nowoutside at 11:05 AM on February 26, 2017

Here is a good online book. It does use some math, but it's very gentle and reminds the reader about much of the math that may have slipped their mind.

Note that the name "neural network" is really just a metaphor. Neural networks don't actually reproduce the physiology or function of the human brain (or other brains). They're much less complicated than that --- which is good news for anyone trying to learn about them!
posted by snowmentality at 11:05 AM on February 26, 2017 [1 favorite]

The best prereqs are several subfields of linear algebra, the physical theory of dynamical systems, numerical computation, and some experience doing actual programming. Notice how none of those things I mentioned are nontechnical. Actually, the experience doing actual programming is depressingly optional and you get a lot of PHD student code.

I guess it's like asking for a nontechnical introduction to a subfield of algorithms. Which neural networks completely and totally are. You can get popularizations (eg: Code by Petzold for coding) but by necessity they will go towards becoming technical.

There is, after about the year 2000, to a solid first and second approximation, no brain content in neural networks.

I guess the least technical content is in histories of people doing things with neural networks. Eric Roberts has some stuff here. Andrey Kurenkov has some stuff here. In addition, AIMA (Russell and Norvig's book) has a really nice afterward on the neural networks chapter.
posted by hleehowon at 11:06 AM on February 26, 2017 [2 favorites]

I was in Surya Ganguli's class about theoretical neuroscience along with lots of less-technical neuroscience kids and someone asked him for a less technical introduction to why neural networks work and he recommended his NIPS paper. Which is hilarious, because you can read it here and that shit ain't nontechnical. This may be more of a Surya Ganguli story than a nontechnical neural networks thing story.
posted by hleehowon at 11:09 AM on February 26, 2017 [1 favorite]

You might take a look at tutorials describing convolutional neural networks (convnets or CNNs), improvements in which are responsible for a lot of the flashy applications you've seen over the last few years. It's also easier to visualize them as fancy trainable image filters. Here's another. You might want to also look at AlphaGo.

Then realize there's a whole zoo of neural architectures, and various ways of connecting them to each other and in ensembles. Also people working with non-deep learning techniques like XGBoost. There's a lot of stuff to sum up!
posted by RobotVoodooPower at 11:30 AM on February 26, 2017

Hmm, more on the nontechnical front.

Hinton, along with the other significant PDP folks (McClelland Rumelhart Sejnowski etc etc) sort of moonlighted as philsophers for a long time. So you can see the philosopher's articles about the philosophical aspects of these dealios, which is decidedly less technical (see the SEP entry).

They called it connectionism because the really old guys (old enough to be dead - Hopfield McCulloch Pitts etc) got called, in retrospection to the "new" connectionism, the "old" connectionism (ok Schmidhuber often goes off about Gauss doing this stuff but he's really talking about numerical linear regression but the original connectionist models were also often linear). There were a lot of big philosophy fights with Fodor, Pylyshyn, Chomsky, etc. The SEP article cites Hinton's 1992 article about it. Very good, but very 1992. I guess the modern version of that popularization is the 2015 Lecun Bengio Hinton review.
posted by hleehowon at 12:40 PM on February 26, 2017

If you want at the stuff that people claimed were some brain stuff with connectionist models before 2000, good review is Medler's "A Brief History"
posted by hleehowon at 12:44 PM on February 26, 2017

Technically, I lied about the brain connection stuff not existing after 2000. But the discipline that looks at "making model for neural stuff", and the discipline that looks at "doing shit", while they were the same discipline in 1987, are not the same discipline in 2017.

So you get neuroscience folks lookin' at liquid state machines, for example, but nobody looking at them in the "getting actual perf out of them" side, because they suck at actually doin' anything.
posted by hleehowon at 12:48 PM on February 26, 2017

Since your question is a yes/no question, I'm gonna go with hleehowon and say "probably not." In my experience, what is interesting, novel, and important about neural networks / deep learning is technical; any nontechnical description I can think of would be a description that applied to every approach to machine learning, not just neural nets.
posted by escabeche at 7:40 PM on February 26, 2017

Response by poster: Thanks everybody! There's lots to comb through here, even if I don't understand a lot of it.

For the record, part of my interest comes from being a sociology student and having learned that there's a niche of cognitive sociologists thinking about human behaviour in terms of neural nets (notably, A Cognitive Theory of Cultural Meaning by Claudia Strauss and Naomi Quinn). The advantage is that neural nets provide a way of thinking about behaviour more in terms of learning through trial and error rather than in terms of specifiable rules (e.g., as in rational choice theory). The Times Magazine article got me thinking about that again and I was wondering if I could build on it by looking more into the CS or cog sci side of neural net research, even if it's just as a heuristic, but only time will tell.
posted by Dr. Send at 11:50 AM on February 27, 2017

if you're interested in that sort of wooly-headed thinking (haha), you should probably poke around Jürgen Schmidhuber's website.

From the technical angle, he's 110% legitimate. Among many other things he studied problems in recurrent neural networks (the kind of thing you need to understand sequences, like videos, written sentences, or audio data). He and his student Hochreiter created the LSTM (long short-term memory) cell, and it can't be overstated how much of a big deal that is. Google Translate, for example, is certainly using LSTMs, as is any decent speech-recognition system.

However, he is kind of the black sheep of neural network luminaries, partly because he claims that people are conspiring to deny him and his students proper credit for ideas, partly because he is willing to get into somewhat kooky ideas about learning and consciousness. To get an idea of the flavor of this, you might enjoy his explanation of all creativity, art, science, and humor in terms of machine learning. (Note that actually this explanation doesn't have to do with any special properties of neural networks as opposed to other classes of functions.)

P.S. I don't know much about sociological theories but if you want to think about this stuff, don't limit yourself to neural networks. They are basically just functions that can be fit from data. They are very flexible and easy to optimize on computers and work well in a lot of domains. But they aren't magic and in terms of "learning from trial and error" there are lots of other approaches that could be swapped out for neural networks.
posted by vogon_poet at 12:33 PM on February 27, 2017