Automated extraction of the gist of an article?

June 16, 2009 6:49 AM

I often have a whole bunch of 500-3000 word articles to read - all reasonably plain English with headings and sub-headings (and occasionally images). Is there any software out there which will take an article (or articles) and write a reasonable one or two paragraph summary of the article, or produce a list of key points?
posted by zaebiz to computers & internet (13 comments total)

I do not believe artificial intelligence has advanced to the point of being able to do that yet. Even machine translation of a document in one language to another—which would seem to me to be less difficult, computationally, than picking key concepts out of an article—remains spotty at best.
posted by DevilsAdvocate at 7:06 AM on June 16

Great Summary is meant to do just this, but I give no guarantees as to how well it will actually perform the task.
posted by pemberkins at 7:12 AM on June 16

From Great Summary: re this page"Is there any software out there which will take an article (or articles) and write a reasonable one or two paragraph summary of the article, or produce a list of key points? (0) "
posted by Debaser626 at 7:33 AM on June 16

This is unpossible. A "good" summary of an article is beyond the reach of many human beings, some of whom have college degrees. So that means that the AI that generated "good" summaries would have to be AI-complete beyond just tricking a person in a 5 minute conversation (the turning test, say); it would have to be better-than-human-intelligence.

As a further thought starter, think about the different kinds of summaries that might need to be written. If your hypothetical AI was reading articles from history, the kinds of summarization data would be quite different than if it were reading engineering articles. Or think about literature / lit-crit journal articles -- those are highly specialized in their own way, too.

Context is a bitch. Read the articles.
posted by zpousman at 7:36 AM on June 16 [1 favorite has favorites]

If you have a Mac, this function is built in to the OS. Look for "Summarize" in the Services menu. I leave it for you to judge whether the results are "reasonable".
posted by nowonmai at 7:43 AM on June 16

Depending on your budget or who is footing the bill, maybe you could look into outsourcing the work to a personal assistant in India or another country with English-speakers.
posted by zerzura at 7:48 AM on June 16

I do not believe artificial intelligence has advanced to the point of being able to do that yet. Even machine translation of a document in one language to another—which would seem to me to be less difficult, computationally, than picking key concepts out of an article—remains spotty at best.

True, but the work has been coming along. My gf has assisted on this project at BBN.
posted by HumuloneRanger at 8:00 AM on June 16

In Word, under Tools, there is AutoSummarize with various options. I leave it for you to judge whether the results are "reasonable". Exactly.
posted by kch at 8:02 AM on June 16

(the above was written about version previous to Word 2007. YMMV).
posted by kch at 8:04 AM on June 16

I think what you're looking for is called an "intern" - or else zerzura's suggestion is pretty good too.
posted by From Bklyn at 8:08 AM on June 16 [1 favorite has favorites]

The results of Apple's "Summarize" function on this page:

Context is a bitch. Read the articles.

It works better than I expected.
posted by ook at 8:41 AM on June 16 [2 favorites has favorites]

You could set up mechanical turk to do this, but it's probably not worth what it would cost.
posted by qxntpqbbbqxl at 9:17 AM on June 16

I had an entire course in library school in how to do this and indexing. Both are much harder than you might think to do decently than you might think. I strongly doubt software is anywhere near being able to do it, and I'd check the credentials and get samples from anyone I was considering hiring.
posted by QIbHom at 10:05 AM on June 16