Google search with many results

November 9, 2008 9:20 PM

A Google Books search of "charles dickens" (Full view only) returns "Books 1 - 10 of 156,600 " - but when I click through the Result Pages, it stops after the 70th or sometimes 100th hit and won't show any more. How do I view all results - is 156,600 even a reliable number?
posted by stbalbach to computers & internet (10 comments total)

How does Google calculate the number of results?
...Google's calculation of the total number of search results is an estimate. We understand that a ballpark figure is valuable, and by providing an estimate rather than an exact account, we can return quality search results faster....
posted by niles at 9:30 PM on November 9, 2008

I concur 156,600 is in the right ball park - the only problem is I can only see the first 70 or 100 hits. Try it - try to see the 200 or 300th search result.
posted by stbalbach at 10:13 PM on November 9, 2008

Crosschecking numbers, 156,600 seems far too high for the number of unique books that Charles Dickens either wrote, or books that were written about him.

One thought is that the search results aren't showing full-text search results, just the 70-ish books where "Charles Dickens" appears in the title or author field, and the 156K number corresponds to the number of books where "Charles Dickens" appears anywhere inside the book, but for some reason those results don't show up.

Crosscheck: A "charles dickens" search on archive.org shows 723 results, which seems to correspond to books where only "Charles Dickens" is in the title or author fields. Bibliography of Dickens on the wiki lists about 100 works.
posted by rajbot at 10:41 PM on November 9, 2008

When I hit the link, Google says "Books 1 - 10 of 1,075". And I can browse to #102. I've read an explanation somewhere once about why this stuff happens (I think on Tim Bray's blog, but I can't find it right now) - it's about caching and indexing and computer science stuff.

In short, those numbers are not reliable.
posted by dhoe at 11:45 PM on November 9, 2008

I've found that adding a extra word to a search phrase can actually return more results. For example, "charles dickens wrote" gives me 313 results, and "by charles dickens" gives me 319, while "charles dickens" gives me just 69.
posted by Knappster at 11:54 PM on November 9, 2008

I thought I had this, because when I clicked on your link it was set to show 'full books only', with 1069 results.

But when I changed to show all, it said there were ~548K results, but only showed me a couple of hundred.

Google Book search used to be lousy: I played round with it when they launched, and couldn't find phrases that I knew were in the books, even though I could browse to the relevant page of the book. But I thought it had got much better.

Sorry, can't give an answer, I think rajpot is closest.
posted by Infinite Jest at 1:00 AM on November 10, 2008

Clicking on your link I get " Books 1 - 10 of 671"

When I go to page 11 I get "Books 101 - 110 of 119"

It wont let me get to page 12 though.
posted by missmagenta at 1:41 AM on November 10, 2008

These are all strange and unreliable results. I guess Google Books Search is fundamentally broken. Do they know this, does anyone know this?

rajbot: The beauty of Google is, it's not just books by or about Dickens, it should be any mention of Dickens, in any book - which should be in the tens if not 100s of thousands. It's supposed to be full-text search. Internet Archive is, as you say, only books by or about Dickens, it is not full text search.
posted by stbalbach at 6:17 AM on November 10, 2008

It is a strange way to present the results. I get 112K as the total results as I click through the first five pages, after which it suddenly drops to 66, but if I go to page 10 directly it's 96.

I don't think it's "fundamentally broken", but this particular results reporting figure is not tremendously useful. There are numerous other aspects of GBS I'd rather have them fix first.
posted by dhartung at 3:27 PM on November 10, 2008

Well, for my purposes, it's discouraging. It's not a full text search of the database. I can't rely on it, it seems to pick and choose a selection that changes each time. This was the great thing about GB, the full text search across the entire library. As it is, we can still find entire books by an author (using inauthor:), which is fine, but not rely on much more than that. The deep searching of books is a big appeal of GB, but it doesn't appear to do it completely - worse, it gives the appearance of doing it, but actually doesn't. And it's not even like the database is that large - there are only around 1 to 2 million public domain books (full text) which is very small for Google.
posted by stbalbach at 8:17 PM on November 10, 2008