Why is search so terrible all of a sudden?
March 13, 2023 12:30 PM Subscribe
Is it just me? Yahoo mail search became virtually unusable a few months ago, returning pages and pages of useless search results. Google is now doing the same. Does anyone know why search suddenly doesn't work? Are there search engines that provide the kind of useful result you got when you went to Google a decade ago?
It's not you, every search engine just sucks now. Here's an article about it. Here's a google search engineer's comment on reddit telling their side of the story, which adds up to nothing useful and reads a bit gaslighty to me, but might be of interest to some folks.
One good tip is to append "reddit" to your searches. You'll get real people's answers to most questions that way, not a bunch of clickbait headlines that serve up ads galore. Another tip I've recently come across is to use the more esoteric Google search operators. I can see how this might be strategically used to get better search results from google.
Breaks my heart how the internet used to be and how it is now.
posted by MiraK at 12:51 PM on March 13, 2023 [41 favorites]
One good tip is to append "reddit" to your searches. You'll get real people's answers to most questions that way, not a bunch of clickbait headlines that serve up ads galore. Another tip I've recently come across is to use the more esoteric Google search operators. I can see how this might be strategically used to get better search results from google.
Breaks my heart how the internet used to be and how it is now.
posted by MiraK at 12:51 PM on March 13, 2023 [41 favorites]
IMO, it's because the web currently is sort of 'shrinking', in that pages that displayed results for more obscure topics age off. So google can only search and index what's available, and if some person stops maintaining the specific page that contained the info you want, there's not much it's willing to do. Google used to maintain older versions of websites, but I guess they don't do that any more or don't publicize it the way they used to ('see cached version' below the primary link used to be an option).
If you search for popular, relatively common things, search is pretty good, and in many ways better than ever. The more obscure you get, the worse it gets.
That's what I have personally found.
posted by The_Vegetables at 1:02 PM on March 13, 2023 [4 favorites]
If you search for popular, relatively common things, search is pretty good, and in many ways better than ever. The more obscure you get, the worse it gets.
That's what I have personally found.
posted by The_Vegetables at 1:02 PM on March 13, 2023 [4 favorites]
I assumed it was chat GPT -written articles and similar.
Marginalia:
Marginalia:
Ever feel like the Internet has gotten a bit... I don't know, samey? There's funny images scrolling by and you blow some air through your nose and keep scrolling and then someone has done something upsetting and you write an angry comment and then you scroll some more.posted by aniola at 1:06 PM on March 13, 2023 [13 favorites]
Remember when you used to explore the Internet, when you used to discover cool little websites made by people and it wasn't just a bunch of low effort content mill listicles and blog spam?
I want to show you that the Internet you used to go exploring is still very much there. There are still tons of small personal websites, and a wealth of long form text from both the past and the present.
So it's a search engine. It's perhaps not the greatest at finding what you already knew was there. Instead it is designed to help you find some things you didn't even know you were looking for.
If you are looking for facts you can trust, this is almost certainly the wrong tool. If you are looking for serendipity, you're on the right track. When was the last time you just stumbled onto something interesting, by the way?
I don't expect this will be the next "big" search engine. This is and will remain a niche tool for a niche audience.
I assumed it was chat GPT -written articles and similar.
No. It's allowed lazier to people to now contribute to the problem, but's not doing anything special.
The war between search engines and those who want to influence said search engines has been waged for 25 years. For a long time, SEO was all about backlinks, headers, titles, meta tags, whatever, but the actual content could still be good content.
However, within the last 5ish years, Google and friends have gone hard into trying to parsing the content for not just relevance, but some sort of AI/ML decision about how "good" it is.
So, after like two decades of various strategies left the content alone, now we have entered "garbage time", where you better put out like 10k words to get your point across, or you won't see the first 10 pages of Google results.
To see this practice, search for crossword or Wordle hints. You'll notice that any website that comes near the top has the same 98% garbage content every single day, with just like 1-3 lines specific to whatever you actually asked about. In the good ol' days, you could have a page with just those 3 lines you'd rank first because you were known as the top Worldle hint guys. Now some dickhole with the same info but 15 paragraph of gibberish will blow you page rank out of the water.
posted by Back At It Again At Krispy Kreme at 1:38 PM on March 13, 2023 [22 favorites]
No. It's allowed lazier to people to now contribute to the problem, but's not doing anything special.
The war between search engines and those who want to influence said search engines has been waged for 25 years. For a long time, SEO was all about backlinks, headers, titles, meta tags, whatever, but the actual content could still be good content.
However, within the last 5ish years, Google and friends have gone hard into trying to parsing the content for not just relevance, but some sort of AI/ML decision about how "good" it is.
So, after like two decades of various strategies left the content alone, now we have entered "garbage time", where you better put out like 10k words to get your point across, or you won't see the first 10 pages of Google results.
To see this practice, search for crossword or Wordle hints. You'll notice that any website that comes near the top has the same 98% garbage content every single day, with just like 1-3 lines specific to whatever you actually asked about. In the good ol' days, you could have a page with just those 3 lines you'd rank first because you were known as the top Worldle hint guys. Now some dickhole with the same info but 15 paragraph of gibberish will blow you page rank out of the water.
posted by Back At It Again At Krispy Kreme at 1:38 PM on March 13, 2023 [22 favorites]
Atlantic The Open Secret of Google Search partly on a Google search ‘AI’ that parses searches to 'understand our intent' – and return content based on that. Yeah, that explains a lot, I don't see how a machine can ever guess what I'm trying to get at … many of us are using search to join the dots between very disparate data.
I'm more and more having to put things in “ ” as Google seems to be doing a lot more (very off the wall) wildcarding - the 'AI' above. And even with “ ” Google still sends back rubbish, like this morning searching for
“prostrate clover”
and getting all kinds of adverts and other rubbish.
I do 99% of search using a very locked down firefox with location blocked so Google is naive to my previous searches - and this is a lot more successful.
posted by unearthed at 1:42 PM on March 13, 2023 [5 favorites]
I'm more and more having to put things in “ ” as Google seems to be doing a lot more (very off the wall) wildcarding - the 'AI' above. And even with “ ” Google still sends back rubbish, like this morning searching for
“prostrate clover”
and getting all kinds of adverts and other rubbish.
I do 99% of search using a very locked down firefox with location blocked so Google is naive to my previous searches - and this is a lot more successful.
posted by unearthed at 1:42 PM on March 13, 2023 [5 favorites]
One good tip is to append "reddit" to your searches
So good that sometimes I see reddit-ed queries in Google's own suggestions.
I can't read the article above, but one thing I've heard is that Google has basically put every search position up for sale, and people are buying. Couple that with the intrusive tokenization that unearthed mentions, and it takes multiple iterations of a query just to get close. I get "there weren't very many results for your search" quite often, and it's not because I'm trying to come up with a dissertation topic.
posted by rhizome at 2:25 PM on March 13, 2023 [5 favorites]
So good that sometimes I see reddit-ed queries in Google's own suggestions.
I can't read the article above, but one thing I've heard is that Google has basically put every search position up for sale, and people are buying. Couple that with the intrusive tokenization that unearthed mentions, and it takes multiple iterations of a query just to get close. I get "there weren't very many results for your search" quite often, and it's not because I'm trying to come up with a dissertation topic.
posted by rhizome at 2:25 PM on March 13, 2023 [5 favorites]
Search has been getting worse for ages. Part of that's SEO, part of it is people putting their content behind paywalls or in walled garden networks (Facebook, Discord, Patreon, Substack, etc), and part of it is the search engine priorities changing.
Check out some of the newer search engine projects like Kagi. They really are substantially better, but pay-for-search may not be something you can get behind. They can also only solve some of the problems with modern search and not things like communities moving into semi-private Discords and not being searchable.
IMO, we really need to reboot the culture of personal linkblogs and personal directories.
posted by forbiddencabinet at 2:56 PM on March 13, 2023 [11 favorites]
Check out some of the newer search engine projects like Kagi. They really are substantially better, but pay-for-search may not be something you can get behind. They can also only solve some of the problems with modern search and not things like communities moving into semi-private Discords and not being searchable.
IMO, we really need to reboot the culture of personal linkblogs and personal directories.
posted by forbiddencabinet at 2:56 PM on March 13, 2023 [11 favorites]
Another issue: Google, at least, started ranking non–SSL/HTTPS sites lower. So now someone who just has a little personal website about their projects or a topic of interest is supposed to pay to get an SSL certificate, or they'll get ranked lower by Google and showed with a warning in Chrome. Yes, security is important, but for most basic websites, having to pay for and set up SSL is a bit of an unnecessary monetary and technical barrier to entry. It's really unfortunate.
posted by limeonaire at 3:00 PM on March 13, 2023 [5 favorites]
posted by limeonaire at 3:00 PM on March 13, 2023 [5 favorites]
Nobody outside of Google/Yahoo knows the full answer to this question, so we can only speculate. My own speculation is that Google has started assuming every website is playing the SEO game, so it can set policy about how websites should work (using HTTPS, loading instantly, etc.) and the "good" websites that want to appear at the top of search results will follow these policies. In reality, the people who pay the most attention to these policies are people who are trying to subtly or not-so-subtly sell you something -- so their websites are the results you get now.
posted by panic at 3:43 PM on March 13, 2023 [5 favorites]
posted by panic at 3:43 PM on March 13, 2023 [5 favorites]
The big change I have noticed on Google search is their focus on recency, if you post a comment here on metafilter it will usually appear in the Google search index within about 1 hour, in some places it will update in minutes.
This is great if you are searching to buy a product where prices change every day – just what advertisers look for.
What's not so great is having every page that's more than about a month old being excluded from your search results. This is why the "reddit" trick works, if a page with useful information on it is 10 years old, Google probably won't show it to you, but if someone on reddit has discussed that same page in the last month, then it will at least show you that.
posted by Lanark at 3:46 PM on March 13, 2023 [2 favorites]
This is great if you are searching to buy a product where prices change every day – just what advertisers look for.
What's not so great is having every page that's more than about a month old being excluded from your search results. This is why the "reddit" trick works, if a page with useful information on it is 10 years old, Google probably won't show it to you, but if someone on reddit has discussed that same page in the last month, then it will at least show you that.
posted by Lanark at 3:46 PM on March 13, 2023 [2 favorites]
This passage from the Atlantic article:
In reality, our samenesses are rarer, deeper, and better, and our differences -- our uniquenesses, the parts of us you can't learn by trying to pattern match with other people -- are the best parts, and the parts that can make life bearable, and that could potentially save us if only we gave them room to explore and build things that have never been seen before, by synthesizing things and ideas that have almost never been seen before.
This is the tragedy of AI: it doesn't listen to us saying that we're actually unique individuals. It ignores what we say we want, asserts loudly that we're just a typical example of a type, and gaslights us into thinking that all we ever really wanted was to be like everyone else and watch a video of some poor cat not quite making the leap he attempted.
That attitude is why so many of us don't spend time with our parents. It's very painful to have your humanity bulldozed.
posted by amtho at 4:57 PM on March 13, 2023 [11 favorites]
...[Google now] attempts to understand not just what the searcher is typing, but what the searcher is trying to get at...Matches my experience, and also matches the "I think a little outside the box all the time" frustrating experience of trying to communicate with people who assume you are just as shallowly, immediately focused as they assume everyone is, and who assume everyone is only interested in the things they complain other people are interested in. It's frustrating, and such a lost opportunity to nurture people's exploratory, creative, specialnesses. It assumes we're all largely the same, or that we fall into the same few groups that people have been building and writing for and about for the decades since TV advertising invented "market segments".
In reality, our samenesses are rarer, deeper, and better, and our differences -- our uniquenesses, the parts of us you can't learn by trying to pattern match with other people -- are the best parts, and the parts that can make life bearable, and that could potentially save us if only we gave them room to explore and build things that have never been seen before, by synthesizing things and ideas that have almost never been seen before.
This is the tragedy of AI: it doesn't listen to us saying that we're actually unique individuals. It ignores what we say we want, asserts loudly that we're just a typical example of a type, and gaslights us into thinking that all we ever really wanted was to be like everyone else and watch a video of some poor cat not quite making the leap he attempted.
That attitude is why so many of us don't spend time with our parents. It's very painful to have your humanity bulldozed.
posted by amtho at 4:57 PM on March 13, 2023 [11 favorites]
If you're searching in DuckDuckGo (which still seems OK to me), you can prepend !r to your search to have it search Reddit for that term. There are other bangs available in DDG as well, even !g to search via Google.
posted by lhauser at 5:55 PM on March 13, 2023 [1 favorite]
posted by lhauser at 5:55 PM on March 13, 2023 [1 favorite]
I don't know the answer to your first question, "why", but in answer to your second question:
Kagi is excellent although it's not free
Neeva is an alternative I don't use but some people like it
posted by riddley at 8:14 PM on March 13, 2023 [1 favorite]
Kagi is excellent although it's not free
Neeva is an alternative I don't use but some people like it
posted by riddley at 8:14 PM on March 13, 2023 [1 favorite]
I’ve been using Bing AI for regular search and it is good at weeding out all the SEO cruft.
Too bad soon it’s going to crush those websites it’s currently curating for me.
posted by notyou at 9:03 PM on March 13, 2023
Too bad soon it’s going to crush those websites it’s currently curating for me.
posted by notyou at 9:03 PM on March 13, 2023
nthing the idea that Google is partly to blame for assigning lower rank to non-HTTP/non-SSL websites. Who were they to decide that everyone must use SSL protocols? On the other hand - why (in the US) isn't there a taxpayer-funded Certificate Authority with user-friendly interface to help create the certs we need?
posted by TimHare at 10:47 PM on March 13, 2023 [5 favorites]
posted by TimHare at 10:47 PM on March 13, 2023 [5 favorites]
Google doesn't have to be good while it can still reasonably assert itself as the only game in town.
posted by rikschell at 6:38 AM on March 14, 2023 [1 favorite]
posted by rikschell at 6:38 AM on March 14, 2023 [1 favorite]
Who were they to decide that everyone must use SSL protocols?
I mean, it's easy to see the benefits - more security, less risk of hijacking by nefarious sites, especially because plenty of people using search are not particularly computer literate, and plenty may be very young and not understand computer security. I see the downsides too. Certs are difficult to apply - I've looked at applying them to my home internet site and it's not straight forward.
posted by The_Vegetables at 7:27 AM on March 14, 2023 [3 favorites]
I mean, it's easy to see the benefits - more security, less risk of hijacking by nefarious sites, especially because plenty of people using search are not particularly computer literate, and plenty may be very young and not understand computer security. I see the downsides too. Certs are difficult to apply - I've looked at applying them to my home internet site and it's not straight forward.
posted by The_Vegetables at 7:27 AM on March 14, 2023 [3 favorites]
Certs are a nightmare to work with.
at my dayjob I work with some smart devops and IT folks and cert rollovers still fuck us up on the regular.
posted by Sauce Trough at 3:39 PM on March 14, 2023 [3 favorites]
at my dayjob I work with some smart devops and IT folks and cert rollovers still fuck us up on the regular.
posted by Sauce Trough at 3:39 PM on March 14, 2023 [3 favorites]
If you're doing IT and hosting your own servers etc then maintaining certs can be a hassle. But for small websites, you can just route everything through a free account at CloudFlare or similar and they'll handle the SSL for you. And even if you want to do it yourself certs are easy to get for free.
As for why must everyone use SSL?
Naively, the internet was built on trust with next to no security. Now that it's no longer an insider's club the bad people are doing stuff. And they've automated it. SSL is as important as your antivirus.
Think of it like communism - a nice idea, but doesn't work because people are asshats. So we need checks and balances. AKA SSL.
posted by gible at 11:34 PM on March 14, 2023 [1 favorite]
As for why must everyone use SSL?
Naively, the internet was built on trust with next to no security. Now that it's no longer an insider's club the bad people are doing stuff. And they've automated it. SSL is as important as your antivirus.
Think of it like communism - a nice idea, but doesn't work because people are asshats. So we need checks and balances. AKA SSL.
posted by gible at 11:34 PM on March 14, 2023 [1 favorite]
This thread is closed to new comments.
posted by Gorgik at 12:47 PM on March 13, 2023 [9 favorites]