Can one search only a subset of web pages

February 15, 2022 11:45 AM

There is a recent FPP and associated MetaTalk that together have a long list of promising recipe sites. Is there a way to put together a collection of these (or any other) sites and then search (google or otherwise) and get results only from these sites? I'm guessing not, but a girl can hope...
posted by 2 cats in the yard to Computers & Internet (14 answers total)

This looks to be possible by using Googles site:foo.com construct; you could do something like:

"sourdough bread" site:foo.com OR site:bar.com OR site:qux.com

This could get unwieldy if the list of sites is long. Someone who is more clever than me may be able to hack together a bookmarklet or similar.
posted by jquinby at 11:57 AM on February 15, 2022 [3 favorites]

...or leverage Google's custom search API: https://developers.google.com/custom-search/docs/overview
posted by jquinby at 11:59 AM on February 15, 2022 [1 favorite]

Sorry for spamming. Give this a shot....copy the following into a google search, hit enter, and bookmark the resulting page:

site:www.turkeysforlife.com OR site:www.budgetbytes.com OR site:www.bethcato.com OR site:shaneandsimple.com OR site:mefiwiki.com OR site:www.welcometopalestine.com OR site:www.villajerada.com OR site:www.supercook.com OR site:www.splendidtable.org OR site:www.simplyleb.com OR site:www.seriouseats.com OR site:www.saveur.com OR site:www.recipekeeperonline.com OR site:www.ranchogordo.com OR site:www.plantoeat.com OR site:www.pickuplimes.com OR site:www.omayahcooks.com OR site:www.mamaslebanesekitchen.com OR site:www.maangchi.com OR site:www.loveandlemons.com OR site:www.jamieoliver.com/recipes OR site:www.getmunchedup.com OR site:www.eatyourbooks.com OR site:www.copymethat.com OR site:www.brandnewvegan.com OR site:www.bonappetit.com OR site:www.bbcgoodfood.com OR site:www.barilla.com OR site:www.allrecipes.com OR site:thewoksoflife.com OR site:tasteofpalestine.org OR site:smittenkitchen.com OR site:simpleindianrecipes.com OR site:schoolnightvegan.com OR site:sallysbakingaddiction.com OR site:rainbowplantlife.com OR site:mela.recipes OR site:mefiwiki.com OR site:www.kitchenofpalestine.com OR site:intifooda.com OR site:ibunbury.blogspot.com OR site:hadiaslebanesecuisine.com OR site:food52.com OR site:cooking.nytimes.com OR site:chefsteps.com OR site:chefindisguise.com OR site:blog.themalamarket.com OR site:abeernajjar.com

I grabbed these off the MetaTalk string - you can add sites as needed. I took the youtube playlists off because I wasn't sure if they'd work, but they might. Just prepend what you're looking for before all the sites and you're good to go.

edit: it looks google restricts searches to 32 words and is cutting off the sites:

'"site:tasteofpalestine.org" (and any subsequent words) was ignored because we limit queries to 32 words" :/
posted by jquinby at 12:21 PM on February 15, 2022 [2 favorites]

Neither Bing nor Yahoo does not limit search strings, but their results are not as pretty but YMMV. I'm really done now.
posted by jquinby at 12:28 PM on February 15, 2022

Response by poster: Thanks jquinby for all your thoughts on this! It does seem to work ok on bing - haven't tried yahoo yet. I can save that string that you so kindly put together as text somewhere and edit it as needed.
posted by 2 cats in the yard at 12:49 PM on February 15, 2022

Response by poster: On closer inspection, it doesn't actually work. It does give results on bing, but they are not all results that actually contain the search term. Or maybe they do contain the search term, but in some SEO part of the site that doesn't come up when I do ctrl-f on the page.
posted by 2 cats in the yard at 1:01 PM on February 15, 2022

If you bookmark the sites you like the best, they will come up first(-ish) when you search for recipes if they have them.
But as I wrote in the discussion, you can train the algorithm if you discipline your search for a month or so, and get far better results.
posted by mumimor at 1:31 PM on February 15, 2022

Best answer: Looks like custom Google search engines still exist. I was surprised. You could spin up a free Google Sites website and drop your custom Google search engine into it. If you do it I bet a lot of us here would like to use it.
posted by COD at 2:01 PM on February 15, 2022 [2 favorites]

Best answer: Came here to say what COD said, you can make a custom Google search and add as many of the following as you'd like (or if there is a limit I didn't hit it):

You can add any of the following:
Individual pages: www.example.com/page.html
Entire site: www.mysite.com/*
Parts of site: www.example.com/docs/* or www.example.com/docs/
Entire domain: *.example.com
posted by tiamat at 2:13 PM on February 15, 2022 [2 favorites]

Best answer: The custom search engine seems to be the trick! I converted that list to one here just to see how it works; making your own will necessarily be replicating the work but this way you'll be able to edit the list of sites as you like. For example - my list didn't include any URIs - just the top-level site name, but you can be a little more fine-grained in GSE.
posted by jquinby at 2:13 PM on February 15, 2022

Response by poster: This works! I am not up for making a public search page associated with my google account, but I will slice and dice the web pages listed in the FPP and MetaTalk into groups (all, "international" (or by nationality), mainstream, product-based, vegan, any other groupings that make sense...) and if someone else wants to host this, let me know and I can send along the lists.
posted by 2 cats in the yard at 2:31 PM on February 15, 2022 [1 favorite]

I’ve found a simpler approach is to use -site:badsite.com to exclude the site (or sites) that I don’t want to see in the results. Works fine on duckduckgo.com - no idea about the other, more privacy-invading search engines.
posted by DaveP at 9:12 AM on February 16, 2022

Response by poster: A previous ask had a bunch of additional recipe links, plus Punchfork, which appears to do more-or-less what I asked for above, but with their own curated list of sites.
posted by 2 cats in the yard at 11:36 AM on February 16, 2022

There's an old version of this at https://foodblogsearch.com/
posted by 168 at 10:20 AM on February 20, 2022