Author Topic: Field testing a boatload of search engines.  (Read 28955 times)

Brad

  • Inner Core
  • Hero Member
  • *
  • Posts: 4155
  • What, me worry?
    • View Profile
Field testing a boatload of search engines.
« on: December 31, 2023, 04:38:46 PM »
Fair warning: this is a long long article, but I found it very interesting.

How bad are search results? Let's compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT
https://danluu.com/seo-spam/

Added:

It may be long but it's nice to see a serious and detailed take on this subject and not just a clickbait list you can read in 1 minute and is a really just a rehash of 20 other clickbait lists of search engines.
« Last Edit: December 31, 2023, 04:51:07 PM by Brad »

ergophobe

  • Inner Core
  • Hero Member
  • *
  • Posts: 9297
    • View Profile
Re: Field testing a boatload of search engines.
« Reply #1 on: January 02, 2024, 02:57:43 AM »
Long indeed. I didn't read but a portion of those 17,000 words (possibly more because it was long before the end that he said he was already at 17,000 words).

I am not surprised by his findings in general, but he did change my mind about how to think about ChatGPT hallucinations. If ChatGPT is supposed to give authoritative answers that can be trusted to make decisions, the hallucinations are a problem. But if we are just comparing it to the "hallucination-adjacent" answers you get from Google and Bing, then actually it starts to look pretty good.

And, Brad, did you swoon with love and appreciation when you saw that home page? I don't think it's possible to go more minimalist than that.

And I loved this quote from Paul Graham on one of the other posts: "What nerds like is the kind of town where people walk around smiling. This excludes LA, where no one walks at all, and also New York, where people walk, but not smiling." I had to hunt for the original and find it here

Brad

  • Inner Core
  • Hero Member
  • *
  • Posts: 4155
  • What, me worry?
    • View Profile
Re: Field testing a boatload of search engines.
« Reply #2 on: January 02, 2024, 10:21:41 AM »
> long

I spot checked his other essays and they are all long like this one.

> hallucinations

But it is a good warning with ChatGPT  - garbage in means garbage out.  And ChatGPT can't distinguish Truth any better than Google can.  For example, if you feed AI nothing but wacko conspiracy theories it thinks thats the objective truth.  It's a case for human editors no matter how flawed and unable to scale that is.

>swoon

Oh yeah, there was a whole lot of swooning going on here at Galactic HQ!  I didn't check the HTML code but it looks like he used Tables, Tables!  bless his heart.  It sure is minimalist, but that page will still be rendering years from now when our modern platforms have failed.

I do like it that he included his methodology so that others can try the same test on other search engines that are waiting in the wings.

ergophobe

  • Inner Core
  • Hero Member
  • *
  • Posts: 9297
    • View Profile
Re: Field testing a boatload of search engines.
« Reply #3 on: January 02, 2024, 04:33:37 PM »
And ChatGPT can't distinguish Truth any better than Google can.

The bold indicates the reframing that changed my mind. You always hear about how ChatGPT is kinda junk because it hallucinates. But his reframing of saying that it should be evaluated compared to Google, not compared to a research scholar in the domain in question, is interesting. It underlines the fact that the morass of junk returned by Google is indeed a morass of junk, especially in highly commercialized spaces.