The Core

Sometimes I wonder whether it isn't time for content-only sites (i.e. no e-comm) to simply ban Google and all the AI training bots.

You asked that before when I asked a similar question and I have rattled around various answers in my head without finding anything satisfactory.

I guess my thought is that if Google searches don't end in clicks, then that is ceasing to become a way to be found.

I've recently handed off the last e-comm site I oversaw, but that's at one extreme: they sell a product that mostly you can only get through them though they do have a couple of specialty retailers that sell their stuff. In that case, of course, it's "come one, come all." They don't care if Google and AI bots steal their content because at the end of the day, if someone ultimately decides that this product is what they're looking for, they have to come to the website to get one. So in cases like that, it makes no sense to ban Google of course.

At the other extreme, let's say you have a pure content site that consists of essays - your personal philosophy, political analysis, industry analysis, something like that.  There is a glut of those kinds of sites and ultimately what people are looking for there is a voice that resonates. The problem is, that is precisely the thing you cannot search for using Google.

Looked at another way...

Writing that is friendly to search engines is exactly the writing that is easiest to duplicate with AI. Therefore, most of what passed for "content" in the past 30 years, is simply not viable in the future.

So if you have a voice that somehow cuts through the chaff and rises above AI-generated content, I would say that is decreasingly findable with a search engine. So you have to fall back on old-school, pre-Google marketing

 - you show up at events and talk to people or, if you can, speak
 - you ask people to talk about you
 - once people show up, you try to get them to subscribe and you ask your subscribers to recommend you to friends
 - you get on podcasts if you can
 - you put short versions of your best ideas on social media and use that as a funnel to Substack/RSS

And so on.

I think we got used to a world where you could do some keyword research, find some gaps, write content that filled the gaps, get traffic from Google. The problem is that is exactly the sort of thing AI is good at, so I see that sort of content as dead. If not today, very soon.

But again, the content that falls outside of that is not discoverable by searching on Google because it never really was. If I think of the voices that fall into that category for me, I found most of them because they appeared on a podcast or got mentioned in a book or wrote a book or something like that. To the extent that I ever found any of them via Google, it was usually a second or third order effect.

But much of the Web 1.0 infrastructure has died and is now having to be rebuilt from scratch.  They aren't pro or anti Google, they just don't care.

This.

And this is why I have trouble putting my thoughts into words. It's like the "old Japanese story" from the Marshall McLuhan circle: I don't know who discovered water, but I'm sure it wasn't a fish.

My mind has been so conditioned to think of search as an indispensable part of the DNA of the web with Google being the dominant allele of the search gene. But there is nothing inherent that says that search as we have known it, must be a part of the web and that it is healthy for it to be seen as the main way to access material and discover new material.

Obviously, if you're trying to make a living from ad-supported content, you need to both live in the present and plan for the future, which means still doing classic SEO, but planning for the day when Google simply no longer cares about sending you traffic.

My idea of blocking Google is a "first mover"/"collective action" problem, like many others. Most parents believe kids shouldn't have cell phones and access to social media at young ages, but it's hard to be the oddball parent who denies these things to your kids when every friend they have has access to them.

Ad-supported sites are mostly playing a volume game, right? They need to harvest a tiny amount of money from each of a large number of visitors. So they really can't block any traffic source.

But other sites, supported by subscription or donation or simply passion projects, don't need volume. To use Kevin Kelly's overused phrase, they just need 1000 true fans. For them, unqualified traffic is a cost, not a resource. Those seem like the sites that would pay the lowest first-mover penalty and can most afford to raise a middle finger to Google and just say: "Look, if you are going to use my content to sell your ads without ever sending people to my site, fine. I am going to block you."

How will Google respond?

A semi-analogous situation occurred early in the pandemic. Airbnb had become super guest-centric. It was starting to really upset hosts and I knew hosts who were leaving the platform because *any* guest who called customer service for *any* reason would be given a refund. Airbnb hadn't yet understood the problem. But then the pandemic hit and they absolutely threw hosts under the bus and people started leaving the platform in droves. They realized that without hosts, they could not have guests. Airbnb has no asset other than the ability to connect guests to hosts and without hosts, they have no business.

The made a significant pivot and now have an ecosystem that is much more balanced and fair to both parties. But it took an existential crisis of, effectively, being blocked by the "creators" for them to realize the problem.

If people grumble about Google but nobody has the chutzpah to block Google, why would they ever change and treat content creators better?

At a certain point, it's better to block Google than continue to put up with the abuse, just as it's better sometimes to quit a high-paying job with an abusive boss.

That's all a separate problem, of course, from the fundamental limitation of keyword-based search that I mentioned above.  Namely:

"Writing that is friendly to search engines is exactly the writing that is easiest to duplicate with AI. Therefore, most of what passed for "content" in the past 30 years, is simply not viable in the future."

I think that's something content creators need to prepare for too regardless of how friendly or hostile Google wants to be to creators.

This is an interesting and thought provoking conversation.  I have recently included a Gemini protocol browser in my side project.  It is similar to https, but close to gopher too.  It does seem to be struggling a bit, but there are some cool projects out -- like a full Wikipedia mirror.  I mostly see a lot of micro-blogging, but I think it has potential.  Right now I would say it is mostly an information-centric alternative protocol inhabited nearly 100% by geeks.