SGE is not the answer

Why a half-baked AI search experience was rushed out the door — and won’t cut it for today’s searcher.

We’ve been closely watching the development of new generative AI search features — both Bing’s AI-generated “stories” and Google’s “Search Generative Experiences” (SGEs).

Our first priority is to make sure we’re prepared for any possible courses of action from Google in particular, as it’s the search engine our clients care most about.

That said, while we have to be ready for any eventuality, we’re not expecting SGE in its current form to ever see a full rollout, and in this post I want to explain why.

Wait, what’s an SGE?

Search Generative Experiences are basically Google’s answer to “the New Bing” in Spring of 2023. Here’s what they both look like:

In both cases, this is basically a big block of content at the top of a SERP that is written by a large language model.

Most SEOs are naturally more concerned about the Google version, owing to Google’s overwhelming market share.

To see Google’s SGEs right now, you need to:

• Sign up to Google’s “Search Labs” program
• Be logged in
• Be on Chrome (or the Google app on Android or iOS)
• Have a US, India, or Japan IP address… for now (a VPN works fine, funnily enough)

SGEs for every search?

Before we get into the crux of the argument here, it’s worth sharing some background on what SGEs are typically like right now. All this data is taken from the top few keywords in every MozCast vertical, totalling 100 US desktop keywords.

One thing we’ve noticed in our testing is that Google is not showing SGEs by default as much as it was initially. There’s now often a “generate” button.

In fact, this was the case for the majority of the SERPs we tested.

Organic overlap

If you do see (or choose to see) an SGE, as an SEO your eye might be drawn to the prominent three links in the top right. Those are what the AI-generated snippet is supposedly based on.

You’d be forgiven for assuming these are pages that also appear in the organic top 10, but in fact that isn’t the case. Indeed, in over 40% of cases, there was no overlap whatsoever between these links and the organic top 10, which suggests they’re built on a different system altogether.

What about the content within SGEs?

Within the SGE itself, overwhelmingly we just saw text — reminiscent of a featured snippet. The most common feature besides text is a map result, but often this is done quite clumsily.

Take a look at this example:

Two main issues immediately appear:

1. The three results in the top right overlap completely with the top three map results in the top left. This is a waste of space.
2. There’s also a conventional local pack immediately beneath. This is a further waste of space.

Which brings me to the next point…

Executional problems

If, like me, you’ve jumped through hoops and spent some time playing around with this feature, you’ll have noticed that SGEs are just… not very good.

Firstly, there’s the overlap problem mentioned above.

This functional overlap is fairly common. If you look at SERPs which have some sort of map (either kind), 35% have both. There doesn’t seem to be any good reason for that since the functionality and intent is nearly identical.

Indeed, SGEs are basically just bolted onto the top of existing SERPs regardless of issues like this. We compared SGE-enabled SERPs to their opted out (i.e. logged out) equivalents, and saw basically identical features beneath SGEs to what would appear on an equivalent SGE-free SERP.

This makes it seem like this feature was potentially pushed out in a rush, and perhaps more to the point for Google, it’s also not well monetized (yet).

SGE SERPs are often not monetized, and when they are, the ads are relegated way down the SERP from where you’d normally find them.

This is without going into the numerous accuracy and plagiarism issues highlighted by many in the SEO industry.

To be fair, Google is still iterating and testing new features on SGEs — they’re not done yet. But progress is slow. Ultimately this has been in testing for months, and still feels like there’s a long way to go.

Conceptual problems

It’s not just execution problems, though. My colleague Pete sums it up well here:

And he’s not the only one.

The fact is that SGE, for now, mostly shows similar or worse information to existing Google SERP features, but with a perhaps 5-10 second loading time (Cumulative Layout Shift, anyone?), added cost to Google, and all sorts of monetization and legal issues to solve.

Bing seems to have run into the same UX concerns. The screenshot above of a Bing AI-generated “story” in the SERP is actually from much earlier this year. Here’s what it looks like now:

Bing is still going to the trouble of generating these “stories,” but I really struggled to find a Bing SERP now that put it at the top of the results, like they all were back in March. This won’t be an accident — Bing has realized that their original results were a better answer.

Lastly, I’ve not even gotten into the legal problems in this post, but Google has been successfully legislated against in France for featured snippets, and in both France and Australia to force revenue sharing with news publishers.

Arguably the featured snippets case was a pyrrhic victory for French publishers, but that is a fraction of the challenge posed by either the plagiarism or disinformation aspect of SGEs down the road.

Right now it’s kind of lawless, but eventually regulation will come. (It’s likely not a coincidence that this feature is currently not available in the EU.)

So, why did they do it then?

Under huge pressure from investors, competitors, and the media, keen to portray Google as losing an AI arms race and at risk of losing its dominance of search, Google had to do something — so they pushed out a rush job.

If SGE was really how they wanted to use LLMs, they’d have done it years ago. Instead, we suspect the way this technology will be utilized by Google is far more subtle, as with AI tech in the Google algorithm for years. (My former colleague Tom Anthony spoke about this recently at MozCon.)

For SEOs, then, that means a likely less radical departure than some are fearing right now. SERPs will continue to evolve, gaining sophistication, new features, and ever more nuanced ranking signals.

They’ll also almost certainly wipe out more and more of the low-end, commodity organic results that featured snippets, knowledge graph results, and now, generative AI responses, handle well.

But a huge block of nonsense appearing after a lengthy delay at the top of every SERP? Unlikely.