Soft 404 bug on Google: A crime thriller in four acts

In the period from April 5th, 2021 to April 19th, 2021, a mysterious Google bug preoccupied the SEO world and caused some webmasters and SEOs to worry. Since one of our customer projects was affected by a massive drop in the visibility of the desktop search as a result, I would like to go into the following on the process and our measures to get on the track.

A chronology of the events surrounding the Soft 404 bug

Tuesday, April 5th, 2021: Immediately after the Easter holidays we had to deal with a significant drop in visibility at Sistrix (approx. 10%). The first thoughts that came to our minds were:

• Was there a core update from Google?
• Is it a natural negative fluctuation in the rankings that quickly recovers?
• Is it a bug with Sistrix?

There were no reports on the first question. For a natural fluctuation, 10% is quite a lot. The normal fluctuations in visibility are usually <5%. A look at ahrefs showed a similar picture. In addition, it was found that when important keywords were queried manually, the rankings collapsed massively and in some cases even disappeared completely from the top 100, although they had previously been on page 1 of the SERPs. If rankings were still available, the URL changed. So a complete de-indexing of the affected URLs took place.

The next consideration was whether we are being affected in any form by thematic punishment or EAT devaluation. The link profile of the domain was mostly clean, no abnormalities in the link sources and anchor texts in the last 60 days. We also found that the losses had no thematic patterns or only applied to a specific area (s) of the page. It was a global affair that affected the entire domain, except for a few less important page areas.

The mobile visibility decreased slightly, but the desktop visibility was more noticeable.

We poked around in the dark and waited to see how things continued to develop. As a result, we gradually lost other URLs that were important for the company from the index, which was accompanied by further significant losses in visibility. While the mobile visibility increased again slightly, the desktop visibility plunged into the abyss. It decreased by up to 13% per day, which, with such a high level of visibility as Rossmann, also resulted in significant losses in organic traffic.

Another starting point was a CMS update that was carried out in mid-March. We also took a close look at all the IT tickets that were implemented in the past few weeks. In the meantime, we had also compared a screaming frog crawl back-up from March with a current crawl in order to identify changes to SEO-relevant points. Nothing!

In February we carried out a page experience audit for Rossmann and gave IT some recommendations for action, some of which had already been implemented. We considered two implemented IT tickets that arose as a result of the Page Experience Audit as a possible cause.

We discovered the first really concrete trace on Thursday, April 8th, 2021. When checking some of the affected URLs via the Google Search Console, we discovered a soft 404 error due to not loaded resources in the live URL query, which could only be reproduced for the Google Desktop Bot.

I then posted a tweet to John Müller, where I described our problem, but unfortunately got no answer.

I also came into contact with some SEOs from English-speaking countries via Twitter and LinkedIn who observed similar problems or symptoms.

The possibility that we might be dealing with a Google-side problem could therefore be considered.

Another Facebook post from me and conversations with external SEO colleagues gave some further approaches, why only the Google Desktop Bot had the problems and therefore only the desktop rankings were affected.

eg problems with settings

• in the CDN
• with a user agent / IP switch
• with an A / B testing tool

At that point it was clear that we were dealing with a technical problem. But it was still unclear whether it was a technical problem with us or with Google. It felt like looking for the famous needle in a haystack.

After the visibility crash on Thursday, the domain lost on Friday, April 9th. once again clearly in terms of visibility. I had already entered the Google Webmaster Hangout on the calendar days before in order to confront Google directly with the problem in the form of John Müller.

Here was the next hot lead. In response to the description of our problem, John explained that various webmasters had reported similar problems and that Google was currently checking sample URLs because the Google bot had recently fine-tuned the Soft 404 detection. He also admitted that they discover inexplicable things but still need more examples. I then emailed examples of affected URLs before the weekend.

Over the weekend, various SEOs who had similar problems got in touch with me because of my tweet and the webmaster hangout. This made it clear that this change on the part of Google contributed significantly to the problem.

The big questions now were:

• How quickly does Google recognize and fix the problem?
• What influence does our setting have on the problem?

Little by little , more and more media took up the problem, such as SEOsuedwest , Search Engine Round Table and Search Engine Journal . We hoped that this would put pressure on Google to fix the problem quickly.

The situation had become so acute that even Rossmann’s homepage was de-indexed from the desktop search and a version of the homepage that was actually forwarded via 301 was ranked in the brand search, but without sitelinks.

We couldn’t or didn’t want to wait until Google solved the problem on its own, but continued researching and analyzing together with our Rossmann colleagues. Through a forum thread on the topic in the Google Webmaster Forum, we came across a tip on how to use the Rich Result Testing Tool from Google to actively select the user agent for a URL test and thus analyze a URL for the desktop bot .

We encountered a soft 404 error in the affected URLs, which indicated a page resource that was not loaded. The A / B testing tool used works with frames and leads to broken forwarding. That was on Tuesday, April 13th, 2021.

There were some tests by Rossmann whether the Soft404 errors disappeared when the tool was deactivated, but this was not the case, whereupon it was decided to let the A / B testing continue. Up to now, the desktop visibility had crashed to around 60% of the original visibility by over 30% with a corresponding decrease in organic traffic via desktop and stagnated at the same level. Thank goodness the mobile traffic is significantly higher than the desktop traffic .

In the meantime, SEOs worldwide reported that the rankings since Thursday April 15th. come back slowly. We can do that since Saturday, April 17th. watch after we temporarily removed the A / B testing code from the page.

The SEO thriller and its end

Google obviously took the problem seriously and reacted relatively quickly. We still have to find out to what extent A / B testing is still a problem. Information from Google as to whether they have reversed the change in Soft 404 detection or have only adjusted it would also be helpful with this question. In the meantime, John Müller replied to numerous questions on Twitter as to whether and what Google has changed. John’s response was that the bug would probably have only affected a few pages and in such cases will not go into detail.

From the statement made by John, one can deduce that the bug only affected certain website settings, i.e. that it was a combination of various factors.

This case nicely shows how exciting the life of an SEO can be and that, in addition to experience, commitment, a good nose and a good network, it can also master the trickiest tasks and challenges.