GWT problems indexing Enfold (soft 404) with search activated (?s=)

Hi fromcouch

thanks for raising this issue – I thought I should “second” the issue.

So, having used enfold themes on some sites for around 3 years I suddenly get a lot of soft 404 errors for pages ending in /*?s= that appear on google console/webmaster tools.

I will try adding the disavow to the robots file.

However, any idea why this has suddenly become an issue – sometime in the last few weeks.

Regards

Richard

I’m also experiencing this issue as I suspect is every website using enfold with search. Are we certain there is not another solution besides altering the robots.txt file? This was not happening prior to updating enfold to the newest version.

Also, I received the notification below from Google this morning and all of the URLs in Goggle’s soft 404 report are for url/?=s.

Increase in “soft-404” pages on “my website name here”

To: Webmaster of “my website name here”

Googlebot identified a significant increase in the number of URLs on “my website name here” that should return a 404 (not found) error, but currently don’t. This can cause a bad experience for your users, who might have been looking for a specific page, but end up elsewhere on your website. This misconfiguration can also prevent Google from showing the correct page in search results.

Recommended Actions:

1
Identify the URLs with errors
Open the Crawl Errors report in your Search Console account to review the list of sample URLs.
Check Crawl Errors
2
Fix the issue
Check your server and CMS settings to make sure that these URLs return a 404 (Not Found) or 410 (Gone) HTTP response code in response to requests for non-existent pages. You may need help from your server administrator or hoster for this step.
3
Verify the fix
Once you’ve fixed the URLs with errors, make sure that Googlebot can access and see your content properly, or that they return a proper error result code. You can verify this using Fetch as Google.

Hi Ismael,

In your recommendation for adding to the robots.txt file you include: Disallow: /*?s=

Is there a purpose for the asterisk? I’m asking because the URLs in my GWT soft 404 report do not show an asterisk and I continue to accrue soft 404 errors despite the fact that I have added your recommended text to my robots.txt file. Also, I have verified that GWT has the correct robots.txt file for my site. Here is a sample link from my GWT:

what-is-the-annual-income-for-a-travel-nurse/?s=

Thanks for your help!

I also wanted to chime in and say I’m getting similar URL queries in Search Console. I’m also using the Enfold theme. I have used the robots file to block these search results but am still getting these 404 errors. It would be best to get rid of the bad code (as suggested by fromcouch) to allow Google to crawl the site more efficiently. In my opinion, the removal tool should not be used to get rid of crawl errors (there is even a disclaimer at the bottom of the help page that says not to use it for this).

Hi, I have also suddenly received the same warning email from Google that I have a sudden increase in soft 403 erros and the are all “/?s=” ending urls. After slightly panicking I found this thread, it seems I’m not alone with this issue
I’ve added as recommended above the following to my robot.txt
Disallow: /search
Disallow: /*?s=

thanks for this info.

Googles email at the end said “Once you’ve fixed the URLs with errors, make sure that Googlebot can access and see your content properly, or that they return a proper error result code. You can verify this using Fetch as Google.”

Do I need to do this now or just wait ?… how long does it take?
Many thanks

Hi Andy,

In a previous thread you said…

Most people want their search results appearing in Google

search-icon-causes-soft-404-errors

I still don’t understand this claim. Why would a URL that causes a Soft 404 error be beneficial?

Every error created by Enfold’s search box contains empty searches… that go to an error page or to the same page (duplicate content)?
/my-sample-url/?s=

• This reply was modified 7 years, 6 months ago by corefocusgroup.

Ok, Thanks!

• This reply was modified 7 years, 6 months ago by corefocusgroup.

Hi guys!
Any news on this? The 404s are not as big a deal as the misuse of Google’s crawling budget. Good content cannot be crawled and for large sites it will definitely affect Search indexing – it already is)

I have tried to fix it with robots.txt but it doesn’t solve the problem.

So I thought I would share Yoast’s article: https://yoast.com/blocking-your-sites-search-results/ (Yoast’s plugin does block indexing of non-existent search pages but it doesn’t cover Enfold’s search functionality) This is such a shame because the menu search functionality on mobile is absolutely terrific but I am evaluating what’s more important)

If anybody has a code solution, please feel free to share.

Warmly,

Havi

Thank you @Coreyfocusgroup. I never used the wordfence plugin. Does Enfold use it?

I’m going to keep the crawling of redirects. GoogleBot only crawls 302s and 307s every time. 301s are only crawled once.

Also, I have already implemented Disallow: /*?s= to no avail. The reality is that you need to allow GoogleBot to crawl and add a no index directive to the “page” but as there’s no page, it’s hard to do. This happens because the bot does not enter any data to search for. So if the code would state that if the Search field is empty, to noindex, nofollow, it would be a cleaner fix.

I will test it on a smaller site. Meanwhile I disabled the tooltip on the large site, deleted all of the soft 404s (to save crawling budget) and will see if they re-appear.

I may modify the code like @fromcouch indicated changing the ?s= for # on the small site and test if it works.

I’ll share my findings in about 2 weeks.

I do hope one of these solutions take care of it. :)

Thanks again!!

Warmly,

Havi

Hi Havi,

I just used Fetch as Google in Google Webmaster (Console) and none of my page redirects are being blocked- all can be indexed.

Only URLs with “redirect_to” are being blocked and those don’t seem to have anything to do with 300 level redirects. I assume it’s something being generated by Enfold or another plugin. But it’s definitely not blocking any page redirects that I have setup.

• This reply was modified 7 years, 4 months ago by corefocusgroup.

Hi,

Add this line in your robot.txt

Disallow: /*?s=