Although it has historically been difficult to gather statistics on the topic, it’s undoubtedly true that we all rely on search engines for information gathering. As a website owner, this means it’s vital that your content is visible to search engines. Indexing errors in Google Search Console indicate otherwise.
Fortunately, you can troubleshoot and fix these issues in no time. Google Search Console provides you with many tools that can help, if you know where to find them.
In this article, we’ll discuss some indexing errors that can happen with new websites, as well as tips for resolving them using Google Search Console. Let’s jump in!
Understanding Indexing Errors
In order for search engines such as Google to show relevant data to users, they must crawl, index, and rank websites. Essentially, links to web resources are saved in a database, then returned in the order in which they were previously ranked when a search is made.
Indexing errors occur when search engines cannot add your website to their databases correctly. This can happen for multiple reasons. Such errors are significant because they will negatively affect your rankings and therefore your site’s visibility and organic traffic.
Google Search Console enables you to analyze your webpages’ indexing status and troubleshoot any errors its bots have found. Since most people use Google to perform searches, this can go a long way to ensure your content’s visibility.
Finding Indexing Errors With the Index Coverage Report
Before you can start fixing errors, you first need to know how to find them. Search Console provides the Index Coverage report – formerly Crawl Stats – which shows pages that were indexed successfully, as well as those that were not.
First, log into your Google account and open Search Console. If you have multiple properties verified in this tool, make sure the relevant site is selected in the dropdown in the top left corner.
Next, go to Index > Coverage to view your site’s report. Specifically, look for the Error tab. Ideally, it will say your site has no errors:
If there are indexing errors on your site, you’ll see them in the Details section:
There are eight possible error values you may see on this screen, which we’ve explored in detail below.
How to Fix Common Indexing Errors with Google Search Console (8 Potential Problems)
Fixing indexing errors requires some investigative skills. However, Google Search Console makes the process as easy as possible to help you maximize your site’s visibility.
1. Submitted URL marked ‘noindex’
This first item isn’t an error in the strict sense of the term. It just means a page was tagged ‘noindex’, which is a way to tell search engines not to list a page. You’d usually do this for content you don’t want to make publicly accessible.
There are two ways a page can be blocked with ‘noindex’. One way is by including a meta tag. The second is by sending a ‘noindex’ header via HTTP requests.
If you’re sure you do not want the URLs with this error indexed, then you can leave them as is. Otherwise, you’ll want to either remove the meta tag from your page or check that the ‘noindex’ header is not being sent along with HTTP requests.
2. Submitted URL blocked by robots.txt
In this case, a rule in your robots.txt file is telling search engines not to index a specific page. Again, there are some cases where you may do this on purpose.
However, if you do want the page to appear in search results, then you can edit your robots.txt file to remove it.
Note that your hosting provider may not enable direct access to your robots.txt file. However, there are tools that can help you work around this, including the popular Yoast SEO plugin.
3. Submitted URL has crawl issue
This error occurs when some issue prevents a page from being indexed. It could be a result of the ‘noindex’ meta tag, or one of the other errors on this list.
Crawl issues can also crop up as a result of page loading errors during an attempt at indexing the page. For example, images on your page, CSS files, or JavaScript may have not been loaded properly. In that case, you’ll need to edit your site’s code to clear this issue.
You can use Search Console to help narrow down the source of the problem to a specific cause so that you can begin to fix it. Click on the URL Inspection button. Next, click on View Crawled Page, and then More Info in the sidebar to the right:
At this point, make sure the error has not been resolved already – it may have just been temporary. Open the relevant URL in a new tab to check that you can load it, then click on Test Live URL to force a refresh of the report:
Next, review the information under More Info for further details. After you determine what the issue is and resolve it, you’ll want to re-submit the page to Google by clicking on the Request Indexing button:
Finally, you’ll need to go back to the Coverage report and click on the Validate Fix button next to the error in the Details section. This should officially mark the problem as resolved.
4. Submitted URL not found (404)
This particular error occurs when a page cannot be found at the time of indexing.
To fix it, click on the URL Inspection button as described above. Then open the page in a new tab to confirm whether it’s available. If it is, then you’ll need to request re-indexing.
If the page is unavailable and you don’t want it indexed, then you can leave it be. It’ll eventually become de-indexed. You can also choose to redirect it to a different page.
5. Submitted URL seems to be a soft 404
This occurs when a page was not found at the time of indexing, but a valid status code was returned telling Google it doesn’t exist or that it should be ignored.
For example, this can happen if an e-commerce checkout page isn’t displayed until some action – such as the user adding items to their cart – has been taken.
To resolve this error, you can redirect the URL to a new page or remove the page from your sitemap. Lastly, if the page is invalid, you may return a status 404 code.
6. Submitted URL returns unauthorized request (401)
This error crops up when Google cannot access a page because it is password protected. You probably don’t want such content to appear in search results anyway, so the solution here is simply to formally block crawlers.
Your options include removing the page from your sitemap, adding a ‘noindex’ meta tag, or including a rule in your robots.txt file to exclude it.
7. Redirect Error
The “redirect error” occurs when Google could not access a page because it redirects to another page that isn’t valid. It may not exist or simply not work for some reason.
The steps to take here are pretty much the same as for the crawl error we discussed above. To begin, you’ll want to click on the URL Inspection button and then open the relevant page in a new tab to confirm the redirect error.
If it persists, you’ll need to redirect to a page that works. Then, Request Indexing and Validate Fix in Search Console. When this error is resolved, you’ll receive a Passed confirmation message.
8. Server error (5xx)
Server errors occur when Google cannot access a page because the server was unavailable. First, test that this issue has not been resolved already using the method described above for a crawl error. If the page is now up, then you can request re-indexing.
However, if upon URL inspection you see a message that reads “URL is not on Google: Indexing Error”, this means the URL has been removed from Google’s database for one of two reasons:
- It was unavailable during indexing
- It was inaccessible, and therefore de-indexed
You may need to visit your server to debug any existing errors, after which you can test the live URL and request re-indexing again. If that doesn’t work, another option is to tell Google to remove the page from your sitemap by adding a ‘noindex’ tag. Error reports will then no longer show up for that page.
Conclusion
Having your website indexed by Google is vital for your content’s visibility in search results. If there are errors while crawling or indexing your website, you may struggle to increase your organic traffic.
In this post, we shared tips for resolving eight indexing errors in Google Search Console. Remember to submit your pages for re-indexing after you fix whatever issue was preventing Google from crawling them.
Do you have any questions about finding or fixing indexing errors using Google Search Console? Let us know in the comments section below!
The post How to Fix Common Indexing Errors with Google Search Console appeared first on Torque.
