Crawled but Not Indexed in GSC – Solve it With These Tips

crawled not indexedWhen uploading new and existing content to your website, it can take a couple of weeks for Google to completely discover, crawl, and (hopefully) index your page despite your best efforts. So, what happens if Google doesn’t index your content? That is where we are here to help.

Learn more about crawled but not indexed pages on your website in Google Search Console, and the necessary steps you can take to have these pages appear in SERPs.

What Does Crawled but Not Indexed Mean?

When a page is crawled but not yet indexed, it means that Google has essentially discovered and crawled the page on your website, but they have actively decided not to include it in their index.

This can pose issues for search marketers because a page that is not indexed will not appear in organic search results. As of right now, there is no definite reasoning as to why some pages are indexed over others, but there are steps that you can take to ensure Google reconsiders indexing your page.

We understand that going through the steps to get your page indexed be incredibly frustrating, but our team wants to help you find definitive solutions to your issues, ultimately making it easier for you to rank in the future!

4 Reasons Your Pages Are Crawled but Not Indexed, and How to Solve It

Assessing pages that have been crawled but not indexed, our team highly recommends working with Google Search Console (GSC) to find the most exact and up-to-date results for some of your issues. Simply utilize Google’s URL Inspection Tool to find the indexing stats for your web pages.

On top of that, here are just some of the troubleshooting processes that we recommend.

1. Duplicate or Thin Content

Google consistently values content that is thorough, helpful, and unique. That means Google pages that lack content or pages that have the same content on them are less valuable than pages that are all individual. This can pose issues in the long run because pages that have duplicate content can be completely disregarded by Google.

That is why we recommend the following tips and trips to keep your content authentic and rich:

  • On high-priority pages, make sure content is unique to the topic
  • Remove unnecessary content and replace it with new copy
  • Avoid any AI-generated or machine-generated content

2. 301 Redirects

301 redirects can occur when Google is indexing a page that redirects to the destination page, but not the destination page itself. This can occur because Google does not notice that the page is a redirect and marking the destination URL as duplicate content.

We recommend addressing this issue if 301 redirects have led to a lot of pages on your site being crawled but not indexed. This can help Google identify your existing issues in a timelier manner as opposed to the existing unconsolidated content signals. One way to do this is by creating a temporary sitemap.

Here are the steps to creating one:

  • Export your “Crawled – currently not indexed” URLs from the GSC report
  • Match them with the preexisting redirects that were previously enacted
  • Locate the redirects that include the destination URL among the crawled URLs
  • With Screaming Frog, create a static sitemap.xml

Once completed, you should be able to upload your Screaming Frog sitemap to your Google Search Console and monitor the pages that become indexed over time.

3. Products that Are Not Available

Most notably on ecommerce sites, product pages that are listed as “out of stock” or are expired or seasonal can be subjected to being crawled and not indexed by Google. This is because Google chooses not to include products that are not available for purchase in its index, as it could lead to a dissatisfied user experience for those looking for nonexistent products.

To solve this issue, make sure to check both the content and schema of your pages to ensure that pages that are appearing as out of stock are listed as in stock. You can do so by crawling your site with Screaming Frog and gathering a list of all the products that are currently listed as “out of stock.”

By using this method and identifying the inaccuracies and discrepancies in the data, it should be a quick fix to clean up your pages and update them with the most current information.

4. False Positives

If you notice that your pages are crawled and currently not indexed, there is a possibility that these pages may be indexed and may be reporting inaccurately.

To check whether your pages are being indexed, you can site search through Google. To do so, type “site:” before inserting the URL that you are inquiring about. For example, if we are trying to see if the Sixth City Marketing home page was indexed, you would type in the following:

site:https://www.sixthcitymarketing.com

If your page appears when you complete your site search, you are good to go! It is a false positive and your page is being indexed by Google. In these cases, there is no need to fix anything.

What to Know About Pages That Are Discovered but Not Indexed

As opposed to crawled but currently not indexed, “Discovered – currently not indexed” is when Google is aware of your URL, but they have neither crawled nor indexed the page. This can also affect your SERPs because this means specific pages of your website are not being recognized through organic search, which can be detrimental to your business depending on the page.

If you have realized that some of the pages on your website have been discovered but are not currently indexed, try requesting pages be indexed through GSC. It is important to note you can only submit 10 to 15 pages daily, to prioritize your pages based on relevance.

It is also important to note that underlying causes between pages that are crawled and discovered can be synonymous, so ensure you are also consistently checking the quality and originality of your content for the best results.

Learn More from Our Marketing Blog

Rachel Calvis

About the Author

Rachel Calvis

Rachel is an Online Marketing Specialist at Sixth City Marketing. Her main role is constructing and editing concise marketing copy for our B2B and B2C clients.

Our Partners & Awards