6 Ways You May Be Blocking Search Engines

Accidentally preventing search engines from crawling and indexing your website’s pages is not going to help your presence in the organic search results. If it seems like the engines aren’t indexing your content, it could be one or more of the following.

1. The content requires a login or registration

If visitors can’t get to areas of your website without logging in or creating an account, neither can the search engine bots.

2. No Text

If your page is almost entirely images, graphics, and videos – there is no text for the search engines to use in determining what a page is about. You can help this along by using alt text for your images and using descriptive file names (like “eiffel-tower-sunset-paris.jpg”). If you want the page to come up in search results, some text should be on it.

3. Different versions of pages being served at the same URL

If you’re serving different pages to different users arriving at the same URL, based on their location or some other variable, the search engine bot is only going to crawl and index one version.

4. Incorrectly configured redirects

Have you moved some of your content, or moved your entire site to a new domain, and implemented redirects? If it’s been a while, and the search engines don’t appear to be indexing the new pages, these might be the culprit. Ensure that they are 301 redirects (not 302s) and ensure that they are set up properly for every page involved.

5. The content is Flash or Silverlight

While search engines can now actually crawl and index some Flash, it’s still risky. If you really want this content crawled, it might not be the best way to present it.

6. They can’t get to the content because of Ajax or JavaScript

Search engines only have trouble with javascript when it is poorly implemented. If your site is using a lot of it and you’re having indexing problems, it may not be as accessible as it could be.