Noindex
Noindex is a meta tag that literally indicates that a page does not need indexing in the search engine. This page still has function and can be visited by visitors, only Google or Bing cannot index the page. It is also possible to exclude only parts from pages for indexing. You place the "noindex,nofollow" meta tag in the <head> of a page. There are several ways you can deploy noindex, this looks like this:
- <meta name="robots" content="noindex,nofollow"/> This means that a page is not indexed and links are not followed
- <meta name="robots" content="noindex,follow"/> This means that the links on a page may be followed and the page will not be indexed
- <meta name="robots" content="index,nofollow"/> This means that there are links that should not be followed but the page is still indexed
Noindex functions
It sounds counterintuitive to not allow pages to be indexed by search engine bots. But if, for example, you offer informative pages on your site that are not advertised, Noindex can be a way for a crawler or bot to better navigate your website. Some practical uses for deploying Noindex are:
- Making temporary web pages disappear after a short time
- Robots can skip information in a large database
- Specific mobile or print versions of web pages that do not require separate indexing
- Pages where excessive traffic is not desired that are publicly reachable
Definition noindex
Noindex is different from Disallow in Robots.txt. You place Noindex in your metadata or in the HTTP response header. If you set a page to Disallow in Robots.txt, they will not be read with Noindex information. So make sure you align these well with each other to avoid a (part of a) page not being scanned at all by the search engine when you want it to be. Finally, it's useful to know that not every search engine bot reads Noindex's meta tag. This makes it a less reliable tool for hiding website pages from search engines.
