In technical SEO, the `noindex` and `nofollow` directives are powerful tools for controlling how search engines interact with your site. While often discussed together, they serve distinct and separate functions. Understanding when and how to use them is crucial for managing your site’s indexability, controlling the flow of PageRank, and ensuring only your most valuable content appears in search results.
Think of your website as a museum. The `noindex` directive is a ‘Staff Only’ sign on a door, telling the public (and search engines) not to enter or list the room’s contents in the public directory. The `nofollow` attribute is a note on a specific painting’s label that says, “While this painting is here, we don’t have an official opinion on the other galleries it references.” For a broader look at crawler directives, see our guide on meta robots and directives.
The ‘noindex’ Directive: Controlling Indexation
The `noindex` directive is a value in a meta robots tag that tells search engines not to include a specific page in their search results. This is the definitive way to keep a page out of the index. Common use cases include:
- Low-Value Pages: Thank-you pages, internal search results, and admin login pages.
- Duplicate Content: Printer-friendly versions of pages or staging environments.
The ‘nofollow’ Attribute: Controlling Link Equity
The `nofollow` attribute is a value in a link’s `rel` attribute (`rel=”nofollow”`) that tells search engines not to pass any PageRank or endorsement to the linked URL. This is important for:
- Untrusted Content: Links in user-generated content, like blog comments or forum posts.
- Paid Links: Google requires that all paid or sponsored links use `rel=”nofollow”` or the more specific `rel=”sponsored”` attribute.
Strategic Combinations: ‘noindex, follow’
You can also combine these directives in a meta tag. The most useful combination is `<meta name=”robots” content=”noindex, follow”>`. This tells search engines: “Do not include this page in the search results, but do trust and follow the links on this page and pass PageRank through them.” This is ideal for pages like category archives that you may not want in the SERPs but that link to many of your valuable articles. For more on this, see this guide from Ahrefs.
Example: A `noindex, follow` Meta Tag
<!-- In the <head> of a paginated archive page -->
<meta name="robots" content="noindex, follow">For a complete guide to all the different meta directives, refer to the Google Search Central documentation.
Frequently Asked Questions
What is the difference between a ‘nofollow’ meta tag and a ‘nofollow’ link attribute?
A ‘nofollow’ meta tag (`<meta name=”robots” content=”nofollow”>`) applies to *all* links on the entire page. A ‘nofollow’ link attribute (`<a href=”…” rel=”nofollow”>`) applies only to that *specific* link. It is much more common and precise to use the link-level attribute.
Can I use the X-Robots-Tag instead of a meta tag?
Yes. The X-Robots-Tag is an HTTP header that can contain the same ‘noindex’ and ‘nofollow’ directives as a meta tag. It’s more powerful because it can be used to noindex non-HTML files like PDFs or to apply rules across an entire site.
When should I use ‘noindex, follow’?
This is a powerful combination for pages you don’t want in the index but that link to important content. Good examples are paginated archives or tag pages. You can de-index the archive page itself while still allowing PageRank to flow through it to your individual articles.
Are you sending the right signals to search engines? Start your Creeper audit today to take control of your site’s indexing and link equity.