Crawler directives tell search engines how to behave when crawling and indexing your website.
Examples of crawler directives are the robots meta tag, the x-robots tag and the robots.txt file.
What are Crawler Directives?
The Marketing Dictionary
SEO and Digital Marketing terms and definitions you need to know.
