|
It is simply a matter of using several tags: <meta name="googlebot" content= <meta name= googlebot-news There are hundreds of different user agents. The most common are: Google: Googlebot (you can see a complete list of Google crawlers here ) Bing: Bingbot (you can see a full list of Google crawlers here ) DuckDuckBot Baidu: Baiduspider Yandex: YandexBot Content The content attribute is used to give instructions to the specified user agent.
It's important to know that if you don't specify a . meta tag on a web page, the default is to index the page and follow all links (unless they have a rel="nofollow" attribute specified). The different directives you can use include: index (include Phone Number List the Phone Number List page in the index) [Note: it is not necessary to include this if no index is specified, it is assumed as an index] . (do not include the page in the index or show it in the .) follow (follow the links on the page to discover other pages) .(do not follow the links on the page) none (a shortcut to specify ., nofollow) all (a shortcut to specify index, follow) noimageindex (do not index images on the page) noarchive (do not show a cached version of the page in .

oarchive, but only for MSN) nositelinkssearchbox (do not show a search box for your site in .) nopagereadaloud (do not allow voice services to read your page aloud) notranslate (do not show page translations in .) unavailable_after (specify a time after which the page should not be indexed You can see a full list of the policies that Google has here and those that Bing has here . Robot Meta Tag Code Examples If you're looking for examples of robot meta tags that you can use to control how search engines crawl and index your web pages,
|
|