You might not want selected pages of your web site crawled mainly because they might not be handy to customers if present in a online search engine's search results. Take note that if your site works by using subdomains and you wish to get selected webpages not crawled on a particular subdomain, you will have to produce a separate robots.Amount of