# Tells Scanning Robots Where They Are And Are Not Welcome # User-agent: can also specify by name; "*" is for everyone # Disallow: if this matches first part of requested path, # forget it # applies to all robots User-agent: * Crawl-delay: 10 Disallow: /product/enlarge.asp Disallow: /product/leftinc.asp Disallow: /product/list.asp Disallow: /product/modline.asp Disallow: /product/printitems.asp Disallow: /product/registration.asp