User-agent: * Disallow: Sitemap: http://www.businesstelegraph.co.uk/sitemaps.xml User-agent: * Disallow: / User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / User-agent: slurp Allow: / User-agent: Yandex Disallow: / User-agent: baiduspider Disallow: / User-agent: AhrefsBot Disallow: / User-agent: BLEXBot Disallow: / crawl-delay: 10