User-agent: * Disallow: /ebcart.asp Disallow: /Items.aspx Disallow: /Checkout/* Disallow: /Account/* Disallow: /Cart.aspx Disallow: /spambot.asp Disallow: /blog/wp-admin/ Disallow: /blog/trackback/ Disallow: /blog/xmlrpc.php Disallow: /blog/feed/ Disallow: /blog/?s=* Disallow: /p/0/// Disallow: /ws/* #SEO: trying this out to help with crawl budget (rjb 11/02/23) Disallow: /Search/* User-agent: ShopWiki Disallow: / User-Agent: Charlotte Disallow: / User-Agent: OmniExplorer_Bot Disallow: / User-agent: Nutch Disallow: / User-agent: Yandex Disallow: / User-agent: sistrix Disallow: / User-agent: Bytespider Disallow: / # Proprietary German backlinks service. too hungry User-agent: SEOkicks-Robot Disallow: / User-agent: msnbot Crawl-delay: 1 # Imperva states they use lots of traffic. (11/03/21) User-agent: AhrefsBot Crawl-delay: 1 Sitemap: https://www.opgi.com/sitemapindex.xml