2016-01-29
1.0.0
Grazina
9.1.0.20
All
.NET
Components
[Overview]
Espace to configure webcrawling permissions
[Detail]
Simple eSpace to configure webcrawling permissions for your server domain.
When installed and configured, it's currently blocking (well behaved) bots.
IMPORTANT NOTE: this espace simply exposes a robots.txt file in the root path of the eSpace. In order to make it work for your domain, you need to configure SEO rules to point the root domain to this eSpace by: going into service center
accessing SEO rules
entering the Site Rules list section
adding as many site rules as the hostnames/ips used to access your server (Base URL should be your hostname/IP, root application should be Robots)
Check the screenshot if you're lazy following instructions.
Find more info how to configure robots.txt files here: http://www.robotstxt.org/