The robots.txt file
The robots.txt file restricts access to your website by search engine robots that crawl websites.
Robots.txt files are used by website owners to stop search engines placing certain pages, or resources, in their index that they (the website owner) do not want to appear in results.
Does my site have a robots.txt file?
Yes. Our platform generates the robots.txt file automatically to exclude certain files of your website from search engines' indexes.
The robots.txt file for your site is located at
(substitute yourdomain.com for your website's actual domain name).
Can I change my robots.txt file?
Yes if you want to you can change the contents of this file but you'll need to contact us with the changes that you require. Please note that we cannot advise you on what the robots.txt file should contain aside from the default contents.