robot.txt implementation

Implement robot.txt editing and page linking, allowing it to be exported along with the other files.

The file is robots.txt and is not normally linked directly to from pages with few exceptions.

What is included is allow or disallow and directories with wildcards. Perhaps this site will help:

Thanks for the suggestion! We can add robots.txt support in the Settings Dialog. It can be a text area, where you enter your robots.txt content (similar to Head Content). I am not sure how page linking would work in this case though. Would love to hear your ideas.