zola/test_site/templates/robots.txt
bemyak d4b5c4fdc4
Make robots.txt compliant with RFC (#1895)
The [RFC](http://www.robotstxt.org/orig.html) mentions only `Disallow`
directive, so it must appear in the file.

`Allow` is an ad hoc agreement between search engines that no all of
them follow.
2022-06-12 21:41:38 +02:00

4 lines
77 B
Text

User-agent: zola
Disallow:
Allow: /
Sitemap: {{config.base_url}}/sitemap.xml