Mirrors/ArchiveBox
2
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox synced 2024-11-23 04:33:11 +00:00
Code Issues Projects Releases Packages Wiki Activity

create robots.txt in output dir

Browse source
This commit is contained in:
Nick Sweeting 2018-09-12 19:25:48 -04:00
parent 451cea47e6
commit 8a23358fc8
1 changed files with 3 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
archiver/index.py
View file

@ -72,6 +72,9 @@ def write_html_links_index(out_dir, links):
copy_tree(os.path.join(TEMPLATES_DIR, 'static'), os.path.join(out_dir, 'static'))
with open(os.path.join(out_dir, 'robots.txt'), 'w+') as f:
f.write('User-agent: *\nDisallow: /')
with open(os.path.join(TEMPLATES_DIR, 'index.html'), 'r', encoding='utf-8') as f:
index_html = f.read()