Mirrors/gotosocial
2
0
Fork 0
mirror of https://github.com/superseriousbusiness/gotosocial synced 2024-11-14 00:37:18 +00:00
Code Issues Projects Releases Packages Wiki Activity

[feature] Block Amazonbot (#2692)

Browse source 
Blocks the Amazon crawler bot.

Closes: #2686
This commit is contained in:
Daenney 2024-02-27 14:25:08 +01:00 committed by GitHub
parent 9cadc764b3
commit 6528592dd2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 5 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
internal/web/robots.go
View file

@ -64,6 +64,11 @@ Disallow: /
User-agent: WellKnownBot
Disallow: /
# Block Amazonbot, because Amazon.
# https://developer.amazon.com/amazonbot
User-agent: Amazonbot
Disallow: /
# Rules for everything else.
User-agent: *
Crawl-delay: 500