mirror of
https://github.com/superseriousbusiness/gotosocial
synced 2024-11-26 22:30:25 +00:00
dcab555a6b
This updates the robots.txt based on the list of the ai.robots.txt repository. We can look at automating that at some point. It's worth pointing out that some robots, namely the ones by Bytedance, are known to ignore robots.txt entirely. |
||
|---|---|---|
| .. | ||
| about.go | ||
| assets.go | ||
| confirmemail.go | ||
| customcss.go | ||
| domain-blocklist.go | ||
| etag.go | ||
| index.go | ||
| profile.go | ||
| robots.go | ||
| rss.go | ||
| settings-panel.go | ||
| signup.go | ||
| tag.go | ||
| thread.go | ||
| web.go | ||