mirror of
https://github.com/superseriousbusiness/gotosocial
synced 2024-11-27 14:51:04 +00:00
0cce2c0838
* [feature] Block Google Bard/AI crawlers * [feature] Block the other OpenAI crawler * [feature] Block Common Crawl crawler This is used in research, but also gleefully advertises itself as the training source used in all LLMs and GPT-3. Fixes: #2240 * [feature] Block Omgilikebot Used by some shady big web data engine company. * [feature] Block Meta's language model crawler * [feature] Block well-known.dev crawler |
||
|---|---|---|
| .. | ||
| about.go | ||
| assets.go | ||
| base.go | ||
| confirmemail.go | ||
| customcss.go | ||
| domain-blocklist.go | ||
| etag.go | ||
| opengraph.go | ||
| opengraph_test.go | ||
| profile.go | ||
| robots.go | ||
| rss.go | ||
| settings-panel.go | ||
| tag.go | ||
| thread.go | ||
| web.go | ||