thetoddluci0/DarkVisitors Robots.txt
Add anti-bot methods to your robots.txt using darkvisitors.com
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| outfile | Where to put the final robots.txt file | yes | ./robots.txt |
| infile | File to use as a base for the robots.txt file | no | — |
| deny_types | List of agent types to deny, in ["a", "b"] format. Defaults to all supported | yes | ["AI Assistant", "AI Data Scraper", "AI Search Crawler", "Undocumented AI Agent"] |
| disallow | Base path to block from. Defaults to / | yes | / |
Outputs
no outputs