vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-05 12:09:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Extend Semrush user-agents

Browse source
This commit is contained in:
Melroy van den Berg 2025-01-18 17:58:13 +01:00 • committed by GitHub
parent ace5a4ce9f
commit a69c6f9bb7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 14 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

14
robots.txt/robots.txt
View file

@ -861,6 +861,20 @@ User-agent: Semrush
Disallow:/ Disallow:/
User-agent: SemrushBot User-agent: SemrushBot
Disallow:/ Disallow:/
User-agent: SiteAuditBot
Disallow:/
User-agent: SemrushBot-BA
Disallow:/
User-agent: SemrushBot-SI
Disallow:/
User-agent: SemrushBot-SWA
Disallow:/
User-agent: SplitSignalBot
Disallow:/
User-agent: SemrushBot-OCOB
Disallow:/
User-agent: SemrushBot-FT
Disallow:/
User-agent: SentiBot User-agent: SentiBot
Disallow:/ Disallow:/
User-agent: SenutoBot User-agent: SenutoBot