vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.11.2708 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-11-11 11:21:42 +00:00
parent c0e0f017f6
commit d1bd0e876f
52 changed files with 69021 additions and 68985 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
AUTO-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2707 #### Version: V4.2021.11.2708
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [626](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
MANUAL-CONFIGURATION.md
View file

@ -4,9 +4,9 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2707 #### Version: V4.2021.11.2708
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [626](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

4
README.md
View file

@ -11,9 +11,9 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.11.2707 #### Version: V4.2021.11.2708
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [626](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

8482
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

4
_generator_lists/bad-user-agents.list
View file

@ -1,5 +1,3 @@
adscanner
seocompany.store
01h4x.com 01h4x.com
360Spider 360Spider
404checker 404checker
@ -556,6 +554,7 @@ ZoomBot
ZoominfoBot ZoominfoBot
ZumBot ZumBot
ZyBorg ZyBorg
adscanner
archive.org_bot archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt arquivo.pt
@ -601,6 +600,7 @@ satoristudio.net
scalaj-http scalaj-http
scan.lol scan.lol
seobility seobility
seocompany.store
seoscanners seoscanners
seostar seostar
serpstatbot serpstatbot

4
_google_analytics_ghost_spam/README.md
View file

@ -3,9 +3,9 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.11.2707 #### Version: V4.2021.11.2708
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [626](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)
____________________ ____________________

8490
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8490
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8490
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/_robots_input/robots-input.txt
View file

@ -554,6 +554,7 @@ ZoomBot
ZoominfoBot ZoominfoBot
ZumBot ZumBot
ZyBorg ZyBorg
adscanner
archive.org_bot archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt arquivo.pt
@ -599,6 +600,7 @@ satoristudio.net
scalaj-http scalaj-http
scan.lol scan.lol
seobility seobility
seocompany.store
seoscanners seoscanners
seostar seostar
serpstatbot serpstatbot

8490
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.92 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.92 0;
127.0.0.1 0; 127.0.0.1 0;

8490
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.92 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.92 0;
127.0.0.1 0; 127.0.0.1 0;

8490
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.92 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-user-agents.conf
View file

@ -615,6 +615,7 @@
"~*(?:\b)ZoominfoBot(?:\b|)" 0; "~*(?:\b)ZoominfoBot(?:\b|)" 0;
"~*(?:\b)ZumBot(?:\b|)" 0; "~*(?:\b)ZumBot(?:\b|)" 0;
"~*(?:\b)ZyBorg(?:\b|)" 0; "~*(?:\b)ZyBorg(?:\b|)" 0;
"~*(?:\b)adscanner(?:\b|)" 0;
"~*(?:\b)archive.org_bot(?:\b|)" 0; "~*(?:\b)archive.org_bot(?:\b|)" 0;
"~*(?:\b)arquivo-web-crawler(?:\b|)" 0; "~*(?:\b)arquivo-web-crawler(?:\b|)" 0;
"~*(?:\b)arquivo.pt(?:\b|)" 0; "~*(?:\b)arquivo.pt(?:\b|)" 0;
@ -660,6 +661,7 @@
"~*(?:\b)scalaj-http(?:\b|)" 0; "~*(?:\b)scalaj-http(?:\b|)" 0;
"~*(?:\b)scan.lol(?:\b|)" 0; "~*(?:\b)scan.lol(?:\b|)" 0;
"~*(?:\b)seobility(?:\b|)" 0; "~*(?:\b)seobility(?:\b|)" 0;
"~*(?:\b)seocompany.store(?:\b|)" 0;
"~*(?:\b)seoscanners(?:\b|)" 0; "~*(?:\b)seoscanners(?:\b|)" 0;
"~*(?:\b)seostar(?:\b|)" 0; "~*(?:\b)seostar(?:\b|)" 0;
"~*(?:\b)serpstatbot(?:\b|)" 0; "~*(?:\b)serpstatbot(?:\b|)" 0;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.92 0;
127.0.0.1 0; 127.0.0.1 0;

8490
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2707 2708

8490
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8490
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.122.29.66 0; 40.122.255.32 0;

8490
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.92 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.92 0;
127.0.0.1 0; 127.0.0.1 0;

8490
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.92 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.92 0;
127.0.0.1 0; 127.0.0.1 0;

8490
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.92 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-user-agents.conf
View file

@ -615,6 +615,7 @@
"~*(?:\b)ZoominfoBot(?:\b|)" 0; "~*(?:\b)ZoominfoBot(?:\b|)" 0;
"~*(?:\b)ZumBot(?:\b|)" 0; "~*(?:\b)ZumBot(?:\b|)" 0;
"~*(?:\b)ZyBorg(?:\b|)" 0; "~*(?:\b)ZyBorg(?:\b|)" 0;
"~*(?:\b)adscanner(?:\b|)" 0;
"~*(?:\b)archive.org_bot(?:\b|)" 0; "~*(?:\b)archive.org_bot(?:\b|)" 0;
"~*(?:\b)arquivo-web-crawler(?:\b|)" 0; "~*(?:\b)arquivo-web-crawler(?:\b|)" 0;
"~*(?:\b)arquivo.pt(?:\b|)" 0; "~*(?:\b)arquivo.pt(?:\b|)" 0;
@ -660,6 +661,7 @@
"~*(?:\b)scalaj-http(?:\b|)" 0; "~*(?:\b)scalaj-http(?:\b|)" 0;
"~*(?:\b)scan.lol(?:\b|)" 0; "~*(?:\b)scan.lol(?:\b|)" 0;
"~*(?:\b)seobility(?:\b|)" 0; "~*(?:\b)seobility(?:\b|)" 0;
"~*(?:\b)seocompany.store(?:\b|)" 0;
"~*(?:\b)seoscanners(?:\b|)" 0; "~*(?:\b)seoscanners(?:\b|)" 0;
"~*(?:\b)seostar(?:\b|)" 0; "~*(?:\b)seostar(?:\b|)" 0;
"~*(?:\b)serpstatbot(?:\b|)" 0; "~*(?:\b)serpstatbot(?:\b|)" 0;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.92 0;
127.0.0.1 0; 127.0.0.1 0;

8490
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.122.29.66 0; 40.122.255.32 0;

8490
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.122.29.66 0; 40.122.255.32 0;

8490
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.92 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test_units/blacklist-user-agents.conf
View file

@ -615,6 +615,7 @@
"~*(?:\b)ZoominfoBot(?:\b|)" 0; "~*(?:\b)ZoominfoBot(?:\b|)" 0;
"~*(?:\b)ZumBot(?:\b|)" 0; "~*(?:\b)ZumBot(?:\b|)" 0;
"~*(?:\b)ZyBorg(?:\b|)" 0; "~*(?:\b)ZyBorg(?:\b|)" 0;
"~*(?:\b)adscanner(?:\b|)" 0;
"~*(?:\b)archive.org_bot(?:\b|)" 0; "~*(?:\b)archive.org_bot(?:\b|)" 0;
"~*(?:\b)arquivo-web-crawler(?:\b|)" 0; "~*(?:\b)arquivo-web-crawler(?:\b|)" 0;
"~*(?:\b)arquivo.pt(?:\b|)" 0; "~*(?:\b)arquivo.pt(?:\b|)" 0;
@ -660,6 +661,7 @@
"~*(?:\b)scalaj-http(?:\b|)" 0; "~*(?:\b)scalaj-http(?:\b|)" 0;
"~*(?:\b)scan.lol(?:\b|)" 0; "~*(?:\b)scan.lol(?:\b|)" 0;
"~*(?:\b)seobility(?:\b|)" 0; "~*(?:\b)seobility(?:\b|)" 0;
"~*(?:\b)seocompany.store(?:\b|)" 0;
"~*(?:\b)seoscanners(?:\b|)" 0; "~*(?:\b)seoscanners(?:\b|)" 0;
"~*(?:\b)seostar(?:\b|)" 0; "~*(?:\b)seostar(?:\b|)" 0;
"~*(?:\b)serpstatbot(?:\b|)" 0; "~*(?:\b)serpstatbot(?:\b|)" 0;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BacklinkCrawler 01h4x.com
Calculon AfD-Verbotsverfahren
CrunchBot Fyrebot
Curious Moblie Safari
DataCha0s Niki-bot
Demon SemrushBot
IndeedBot TeleportPro
MIDown tool VB Project
Majestic12 Wotbox
tAkeOut sp_auditbot

322
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404enemy
404checker 80legs
ADmantX
AIBOT AIBOT
Abonti ASPSeek
Aboundex Acunetix
Aboundexbot
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot AiHitBot
Aipbot AlphaBot
AllSubmitter
Alligator
Anarchie Anarchie
Anarchy
Anarchy99 Anarchy99
Ankit Apexoo
Aspiegel
Asterias
AwarioRssBot AwarioRssBot
AwarioSmartBot BDCbot
BBBike
BDFetch BDFetch
BLEXBot BLEXBot
BacklinkCrawler Badass
BetaBot Bandit
Bitacle Barkrowler
Bigfoot
Black Hole Black Hole
BlackWidow
Blackboard Blackboard
Blow
Boardreader Boardreader
Bolt
BotALot
Brandprotect Brandprotect
Brandwatch Buck
Bullseye Buddy
BuzzSumo BuiltBotTough
CCBot Calculon
CazoodleBot
Cegbfeieh Cegbfeieh
CheTeam CherryPicker
CheeseBot Cloud mapping
ChinaClaw Cosmos
Chlooe Crawling at Home Project
Cogentbot CrunchBot
CrazyWebCrawler Curious
Custo Custo
CyotekWebCopy
DSearch
DTS Agent DTS Agent
DataCha0s DataCha0s
Deusu DatabaseDriverMysqli
Digincore Demon
DigitalPebble
Disco Disco
Dispatch
DittoSpyder DittoSpyder
DnyzBot DnyzBot
DomainCrawler DomCopBot
DomainSigmaCrawler DomainStatsBot
Download Wonder Download Wonder
ECCP/1.0 EMail Siphon
EasyDL EMail Wolf
Ebingbong EirGrabber
EroCrawler EroCrawler
ExtLinksBot Express WebPictures
Extractor Extractor
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
Firefox/7.0 FDM
FlashGet FrontPage
Flunky
FyberSpider
G-i-g-a-b-o-t
GT::WWW GT::WWW
GetRight Genieo
GetWeb GermCrawler
GoZilla Gigabot
Go!Zilla
Go-Ahead-Got-It
Gotit Gotit
Grabber Grafula
GrapeshotCrawler
GridBot GridBot
HEADMasterSEO
HMView HMView
HTMLparser
HTTP::Lite HTTP::Lite
HTTrack HTTrack
Haansoft Haansoft
HaosouSpider
Hloader Hloader
Humanlinks
HybridBot HybridBot
IDBTE4M IDBot
IRLbot IRLbot
Id-search Id-search
IlseBot Intelliseek
Image Fetch InterGET
InfoNaviRobot
InfoTekies
InternetSeer InternetSeer
Iria
Iskanie
IstellaBot IstellaBot
JennyBot JOC Web Spider
JetCar Jbrofuzz
Jetty Jetty
Joomla
Jorgee
JustView
Jyxobot Jyxobot
Keyword Density Keyword Density
Kozmosbot
LNSpiderguy
LWP::Simple
Larbin
Leap Leap
LeechFTP
Lftp
LibWeb LibWeb
LieBaoFast Libwhisker
Lightspeedsystems Lightspeedsystems
LinkWalker Likse
Linkdexbot Lipperhey Spider
LinkpadBot
LinqiaRSSBot
Lipperhey
Litemage_walker
Lmspider
Ltx71 Ltx71
MIIxpc MFC_Tear_Sample
MJ12bot MIDown tool
MSFrontPage MTRobot
MSIECrawler Mag-Net
Majestic-SEO Majestic-SEO
Majestic12
MarkMonitor MarkMonitor
MarkWatch Masscan
Mass Downloader MauiBot
Mb2345Browser MegaIndex.ru
Meanpathbot
Microsoft Data Access Microsoft Data Access
Mojeek Microsoft URL Control
Mr.4x3 Mojolicious
Musobot
NICErsPRO NICErsPRO
NPbot NPbot
NearSite Name Intelligence
NetAnts Nameprotect
NetLyzer Navroad
NetZIP Needle
Netcraft Nessus
NetSpider
Nettrack Nettrack
Netvibes Netvibes
Nimbostratus Nibbler
Nmap Niki-bot
Nutch NimbleCrawler
OpenLinkProfiler Ninja
OrangeBot Nuclei
Octopus
Offline Navigator
OrangeSpider OrangeSpider
OutclicksBot PECL::HTTP
POE-Component-Client-HTTP Page Analyzer
PageAnalyzer PageAnalyzer
PageScorer
PageThing.com PageThing.com
Panscient PeoplePal
Petalbot Pi-Monster
Picscout
Picsearch
PictureFinder PictureFinder
Piepmatz
Pimonster Pimonster
Pixray Pu_iN
PleaseCrawl QueryN Metasearch
Pockey
Probethenet
PxBroker
RankActive RankActive
RankActiveLinkBot RankingBot2
RankingBot RankurBot
ReGet Re-re
RealDownload Reaper
RedesScrapy SEOkicks-Robot
SEOkicks SMTBot
SEOprofiler Scanbot
SISTRIX ScrepyBot
SalesIntelligent Searchestate
SearchmetricsBot
Semrush Semrush
Site Sucker SemrushBot
SeobilityBot
Siphon
SiteExplorer
SiteLockSpider
Sitebeam Sitebeam
Siteimprove
Sitevigil
SlySearch SlySearch
SmartDownload SmartDownload
Sociscraper Snake
Snapbot
Snoopy
Sogou web spider Sogou web spider
SpaceBison
SputnikBot
Sqlmap
Sqlworm Sqlworm
Sqworm Sqworm
Steeler
Sucuri
SuperBot
SurveyBot SurveyBot
Swiftbot
T0PHackTeam T0PHackTeam
T8Abot Teleport
The Intraformant Telesoft
TheNomad TheNomad
TightTwatBot TurnitinBot
Tracemyfile
Trendiction
Trendictionbot
Turnitin
TwengaBot
Twice Twice
Typhoeus Upflow
URLy.Warning VB Project
UnisterBot
VCI
Vagabondo Vagabondo
VelenPublicWebCrawler
VoidEYE VoidEYE
WASALive-Bot Voil
WPScan WEBDAV
WWW::Mechanize WWW-Collector-E
WallpapersHD
WeSEE
Web Auto
Web Enhancer Web Enhancer
Web Fetch
Web Fuck Web Fuck
Web Sauger Web Pix
Web Sucker
WebCollage
WebEnhancer WebEnhancer
WebFuck
WebGo IS
WebImageCollector
WebReaper WebReaper
WebStripper WebSucker
WebWhacker
WebZIP WebZIP
WebsiteExtractor Webalta
WebsiteQuester
Webster Webster
Woobot Whack
Widow
WinHTTrack
Xaldon WebSpider
Xaldon_WebSpider
YoudaoBot YoudaoBot
Zauba Zauba
Zermelo Zeus
ZmEu ZmEu
ZumBot ZoomBot
ZyBorg adscanner
arquivo-web-crawler autoemailspider
arquivo.pt cah.io.community
check1.exe check1.exe
clark-crawler clark-crawler
coccocbot-web coccocbot-web
com.plumanalytics cognitiveseo
crawl.sogou.com crawler4j
crawler.feedback
demandbase-bot demandbase-bot
eCatch eCatch
evc-batch internetVista monitor
facebookscraper ips-agent
heritrix lwp-request
lwp-trivial lwp-trivial
magpie-crawler magpie-crawler
meanpathbot page scorer
muhstik-scan
pcBrowser pcBrowser
plumanalytics probe-image-size
s1z.ru
scalaj-http scalaj-http
sexsearcher scan.lol
siteripz seobility
sogouspider seocompany.store
sp_auditbot sp_auditbot
tAkeOut tAkeOut
ubermetrics-technologies.com trendiction.de
voyagerx.com x22Mozilla
webmeup-crawler xpymep1.exe
webpros.com zauba.io
zgrab

302
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 01h4x.com
AIBOT 404enemy
ASPSeek ASPSeek
Aboundex AhrefsBot
Aboundexbot
AiHitBot AiHitBot
Alligator
Anarchie Anarchie
Anthill Anarchy99
Apexoo
Aspiegel Aspiegel
Attach
AwarioSmartBot AwarioSmartBot
BBBike BackDoorBot
BackStreet BackStreet
BackWeb
Backlink-Ceck
Badass Badass
Barkrowler Barkrowler
Bigfoot Battleztar Bazinga
Bitacle BetaBot
BlackWidow Black Hole
Blackboard Blackboard
Boardreader
Brandwatch
Buck
BuiltBotTough BuiltBotTough
Bullseye Bullseye
BuzzSumo
CATExplorador CATExplorador
CCBot
CSHttp CSHttp
Calculon Calculon
CazoodleBot
CheeseBot CheeseBot
CherryPicker
ChinaClaw
Cliqzbot
Cloud mapping Cloud mapping
Collector Cogentbot
Cosmos Copyscape
Craftbot
Crawling at Home Project
Crescent
CyotekWebCopy CyotekWebCopy
DSearch DBLBot
DataCha0s DIIbot
Demon DatabaseDriverMysqli
Devil Devil
Dirbuster
Disco Disco
Discobot DomCopBot
DittoSpyder
DomainAppender DomainAppender
Dragonfly
ECCP/1.0
EMail Siphon EMail Siphon
EMail Wolf
Ebingbong
EirGrabber
EroCrawler
Evil
Exabot
Express WebPictures Express WebPictures
ExtractorPro ExtractorPro
Extreme Picture Finder
Ezooms Ezooms
FHscan FemtosearchBot
Fimap
FlashGet FlashGet
Flunky Foobot
Freeuploader Fuzz
FyberSpider
Fyrebot Fyrebot
GT::WWW
Genieo Genieo
GermCrawler
Gigabot Gigabot
Go-Ahead-Got-It
GoZilla GoZilla
GrabNet Gotit
Grafula GridBot
HMView
HTTrack
Haansoft Haansoft
HaosouSpider Hloader
Harvest HybridBot
Heritrix IDBot
IDBTE4M IlseBot
Iblog Indy Library
Image Sucker
IndeedBot
InfoNaviRobot
Internet Ninja
InternetSeer InternetSeer
Iria
IstellaBot IstellaBot
JOC Web Spider Jbrofuzz
JamesBOT
JetCar JetCar
Jetty Jetty
Jorgee JikeSpider
JustView
Jyxobot
Kinza Kinza
Kozmosbot Lanshanbot
LNSpiderguy Larbin
LWP::Simple
LeechFTP
LeechGet LeechGet
LexiBot
Lftp Lftp
LibWeb LibWeb
Libwhisker Libwhisker
Lightspeedsystems
LinkWalker
Linkdexbot
LinkextractorPro LinkextractorPro
LinksManager LinksManager
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
Litemage_walker Litemage_walker
MFC_Tear_Sample Lmspider
MIDown tool MIDown tool
MIIxpc MIIxpc
MQQBrowser MQQBrowser
Magnet MSIECrawler
Mail.RU_Bot Majestic SEO
MarkMonitor MarkMonitor
MarkWatch Masscan
Mata Hari Mata Hari
MeanPath Bot Microsoft URL Control
Meanpathbot Moblie Safari
MegaIndex.ru Mojeek
Mr.4x3 Morfeus Fucking Scanner
Msrabot Mozlila
Musobot
NICErsPRO
NPbot
Nameprotect Nameprotect
Nessus Navroad
Net Vampire
NetLyzer NetLyzer
NetMechanic NetSpider
NetZIP Netcraft
Nettrack Nettrack
Netvibes Nibbler
NextGenSearchBot
Niki-bot
Nikto
Nimbostratus Nimbostratus
Nuclei Octopus
OnCrawl Offline Explorer
OrangeBot Offline Navigator
OpenLinkProfiler
OrangeSpider
OutclicksBot OutclicksBot
POE-Component-Client-HTTP PHPCrawl
Page Analyzer
PageThing.com
Pandalytics Pandalytics
Papa Foto Panscient
PeoplePal
Petalbot
Pi-Monster Pi-Monster
Picsearch
PictureFinder PictureFinder
Pimonster
Pixray Pixray
Pockey ProPowerBot
Psbot Pu_iN
QueryN Metasearch
RSSingBot RSSingBot
RankingBot2 RankFlex
RankingBot
RankurBot RankurBot
Re-re Re-re
RealDownload ReGet
Recorder Reaper
Ripper RebelMouse
RedesScrapy
RepoMonkey
RocketCrawler RocketCrawler
Rogerbot
SBIder SBIder
SEOprofiler
SEOstats SEOstats
SISTRIX
SMTBot SMTBot
SalesIntelligent ScanAlert
Scanbot Scanbot
ScoutJet Screaming
ScreenerBot
ScrepyBot ScrepyBot
SearchmetricsBot SentiBot
SemrushBot SeobilityBot
Seomoz
Shodan Shodan
Siphon Siphon
SiteCheckerBotCrawler Site Sucker
SiteSnagger SiteLockSpider
SiteSucker SiteSucker
Sitebeam
Siteimprove Siteimprove
SlySearch Snake
SmartDownload
Snapbot Snapbot
Snoopy
Sogou web spider
Sosospider Sosospider
Sottopop
Spanner
SputnikBot SputnikBot
Sqlmap
Sqlworm
Stripper Stripper
Sucuri Suzuran
T0PHackTeam
T8Abot T8Abot
TeleportPro
Telesoft
Telesphoreo
Telesphorep
The Intraformant The Intraformant
Toata TheNomad
Tracemyfile Trendiction
Trendictionbot
True_Robot True_Robot
Turingos
Turnitin
TurnitinBot
Twice Twice
Typhoeus
URLy.Warning URLy.Warning
UnisterBot V-BOT
Vacuum Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie Virusdie
WBSearchBot Voil
WWW-Collector-E WPScan
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWWOFFLE
Wallpapers Web Collage
Wallpapers/3.0 Web Pix
Web Auto
Web Fuck
Web Sauger Web Sauger
WebCollage Web Sucker
WebImageCollector WebBandit
WebLeacher WebEnhancer
WebZIP WebGo IS
WebPix
WebStripper
WebWhacker
Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
Website Quester Webshag
WebsiteExtractor Webster
Whacker Whacker
Whatweb Whatweb
WinHTTrack Wprecon
WiseGuys Robot
Wonderbot
Wotbox
Xaldon_WebSpider
Xenu Xenu
YoudaoBot
Zade
Zauba Zauba
Zitebot Zermelo
ZmEu
ZoomBot
ZoominfoBot ZoominfoBot
archive.org_bot archive.org_bot
autoemailspider
cah.io.community cah.io.community
coccocbot-web check1.exe
cognitiveseo crawl.sogou.com
com.plumanalytics crawler.feedback
dataforseo.com evc-batch
instabid gopher
internetVista monitor heritrix
isitwp.com
lwp-request lwp-request
lwp-trivial magpie-crawler
meanpathbot muhstik-scan
mediawords netEstate NE Crawler
oBot pcBrowser
plumanalytics
polaris version polaris version
s1z.ru s1z.ru
satoristudio.net satoristudio.net
scalaj-http
scan.lol scan.lol
seostar serpstatbot
trendiction.com sitechecker.pro
ubermetrics-technologies.com sogouspider
webpros.com sp_auditbot
webprosbot spyfu
sysscan
webmeup-crawler
x09Mozilla x09Mozilla
x22Mozilla
xpymep1.exe xpymep1.exe
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
clubfashionista.com aquarium-pro.ru
compliance-mark.top arabsexxxtube.com
mybinaryoptionsrobot.com azbukadiets.ru
ngps1.ru fbdownloader.com
piccdata.com hustoon.over-blog.com
qwarckoine.com iqs.biz.ua
uogonline.com nofreezingmac.click
vsesubwaysurfers.com pufip.com
vykupavto-krasnodar.ru stuff-about-money.com
wildworld.site tongkatmadura.info

932
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0n-line.tv 0c47f8422d3f.com
1001watch.com.ua 101lesbian.xyz
2drittel.de 1j7740kd.website
2x2fan.ru 2728fb936f0.com
3th.co.in 2pxg8bcf.top
4k-player.pl 45en.ru
4kpics.rocks 4replicawatch.net
57883.net 5-steps-to-start-business.com
7a2rnuey1tw9ar.ru aa9d046aab36af4ff182f097f840430d51.com
8xv8.com aad1f4acb0a373420d9b0c4202d38d94fa.com
98oi.ru abiente.ru
a1security.com.ua academiacsmendoza.org
aa8b68101d388c446389283820863176e7.com addray.pro
adcash.com adsloads.com
adconscious.com advokateg.ru
addtoadd.men aero2.ru
adsfresh.men affiliate-fr.com
adtech.fr ahmedabadwebs.com
adult-shop.com.ua aihelen.net
advancedcleaner.com airmaxshoes-2016.com
affrh2015.com ald2014.org
afmuseum.com
albuteroli.com
algarveglobal.com
alldezservice.kz
allkrim.com
allnews24.in allnews24.in
alloysteel.ru
allpdfmags.net
allsilver925.co.il
alvtank.se alvtank.se
amazingninja.com am-se.com
amazon-adsystem.com angkortours.vn
amazon-seo-service.com animebox.com.ua
anniemation.com apartmentratings.com
anonymizeme.pro asia-forum.ru
ap.senai.br asophoto.com
apifasterlightin-a.akamaihd.net asrvrep-a.akamaihd.net
apioutoboxnet-a.akamaihd.net atmovs.com
appfixing.space audiobangout.com
apple.com-cleaner.systems availit.weebly.com
applicationg29.com avek.ru
arvut.org avtoarenda.by
asdfg.pro azadnegar.com
asmxsatadriverin.aircus.com azazaporn.com
asrv-a.akamaihd.net azbuka-mo.ru
autochoixspinelli.com babieca.com
autoseo-traffic.com balkanfarma.ru
aviav.eu bannerconnect.net
avon-severozapad.ru bavsac.com
avramstroy.ru bdsmgalls.net
ayerbo.xhost.ro beachpics.com
babs.com.ua bharatdefencekavach.com
back.dog biglistofwebsites.com
backlinkwatch.com binomo.com
balans.shahterworld.org bizfly.info
bangla-chat-uk-paralud.ga bizlist.com.de
bankofthewext.com bizzliving.com
bcmp.org blockety.co
beatifullandscape.co blpmovies.com
beauty-b0x.pl bmw-ark.ru
beauty-clinic.ru bochemit.com.ua
begalka.xyz boo-arts.com
berdasovivan.ru
berrymall.ru
best-seo-offer.com
bestkfiledxd.cf
bestofpicture.com
bestpornuha.com
bezlimitko.xyz
bezsporno.ru
bio.trade-jp.net
bitcoinpile.com
bjgugu.net.ua
blackcurranthumidifiers.site
blackle.com
blackplanet.com
blobar.org
blogos.kz
bnt-team.com
bocoarchives.org
bookmark4you.com.biz
bouda.kvalitne.cz bouda.kvalitne.cz
braintobe.win
braip.com.br
brandedleadgeneration.com brandedleadgeneration.com
brandov.ru brg8.com
breastaugmentation.co.za buyfriend.ru
briomotor.co buynorxx.com
browsepulse-a.akamaihd.net bzero1jewelry.net
browserprotecter.com
bugshoot.cn
burger-imperia.com
buron.pw
business-suggested.tk
buyessayonline19.blogspot.ru
buytizanidineonlinenoprescription.blogspot.com
bvlgariring.xyz
byme.se
callejondelpozo.es callejondelpozo.es
camdenmemorials.com candy-glam-hp.com
capture-room.com cardsdumps.com
carrauterie.be carsdined.org
carstrends2015.com cartierlove.xyz
cartierloveringreplica.xyz cartierreplica.xyz
cartierreplica.top cashonads.com
catalogs-parts.com caveavins.fr
cb.iphantom.com
cbox.ws cbox.ws
cejewelry.xyz
cellfun.mobi
cercacamion.it cercacamion.it
cherrypointplace.ca cezartabac.ro
chidporn.com chatroulette.si
cheapbarbour.online
cheapjerseysa.com
chee-by.biz
chinaelitecheapjerseys.com chinaelitecheapjerseys.com
chinajerseyswholesalecoupons.com christianlouboutinoutlet.win
choice-credit.ru christianlouboutinsaleoutletonline.us
cialgenisrx.com cienum.fr
cleanallvirus.com cinemaenergy-hd.ru
cleanpcnow.com cinemaflix.website
clmforexeu.com clickprotects.com
compiko.info clothesforcash.com
communisave.co.za
compliance-alex.top
compliance-andrew.xyz compliance-andrew.xyz
compliance-brian.top compliance-brian.xyz
compliance-julianna.top compliance-checker-7.info
compliance-walter.top compliance-ivan.top
continent-e.tv compliance-jack.top
compliance-willy.top
conboy.us
cookie-law-enforcement-hh.xyz cookie-law-enforcement-hh.xyz
cosmetic.donna7753191.ru coolgamechannel.com
counciltally.com coop-gamers.ru
covadhosting.biz cornerstone-countertops.com
countercrazy.com
courtshipgift.com
cpajunkies.com
cv.wallhade.co cv.wallhade.co
cwetochki.ru cyse.tk
db.speedup.gdn dailyfinancefix.com
dcj-nn.ru dalavia.ru
dealitright.click dame-ns.kz
deniven.1bb.ru daretodonate.co
design-lands.ru delta-line.men
descargar-musica-gratis.net
deshevo-nedorogo.ru
destinationrealestate.com
detsky-nabytek.info detsky-nabytek.info
dev33.dioniqlabb.se dev.citetick.com
dichvuvesinhhanoi.com diesel-parts28.ru
distver.ru dieswaene.com
diusyawiga.tk dimaka.info
dosug-lux.ru div.as
downvids.net djekxa.ru
driving.kiev.ua dkvorota.ru
drugs-no-rx.info doctissimo.fr
doctormakarova.ru
domain-submit.info
domaincrawler.com
donna7753191.ru
dood.live
dostavimvdom.ru
dosugrostov.site
download-of-the-warez.blogspot.com
downtuptv.gq
drev.biz
duawitchrarato.tk
e-avon.ru e-avon.ru
ed-shop01.ru easync.io
egvar.net easyshoppermac.com
electrik-avenue.com ekaterinburg.xrus.org
elektrischezigarette1.onsugar.com el-nation.com
elektrozigaretten2.yn.lt eldiariodeguadalajara.com
elvel.com.ua elektronischezigarettekaufen1.beeplog.com
energy-ua.com elidelcream.weebly.com
enginebay.ru elmifarhangi.com
essayassist.com en.altezza.travel
essaypro.com eng-lyrics.com
europeanwatches.ru englate.com
eyes-on-you.ga enternet.ee
ezb.elvenmachine.com eredijovon.com
ezigarettekaufen.myblog.de eropho.com
faracontrol.ir escortslet.net
euronis-free.com
euroskat.ru
exoneration-project.us
ezigarettenshop1.yolasite.com
ezigaretteshop.webs.com
ezigaretteshop2.vefblog.net
ezofest.sk
facialporntube.com
falcon-images.blogspot.com
fanpagerobot.com
fashion.stellaconstance.co
fealq.com
fearcrow.com fearcrow.com
fettefrauen.net fenoyl.batcave.net
fickenprivat.info fetishinside.com
fikasound.tk
filkhbr.com
film-one.ru
findpics.pw findpics.pw
findsexguide.com fodelsedagspresenter.nu
foodcrafts.website
forensicpsychiatry.ru forensicpsychiatry.ru
foto-sisek.porngalleries.top forms-mtm.ru
free-gluten.ru foxweber.com
free-laptop-reward.com foxydeal.com
free-share-buttons.com frcls.fr
freemaintenancesysforpcandmac.top free-today.com
freewlan.info freenode.info
fres-news.com freewareseek.com
funcrushgames.com fxtips.ru
fyl.com.ru g.starmoe.xyz
gabeshop.ru gallery.rennlist.com
gambarkatabaru.com gameprimary.com
gambarxkata.co gaming-journal.com
gamesprite.me gayxperience.com
gamevalue7.weebly.com geile-lelly.eu
geoads.com
get-seo-domain.com get-seo-domain.com
girlporn.ru gettpromos.com
globetrotting-culture.ru gfaq.ru
gomusix.com ghazel.ru
greatidea.marketing globalscam.ga
gsasearchengineranker.pw glopages.ru
gsasearchenginerankerhelp.com gloverid.site
gtopstats.com go2album.com
gwebtools.com golfresa.lucania.se
h2monline.com good-mummy.ru
hamacapty.com green-tea.tv
hard-porn.mobi greenzaim.ru
grupografico-pilar.com.ar
gsasearchengineranker.space
gsasearchenginerankerdiscount.com
gta-top.ru
gwhwpxbw.bloger.index.hr
handicapvantoday.com
hanink.biz.ly
hasfun.com hasfun.com
hausfrauensex18.com histock.info
hdimagelib.com hollywoodactress.info
healgastro.com hosting-tracker.com
hem.passagen.se hotelcrocenzi.sm
homik.pw hotenergy.ru
housediz.com houseofrose.com
hoverboard360.es hoztorg-opt.ru
hoverboardforsaledirect.com humanelydrew.com
howlongdoestizanidinestayinyoursystem.blogspot.com hustoon.over-blog.com
hreade.com i-hobot.ru
hscsscotland.com ibb.com.ua
hulfingtonpost.com
idc.com.ua idc.com.ua
iframe-toloka.com ideawheel.com
iinstalll-fii1leis.jus0wil.pp.ua iflycapetown.co.za
ilmen.net iklysha.ml
ilmexico.com ilikevitaly.com
ilovevitaly.xyz imagui.eu
imadedinner.net imicrovision.com
images.gyffu.com imperiafilm.ru
img.wallpaperstock.net inboundlinks.win
impresagaia.it infospot.pt
incanto.in.ua innoslicon.com
incep.imagine-ex.co inspiring-desperate.tk
intim-uslugi.info intelhdgraphicsgtdrive6w.metroblog.com
investmac.com iqs.biz.ua
investsuccess.org iqupdatetmz.win
islandminingsupply.wordpress.com istizanidineanarcoticdrug.blogspot.com
izhevsk.xrus.org j33x.com
janerikholst.se james13prix.info
javrip.net jamiembrown.com
jbl-charge.info jerseyswholesalechinalimited.com
jetsli.de jerseywholesalebizchina.com
johnnyhaley.top jillepille.com
jumptap.com jose.mulinohouse.co
jyrxd.com joy-penguin.com
jyvopys.com jpcycles.com
kalb.ru kamagragelusa.net
kazinogames.lv karadene.com
kemerovo.zrus.org kazan.xrus.org
ketoanhanoi.info keywordbasket.com
keyhantercume.com kinozapas.com
keywordsdoctor.com klejonka.info
keywordspay.com kliksaya.com
kinobaks.com kndxbkdx.bloger.index.hr
kiprinform.com knogg.net
kiwi237au.tk komputernaya-pomosh-moscow.ru
kladrus.ru kopihijau.info
kletkimehan.ru
knigonosha.net
kongoultry.net
konica.kz
koptims.tiu.ru koptims.tiu.ru
krasivoe-hd.com kostenloser-sex.com
krasnodar.ru kosynka-games.ru
krasnodar.zrus.org kredit-pod-zalog-krasnodar.ru
kredit-blog.ru kunstaktien.info
kskjco.club larutti.ru
kung-fu-ru.com legionalpha.com
kustanay.kz lemon-ade.site
kuzinsp.ru letslowbefast.today
kvartiry-remont.ucoz.ru letsrepair.in
ladsblue.com lexixxx.com
languagecode.com lezbiyanki.net
lapolis.it lietaer.com
lecbter-relationships.ga lignofix.ua
ledis.top
lenpipet.ru
lernur.net
linerdrilling.com linerdrilling.com
lion.conboy.us lineshops.biz
live-sexcam.tk linkbux.ru
liveinternet.ru longlifelomilomi.info
liverpool.gsofootball.com lostfilm-online.ru
lob.com.ru lowephotos.info
lombia.co macotool.com
louboutinreplica.pw
ltvperf.com
lulea-auktionsverk.se
mackeeper-center.club
magicalfind-a.akamaihd.net
mail.allnews24.in
mainhp.net mainhp.net
marketingtechniques.info makedo.ru
matrixalchemy.com masturbate.co.uk
medicines-choice.com max-p.men
medictube.ru maxxtor.eu
meds-online24.com medanestesia.ru
meltwater.com mejoresfotos.eu
mensandals.xyz mezaruk.info
minecraft-neo.ru micasainvest.com
mini.7zap.com microstatic.pl
mister-shop.com minegam.com
mistr-x.org minharevisao.com
mmostrike.ru misterjtbarbers.com
mnimmigrantrights.net mobplayer.ru
mobile-appster.ru mockupui.com
moje-recenze.cz moi-glazki.ru
mompussy.net mosrif.ru
moncleroutletonline.pw mp3ringtone.info
montredemarque.nl mrinsidesales.com
moscow-region.ru mrlmedia.net
muschisexbilder.com mrsdalloways.com
mrwhite.biz
msk.afora.ru
musicdaddy.net musicdaddy.net
my-floor.in.ua musicvidz.ru
mydirtystuff.com my-big-family.com
mydownloadengine.com mydownloadengine.com
mydownlodablefiles.com myfreecams.com
mykings.pw myfreetutorials.com
myonlinepayday.co mygameplus.com
myplaycity.com myhydros.org
myra.top mylida.org
mysex21.com mymobilemoneypages.com
nacap.ru naphukete.ru
nardulan.com naturtreenspicerx.pw
nelc.edu.eg nfljerseyscheapbiz.us
nomuos.it nfljerseysforsalewholesaler.com
novodigs.com nikitabuch.com
o.light.d0t.ru nippon-bearings.ru
obsessionphrases.com no-fuel.org
offtime.ru nodup.ru
noscrapleftbehind.co
ofanda.com
ohmyrings.com
oktube.ru oktube.ru
onemactrckr.com oneclickfiles.com
onlinadverts.com onefilms.net
onlinetvseries.me onko-24.com
opinionreelle.com onlainbesplatno.ru
orenburg-gsm.ru online-sbank.ru
orhonit.com onlinemegax.com
ozas.net osb.se11.ru
pagesense.com owathemes.com
panicatack.com ownshop.win
parfusale.se paramountmarble.co.uk
pic2fly.com partner-high.men
picswe.com perfectpracticeweb.com
pila.pl perm.xrus.org
play-movie.pl picurams.pw
polimga.pw pinapchik.com
porn555.com pkr1hand.com
porno-chaman.info plastjulgranar.se
porno-dojki.net pogosh.com
pornogad.com polcin.de
pornogig.com porndl.org
poshiv-chehol.ru pornokajf.com
pr-ten.de pornosok.ru
predmety.in.ua pornozhara.com
predominant-invent.tk priora-2.com
preg.marketingvici.com pro-tec.kz
preparevideosafesystem4unow.space profitsport.club
printdirectforless.com progonrumarket.ru
privatefx-in.ru pronekut.com
profitfx.online prostitutki-kiev.org
pufip.com prostitutki-tyumeni.org
pukaporn.com proxyelite.biz
puzo2arbuza.ru psvita.ru
qpypcx.com push-ad.com
ranksonic.com pyrodesigns.com.au
refads.pro qcstrtvt.bloger.index.hr
remontbiz.ru qld10000.net
ringtonepartner.com quick-offer.com
rn-to-bsn.com quizzitch.net
roma-kukareku.livejournal.com qwesa.ru
rossmark.ru rank-checker.online
rozalli.com razamicroelectronics.com
rrutw.com realnye-otzyvy.info
runstocks.com renecaovilla.online
rus-pornuha.com rezeptiblud.ru
rus-teh.narod.ru ryetaw.com
rvzr-a.akamaihd.net s1z.ru
samara.rosfirm.ru sack.net
sarm.tk sajatvelemeny.com
sashagreyblog.ga samolet.fr
saveindex.xyz savememoney.co.za
sbdl.no scanner-josh.top
scanner-jess.top scanner-julianna.top
scanner-jessica.top score-ads.men
scanner-walt.top searchlock.com
search.pch.com seeresultshub-a.akamaihd.net
searchkut.com seo-smm.kz
searchmywindow-a.akamaihd.net
secret.xn--oogle-wmc.com
seminarykansas904.ml
seo-traffic-ranking.info
seojokes.net seojokes.net
seorussian.ru seorankinglinks.us
serdcenebolit.com seozoom.it
sevendays.com.ua servisural.ru
sexpornotales.net seryeznie-znakomstva.ru
sexvideo-sex.com sex-pr.net
sfd-chess.ru sex-watch.com
shitmovs.com sexobzor.info
sientalyric.co sexsaoy.com
simplesite.com shohanb.com
site3.free-share-buttons.com shopperifymac.com
siteimprove.com siha.de
skanninge.se simple-image.com.ua
slavkokacunko.de site-auditor.online
smadihome.com sizeplus.work
sms2x2.ru sky-mine.ru
smsactivator.ru skylta.com
snts.shell-pmr.ru skyway24.ru
social-s-ggg.xyz smstraf.ru
socialsignals24.com snow.nvr163.com
soft1.ru sophang8.com
solinf.co
solnplast.ru
soundfrost.org
spb-plitka.ru
spb.ru
spidtest.space spidtest.space
spinnerco.ca sports-supplements.us
spywarebegone.com srdrvp.com
stackthatbucks.com staff.prairiesouth.ca
stard.shop stathat.com
steelmaster.lv steebook.com
sterva.cc studworks.org
stpicks.com suggest-keywords.com
stretchingabuckblog.com supermama.top
strfls.com surgut.zrus.org
stroy-portal22.ru suture.co
supernew.org swingerseiten.com
superstats.com tacbelarus.ru
swagbucks.com tagil.zrus.org
swinginwithme.ru takeflyte.com
swiped.su
talant-factory.ru talant-factory.ru
tds-advert002.info tech4master.com
techart24.com
technopellet.gr
teenfuck.tv teenfuck.tv
telefonsexkostenlos.tk tehngr.ru
tengohydar.tk thebluffs.com
thefotosgratis.eu thecounter.com
themeforest.net thefarmergame.com
thetardistimes.ovh thepantonpractice.co.uk
timhost.ru tilido.com
tizanidinesleepaid.blogspot.com tizanidinecost.blogspot.com
tizanidinevsvalium.blogspot.com tizanidinestreetprice.blogspot.com
tksn.ru tizanidinestreetvalue.blogspot.com
tooplay.com tizanidinewithdrawalsymptoms.blogspot.com
top10-online-games.com tmtrck.com
top10registrycleaners.com topbestgames.com
topkarkas.com torrentgamer.net
topvidos.ru torrents.cd
torrentdownloadhub.com trafficshaper.com
torrnada.ru traffique.net
tourcroatia.co.uk traffmonster.info
traffic100.com tramadolandtizanidine.blogspot.com
trafficjunky.net truck-land.life
trafficmonetize.org trustaffs.com
trafficstars.com tuberkulezanet.ru
traxdom.ru tuckermktg.com
trustedmaccleaner.com tyumen.xrus.org
tt-ipd.info ultimateclassicrock.com
ttrraacckkrr.com unece.org
ttsq.fr uptime-gamma.net
tuckpointingmasonrysystems.com urlopener.blogspot.com.au
tv-spoty.info utrolive.ru
twelvevisionspartyofcolorado.com v24s.net
u17795.netangels.ru validccseller.com
ublaze.ru vapsy.com
ugogo.info velen.io
unrealcommander.org venerologiya.com
uroki.net via-energy-cumpara.com
usacasino.com via-energy-order.com
usadacha.net viagra.pp.ua
usbggettwku.ga viagroid.ru
vandrie-ict.nl viandpet.com
vbtracker.net vikistars.com
vektorpress.ru villacoloniale.com
verymes.xyz vita.com.hr
vesnatehno.com vitalads.net
via-gra.webstarts.com viteonlusarezzo.it
viagengrarx.com vkak.ru
videtubs.pl vladimir.xrus.org
vigrx-original.ru vodaodessa.com
vinsit.ru vykup-avto-krasnodar.ru
vkgaleria.com vykupavto-krasnodar.ru
vladimir.zrus.org vzubah.com
vut.com.ru warningwar.ru
wallpaperaccess.com webmasterhome.cn
webscouter.net webnode.me
website-datenbank.de website-datenbank.de
website-speed-up.top wholesalejerseychinaoutlet.com
weekes.biz.tc wormix-cheats.ru
white-truck.life www.solartek.ru
woman-h.ru www.souvenirua.com
worldoffiles.ru www.timer4web.com
writingservices17.blogspot.ru wygraj-teraz.com
wzgyyq.com xdoza.com
x5market.ru xlolitka.com
xcombear.ru xlovecam.com
xn----btbdvdh4aafrfciljm6k.xn--p1ai xn----9sbebi2bvzr7h.xn--p1ai
xn----itbkqkfiq.xn--p1ai xn--80abgj3a5acid6ghs.top
xn--80ab4aa2g.xn--p1ai xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai
xn--80atua3d.xn--p1ai xn--80atua3d.xn--p1ai
xnxx-n.com xn--h1aakne2ba.xn--p1ai
xvideospornoru.com xn--hxazdsfy.blogspot.com
xwatt.ru xxxdatinglocal.us
xxlargepop.com xxxkaz.org
yogamatsexpert.com yaoguangdj.com
you-shall-not-pass.is74.ru you-shall-not-pass.is74.ru
zdesformula.ru youdesigner.kz
yugo-star.ru
zakazfutbolki.com
zeikopay.com
zerocash.msk.ru
zeroredirect10.com
zhorapankratov7.blogspot.com zhorapankratov7.blogspot.com
zigzog.ru
zoogdinsney.com
zoolubimets.ru
zoominfo.com zoominfo.com
zoomovies.org
zryydi.com
zs2vm.top
zskdla.site
zvetki.ru
zzlgxh.com

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
12masterov.com 123kuma.com
3-letter-domains.net 4kplayer.pl
7search.com 7a2rnuey1tw9ar.ru
acheterviagrafr24.com 7wind.ru
adpremium.org a.frcls.fr
ads-cool.pro aa81bf391151884adfa3dd677e41f94be1.com
alldubai.biz adclickthru.net
allstatesugarbowl.org adman.gr
anjalika.co.in admatic.com.tr
apispringsmartne-a.akamaihd.net adultfullhd.com
appleid-verification.com ageofclones.com
arkartex.ru aktobe.xkaz.org
artdestshop.eu albamargroup.com
atelielembrancaqueencanta.com.br alert-fdm.xyz
autowebmarket.com.ua ample-awards-today.us
babespcs.com amung.us
babieca.com animali.deagostinipassion.it
bad-stars.net anlimebel.kiev.ua
bankofthewext.com apibetweenlinesn-a.akamaihd.net
batanga.net apidiamondatanet-a.akamaihd.net
belsetirehafi.tk arenda-avtoprokat-krasnodar.ru
bestchoice.cf artpicso.com
bestofferswalkmydogouteveryday.gq arvut.org
bestwebsitesawards.com atleticpharm.org
beta.hotkeys.com autoloans.com
bidr.trellian.com automobile-spec.com
birzha-truda.eu aviapanda.ru
blog.axant.it aviav.org
blogrankers.com avtochehli.by
bobinoz.com avtovolop.ru
bodybuilding-shop.biz azartmix.com
bolegapakistan.com badavit.com.ua
bonkers.name bankmib.ru
bonzibuddi.com bannerpower.com
book-bets.com barnfurnituremart.com
boostmyppc.com bbsoldes.fr
braindaily.xyz beachtoday.ru
brandov.ru beatifullandscape.co
bestcurs.org
biteg.xyz
bloglag.com
bmwhighperformers.com
bolezniorganov.ru
brains2.biz
breget74.ru
brillianty.info brillianty.info
bum.com.ru btvn.ru
buybest1.biz bukleteg.xyz
buypillsorderonline.com buttons-for-your-website.com
cardiosport.com.ua buyessayonline19.blogspot.ru
cartechnic.ru call-of-duty.info
chatroulette.online candelluxsklep.pl
chatroulette.video cartierlove.xyz
cheapjerseysbizwholesale.us cbb1smartlist12.click
cheapjerseysfootballshop.com chinese-amezon.com
cialischstgerts.com
ciekawinki.pl ciekawinki.pl
citizenclsdriveri7.pen.io
civilwartheater.com civilwartheater.com
clarithromycin500mg.com collegerentals.com
compliance-barak.xyz compliance-barak.top
cypernhuset.se cordstrap.cc
darkages.info countercrazy.com
davebestdeals.com cracksplanet.com
ddlmega.net craftinsta.ru
dividendo.ru custom-electric-guitar.com
docstoc.com customerpromos-a.akamaihd.net
documentsite.net cutalltheshit.com
dodlive.mil dbmkfhqk.bloger.index.hr
domain2008.com detki-opt.ru
download-of-the-warez.blogspot.com diegolopezcastan.com
dvd-famille.com dimaka.info
dynainbox.com diy-handmade-ideas.com
eldiariodeguadalajara.com djstools.com
elektronischezigarettekaufen2.tumblr.com dogsrun.net
emailaccountlogin.co dollartree.info
empirepoker.com domainsatcost.com
euromasterclass.ru domashneeruporno.com
everypony.ru dotmass.top
excitacion.info downtuptv.gq
exmasters.com duawitchrarato.tk
expertblog.info dverimegapolis.ru
faithe.top educontest.net
familyphysician.ru ege-essay.ru
financepoints.eu ege-russian.ru
findpics.pw electricwheelchairsarea.com
fitnesspiks.com eredijovon.com
fiverr.com ezigarettenkaufen1.deviantart.com
forum.doctissimo.fr ezigaretteshop2.vefblog.net
forum.poker4life.ru falllow.gq
forum.tvmir.org filesdatabase.net
forum20.smailik.org filesmonster.porn
forums.toucharcade.com findthe.pet
foxjuegos.com flashahead.info
fullfileaccess.com fm-upgrade.ru
gameonasia.com frankofficial.ru
gepezz.info free-deals.faith
gettpromos.com free-floating-buttons.com
google-liar.ru gallery.rennlist.com
greatzip.com gemara.com
greenzaim.ru girlspicsa.com
gsasearchenginerankerhelp.com gopixdatabase.com
hanwei.us gratuitxblcodes.com
hao123.com greenidesign.co
hawaiielectriclight.com guge.io
hem.passagen.se haveinc.xyz
hobild.net hd720kino.ru
homeinns.com helicalpile.us
idegenvezeto.eu helvetia.com.ua
inbabes.sexushost.com hermesreplica.win
interferencer.ru hit-men.men
interfucks.net home-task.com
isitwp.com hoterika.com
itag.pw i-hobot.ru
ilovevitaly.ru
instakink.com
ip-guide.com
jackpotchances.com jackpotchances.com
javlibrary.cc jerseychinabizwholesale.us
jose.mulinohouse.co kamalsinha.com
journeydownthescale.info keenoutlet.online
kaz.kz keywordblocks.com
kirov.zrus.org kinopolet.net
komputers-best.ru kometars.xyz
konkursov.net kongruan.com
kosova.de ladsblue.com
krasivoe-hd.net laptop-4-less.com
krasnodar-avtolombards.ru letto.by
krynica.info likesuccess.com
ks1234.com localflirtbuddies.com
kumuk.info locustdesign.co
landscapebackgrounds.blogspot.com losangeles-ads.com
le-clos-des-alouettes.com mackeeper-center.club
libertybilisim.com mahnwachen-helfen.info
mediaoffers.click makis.nu
medizinreisen.de malwareremovalcenter.com
meteostate.com matpre.top
micasainvest.com mebel-alait.ru
mirzonru.net medanestesia.ru
misandesign.se medkletki.ru
mere.host.sk
mnimmigrantrights.net mnimmigrantrights.net
moi-glazki.ru modenamebel.ru
moviemail-online.co.uk money-every-day.com
moxo.com moneyteam24.com
musicpro.monster mowser.com
muycerdas.xxx musezone.su
myfreemp3.eu mylovelibrabry.com
mykings.pw najaden.no
namecrumilchlet.tk neodownload.webcam
newstaffadsshop.club netfacet.net
nfljerseyscheapbiz.us newtechspb.ru
nonews.co nisuturnetdgu.tk
obsessionphrases.com nodup.ru
offer.camp notebook-pro.ru
oksrv.com nuker.com
omgtnc.com o-dachnik.ru
onlineserialy.ru omsk.xrus.org
ooomeru.ru oops-cinema.ru
openmultipleurl.com
outshop.ru outshop.ru
ozoz.it panchro.co.uk
panchro.xyz pandarastore.top
pawli.eu partnerworkroom.men
pc-test.net picture-group.com
pepperstyle.ru plohaya-kreditnaya-istoriya.ru
pictures-and-images.net pokemon-go-play.online
popads.net popunder.ru
poppen-nw.net pornhub-forum.ga
porno-raskazy.ru pornoreino.com
pornogad.com potolokelekor.ru
priceg.com pr-ten.de
printdirectforless.com printie.com
prize44.com
proekt-gaz.ru proekt-gaz.ru
profitkode.com rebrand.ly
prospekt-st.ru reimageplus.com
prostitutki-astana.org replica-watch.ru
proxyradar.com ring4rhino.com
rapidokbrain.com royalcar-ufa.ru
reklama1.ru rucrypt.com
remybutler.fr runstocks.com
renecaovillasale.online russian-postindex.ru
rentehno.ru samara.rosfirm.ru
ric.info sape.top
ringtonepartner.com
sad-torg.com.ua
sakhboard.ru
sardinie.us
sbdl.no
sbprabooks.com sbprabooks.com
sbtdesign.co.uk sbwealthsolutions.ca
scenicmissouri.us scanner-alex.top
school-diplomat.ru scanner-alexa.top
search.alot.com scansafe.net
seminarygeorgia59.ga sdrescher.net
serw.clicksor.com semxiu.com
sex-pr.net seolab.top
sherlock.se sharebutton.org
shoesonlinebuy.xyz sigmund-freud.co.uk
sitebeam.net sneakyboy.com
sitechecker.pro snip.tw
skachat-besplatno-obrazcy.ru svolze.com
slim.sellany.ru t3chtonic.com
slimcdn.com tacbelarus.ru
smart-balancewheel.com talant-factory.ru
smartshoppymac.com tasteidea.com
smokewithrabbits.com techart24.com
sobecjvuwa.com.ru templates.franklinfire.co
sofit-dmd.ru thecoolimages.net
solmarket.by thewomenlife.com
sortthemesitesby.com tizanidine4mgtablets.blogspot.com
spravka-medosmotr.ru tizanidineandcipro.blogspot.com
srgwebmail.nl tizanidinecost.blogspot.com
stjamesschool.info tizanidinehcl4mgtablet.blogspot.com
strangeduckfilms.com tizanidineoral.blogspot.com
stretchingabuckblog.com tizanidineorflexeril.blogspot.com
stroydetali.ru tksn.ru
supermama.top top-study.work
support.nopeas.sk touchmods.fr
sureone.pro track112.site
swiped.su tracksurf.daooda.com
teencastingporn.com tracksz.co
tizanidineandgabapentin.blogspot.com trafficmp.com
tizanidinenarcotic.blogspot.com twincitiescarservice.com
tizanidinezanaflex.blogspot.com tyumen.xrus.org
top10-online-games.com tzritel.tk
topads.men uggbootsoutletsale.us
toproadrunner5.info uggsale.online
upproar.com unionmarkt.de
urdoot.win unlimitdocs.net
urengoy.pro viagra.pp.ua
urldelivery.com videochat.world
urlopener.blogspot.com.au vizag.kharkov.ua
ustion.ru vodkoved.ru
vclicks.net voloomoney.com
vegascosmetics.ru voltrknc1.com
viddyoze.com vroze.com
viktoria-center.ru warezaccess.com
vitoriacabos.com watchinf.com
vladimir.zrus.org webcamtalk.net
voloslove.ru webmasters.stackexchange.com
vote-up.ru website-speed-checker.site
vriel.batcave.net whatzmyip.net
webextract.profound.net wowas31.ucoz.ru
webnode.me xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
weboptimizes.com xn----7sbifcamovvfggw9d.xn--p1ai
wedding0venues.tk xn--80ahvj9e.xn--p1ai
wetgames.ru yaderenergy.ru
wheelchairliftsarea.com youandcredit.ru
wurr.voila.net yugk.net
www.kabbalah-red-bracelets.com yurgorod.ru
www.wohnkabinen-shop.de yuweng.info
xlolitka.com zatjmuzu.info
xvideosporn.biz zero1.it
xvideospornoru.com zmoda.hostreo.com
xxx-treker.ru zscalertwo.net
yopoint.in zvooq.eu
yourserverisdown.com
zhorapankratov7.blogspot.com
zigarettenonlinekaufen1.bloog.pl
zverokruh-shop.cz

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.92 0;
127.0.0.1 0; 127.0.0.1 0;

10
dev-tools/tmp/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2707 ### Version: V4.2021.11.2708
### Updated: Thu Nov 11 10:03:29 UTC 2021 ### Updated: Thu Nov 11 11:15:41 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 626
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -1127,6 +1127,8 @@ User-agent: ZumBot
Disallow:/ Disallow:/
User-agent: ZyBorg User-agent: ZyBorg
Disallow:/ Disallow:/
User-agent: adscanner
Disallow:/
User-agent: archive.org_bot User-agent: archive.org_bot
Disallow:/ Disallow:/
User-agent: arquivo-web-crawler User-agent: arquivo-web-crawler
@ -1217,6 +1219,8 @@ User-agent: scan.lol
Disallow:/ Disallow:/
User-agent: seobility User-agent: seobility
Disallow:/ Disallow:/
User-agent: seocompany.store
Disallow:/
User-agent: seoscanners User-agent: seoscanners
Disallow:/ Disallow:/
User-agent: seostar User-agent: seostar

10
robots.txt/robots.txt
View file

@ -6,9 +6,9 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2707 ### Version: V4.2021.11.2708
### Updated: Thu Nov 11 10:03:29 UTC 2021 ### Updated: Thu Nov 11 11:15:41 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 626
################################################### ###################################################
### Version Information ## ### Version Information ##
@ -1127,6 +1127,8 @@ User-agent: ZumBot
Disallow:/ Disallow:/
User-agent: ZyBorg User-agent: ZyBorg
Disallow:/ Disallow:/
User-agent: adscanner
Disallow:/
User-agent: archive.org_bot User-agent: archive.org_bot
Disallow:/ Disallow:/
User-agent: arquivo-web-crawler User-agent: arquivo-web-crawler
@ -1217,6 +1219,8 @@ User-agent: scan.lol
Disallow:/ Disallow:/
User-agent: seobility User-agent: seobility
Disallow:/ Disallow:/
User-agent: seocompany.store
Disallow:/
User-agent: seoscanners User-agent: seoscanners
Disallow:/ Disallow:/
User-agent: seostar User-agent: seostar