vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.03.3001 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-03-25 12:50:15 +00:00
parent 0a91422d10
commit fcba961ef9
51 changed files with 81948 additions and 81948 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.03.3000 #### Version: V4.2022.03.3001
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.03.3000 #### Version: V4.2022.03.3001
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.03.3000 #### Version: V4.2022.03.3001
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10106
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.03.3000 #### Version: V4.2022.03.3001
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10110
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10110
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10110
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

6
dev-tools/beta_conf_files/bots.d/blacklist-user-agents.conf
View file

@ -5,8 +5,8 @@
# VERSION INFORMATION # # VERSION INFORMATION #
#---------------------- #----------------------
# Version: V4.2019.08 # Version: V4.2022.03
# Updated: 2019-07-05 # Updated: 2022-03-25
#---------------------- #----------------------
# VERSION INFORMATION # # VERSION INFORMATION #
@ -74,6 +74,6 @@
# ------------ # ------------
"~*(?:\b)x22(?:\b)" 3; "~*(?:\b)x22(?:\b)" 3;
"~*(?:\b){|}(?:\b)" 3; #"~*(?:\b)|{|}(?:\b)" 3;
"~*(?:\b)mb_ereg_replace(?:\b)" 3; "~*(?:\b)mb_ereg_replace(?:\b)" 3;
"~*(?:\b)file_put_contents(?:\b)" 3; "~*(?:\b)file_put_contents(?:\b)" 3;

10110
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 1; 10.1.0.232 1;
127.0.0.1 1; 127.0.0.1 1;

6
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-user-agents.conf
View file

@ -5,8 +5,8 @@
# VERSION INFORMATION # # VERSION INFORMATION #
#---------------------- #----------------------
# Version: V4.2019.08 # Version: V4.2022.03
# Updated: 2019-07-05 # Updated: 2022-03-25
#---------------------- #----------------------
# VERSION INFORMATION # # VERSION INFORMATION #
@ -74,6 +74,6 @@
# ------------ # ------------
"~*(?:\b)x22(?:\b)" 3; "~*(?:\b)x22(?:\b)" 3;
"~*(?:\b){|}(?:\b)" 3; #"~*(?:\b)|{|}(?:\b)" 3;
"~*(?:\b)mb_ereg_replace(?:\b)" 3; "~*(?:\b)mb_ereg_replace(?:\b)" 3;
"~*(?:\b)file_put_contents(?:\b)" 3; "~*(?:\b)file_put_contents(?:\b)" 3;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 0; 10.1.0.232 0;
127.0.0.1 0; 127.0.0.1 0;

10110
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 1; 10.1.0.232 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 0; 10.1.0.232 0;
127.0.0.1 0; 127.0.0.1 0;

10110
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 1; 10.1.0.232 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 0; 10.1.0.232 0;
127.0.0.1 0; 127.0.0.1 0;

10110
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3000 3001

10110
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10110
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

6
dev-tools/test1_conf_files/bots.d/blacklist-user-agents.conf
View file

@ -5,8 +5,8 @@
# VERSION INFORMATION # # VERSION INFORMATION #
#---------------------- #----------------------
# Version: V4.2019.08 # Version: V4.2022.03
# Updated: 2019-07-05 # Updated: 2022-03-25
#---------------------- #----------------------
# VERSION INFORMATION # # VERSION INFORMATION #
@ -74,6 +74,6 @@
# ------------ # ------------
"~*(?:\b)x22(?:\b)" 3; "~*(?:\b)x22(?:\b)" 3;
"~*(?:\b){|}(?:\b)" 3; #"~*(?:\b)|{|}(?:\b)" 3;
"~*(?:\b)mb_ereg_replace(?:\b)" 3; "~*(?:\b)mb_ereg_replace(?:\b)" 3;
"~*(?:\b)file_put_contents(?:\b)" 3; "~*(?:\b)file_put_contents(?:\b)" 3;

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.232.107.4 0; 20.232.42.115 0;

10110
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 1; 10.1.0.232 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 0; 10.1.0.232 0;
127.0.0.1 0; 127.0.0.1 0;

10110
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 1; 10.1.0.232 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 0; 10.1.0.232 0;
127.0.0.1 0; 127.0.0.1 0;

10110
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 1; 10.1.0.232 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 0; 10.1.0.232 0;
127.0.0.1 0; 127.0.0.1 0;

10110
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

6
dev-tools/test2_conf_files/bots.d/blacklist-user-agents.conf
View file

@ -5,8 +5,8 @@
# VERSION INFORMATION # # VERSION INFORMATION #
#---------------------- #----------------------
# Version: V4.2019.08 # Version: V4.2022.03
# Updated: 2019-07-05 # Updated: 2022-03-25
#---------------------- #----------------------
# VERSION INFORMATION # # VERSION INFORMATION #
@ -74,6 +74,6 @@
# ------------ # ------------
"~*(?:\b)x22(?:\b)" 3; "~*(?:\b)x22(?:\b)" 3;
"~*(?:\b){|}(?:\b)" 3; #"~*(?:\b)|{|}(?:\b)" 3;
"~*(?:\b)mb_ereg_replace(?:\b)" 3; "~*(?:\b)mb_ereg_replace(?:\b)" 3;
"~*(?:\b)file_put_contents(?:\b)" 3; "~*(?:\b)file_put_contents(?:\b)" 3;

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.232.107.4 0; 20.232.42.115 0;

10110
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

6
dev-tools/test3_conf_files/bots.d/blacklist-user-agents.conf
View file

@ -5,8 +5,8 @@
# VERSION INFORMATION # # VERSION INFORMATION #
#---------------------- #----------------------
# Version: V4.2019.08 # Version: V4.2022.03
# Updated: 2019-07-05 # Updated: 2022-03-25
#---------------------- #----------------------
# VERSION INFORMATION # # VERSION INFORMATION #
@ -74,6 +74,6 @@
# ------------ # ------------
"~*(?:\b)x22(?:\b)" 3; "~*(?:\b)x22(?:\b)" 3;
"~*(?:\b){|}(?:\b)" 3; #"~*(?:\b)|{|}(?:\b)" 3;
"~*(?:\b)mb_ereg_replace(?:\b)" 3; "~*(?:\b)mb_ereg_replace(?:\b)" 3;
"~*(?:\b)file_put_contents(?:\b)" 3; "~*(?:\b)file_put_contents(?:\b)" 3;

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.232.107.4 0; 20.232.42.115 0;

10110
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 1; 10.1.0.232 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BetaBot Badass
Chlooe Fuzz
FemtosearchBot FyberSpider
Gotit RocketCrawler
Haansoft Rogerbot
Mail.RU_Bot UnisterBot
Sottopop Web Pix
TheNomad WebSucker
archive.org_bot Whack
lwp-request polaris version

316
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider AIBOT
404enemy ASPSeek
80legs Acunetix
ADmantX Apexoo
Aboundexbot Asterias
AiHitBot
Aipbot
AllSubmitter
Anarchy
Anarchy99
Ankit
Atomseobot
Attach
AwarioRssBot
BBBike
BDCbot BDCbot
BackDoorBot BLEXBot
BackWeb Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Barkrowler Badass
BetaBot Bandit
Black Hole Battleztar Bazinga
Blackboard Bigfoot
BlowFish Bitacle
Bolt
Brandprotect
Buck Buck
BuiltBotTough BuiltWith
CATExplorador CATExplorador
Chlooe CODE87
Cocolyzebot CazoodleBot
Cegbfeieh
CensysInspect
ChinaClaw
Citoid
Cliqzbot
Cloud mapping
Copier Copier
Copyscape
Cosmos Cosmos
Crescent Craftbot
CrunchBot CrazyWebCrawler
Custo Custo
CyotekWebCopy CyotekWebCopy
DIIbot DBLBot
DSearch DSearch
DataCha0s DTS Agent
Demon
Devil
Digincore
Dirbuster
Disco
Discobot Discobot
DittoSpyder
DnBCrawler-Analytics
DnyzBot DnyzBot
DomainAppender DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
Domains Project Drip
Download Wonder Ecxi
EMail Wolf
Ebingbong
EirGrabber EirGrabber
EroCrawler
Express WebPictures
ExtLinksBot ExtLinksBot
Extractor
Extreme Picture Finder Extreme Picture Finder
FDM EyeNetIE
FHscan FHscan
Firefox/7.0 Fimap
Flunky Flunky
Freeuploader Foobot
FrontPage FrontPage
Fuzz
FyberSpider
GT::WWW GT::WWW
Genieo
GetRight GetRight
GrapeshotCrawler Getintent
Gigabot
Gotit
Grafula
GrapeFX
GridBot GridBot
HEADMasterSEO HTMLparser
HMView HTTrack
HaosouSpider
Harvest Harvest
Havij Havij
Heritrix Hloader
HonoluluBot HonoluluBot
HybridBot Humanlinks
IDBot IDBot
IRLbot
Iblog Iblog
Id-search Intelliseek
IlseBot Iria
Image Fetch
IndeedBot
InfoTekies
Iskanie Iskanie
JOC Web Spider
JamesBOT JamesBOT
JikeSpider
JustView JustView
Jyxobot
Kozmosbot Kozmosbot
LWP::Simple LWP::Simple
LeechFTP Lanshanbot
LibWeb LeechGet
Linkbot Lftp
LinqiaMetadataDownloaderBot Linkdexbot
Lipperhey LinkextractorPro
Litemage_walker LinksManager
Ltx71 LinqiaRSSBot
MFC_Tear_Sample LinqiaScrapeBot
Lmspider
MIDown tool MIDown tool
Majestic-SEO MJ12bot
MarkMonitor MQQBrowser
Mag-Net
Majestic SEO
Majestic12
MarkWatch MarkWatch
Mass Downloader
Masscan
MauiBot MauiBot
MeanPath Bot
Meanpathbot Meanpathbot
Metauri
MicroMessenger MicroMessenger
Microsoft Data Access
Mister PiX Mister PiX
Mojeek Mr.4x3
Mojolicious
MolokaiBot
Mozlila
NICErsPRO NICErsPRO
NPbot
Name Intelligence
Nameprotect
NearSite NearSite
Needle Needle
Net Vampire NetLyzer
NetAnts
NetMechanic NetMechanic
NetSpider NetSpider
NetZIP NetZIP
Nettrack Nettrack
NextGenSearchBot Nibbler
Niki-bot Nmap
Not Not
Offline Navigator Nuclei
Octopus
OpenLinkProfiler
OpenVAS
Openfind Openfind
Page Analyzer Openvas
Pavuk OrangeBot
OutfoxBot
PECL::HTTP
PageAnalyzer
PageScorer
PageThing.com
Panscient
Papa Foto
Petalbot Petalbot
Picscout
Picsearch Picsearch
Piepmatz PictureFinder
Pimonster Pimonster
Pixray Probethenet
Psbot Psbot
PxBroker Pu_iN
PyCurl Quick-Crawler
RSSingBot RankingBot
RankActiveLinkBot
RankFlex
RankingBot2
Rankivabot Rankivabot
RankurBot Re-re
RedesScrapy ReGet
RocketCrawler RealDownload
Recorder
Rogerbot Rogerbot
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SMTBot
ScanAlert
Scanbot
Screaming
ScreenerBot ScreenerBot
ScrepyBot ScrepyBot
SearchmetricsBot Searchestate
Seekport
SemanticJuice SemanticJuice
SemrushBot SeoSiteCheckup
SentiBot SeobilityBot
Siphon Seomoz
SiteExplorer Site Sucker
SlySearch SiteCheckerBotCrawler
SmartDownload SiteSnagger
Snake Siteimprove
Sociscraper Sociscraper
Spammen Sosospider
Sottopop
SpaceBison
SpankBot SpankBot
Spanner Stripper
Sqlmap Sucker
Sqlworm
Sucuri Sucuri
Swiftbot Szukacz
Teleport T8Abot
TeleportPro
Telesoft Telesoft
Telesphoreo
Telesphorep Telesphorep
The Intraformant
TightTwatBot TightTwatBot
Toata Trendictionbot
Trendiction
True_Robot True_Robot
Turnitin Turnitin
TwengaBot TwengaBot
URLy.Warning
Upflow Upflow
Vagabondo V-BOT
VelenPublicWebCrawler VidibleScraper
VeriCiteCrawler Voil
Virusdie WBSearchBot
Voltron WEBDAV
WASALive-Bot
WPScan WPScan
WWW-Mechanize WWW-Mechanize
Wallpapers WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WeSEE WeSEE
Web Auto
Web Collage Web Collage
Web Fetch Web Sucker
WebAuto WebAuto
WebBandit
WebCollage WebCollage
WebCopier WebFuck
WebEnhancer WebGo IS
WebFetch WebPix
WebImageCollector WebReaper
WebSauger WebStripper
WebZIP WebSucker
WebmasterWorldForumBot Webalta
WebsiteExtractor WebsiteExtractor
Whatweb WebsiteQuester
Widow Whacker
Wonderbot Who.is Bot
WinHTTrack
WiseGuys Robot
Wotbox
Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
YoudaoBot
Zade
Zauba Zauba
Zermelo
Zeus Zeus
Zitebot
ZmEu
ZoomBot ZoomBot
arquivo-web-crawler ZoominfoBot
ZyBorg
archive.org_bot
arquivo.pt arquivo.pt
autoemailspider autoemailspider
backlink-check cah.io.community
check1.exe
cognitiveseo
com.plumanalytics com.plumanalytics
eCatch crawler.feedback
evc-batch dataforseo.com
gopher gopher
instabid instabid
internetVista monitor internetVista monitor
lwp-trivial magpie-crawler
netEstate NE Crawler muhstik-scan
oBot plumanalytics
polaris version ripz
probe-image-size s1z.ru
satoristudio.net
seocompany.store seocompany.store
seoscanners seoscanners
seostar seostar
serpstatbot
sexsearcher sexsearcher
siteripz sitechecker.pro
sp_auditbot
tAkeOut tAkeOut
ubermetrics-technologies.com trendiction.com
webmeup-crawler trendiction.de
webpros.com
webprosbot
x09Mozilla x09Mozilla
x22Mozilla x22Mozilla
xpymep1.exe
zauba.io
zgrab

310
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404checker 404checker
404enemy 80legs
AIBOT AIBOT
Abonti
Aboundex Aboundex
Aboundexbot Aboundexbot
AiHitBot AfD-Verbotsverfahren
Anarchy Alexibot
Aspiegel AllSubmitter
Alligator
Anarchy99
Anthill
Atomseobot Atomseobot
AwarioSmartBot
BDCbot
BDFetch BDFetch
BackDoorBot BackStreet
BackWeb
BacklinkCrawler BacklinkCrawler
Badass Badass
Barkrowler Bandit
Battleztar Bazinga BatchFTP
Bigfoot BetaBot
Blackboard Blackboard
Blow BlowFish
Bolt Bolt
BotALot
Brandprotect
Brandwatch
Buddy Buddy
BuiltWith CATExplorador
Bullseye
BunnySlippers
CCBot
CODE87
CSHttp
CazoodleBot CazoodleBot
Cegbfeieh CheeseBot
CherryPicker CherryPicker
ChinaClaw Chlooe
Cliqzbot Claritybot
Cloud mapping Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot Collector
Curious Copier
CyotekWebCopy Copyscape
DBLBot Crescent
DSearch DIIbot
DTS Agent DTS Agent
DatabaseDriverMysqli DataCha0s
Demon Deusu
Devil Devil
Digincore DigitalPebble
Disco Discobot
Discoverybot DittoSpyder
Dispatch DnBCrawler-Analytics
DomCopBot DomCopBot
Domains Project DomainCrawler
Dotbot DomainStatsBot
Dragonfly Dragonfly
EMail Siphon ECCP/1.0
EMail Wolf EMail Wolf
EroCrawler
Exabot Exabot
Express WebPictures
Extreme Picture Finder
Ezooms
FemtosearchBot FemtosearchBot
Flunky Flunky
Fuzz
G-i-g-a-b-o-t
GT::WWW
GalaxyBot
Genieo
GermCrawler GermCrawler
Go!Zilla Go!Zilla
Go-Ahead-Got-It GrapeshotCrawler
Grafula GridBot
GrapeFX HMView
HEADMasterSEO HTTP::Lite
HaosouSpider HaosouSpider
Harvest Harvest
Heritrix Heritrix
HonoluluBot HybridBot
IDBTE4M
IDBot IDBot
IRLbot IRLbot
IlseBot IlseBot
Image Fetch InfoTekies
Image Sucker
InfoNaviRobot
Intelliseek Intelliseek
Iria Iria
JOC Web Spider Iskanie
Jetty JamesBOT
Jyxobot JikeSpider
Keybot Translation-Search-Machine
Keyword Density
Kozmosbot
LWP::Simple LWP::Simple
Larbin Lanshanbot
Leap
LexiBot LexiBot
Lftp
LibWeb LibWeb
Lightspeedsystems Lightspeedsystems
Likse Likse
LinkWalker LinqiaRSSBot
LinkpadBot LinqiaScrapeBot
Litemage_walker
Lmspider Lmspider
Ltx71 MFC_Tear_Sample
MJ12bot MIIxpc
MSFrontPage
MSIECrawler MSIECrawler
Mag-Net MTRobot
Magnet Majestic12
Mail.RU_Bot MarkMonitor
Majestic SEO
MarkWatch MarkWatch
MauiBot Mass Downloader
Masscan
Mediatoolkitbot Mediatoolkitbot
Metauri MegaIndex.ru
Microsoft Data Access MicroMessenger
Minefield Minefield
Mister PiX Mister PiX
Mojeek Moblie Safari
Mozlila
Musobot
NPbot NPbot
Name Intelligence Nameprotect
NearSite Navroad
Needle Nessus
Net Vampire Net Vampire
NetSpider Nettrack
NetZIP NextGenSearchBot
Netcraft
Netvibes
Niki-bot
Nikto Nikto
Nutch
Octopus Octopus
Offline Navigator Openfind
OpenLinkProfiler OrangeBot
OutclicksBot
Page Analyzer Page Analyzer
PageGrabber PageGrabber
PeoplePal PageScorer
Pandalytics
Petalbot Petalbot
Piepmatz Pi-Monster
Picscout
Picsearch
PictureFinder
Pimonster
Pixray Pixray
PleaseCrawl PleaseCrawl
Pockey ProPowerBot
Probethenet Probethenet
Psbot
Pu_iN Pu_iN
Pump
PxBroker
QueryN Metasearch QueryN Metasearch
RankActive Quick-Crawler
RankActiveLinkBot
RankFlex RankFlex
RankingBot RankingBot
RankingBot2 RankingBot2
Re-re RankurBot
ReGet
Reaper Reaper
RedesScrapy RebelMouse
SEOlyticsCrawler Recorder
SISTRIX SEOkicks-Robot
SEOstats
SMTBot SMTBot
SalesIntelligent SalesIntelligent
ScanAlert ScanAlert
Scanbot
ScoutJet ScoutJet
Semrush Screaming
SemrushBot
SentiBot SentiBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot
Seomoz
Shodan
Siphon
Site Sucker
SiteSnagger SiteSnagger
SiteSucker
Sitevigil
SlySearch SlySearch
SmartDownload SmartDownload
Snapbot Snake
Snoopy Sogou web spider
Sociscraper
Sosospider Sosospider
Sottopop Sqlmap
SpaceBison Steeler
Spbot Sucker
Sqworm
Stripper
Sucuri Sucuri
SuperHTTP Surfbot
Suzuran SurveyBot
T8Abot T8Abot
Telesoft TeleportPro
Telesphorep The Intraformant
Thumbor TheNomad
Titan TightTwatBot
Toata
Toweyabot Toweyabot
Tracemyfile
Trendiction Trendiction
Turingos Turnitin
TurnitinBot
TwengaBot TwengaBot
Typhoeus Twice
URLy.Warning URLy Warning
VB Project Upflow
VCI
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VidibleScraper VeriCiteCrawler
Virusdie
Voltron Voltron
WASALive-Bot
WBSearchBot WBSearchBot
WPScan WEBDAV
WWW-Collector-E WISENutbot
Wallpapers/3.0 WWW::Mechanize
WallpapersHD WallpapersHD
Web Collage Web Collage
Web Enhancer Web Fuck
Web Fetch Web Pix
Web Sauger Web Sauger
WebAuto WebCopier
WebCollage WebEnhancer
WebGo IS
WebPix
WebReaper
WebSauger WebSauger
WebStripper
WebSucker
WebWhacker
WebZIP
Webalta
Website Quester
WebsiteQuester WebsiteQuester
WinHTTrack Webster
WiseGuys Robot Whacker
Wotbox Wonderbot
Wprecon Xaldon WebSpider
Xaldon_WebSpider
YoudaoBot
Zade Zade
Zauba
Zermelo Zermelo
Zeus ZoomBot
Zitebot
ZmEu
ZyBorg ZyBorg
archive.org_bot archive.org_bot
arquivo-web-crawler
arquivo.pt
autoemailspider autoemailspider
clark-crawler check1.exe
crawl.sogou.com coccocbot
dataforseo.com com.plumanalytics
domainsproject.org crawler.feedback
demandbase-bot
eCatch eCatch
heritrix gopher
instabid internetVista monitor
ips-agent ips-agent
magpie-crawler iubenda-radar
meanpathbot lwp-trivial
muhstik-scan muhstik-scan
netEstate NE Crawler
oBot
pcBrowser
plumanalytics
polaris version polaris version
probe-image-size ripz
serpstatbot scalaj-http
seocompany.store
seoscanners
sexsearcher sexsearcher
sitechecker.pro
sp_auditbot
sysscan
trendiction.com trendiction.com
webgains-bot webpros.com
webprosbot webprosbot
x22Mozilla x09Mozilla
xpymep1.exe
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
aac52006ec82a24e08b665f4db2b5013f7.com cartierlovebraceletreplica.xyz
adprotect.net clickintext.net
ads.gold d2jsp.org
bouda.kvalitne.cz happy.new.yeartwit.com
claytransformations.info listiseltemournlan.gq
franch.info park.above.com
ltvperf.com powc.r.ca.d.sendibm2.com
oz-offers.com sexuria.net
postclass.com usadacha.net
xboxster.ru wildcattube.com

912
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0c47f8422d3f.com 1008.su
100dollars-seo.com 123kuma.com
123cha.com 12bet.com
273-fz.ru 1bet.com
45en.ru 2daytrendingnews.com
7search.com 2nt.ru
7zap.com 34.gs
8kisses.eu 3rbseyes.com
aa625d84f1587749c1ab011d6f269f7d64.com 4ur.click
adidas.frwebs.fr 8lufu.com
adimmix.com 999webdesign.com
admanaerofoil.com 9icmzvn6.website
ads-cool.pro abclauncher.com
adserver-e7.com adanih.com
afftrack001.com adcash.com
afora.ru adclickservice.com
alienwheels.de adloads.com
alldownload.pw admatic.com.tr
allmarketsnewdayli.gdn advertisingtag.net
allornamenti.com agadelha.com.br
alta-realestate.com agapovdg.ru
ameblo.top aibolita.com
amung.us alldubai.biz
aosexkontakte.net allfinweb.com
allkrim.com
alphaforum.ru
amospalla.es
analytics-ads.xyz
animalia-life.club
anime.dougasouko.com
animebox.com.ua
apartmentbay.ru
apibatbrowsecom-a.akamaihd.net
apilinkswiftco-a.akamaihd.net
apimossnetinfo-a.akamaihd.net
apishops.ru apishops.ru
arabgirls.adultgalls.com appfastplay.com
arabsexxxtube.com apptester.tk
arate.ru artavenuegardenstudios.com
arkkivoltti.net as5000.com
arraty.altervista.org atlant-auto.info
ausmepa.org.au audiofree.ru
australia-opening-times.com auto-moto-elektronika.cz
autobrennero.it auto-zapchasti.org
autochoixspinelli.com auto.rusvile.lt
aviva-limoux.com
avon-novosib.ru avon-novosib.ru
ayodhya.co avon-severozapad.ru
babs.com.ua avtolombard-krasnodar.ru
backiee.com azartmix.com
bang-hotties.com azte.ch
basisches-wasser.net babyguns.ru
benchmarkcommunications.co.uk backuperwebcam.weebly.com
baersaratov.ru
bagsonsale.online
balook.com
best-seo-offer.com best-seo-offer.com
bestchoice.cf bestplacetobuyeriacta.jw.lt
bget.ru betune.onlinewebshop.net
bidr.trellian.com beyan.host.sk
bitcoins-live.ru bezsporno.ru
beztuberkuleza.ru
bigcareer.info
bingo8888.com
bitporno.sx
bizfly.info
bizlist.com.de
biznesluxe.ru biznesluxe.ru
blackhatworth.com bloggers.nl
blogs.rediff.com
bmw-ark.ru bmw-ark.ru
bmwhighperformers.com bombla.org
bolezniorganov.ru braip.com.br
bonvillan.com brandehk.dk
bookmark4you.com browserprotecter.com
bradleylive.xyz buypuppies.ca
brendbutik.ru buyscabiescream.com
briomotor.co c-english.ru
bronzeaid-a.akamaihd.net calc-for-credit.ru
bugshoot.cn callawaygolfstore.online
bum.com.ru
buypillsorderonline.com
buzzurl.jp
byme.se
c-gainsbourg.com
calcularpagerank.com
camdolls.com camdolls.com
candcstuccoandstone.com canrioloadm.gq
capsderfudd.tk cartierloveringreplica.xyz
cardsdumps.com cartierreplica.pw
carrauterie.be cartiertrinity.xyz
cartierlovebracelet.xyz chatroulette.online
cfcl.co.uk chocolateslim-en-france.com
cheap-trusted-backlinks.com
cheapmoncler.win
cheapmoncler.xyz
cheapwebsitehoster.com
cherubinimobili.it
children-learningreading.info
chinacheapelitejerseys.com
cibpenonptib.flu.cc cibpenonptib.flu.cc
cienum.fr compliance-brian.xyz
citetick.com compliance-jack.top
cityadspix.com compliance-jessica.top
cjmc.info compliance-julianna.top
classicakuhni.ru compliance-margo.top
cleansearch.net contentlook.co
clipartnew.com cosmetic.donna7753191.ru
cobaltpro.ru
coin-hive.com
cometorussia.net
compliance-donald.xyz
contentsexpress.com
cookielawblog.wordpress.com
coolbar.pro
coolyarddecorations.com
cougfan.info
counterbot.com counterbot.com
cracksplanet.com cpajunkies.com
crafthubs.com cranly.net
crystalslot.com creams.makeforum.eu
culad.com
cupday.com
customboxes4less.com
customsua.com.ua customsua.com.ua
daptravel.com dafatiri.com
dbmkfhqk.bloger.index.hr damasarenai.info
ddpills.com datadepths.com
de.zapmeta.com dating-app.ru
dcj-nn.ru
ddlmega.net
deanmoore.ie
decoratinghomes.ga decoratinghomes.ga
design-lands.ru deda-moroza-zakaz.ru
detskie-konstruktory.ru dekorkeramik.ru
deutschland123.de delfin-aqua.com.ua
diesel-parts28.ru deniven.1bb.ru
digitalfaq.com detalizaciya-tut.biz
dipstar.org detoxmed24.ru
disruptingdinnerparties.com dev78.dioniqlabb.se
dividendo.ru dichvuvesinhhanoi.com
djonwatch.ru directivepub.com
dogbreedspicture.net diusyawiga.tk
dogoimage.com dnepr-avtospar.com.ua
dojki365.online doktoronline.no
domaincdn.xyz dvd-famille.com
dominateforex.ml dvrlists.com
domination.ml e-collantes.com
doska-vsem.ru
downloader12.ru
doxyporno.com
drugspowerstore.com
dtnlyss.com
dumpsmania24.com
e-biznes.info
ekspertmed.com ekspertmed.com
electricwheelchairsarea.com elektir.ru
elektronischezi.livejournal.com eloxal.ru
elektrozigarette1.dreamwidth.org
elektrozigarettekaufen1.devhub.com
elitedollars.com
elvel.com.ua elvel.com.ua
errorfixing.space eropho.com
essay-writing.work erotiktreff24.info
este-line.com.ua esopini.com
evening-dating-club.info excitacao.com
exchanges-bet.com expresstoplivo.ru
factorynightclub.com extremez.net
famix.xyz ezigarettenshop2.postbit.com
faracontrol.ir f-loaded.de
favoritemoney.ru f0815.de
feargames.ru falllow.gq
fetishinside.com fandlr.com
fetlifeblog.com fanpagerobot.com
filerockstar298.weebly.com farmprofi.net
filesclub.net fealq.com
filkhbr.com fetroshok.ru
fillmewithhappiness.com filmidivx.com
filmci.pro financehint.eu
filmetricsasia.com firesub.pl
fishtauto.ru fishtauto.ru
fr-bearings.ru flipper.top
frameimage.org foodgid.net
franecki.net forensicpsychiatry.ru
free-traffic.xyz fortwosmartcar.pw
free-video-tool.com forum.tvmir.org
freshwallpapers.info forumprofi.de
frustrated-favorable.gq free-social-buttons.com
fullfileaccess.com freelotto.com
fun2cell.net freshmac.space
fxtips.ru frvo.alptandem.ru
fxund.us fym.com.ru
fyl.com.ru
g.starmoe.xyz
galeria-zdjec.com galeria-zdjec.com
gamevalue7.weebly.com gamblingnerd.com
gdebestkupit.ru games.kolossale.ru
generalporn.org gate5.co.za
germetiki.com.ua ge0ip.com
gk170.ru
glastecfilms.com.my
glavtral.ru glavtral.ru
gojiberriess.apishops.ru gomusix.com
gombita.info goodhumor24.com
gpirate.com gosmeb.ru
gratis-sexkontakte.com grupografico-pilar.com.ar
gsasearchengineranker.pw gstatey.net
gtopstats.com guardlink.com
guidefs.ru
hao123.com
happy.new.yeartwit.com happy.new.yeartwit.com
healgastro.com hatdc.org
healthcarestore.info hcate.com
hentaiheroes.com hdpixion.com
hikesearch.net hermesbracelets.xyz
hostingclub.lk hermesreplica.win
hotxnights.info heygidday.biz
howtoclean.club hidefiles.org
https-legalrc.biz hopeonthestreet.co.uk
humbmosquina.tk hoporno.com
i4track.net husky-shop.cz
ikearugs.xyz idegenvezeto.eu
imagefinder.site igrovyeavtomaty777.ru
images-graphics-pics.com ihtec2019.org
iminent.com imadedinner.net
incep.imagine-ex.co imk.com.ua
indetiske.ya.ru imperiafilm.ru
instabid.tech importchinacoach-teach.com
intelhdgraphicsgtdrive6w.metroblog.com incomekey.net
intervsem.ru insider.pro
investmac.com
inzn.ru inzn.ru
iopeninghours.co.uk iqoption-bin.com
ipostroika.ru
isitpaleo.info
islamtoday.co.za
ispac.org
itag.pw itag.pw
ivearchenceinflu.cf iwantedmoney.com
jackwolfskinoutlet.online javlibrary.cc
jeremyeaton.co jerseysbizwholesalecheap.com
jmat.cn juliaworld.net
justbcause.com
kadashihotel.com
kaidalibor.de kaidalibor.de
katadhin.co kartiniresto.com
kemerovo.zrus.org katushka.net
kerch.site khadastoafarde.tk
keyword-suggestions.com kinofak.net
kfon.eu
kinky-fetishes.com
kino-key.info
kinobaks.com
kinoflux.net
kinohit1.ru
kinosed.net
kinotorka.ru
kmd-pto.ru kmd-pto.ru
knowsitall.info kollekcioner.ru
konica.kz kredit-blog.ru
konpax.com kvartiry-remont.ucoz.ru
kovrenok.ru lacasamorett.com
krafte.ru
lafriore.ru lafriore.ru
lanasshop.ru
landmania.ru
larutti.ru
lashstudia.ru lashstudia.ru
law-check-nine.xyz le-clos-des-alouettes.com
lazy-z.com lechenie-gemorroya.com
lescinq.com lerporn.info
letmacwork.world letslowbefast.today
lietaer.com lietaer.com
limtu.ifmo.ru life-instyle.com
littlesexdolls.com likrot.com
llastbuy.ru linkbolic.com
longadventure.com linkpulse.com
louboutinreplica.xyz lion.playtap.us
lubetube.com livingroomdecor.info
macdamaged.space ljusihus.se
mackeeper-center.club lombia.co
mafa-free.com long-beach-air-conditioning.com
mafcards.ru lordzfilmz.me
magazintiande.ru lostaruban.com
magicdiet.gq mabdoola.blogspot.com
mantramusic.ru magnetpress.sk
map028.com makemoneyonline.com
matpre.top marblestyle.ru
maximilitary.ru mb140.ru
maxthon.com medicine-4u.org
meds-online24.com merryhouse.co.uk
medtherapy.ru mikrobiologies.ru
members.ghanaweb.com mindeyegames.com
michaelkorsoutlet.store missis.top
mikozstop.com mmgq.ru
mir-business-24.ru mmog-play.ru
mirtorrent.net moivestiy.biz
mjchamonix.org mojowhois.com
mmostrike.ru money-every-day.com
mobicover.com.ua money-for-placing-articles.com
mobioffertrck.com moscow.online-podarki.com
mobstarr.com moscow.xrus.org
mompussy.net mototsikl.org
monclerjacketsoutlet.win movies-in-theaters.net
moomi-daeri.com moviezbonkerssk.cf
morpicert.pw moyakuhnia.ru
mrbojikobi4.biz mrlmedia.net
msfsaar.de mrsdalloways.com
mybinaryoptionsrobot.com mttwtrack.com
mycouponizemac.com multgo.ru
myfreetutorials.com music.utrolive.ru
mymercy.info musicvidz.ru
muz-tracker.net
my-aladin.com
mydownlodablefiles.com
mygameplus.com
myra.top
myshopmatemac.com
mystats.xyz
na-telefon.biz
nalogovyy-kodeks.ru nalogovyy-kodeks.ru
nardulan.com naturalpharm.com.ua
net-profits.xyz negral.pluto.ro
newfilmsonline.ru networkad.net
notfastfood.ru nfljerseyscheapbiz.us
novatech.vn nl.netlog.com
nsatc.net nofreezingmac.click
o-dachnik.ru norththeface.store
oddamzadarmo.eu o-o-6-o-o.com
okeinfo.online ok.ru
okmusic.jp omgtnc.com
okroshki.ru onclickpredictiv.com
olgacvetmet.com onclkads.com
online-hit.info onescreen.cc
online-podarki.com onlineku.com
onlinewritingjobs17.blogspot.ru
onload.pw
openfrost.com openfrost.com
osoznanie-narkotikam.net origin-my.ru
ostrovtaxi.ru orion-code-access.net
ownshop.cf oxford-book.com.ua
oxotl.com
oyster-green.com
paccohichetoti.ml paccohichetoti.ml
pagesense.com paceform.com
panchro.xyz pammik.ru
paradontozanet.ru
partner-host.men partner-host.men
partner-pop.men patterntrader-en.com
paydayloanslocal.com pawli.eu
pcboa.se pc-services.ru
pdn-4.com pearlisland.ru
pdns.download pelfind.me
pekori.to pflexads.com
perm-profnastil.ru phelissota.xyz
petsblogroll.com picsearch.com
phpdevops.com pillscheap24h.com
picurams.pw pivka.xyz
pio.polytopesexempt.com placid-rounded-coast.glitch.me
pizda.lol pliks.pl
pkr1hand.com poligon.com
pl-vouchers.com polybuild.ru
pomoc-drogowa.cba.pl pons-presse.com
popads.net pontiacsolstice.info
popander.mobi
popcash.net
popmarker.com
poptool.net
popugauka.ru popugauka.ru
popugaychiki.com pornhub-forum.ga
porngalleries.top porno.simple-image.com.ua
porno-raskazy.ru powitania.pl
pornolook.net preconnubial.usuby.site
pornophoto.xyz pretty-mart.com
pornosemki.info privatov-zapisi.ru
pornosok.ru profitfx.online
pozdravleniya-c.ru prosmibank.ru
preparevideosafesystem4unow.site prostitutki-almata.org
princeadvantagesales.com proxyelite.biz
privacylocationforloc.com pukaporn.com
prizefestival.mobi pushdata.sendpulse.com
pron.pro pussysaga.com
pronorm.fr quality-traffic.com
prostitutki-tolyatti.org quickloanbank.com
pst2017.onlinewebshop.net raavidesigns.com
puserving.com rainbowice.ru
py100.ru
queerspace.com
quelle.ru
qwarckoine.com
rada.ru
radiodigital.co
ragecash.com
ranksays.com
ranksignals.com
rapidgator-porn.ga rapidgator-porn.ga
raschtextil.com.ua reining.lovasszovetseg.hu
rating-bestcasino.com relax.ru
refudiatethissarah.info remont-mobile-phones.ru
rfid-locker.co remontbiz.ru
russkoe-zdorovie.ru renhacklids.tk
rent2spb.ru
repeatlogo.co.uk
richinvestmonitor.com
rosbalt.com.ua
rospromtest.ru
rossanasaavedra.net
royal-investments.net
rucrypt.com
runofilms.ru
russia-tao.ru
rvi.biz
rvtv.ru rvtv.ru
sajatvelemeny.com rvzr-a.akamaihd.net
salut-camp.ru rybalka-opt.ru
sampleletters.net sale-japan.com
sanatorrii.ru samara.rosfirm.ru
sanidumps.com sandhillsonline.com
sarahmilne.top sankt-peterburg.nodup.ru
sbwealthsolutions.ca sawin.beth.webd.pl
scanner-margo.top scanner-brian.top
scrapinghub.com scat.porn
scrnet.biz.ua scopich.com
sdelatmebel.ru
search.1and1.com search.1and1.com
seinterface.com searchencrypt.com
semalt.com senger.atspace.co.uk
seo18.su seojokes.net
seorank.info seorankinglinks.us
seozoom.it seotoolsagency.com
service-core.ru service.adtech.us
sex-foto.pw sex-foto.pw
sexblog.pw sexad.net
sfd-chess.ru sexobzor.info
sheerseo.com sexspornotub.com
shinikiev.com.ua sharebutton.net
sharebutton.to
shell-pmr.ru
shodanhq.com shodanhq.com
shoppingjequiti.com.br siteimprove.com
shopsellcardsdumps.com sitevaluation.org
shtora66.ru skatestick.bid
sierraapps.com skuteczna-dieta.co.pl
sitebeam.net skytraf.xyz
sitevaluation.com sledstvie-veli.net
sizeplus.work
skanninge.se
smallseotools.com
smartadserver.com smartadserver.com
snegozaderzhatel.ru snworks.com
socialbookmarksubmission.org soaksoak.ru
soundfrost.org social-vestnik.ru
space4updating.win space-worry.ml
spitfiremusic.com sprttrack.com
sportbetfair.com spy-app.info
spy-sts.com
spyfu.com
srecorder.com
starpages.net starpages.net
stmassage.ru statustroll.com
store-rx.com steamoff.net
storehouse.ua steebook.com
strangeduckfilms.com stevemonsen.com
success-seo.com stop-zavisimost.com
superstarfloraluk.com stroimajor.ru
surcentro.com superinterstitial.com
sureone.pro symphonyintegratedhealthcare.com
suture.co tabakur77.com
takethatad.com tackletarts.co
tam-gde-more.ru takeprofitsystem.com
tattooreligion.ru tedy.su
taxi-v-eisk.ru teenforporn.com
telefonsexkostenlos.tk thetardistimes.ovh
telemetryverification.net thewebsitetemplate.info
thebluenoodle.com titelhelden.eu
theporndude.com tizanidine6mg.blogspot.com
timhost.ru tizanidineinteractions.blogspot.com
tizanidinehcl2mgsideeffects.blogspot.com tizanidinemusclerelaxant.blogspot.com
tizanidinesideeffects.blogspot.com tizanidineorflexeril.blogspot.com
traderzplanet.in tizanidinerecreationaluse.blogspot.com
traffic100.com tizanidineshowupondrugtest.blogspot.com
traffic2cash.org tnaionline.org
trk-4.net tonivedu.it
try-rx.com top-study.work
tsan.net top10-way.com
tubeline.biz topvidos.ru
toy-shop.top
toyota.7zap.com
tracksz.co
traffic-club.info
trafficnetzwerk.de
traffique.net
trahvid.com
trucri.me
trudogolik.net
trumpetedextremes.com
tuberkulezanet.ru
tubo360.com tubo360.com
tula.howotorg.ru tula.mdverey.ru
turist-strani.ru turbodsp.com
turn-up-life.life turizm.bz
tvoystartup.ru tv-spoty.info
ua-company.ru txxx.com
uchetunet.su uhdtv.website
ukrtvory.in.ua uni.me
umg-stroy.ru univerfiles.com
unitygame3d.com v720hd.ru
unrealcommander.biz validdomain.xyz
uptime-gamma.net valmetrundan.se
upupa.net vasileostrovsky-rayon.ru
usacasino.com vbabule.net
usiad.net vclicks.net
utrolive.ru velen.io
uvozdeckych.info vepad.com
uytmaster.ru viberdownload10.com
uzungil.com video-chat.in
varikoz24.com videochat.mx
viagraneggrx.com videonsk.com
videokrik.net
videosbox.ru videosbox.ru
vids18.site vids18.site
vielporno.net
vigrx-original.ru
vip-parfumeria.ru
vipps.com.my
visa-china.ru
vitanail.ru vitanail.ru
vkak.ru vrnelectro.ru
voditeltrezviy.ru vse-pesni.com
voronezh.xrus.org
vtcdns.com
vysigy.su vysigy.su
w-journal.ru w3data.co
walkme.com webshoppermac.com
wdrake.com
website-audit.com.ua website-audit.com.ua
webstatsdomain.org wedding0venues.tk
whatistizanidine2mg.blogspot.com weightatraining.com
whatsappbot.flyland.ru welcomeauto.ru
whereverdesperate.gq wholesalejerseysgaa.com
whiteelephantwellington.com
wholinkstome.com wholinkstome.com
wpthemedetector.co.uk winner7777.net
wrona.it winterclassichockeyjerseys.com
ww2awards.info withstandingheartwarming.com
www.solartek.ru wleuaprpxuvr.ga
www.souvenirua.com wnoz.de
x-diesel.com womama.ru
xblog.in woman-h.ru
xdoza.com worldhistory.biz
xep.info www.get-free-traffic-now.com
xerox-douglas.cf x-porno.video
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai xjrul.com
xn----7sbahjd3btneuw1joc.xn--p1ai xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai
xn--80aeahghtf8ac5i.xn--p1ai xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xn--b1addnj3cah.xn--p1ai xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xolodremont.ru xn--80ahvj9e.xn--p1ai
xportvusbdriver8i.snack.ws xn--80ajbshivpvn2i.xn--p1ai
xuki.us xn--d1aifoe0a9a.top
xn--hxazdsfy.blogspot.com
xsfetish.org
xvideosporn.biz xvideosporn.biz
xxxdatinglocal.us yoshkarola.zrus.org
xxxnatelefon.ru yougotanewdomain.com
yadro.ru yoursite.com
yellocloud.be yugo-star.ru
yellowads.men zakonobosago.ru
yougetsignal.com zaloadi.ru
yourads.website zarepta.com
yousense.info zhacker.net
za-fun-offer.com zona-aqua.ru
zarajbuilders.com
zastroyka.org
zigarettenonlinekaufen1.blox.pl
zocaparj.kz
zog.link
zoogdinsney.com zoogdinsney.com
zoominfo.com zvooq.eu
zrizvtrnpale.tk

480
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
101billion.com 1-free-share-buttons.com
3w24.com 4ur.click
5kstore.com 8558l.jobs.net
aac500b7a15b2646968f6bd8c6305869d7.com 8lufu.com
abiente.ru aadroid.net
abwa.tk addbin.men
actices.com adf.ly
activecampaign.dreamhosters.com adsland.men
adrunnr.com adtech.us
ads.tremorhub.com aktivator-windows10.blogspot.com
adviceforum.info alfa-img.com
agroeconom.kz ananumous.ru
alert-fjg.xyz
alfa-medosmotr.ru
alienwheel.es
all4invest.info
animal-drawings.com animal-drawings.com
apiitsthirteende-a.akamaihd.net anime.dougasouko.com
arabsexxxtube.com apimossnetinfo-a.akamaihd.net
ascat.porn appartement-stumm.at
atleticpharm.org appleid-verification.com
approved.su
aquarium-pro.ru
arcadepages.com
atlasvkusov.ru
aucoinhomes.com aucoinhomes.com
avtochehli.by autoloans.com
awaybird.ru awaybird.ru
azbuka-mo.ru azartclub.org
barbourjackets.online azartmix.com
beachtoday.ru bablonow.ru
bestcalovebracelet.cn banan.tv
bankofthewext.com
bard-real.com.ua
battlecarnival.su
beremenyashka.com
bestcurs.org
bestempresas.es
bget.ru bget.ru
biglistofwebsites.com bgviagrachrx.com
bigpenisguide.com bimatoprost-careprost.com.ua
bingo8888.com bjgugu.net.ua
bioinnovate.co bmsco.co
bitcoin-ua.top buyviagraa.blogspot.com
blackcurranthumidifiers.site byme.se
blackle.com candcstuccoandstone.com
blacktwhite.com capture-room.com
blog.koreadaily.com carabela.com.do
bolitgorlo.net carfax.com.ua
bracketsmackdown.com castingbank.ru
bradleylive.xyz caveavins.fr
buron.pw cementaresearch.se
buy-forum.ru cfcl.co.uk
c-gainsbourg.com cgi2.nintendo.co.jp
cactussoft.biz chastnoeporno.com
camdolls.com choosecuisine.com
candelluxsklep.pl clknsee.com
cartechnic.ru cmhomestayagency.com
cdnanalytics.xyz commoncrawl.org
cercacamion.it compliance-mary.top
chatseo.com corta.co
chinajerseyswholesalecoupons.com creditwell.ru
cialischsrx.com customcatchcan.com
cineacademy.ru
cleanersoft.com
click2pawn.com
clickbanksites.info
cocyq.inwtrade.com
conocer-sanabria.com
continent-e.tv
counciltally.com
czat.wp.pl czat.wp.pl
d0t.ru damedingel.ya.ru
daymusam.com
dbmkfhqk.bloger.index.hr
decoratinghomes.ga
degerlund.net degerlund.net
deluxedumps.com dgfitness.co
derevesendeco.com diesel-parts28.ru
design-ideas.info domainsphoto.com
dinosaurus.site
distver.ru
docs4all.com
domik-derevne.ru
dosugrostov.site dosugrostov.site
download-of-the-warez.blogspot.com draniki.org
elvel.com.ua drpornogratisx.xxx
engines-usa.com eandsgallery.com
esopini.com eduserver.net
ezigarettekaufen2.blox.pl ekto.ee
ferieboligkbh.dk elektir.ru
fortevidyoze.net errorfixing.space
fr-bearings.ru estelight.ru
franklinfire.co everflora.ru
free-gluten.ru f07.de
free-share-buttons.blogspot.com fakehandbags.xyz
ftns.ru feargames.ru
fz139.ttk.ru fellowshipoftheminds.com
gamezblox.com fickblock18.com
gardene.ru formulaf1results.blogspot.com
gerhardhealer.com free-social-buttons.com
getlaid-xxxhookupdirect.com free-today.com
gribokstop.com freewhatsappload.com
haikuware.com freza-sverlo.ru
heatpower.ru gaygalls.net
hjaoopoa.top ghostvisitor.com
hop.clickbank.net gogalleryawesome.com
iboss.com gratuitxblcodes.com
ideashome.id guitar-master.org
ilovevitaly.xyz haveinc.xyz
imgarcade.com hdpixent.com
infobabki.ru hoporno.com
jjbabskoe.ru hostsshop.ru
jo24news.com hotel-mkad.ru
johannesburgsingles.co.za housemilan.ru
junketjuice.blogspot.com houtings.xyz
kamalsinha.com hyiphunter.org
katushka.net imperiafilm.ru
keenoutlet.online inmate-locator.us
kemerovo.zrus.org instasexyblog.com
kicknights.gq investzalog.ru
kiprinform.com jerseysbizwholesalecheap.com
kmd-pto.ru job.icivil.ir
kochanelli.com kaac.ru
lasvegaslockandsafe.com kaz.kz
laudit.ru keywordblocks.com
laxob.com kiinomaniak.pl
le-clos-des-alouettes.com kinostorm.net
leboard.ru kinozapas.com
lechenie-gemorroya.com klitimg.pw
life-instyle.com kretpicf.pw
link.ac lafriore.ru
liverpool.gsofootball.com lashstudia.ru
loginduepunti.it layola.biz.tc
lombia.co lecbter-relationships.ga
louboutinreplica.xyz lightinghomes.net
magicplayer-s.acestream.net linkpulse.com
marketingvici.com liupis.com
mebelcomplekt.ru loadopia.com
metarip.ru lob.com.ru
methodsmarketing.com locustdesign.co
midst.eu luxmagazine.cf
mirzonru.net m4ever.net
moje-recenze.cz magazin-pics.ru
mojowhois.com manuscript.su
monarchfind-a.akamaihd.net max-eclat.men
moneyzzz.ru mbiologi.ru
morf.snn.gr mcar.in.ua
mp3ringtone.info meduza-consult.ru
musezone.su mhi-systems.ru
muznachas-service.ru migente.com
mydirtystuff.com missvietnam.org
neks.info mister-shop.com
new-apps.ru mitrasound.ru
news-readers.ru mmoguider.ru
nyfinance.ml mygameplus.com
o-o-6-o-o.com nash-krym.info
oddamzadarmo.eu national-today-winning-winner.club
olvanto.ru needtosellmyhousefast.com
onemantrip.com newfilmsonline.ru
onlyporno.ru news-speaker.com
opinionreelle.com nextlnk12.com
orhonit.com noclegonline.info
paradontozanet.ru nrjmobile.fr
partnerline.men nzfilecloud.weebly.com
partnerworkroom.men o-o-8-o-o.ru
petrovka-online.com obiavo.by
phobia.us orgasmus-virtual.com
ourtherapy.ru
ownshop.win
palvira.com.ua
partner-pop.men
pearlisland.ru
pelfind.me
petedrummond.com
photokitchendesign.com photokitchendesign.com
pics-group.com play-mp3.com
piratecams.com popup-hgd.xyz
pogosh.com pornofeuer.com
popserve.adscpm.net pornowarp.info
poptool.net preparevideosafesystem4unow.space
porn555.com privacyassistant.net
pp-budpostach.com.ua progress-upakovka.ru
pr-ten.de prokotov.com
preparevideosafesystem4unow.site
printingpeach.com
progonrumarket.ru
pron.pro pron.pro
pufip.com prostitutki-rostova.org
q-moto.ru prostitutki-yaroslavlya.org
r-e-f-e-r-e-r.com pskcijdc.bloger.index.hr
ranksonic.info ptr.ruvds.com
realtytimes.com purplestats.com
remontvsamare.su rada.ru
restorator-msk.ru radiogambling.com
ricorsogiustizia.org ratemodels.net
rosbalt.com.ua realresultslist.com
runofilms.ru rezeptiblud.ru
salutmontreal.com roma-kukareku.livejournal.com
scanner-george.top sanatorrii.ru
scanner-john.top saneitconsulting.com
seadragonherbery.com scanner-josh.top
seansonline24.pl sdelatmebel.ru
seccioncontrabajo.com seccioncontrabajo.com
semxiu.com seolab.top
serw.clicksor.com sexyali.com
sexkvartal.com shmetall.com.ua
sexphoto.site silktide.com
sexyteens.hol.es site5.com
sierraapps.com sk.golden-praga.ru
simplepooltips.com slavia.info
sims-sims.ru smokewithrabbits.com
site-auditor.online snip.tw
siteheart.net soc-proof.su
slujbauborki.ru speedup-my.site
snowplanes.com spidtest.org
sodexo.com splendorsearch-a.akamaihd.net
sovinsteel.ru sptslmtrafms.com
spb.ru stard.shop
staff.prairiesouth.ca supermesta.ru
sticken.co swingerseiten.com
streetfooduncovered.com swinon.site
streha-metalko.si syvertsen-da.no
sucsesofinspiration.com tamada69.com
sumo.com tecnoteakviareggio.it
taqywu51.soup.io thaismartloan.com
templates.radiodigital.co thfox.com
thebluenoodle.com titslove.yopoint.in
thetardistimes.ovh tizanidinedruginteractions.blogspot.com
tizanidine4mgstreetvalue.blogspot.com tizanidinehydrochloride2mg.blogspot.com
tizanidineonline.blogspot.com toloka.hurtom.com
tnctrx.com toonfamilies.net
tramadolandtizanidine.blogspot.com torrnada.ru
trkdf.com traffic2money.com
tubo360.com trafficnetzwerk.de
twojebook.pl trafficstars.com
unitygame3d.com tryrating.com
unlimitdocs.net tt-ipd.info
uptime-as.net tvnewsclips.info
uptimebot.net uamtrk.com
via-gra.webstarts.com udsgame.online
viteonlusarezzo.it ukkala.xyz
volgograd.xrus.org uprour.com
vykup-avto-krasnodar.ru ural-buldozer.ru
wait3sec.org urlopener.com
wapsite.me uroffer.link
watsonrealtycorp.com v-doc.co
webads.co.nz velpanex.ru
webnode.me vinsit.ru
webs.com voditeltrezviy.ru
webtherapy.ru voluumtracker1.com
whatsappdownload10.com wanker.us
whatzmyip.net warning.or.kr
x-lime.com webmasters.stackexchange.com
xep.info website-analyzer.info
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai whatsappbot.flyland.ru
xn--80ajbshivpvn2i.xn--p1ai whiteproduct.com
xn--d1aifoe0a9a.top worldhistory.biz
xn--oogle-wmc.com x-diesel.biz
xtubeporno.net x-lime.net
xuki.us xjrul.com
youjizz.vc xmlinde.com
yoursearch.me yeartwit.com
yunque.pluto.ro ynymnwbm.bloger.index.hr
yuweng.info yurgorod.ru
zaidia.xhost.ro zajm-zalog-krasnodar.ru
zerocash.msk.ru zaobao.com.sg
zeroredirect9.com zophim.me
zoosexart.com zs2vm.top
zynax.ua

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.75 0; 10.1.0.232 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.03.3000 ### Version: V4.2022.03.3001
### Updated: Fri Mar 25 10:03:32 UTC 2022 ### Updated: Fri Mar 25 12:44:30 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.03.3000 ### Version: V4.2022.03.3001
### Updated: Fri Mar 25 10:03:32 UTC 2022 ### Updated: Fri Mar 25 12:44:30 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##