vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.10.2642 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-10-10 10:17:50 +00:00
parent fe36806fd7
commit a14634a958
48 changed files with 68282 additions and 68302 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2641 #### Version: V4.2021.10.2642
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2641 #### Version: V4.2021.10.2642
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.10.2641 #### Version: V4.2021.10.2642
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8400
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.10.2641 #### Version: V4.2021.10.2642
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8404
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8404
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8404
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8404
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.148 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.148 0;
127.0.0.1 0; 127.0.0.1 0;

8404
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.148 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.148 0;
127.0.0.1 0; 127.0.0.1 0;

8404
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.148 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.148 0;
127.0.0.1 0; 127.0.0.1 0;

8404
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2641 2642

8404
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8404
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.255.203.218 0; 40.71.82.105 0;

8404
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.148 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.148 0;
127.0.0.1 0; 127.0.0.1 0;

8404
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.148 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.148 0;
127.0.0.1 0; 127.0.0.1 0;

8404
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.148 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.148 0;
127.0.0.1 0; 127.0.0.1 0;

8404
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.255.203.218 0; 40.71.82.105 0;

8404
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.255.203.218 0; 40.71.82.105 0;

8404
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 1; 10.1.0.148 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Anthill Calculon
Dirbuster Domains Project
FDM GetWeb
Gotit SEOstats
Image Sucker Szukacz
Joomla Titan
Pavuk TurnitinBot
VelenPublicWebCrawler Web Fetch
VidibleScraper lwp-request
crawl.sogou.com satoristudio.net

290
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404checker 404checker
404enemy AIBOT
ALittle Client ALittle Client
Aboundexbot ASPSeek
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot Alligator
Anarchie Anarchie
Anarchy99 Anarchy99
Anthill Anthill
BBBike Apexoo
Aspiegel
AwarioSmartBot
BDCbot BDCbot
BDFetch BacklinkCrawler
Badass Badass
BatchFTP BatchFTP
Battleztar Bazinga
Bitacle
Black Hole
BlackWidow
BlowFish BlowFish
Boardreader BotALot
Brandprotect Brandprotect
Buddy Brandwatch
BuiltBotTough BuiltWith
Bullseye
BuzzSumo BuzzSumo
CODE87 Calculon
CazoodleBot CazoodleBot
CheTeam Chlooe
CherryPicker Cocolyzebot
ChinaClaw
Cloud mapping
CopyRightCheck CopyRightCheck
Cosmos Cosmos
Crawling at Home Project Craftbot
CrazyWebCrawler
Crescent Crescent
Custo
DBLBot
DIIbot DIIbot
Demon DSearch
Devil DataCha0s
Digincore Disco
DittoSpyder
DomCopBot
DomainAppender
DomainCrawler DomainCrawler
Dotbot Dotbot
EMail Wolf Dragonfly
ECCP/1.0
EMail Siphon
EasyDL EasyDL
Ebingbong Ebingbong
EroCrawler EroCrawler
Evil
Exabot
Extractor
ExtractorPro ExtractorPro
FHscan Extreme Picture Finder
EyeNetIE
Ezooms
Fimap Fimap
FlashGet
Flunky Flunky
Foobot Freeuploader
GT::WWW Fyrebot
GalaxyBot G-i-g-a-b-o-t
Genieo Genieo
GermCrawler GetRight
Getintent Go!Zilla
Gigabot
GoZilla
Gotit
GrabNet GrabNet
Grabber Grabber
GrapeshotCrawler
GridBot GridBot
HEADMasterSEO HTMLparser
HMView HTTP::Lite
HTTrack HTTrack
Harvest HaosouSpider
HybridBot Havij
Hloader
Iblog
Id-search Id-search
IlseBot IlseBot
IndeedBot IndeedBot
Indy Library InfoNaviRobot
InfoTekies InfoTekies
Internet Ninja JetCar
IstellaBot Jetty
JikeSpider JikeSpider
JustView Jorgee
Keyword Density Kenjin Spider
Kinza LWP::Simple
Larbin Leap
LibWeb
Libwhisker Libwhisker
LieBaoFast
Likse Likse
LinkWalker
Linkdexbot
LinkextractorPro LinkextractorPro
LinqiaScrapeBot Lipperhey
Ltx71 Lipperhey Spider
MJ12bot Litemage_walker
MSFrontPage MQQBrowser
MSIECrawler MSIECrawler
Mail.RU_Bot MTRobot
Magnet
Majestic SEO Majestic SEO
Majestic12 Majestic-SEO
MarkWatch MarkWatch
Mass Downloader
Mata Hari Mata Hari
MeanPath Bot Mb2345Browser
Mediatoolkitbot
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Mister PiX Moblie Safari
Morfeus Fucking Scanner Msrabot
Mr.4x3 NICErsPRO
Musobot
Name Intelligence Name Intelligence
Nameprotect
NearSite NearSite
Nessus Nessus
NetAnts NetAnts
NetLyzer NetMechanic
NetZIP NetSpider
Nikto Netcraft
NextGenSearchBot
NimbleCrawler NimbleCrawler
Nimbostratus Ninja
Nmap
Nuclei Nuclei
Nutch Offline Explorer
Octopus OpenVAS
Openfind Openvas
OrangeBot OrangeSpider
Page Analyzer OutclicksBot
PageAnalyzer PECL::HTTP
PageScorer PageScorer
Pandalytics
Pavuk Pavuk
Pi-Monster PeoplePal
Picsearch Picscout
Piepmatz
Pimonster
Pixray
ProPowerBot
ProWebWalker
Probethenet Probethenet
Pump Pu_iN
PyCurl QueryN Metasearch
Quick-Crawler RankActive
RankingBot2 RankActiveLinkBot
Rankivabot RankurBot
Re-re Re-re
Reaper ReGet
RebelMouse RebelMouse
Recorder
RedesScrapy RedesScrapy
Ripper Ripper
Rogerbot
SBIder
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SMTBot
Scrapy
Screaming
ScreenerBot ScreenerBot
SearchmetricsBot
Semrush
SemrushBot SemrushBot
SentiBot SentiBot
SeobilityBot SeobilityBot
Seomoz
Shodan Shodan
Siphon
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSnagger
SiteSucker
Siteimprove Siteimprove
Sitevigil
SlySearch SlySearch
SmartDownload SmartDownload
Snoopy Snapbot
SocialRankIOBot
Sociscraper Sociscraper
Sosospider
SpaceBison SpaceBison
Spammen
SpankBot SpankBot
SputnikBot
Sqlmap Sqlmap
Sqlworm
Sqworm
Stripper
SuperBot SuperBot
SuperHTTP SuperHTTP
Surfbot
SurveyBot SurveyBot
Swiftbot Suzuran
Szukacz T8Abot
TeleportPro TeleportPro
Telesoft
Telesphorep
The Intraformant
Thumbor
TightTwatBot
Toata
Toweyabot Toweyabot
Tracemyfile Trendiction
Trendictionbot Trendictionbot
True_Robot
Turingos Turingos
Turnitin
TurnitinBot TurnitinBot
TwengaBot
Twice Twice
Typhoeus
URLy Warning
UnisterBot UnisterBot
Upflow VelenPublicWebCrawler
V-BOT VeriCiteCrawler
Vagabondo Voil
VidibleScraper Voltron
Virusdie
VoidEYE
WISENutbot WISENutbot
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE Wallpapers
Wallpapers/3.0 WallpapersHD
Web Auto
Web Collage Web Collage
Web Enhancer Web Enhancer
Web Fetch Web Fuck
Web Sauger Web Sauger
WebAuto
WebBandit
WebCopier WebCopier
WebEnhancer
WebFetch
WebGo IS WebGo IS
WebImageCollector WebLeacher
WebPix WebReaper
WebSucker
WebWhacker WebWhacker
WebZIP WebsiteQuester
WebmasterWorldForumBot Webster
WebsiteExtractor Who.is Bot
Widow WinHTTrack
Woobot
Wprecon Wprecon
Xaldon WebSpider Xaldon WebSpider
YoudaoBot Xaldon_WebSpider
Zauba ZmEu
Zeus ZumBot
Zitebot archive.org_bot
ZoomBot clark-crawler
ZoominfoBot crawl.sogou.com
arquivo.pt domainsproject.org
coccocbot-web
com.plumanalytics
crawler.feedback
crawler4j
dataforseo.com
evc-batch evc-batch
heritrix gopher
instabid
internetVista monitor
ips-agent
isitwp.com
lwp-request
magpie-crawler
meanpathbot meanpathbot
muhstik-scan
netEstate NE Crawler
oBot
page scorer
plumanalytics plumanalytics
probe-image-size
s1z.ru
satoristudio.net satoristudio.net
scalaj-http
seobility seobility
sexsearcher seoscanners
serpstatbot
sitechecker.pro sitechecker.pro
siteripz siteripz
sogouspider sogouspider
spyfu
tAkeOut tAkeOut
trendiction.com trendiction.com
trendiction.de trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
x09Mozilla webmeup-crawler
webpros.com
xpymep1.exe xpymep1.exe
zgrab

312
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 01h4x.com
404enemy 404checker
80legs
ADmantX
AIBOT
ALittle Client ALittle Client
ASPSeek
AfD-Verbotsverfahren AfD-Verbotsverfahren
Alexibot AhrefsBot
Alligator Aipbot
Apexoo AlphaBot
Aspiegel Anarchie
BDCbot Anarchy99
BackStreet Anthill
Attach
BBBike
BLEXBot
BackDoorBot
BackWeb
Backlink-Ceck Backlink-Ceck
Badass
Bandit Bandit
BatchFTP Barkrowler
Bigfoot Bitacle
Blackboard BlackWidow
BlowFish
Bolt Bolt
Brandwatch BuiltWith
BuiltBotTough
Bullseye Bullseye
CATExplorador BuzzSumo
CCBot
CODE87 CODE87
CSHttp CSHttp
CheTeam CheeseBot
CherryPicker CherryPicker
Claritybot Chlooe
Cliqzbot Cliqzbot
Cocolyzebot Collector
Copier Copier
Crawling at Home Project
CrazyWebCrawler
Crescent
Curious Curious
Custo
DBLBot
DSearch DSearch
DTS Agent
DatabaseDriverMysqli
Demon
Deusu Deusu
DigitalPebble Dirbuster
Disco Discoverybot
Discobot DomainCrawler
DnyzBot DomainStatsBot
DomCopBot
DomainSigmaCrawler
Dotbot
Download Wonder
ECCP/1.0
EasyDL
Ecxi Ecxi
EirGrabber EroCrawler
Extractor Exabot
Express WebPictures
ExtLinksBot
EyeNetIE
Ezooms Ezooms
FDM FHscan
FemtosearchBot FemtosearchBot
Firefox/7.0 Firefox/7.0
FlashGet Flunky
FyberSpider Fuzz
GermCrawler Fyrebot
Go!Zilla GalaxyBot
GrabNet Genieo
GetRight
GetWeb
Getintent
Go-Ahead-Got-It
GoZilla
Gotit
Grabber Grabber
Grafula
GrapeFX GrapeFX
GridBot
HEADMasterSEO HEADMasterSEO
HMView
HTMLparser HTMLparser
HTTP::Lite
HTTrack HTTrack
Haansoft
Harvest Harvest
Havij Havij
Heritrix
Hloader
IRLbot IRLbot
Id-search
IlseBot IlseBot
Image Fetch IndeedBot
Image Sucker
Indy Library
InfoTekies
Intelliseek Intelliseek
InterGET Internet Ninja
IstellaBot InternetSeer
JOC Web Spider JOC Web Spider
Jbrofuzz JamesBOT
JennyBot JetCar
Jetty
Joomla
Jorgee Jorgee
JustView
Keyword Density Keyword Density
LNSpiderguy Larbin
Lanshanbot Lftp
Leap
LeechGet
LibWeb LibWeb
LieBaoFast
Likse Likse
Linkdexbot Linkbot
LinkpadBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Spider
Litemage_walker
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MIDown tool
MIIxpc MTRobot
MJ12bot Majestic12
MQQBrowser
MSIECrawler
Magnet
Majestic SEO
MarkMonitor
MarkWatch MarkWatch
Mass Downloader Masscan
Mata Hari
MauiBot
Mb2345Browser
MeanPath Bot MeanPath Bot
MegaIndex.ru MegaIndex.ru
MicroMessenger Mister PiX
Microsoft URL Control
Moblie Safari Moblie Safari
Mozlila Mojeek
Musobot Mr.4x3
Name Intelligence NPbot
Nameprotect
Navroad Navroad
NearSite NearSite
Needle Needle
NetLyzer Net Vampire
NetSpider
NetZIP NetZIP
Netcraft
Nettrack Nettrack
Nibbler Netvibes
NimbleCrawler Niki-bot
Nimbostratus Nimbostratus
Ninja Nmap
Nuclei
Nutch Nutch
OpenLinkProfiler Offline Navigator
Openfind OutclicksBot
Openvas POE-Component-Client-HTTP
OrangeSpider PageGrabber
OutfoxBot PageScorer
PageAnalyzer
Panscient Panscient
Papa Foto Papa Foto
Pavuk
Petalbot Petalbot
Picsearch Pi-Monster
Pimonster PictureFinder
ProPowerBot ProPowerBot
ProWebWalker
Pump Pump
RankActiveLinkBot PxBroker
QueryN Metasearch
Quick-Crawler
RSSingBot
RankingBot2 RankingBot2
Rankivabot
RankurBot
Re-re
ReGet
RealDownload RealDownload
Reaper Recorder
RebelMouse RepoMonkey
RedesScrapy
Ripper Ripper
RocketCrawler
Rogerbot
SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler
SISTRIX
ScanAlert
ScoutJet
Scrapy Scrapy
SeobilityBot ScrepyBot
Searchestate
SearchmetricsBot
Semrush
Shodan Shodan
Siphon
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer Snapbot
SiteSnagger Snoopy
SiteSucker
SlySearch
SmartDownload
Sociscraper Sociscraper
SpaceBison Sosospider
SpankBot
Spanner Spanner
Spbot
Spinn3r Spinn3r
Sqlmap Sqlmap
Stripper Stripper
Sucker Sucker
SuperHTTP Sucuri
Surfbot
Suzuran Suzuran
Swiftbot T0PHackTeam
Szukacz TeleportPro
T8Abot Telesoft
Teleport
Telesphorep
The Intraformant The Intraformant
Thumbor TightTwatBot
Toata Titan
Tracemyfile Trendiction
TwengaBot TurnitinBot
URLy Warning
UnisterBot UnisterBot
VB Project
Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
Virusdie
VoidEYE VoidEYE
Voil Voltron
WASALive-Bot
WBSearchBot
WEBDAV WEBDAV
WISENutbot
WWW-Collector-E WWW-Collector-E
WWW-Mechanize WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WeSEE WallpapersHD
Web Fetch Web Auto
Web Fuck
Web Pix Web Pix
Web Sucker WebCopier
WebEnhancer WebFetch
WebGo IS WebReaper
WebImageCollector WebStripper
WebLeacher WebSucker
WebWhacker WebWhacker
Webalta WebmasterWorldForumBot
Whacker Website Quester
WebsiteExtractor
Widow
WiseGuys Robot WiseGuys Robot
Wonderbot
Woobot
Wotbox Wotbox
Xaldon_WebSpider Zeus
Zermelo Zitebot
ZmEu
ZoomBot ZoomBot
ZumBot ZumBot
ZyBorg ZyBorg
autoemailspider backlink-check
check1.exe check1.exe
coccocbot-web clark-crawler
com.plumanalytics
crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j
dataforseo.com dataforseo.com
eCatch
facebookscraper facebookscraper
gopher heritrix
instabid internetVista monitor
isitwp.com
lwp-trivial
magpie-crawler
meanpathbot meanpathbot
oBot muhstik-scan
netEstate NE Crawler
page scorer page scorer
pcBrowser
polaris version polaris version
probe-image-size probe-image-size
ripz
satoristudio.net
scan.lol scan.lol
seoscanners seobility
seostar seostar
sogouspider sogouspider
sp_auditbot
spyfu
sysscan
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com webpros.com
webmeup-crawler xpymep1.exe
x22Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
1001desert.com alphacarolinas.com
101lesbian.xyz animenime.ru
ap.senai.br approvedlocal.co.za
hoterika.com bestofpicture.com
klejonka.info findacheaplawyers.com
luciddiagnostics.in onlyforemont.ru
members.ghanaweb.com replicaclub.ru
pogosh.com seksotur.ru
privatamateure.com threecolumnblogger.com
ukr-lit.com trafficmp.com

936
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00webcams.com 2020iscoming.info
1millionusd.xyz 2345.com
2000k.ru 23kw.ru
21h2o.com 2728fb936f0.com
2drittel.de 2drittel.de
2x2fan.ru 2girls1cup-free.com
3rbseyes.com 2pxg8bcf.top
3w24.com 40cg.com
4k-player.pl 4ureyesonly.com
7xc4n.com 51.la
abusora.com 55wmz.ru
add-add.men 7yue.org
8kisses.eu
a96527gi.beget.tech
aa9bd78f328a6a41279d0fad0a88df1901.com
aarbur.com
aasoldes.fr
abovetherivernc.com
academia-nsk.org
activepr.ru
adloads.com adloads.com
ads.gold adnotbad.com
adsloads.com adsref.men
adultgalls.com adssafeprotected.com
advancedcleaner.com adultwebhosting.info
advertex.info adventureparkcostarica.com
advertisingtag.net aero2.ru
advokateg.xyz affordablewebsitesandmobileapps.com
ahhjf.com afftrack001.com
akama.com ald2014.org
aksonural.ru alienwheels.de
allbizne.co.ua all4invest.ru
allblogroll.com all4wap.ru
amanda-porn.ga alldubai.biz
apessay.com allkrim.com
api.stathat.com allornamenti.com
apple.com-webbrowsing-security.science allwomen.info
ardimobileinfo.ml ameblo.jp
arendakvartir.kz amigobulls.com
arraty.altervista.org anal-acrobats.com
artel116.ru anal-acrobats.hol.es
asscenihotosidea.blogspot.com angry-fermi-7633.arukascloud.io
atelielembrancaqueencanta.com.br animaltoplist.com
autoblogger24.info apartmentratings.com
autobrennero.it apiadanaknet-a.akamaihd.net
autoseo-traffic.com apibrowsesmartne-a.akamaihd.net
autowebmarket.com.ua apimossnetinfo-a.akamaihd.net
azbukadiets.ru apiwoodensealcom-a.akamaihd.net
appfixing.space
apptester.tk
aproposde.com
apxeo.info
articlesdirectoryme.info
artpicso.com
aucoinhomes.com
autochoixspinelli.com
avitocars.ru
avon-novosib.ru
azazaporn.com
bag77.ru
balkanfarma.org balkanfarma.org
bannerconnect.net beauby.ru
bannerswap.com bedcapdealers.com
batanga.net
bayareaaeroclub.org
bbsoldes.fr
beatifullandscape.co
beget.tech beget.tech
belstaffstore.online belinvestdom.by
bestdooz.com besthoro.ru
bestserials.com bestserials.com
besttorrentknifta.weebly.com bestssaker.com
bestwaystogetpaid.us bfz.biz
beyan.host.sk bharatdefencekavach.com
bezcmexa.ru big-cash.net
bg6s0.com bigcareer.info
bioinnovate.co blogig.org
birzha-truda.eu
biznesluxe.ru
bkgr.se
blog.remote-computer.de
bloggers.nl
blogos.kz blogos.kz
bluesalt.co bocoarchives.org
bmwhighperformers.com boole.onlinewebshop.net
bonux.nextview.ru brainboosting.club
braip.com.br bristolhostel.com
breget74.ru bugof.gq
brillianty.info bugshoot.cn
browserprotecter.com buqyxa.rincian.info
bum.com.ru buynorxx.com
business-made-fun.com c1.onedmp.com
buyhoverboard.com cablecar.us
buypillsorderonline.com candypeople.se
buyviagraa.blogspot.com canrole.com
call-of-duty.info catalogs-parts.com
camdolls.com cbox.ws
camsex.xxx-cam.webcam cbozhe.com
carsdined.org cerev.info
cartierlovebraceletreplica.xyz cheapestjerseys-wholesale.com
cartierwatch.xyz cheapmarmot.online
censys.io chinacheapelitejerseys.com
chocolateslim-original.com christianlouboutinreplica.win
chocolateslim-u-srbiji.com cialischmrx.com
chocoslim.pro clickaider.com
christianlouboutinshoes.xyz clickhype.com
citysecurity.nu clickzzs.nl
clayvasedesigns.tk
clicksor.net
cmhomestayagency.com cmhomestayagency.com
community.allhiphop.com coldfilm.ru
compliance-jane.top compliance-alex.xyz
compliance-josh.top compliance-elena.top
compliance-mark.top compliance-fred.xyz
cookie-law-enforcement-aa.xyz compliance-jessica.top
coop-gamers.ru compliance-john.top
cottageofgrace.com compliance-mary.top
covetnica.com contentlook.co
custom-electric-guitar.com continent-e.tv
customergrowthsystems.com cookie-law-enforcement-ff.xyz
customerpromos-a.akamaihd.net cosmeticswomens-womensports.rhcloud.com
danceuniverse.ru countercrazy.com
dangphoto.trade csgo4.win
dayibiao.com cubook.supernew.org
daymusam.com daisye.top
ddospanel.com dating-time-now.com
ddpills.com detalizaciya-tut.biz
dentfidemountpreach.tk detoxmed24.ru
detki-opt.ru
devochki.top devochki.top
div.as digital-video-processing.com
divatraffic.com dividendo.ru
docs4all.com dogoimage.com
docstoc.com dojki365.online
doska-vsem.ru doktoronline.no
dostavka-v-krym.com domaincdn.xyz
domaincheck.io
download-of-the-warez.blogspot.com
doxysexy.com doxysexy.com
duawitchrarato.tk dyt.net
dumpsmania24.com easyshoppermac.com
dwomlink.info ebooknovel.club
e-commerce-seo1.com ekspresihati.info
edge.sharethis.com elektronischezi.livejournal.com
educontest.net elektronischezigarette2.mex.tl
ege-russian.ru elvenmachine.com
ekobata.ru enews.tech
electrik-avenue.com enginebay.ru
englishgamer.com envaseslotusama.com
erotik0049.com eredijovon.com
ertelecom.ru erotiktreff24.info
eshop4u.jp europeanwatches.ru
essay-edu.biz everytalk.tv
essay-writing.work executehosting.com
essayservicewriting.org extrabot.com
eu-cookie-law.info face.hostingx.eu
evehemming.blogspot.com.au falllow.gq
evening-dating-club.info fapgon.com
eyessurgery.ru fastfixing.tech
ez8motelseaworldsandiego.com fatmaelgarny.com
ezigarettekaufen2.blox.pl fba-mexico.com
ezigarettenkaufen2.dreamwidth.org
failingmarriege.blogspot.com
fandlr.com
ferieboligkbh.dk ferieboligkbh.dk
ferretsoft.com ferrotodo.com
fickluder69.com filmidivx.com
filmetricsasia.com financepoints.eu
finansov.info findacheaplawyers.com
finstroy.net fitfloponline.store
fitness-video.net foodcrafts.website
florida-tourism.net formseo.com
focalink.com forpackningsutveckling.se
footballfarrago.com forpostlock.ru
formularz-konkurs.tk fortwosmartcar.pw
free-today.com free-gluten.ru
freelinkbuilding.website.tk freeuploader.ml
freelotto.com frvo.alptandem.ru
fuckbuddybestgilf.info fuck-paid-share-buttons.xyz
fullgirl.ru game300.ru
g7m.pl gamerextra.com
gambarxkata.co gamesprite.me
gameplexcity.com gardene.ru
games.kolossale.ru gay-file.com
gay-site.store ge0ip.net
gaz-voshod.ru get-your-social-buttons.info
getmiro.com
getmyads24.com
getpopunder.com
gilbertbanda.net gilbertbanda.net
gk-atlant.info given2.com
glass-msk.ru gobongo.info
golden-praga.ru goo.ne.jp
goodhumor24.com googlefeud.com
googlsucks.com
gratis-sexkontakte.com
gratuitxblcodes.com
grizzlysgrill.com grizzlysgrill.com
gtopstats.com gsou.cf
guge.io guigyverpo.cf
gwebtools.com gwagka.com
handicapvansarea.com gz2.bbsoldes.fr
hausfrauensex18.com handsandlegs.ru
hd720kino.ru hasshe.com
highspeed5.net hdimagegallery.net
homedecorpicture.us healmytrauma.info
healthcarestore.info
heartofpayne.xyz
hentaiheroes.com
hkdiiohi.skyrock.com
hol.es
holidaypics.org
hopeonthestreet.co.uk hopeonthestreet.co.uk
hostingclub.lk hostcritique.com
hoztorg-opt.ru hotloans.ru
hspline.com howmuchdoestizanidinecost.blogspot.com
hybrid.ru iccornacircri.cf
igtools.club ilikevitaly.com
incanto.in.ua imagerydatabase.com
indianmedicaltourismshop.com imgarit.pw
indo-export.ru inclk.com
innoslicon.com infazavr.ru
intellekt21.ru infektsii.com
internetartfair.com insomniagamingfestival.com
investmac.com instasexyblog.com
inzn.ru insurple.com
iradiology.ru investzalog.ru
itbc.kiev.ua istizanidineanarcoticdrug.blogspot.com
itis4you.com istock-mebel.ru
itservicesthatworkforyou.com iwanttodeliver.com
jagg.info j-times.ru
janettabridal.com jamiembrown.com
javstock.com japan-bearings.ru
jennyfire.ru jav-fetish.site
jerseywholesalebizchina.com javlibrary.cc
joessmogtestonly.com jikoman.info
jpcycles.com jofucipiku.tk
jrpmakati.com johnrobertsoninc.com
journalhome.com
kaidalibor.de
kakadu-interior.com.ua kakadu-interior.com.ua
kambasoft.com karachev-city.ru
kazan.xrus.org karaganda.xkaz.org
kartiniresto.com
katadhin.co
kchaxton.com kchaxton.com
kerch.site
kerwinandcariza.com kerwinandcariza.com
kevblog.top kiinomaniak.pl
kino-fun.ru kinky-fetishes.com
kiprinform.com kinomaniatv.pl
kirov.zrus.org
kit-opt.ru
klejonka.info klejonka.info
kongruan.com kmgamex.cf
konteiner24.com kopihijau.info
koptims.tiu.ru kosmetyki.tm.pl
krafte.ru krafte.ru
kraljeva-sutjeska.com
krasula.pp.ua krasula.pp.ua
ktotut.net kursy-ege.ru
labplus.ru
lafourmiliaire.com
lanadelreyfans.us
landmania.ru
languagecode.com languagecode.com
law-check-seven.xyz laparfumotec.com
letmacworkfaster.site laptoper.net
letslowbefast.site larutti.ru
letsrepair.in lazy-z.com
levaquin750mg.blogspot.com leboard.ru
libertybilisim.com ledpolice.ru
lifeinsurancekb.com linkarena.com
liky.co.ua linkrr.com
linkbuddies.com liupis.com
livefixer.com liveinternet.ru
lizace.com livejasmin.com
llastbuy.ru livingroomdecor.info
loadopia.com lost-alpha.ru
luchshie-topcasino.ru louboutinreplica.xyz
luciddiagnostics.in lovelycraftyhome.com
m-google.xyz lsex.xyz
maglid.ru lulea-auktionsverk.se
magnetic-bracelets.ru m4ever.net
mahnwachen-helfen.info macdamaged.space
marcoislandvacations.net mackeeper-land-672695126.us-east-1.elb.amazonaws.com
maridan.com.ua magazin-pics.ru
mbiologi.ru mamasuper.prom.ua
mcadamssupplyco.com marmotstore.online
meble-bogart.info marwer.info
medosmotr-ufa.ru masthopehomes.com
meetingrainstorm.bid mature.free-websites.com
mega-bony2017.pl med-zdorovie.com.ua
megaindex.ru millionare.com
memberty.com mir-business-24.ru
microsoftportal.net mixed-wrestling.ru
minyetki.ru mixx.com
mir-limuzinov.ru mobot.site
mistr-x.org mockupui.com
mjchamonix.org modenamebel.ru
mlf.hordo.win moinozhki.com
mmoguider.ru monarchfind-a.akamaihd.net
mmstat.com monetizationking.net
mojpreskumanie.com
monclerjacketsoutlet.win
moncleroutletonline.xyz
moneyviking-a.akamaihd.net
monsterdivx.tv monsterdivx.tv
morf.snn.gr moomi-daeri.com
morlat.altervista.org mosdverka.ru
moroccosurfadventures.com movpod.in
moviemail-online.co.uk moxo.com
movies-in-theaters.net mrlmedia.net
msk.afora.ru mtmtv.info
muz-baza.net muabancantho.info
mvpicton.co.uk mulberryoutletonlineeu.com
mybackgroundlandscape.blogspot.com myanyone.net
mydearest.co mybinaryoptionsrobot.com
mygameplus.ru mycouponizemac.com
myiptest.com myplaycity.com
myonlinepayday.co myyour.eu
na-telefon.biz nacap.ru
nagdak.ru najaden.no
napalm51.nut.cc nakozhe.com
naperehresti.info neodownload.webcam
naphukete.ru new-apps.ru
narutonaruto.ru notfastfood.ru
nastroyke.net oecnhs.info
naturtreenspicerx.pw offtopic.biz
naval.jislaaik.com oklogistic.ru
networkad.net onlainbesplatno.ru
nevansk.ru online7777.com
newhairstylesformen2014.com onlinebay.ru
newstudio.tv onlinemeetingnow.com
no-rx.info onlyforemont.ru
notebook-pro.ru ons-add.men
nufaq.com onstrapon.purplesphere.in
obesidadealgarve.com ontargetseo.us
obiavo.kz outrageousdeal-a.akamaihd.net
oddamzadarmo.eu paradontozanet.ru
of-ireland.info parvezmia.xyz
onemactrckr.com pattersonsweb.com
oneshotdate.com peekyou.com
openfrost.net pertlocogasilk.tk
openstat.com petrovka-online.com
optitrade24.com pflexads.com
orhonit.com phelissota.xyz
panchro.xyz photo.houseofgaga.ru
paparazzistudios.com.au physfunc.ru
paperwritingservice17.blogspot.ru piccshare.com
partner-host.men picsforkeywordsuggestion.com
pcimforum.com picturesboss.com
penisvergrotendepillennl.ovh pinstake.com
pinwallpaper.top pix24x7.com
pio.polytopesexempt.com play-movie.pl
pivka.xyz plohaya-kreditnaya-istoriya.ru
pixshark.com popup.matchmaker.com
playboyfiles.xblog.in porndairy.in
playmsn.com
pojdelo.weebly.com
poppen-nw.net
poquoson.org
porno-dojki.net
porno-play.net porno-play.net
pornowarp.info pornoelita.info
pornstartits.xblog.in pornosmola.info
preg.marketingvici.com portside.cc
privat-girl.net privatov-zapisi.ru
producm.ru privhosting.com
prodvigator.ua profitsport.club
proftests.net progress-upakovka.ru
promodj.com pronorm.fr
proprostatit.com proprostatit.com
psa48.ru pts163.ru
psoriasis-file.trade puserving.com
pukaporn.com pussyspace.net
pulse33.ru qor360.com
qitt.ru qpypcx.com
quickchange.cc quality-traffic.com
rabot.host.sk quickbuck.com
rczhan.com rangoman.date
redirlock.com rankingchart.de
reeyanaturopathy.com ranksonic.com
registrationdomainsite.com rapidgator-porn.ga
remmling.de rating-bestcasino.com
remybutler.fr rcpmda.ikan1080.xyz
riders.ro regionshop.biz
rock-cafe.info remontvsamare.su
rosbalt.com.ua rentehno.ru
russintv.fr rightenergysolutions.com.au
rutor.group roma-kukareku.livejournal.com
rvtv.ru royal-investments.net
salutmontreal.com rukino.org
sammlungfotos.online russia-tao.ru
santasgift.ml s-iwantyou.com
s.lollypopgaming.com
saitevpatorii.com
savefrom.com savefrom.com
scanner-donald.top sbprabooks.com
scanner-jane.top sc-specialhost.com
scanner-walt.top scanner-george.top
scenicmissouri.us scanner-jessica.top
scanner-mark.top
scanner-mary.top
schlampen-treffen.com schlampen-treffen.com
sdelai-prosto.ru scoopquest.com
se-welding.ru scrapinghub.com
search.pch.com screpy.com
search.xtconnect.com searchimpression.com
searchencrypt.com segol.tv
searchwebknow-a.akamaihd.net seorankinglinks.com
seksotur.ru servicecenter.co.ua
selectads.men sex-sex-sex5.com
seocdvig.ru sextracker.be
seokicks.de sexvideo-sex.com
seomarketings.online shop.acim.org
seotoolsagency.com shopperifymac.com
sexstream.pl shtora66.ru
sexvporno.ru sideeffectsoftizanidine.blogspot.com
sexy-pings.com sinestesia.host.sk
sexyebonyteen.com site3.free-share-buttons.com
sexytrend.ru slavia.info
shijian.ac.cn social-button.xyz
shop.xz618.com social-buttons.xyz
shopcheermakeup.info social-s-ggg.xyz
sierraapps.com socialbuttons.xyz
siteaero.com sodexo.com
siteexpress.co.il songoo.wz.cz
skachat-besplatno-obrazcy.ru soserfis.com
slayerlife.com soundfrost.org
snip.tw space4update.pw
snow.nvr163.com sportbetfair.com
socialmonkee.com srdrvp.com
socialseet.ru stockspmb.info
softonicads.com stopnarco.ru
solution4u.com strana-krasoty.ru
spaceshipad.com strana-solnca.ru
sptslmtrafms.com strongsignal-a.akamaihd.net
stathat.com stroyplus.ru
steame.ru sukarame.net
strv.se sumo.com
sugarkun.com superstarfloraluk.com
supermodni.com.ua td-33.ru
surffoundation.nl tech4master.com
surflinksmedical.com telsis.com
survival.betteroffers.review testbotprocessor44.com
taboola.com
taqywu51.soup.io
tattooreligion.ru
tedxrj.com
teencastingporn.com
templates.franklinfire.co
tengohydar.tk
texbaza.by
the-usa-games.blogspot.com the-usa-games.blogspot.com
theendivechronicles.com theprofitsmaker.net
thejournal.ru thewebsitetemplate.info
thexart.club titelhelden.eu
threecolumnblogger.com titslove.yopoint.in
timhost.ru tizanidinedosage.blogspot.com
tizanidine4mgtablets.blogspot.com tizanidinedrugtest.blogspot.com
top1-seo-service.com tizanidinehcl4mgtab.blogspot.com
tizanidinehcl4mgtabinfo.blogspot.com
tizanidinehcl4mgtablet.blogspot.com
tizanidineoral.blogspot.com
tizanidineorflexeril.blogspot.com
tizanidinestreetprice.blogspot.com
tizanidinewithdrawal.blogspot.com
tk-assortiment.ru
top-karkas.ru
topads.men topads.men
topanasex.com torrent-to-magnet.com
topsiteminecraft.com torture.ml
track.deriv.com
traffixer.com traffixer.com
trahic.ru
trahvid.com trahvid.com
traxdom.ru tuckermktg.com
triumf-realty.ru tversvet.ru
trucri.me tvory.predmety.in.ua
tsan.net twiclub.in
tula.mdverey.ru twittruth.com
turbabitload.weebly.com u555u.info
tvand.ru uac.net.au
twu.com.ua uchil.net
udsgame.online ucsol.ru
uhdtv.website unitexindia.com
ukr-lit.com urlopener.blogspot.com.au
umityangin.net
unmaroll.ya.ru
userequip.com
v-doc.co
valoresito.com valoresito.com
vchulkah.net vellings.info
via-energy-order.com velobikestock.com
viagroid.ru veopornogratis.xxx
video-chat.in vepad.com
videochat.mx viagra.pp.ua
videochat.tv.br videonsk.com
viel.su viel.su
vipps.com.my vilingstore.net
virus-respirators.com vipsiterip.org
viveresaniesnelli.it voloslove.ru
vkak.ru voronezh.xrus.org
vozbujdenie.com vrotike.ru
vroze.com vseigry.fun
vseuznaem.com vuclip.com
vucms.com vzlom-na-zakaz.com
w3data.co wcb.su
wanker.us webnode.me
wareseeker.com webscouter.net
waterefficiency.co websiteexplorer.info
websiteaccountant.de weburok.com
whatzmyip.net wechatdownload10.com
williamrobsonproperty.com whatsappbot.flyland.ru
windowssearch-exp.com whiteproduct.com
wiosenny-bon-1500.pl wma-x.com
wonderfulflowers.biz wordpress-crew.net
word-vorlagen.net worldtraveler.world
worldis.me www.solartek.ru
wowas31.ucoz.ru xblog.in
wrz0iuebwhp5fg.freeddns.com xitjw.info
wufak.com xmlinde.com
wurr.voila.net xmnb.net
www.pinnacle-bets.com xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
x-lime.com xn----btbdvdh4aafrfciljm6k.xn--p1ai
x-mix.info xn--80aaagvmjabrs1aoc9luc.xn--p1ai
x-rates.ru xn--80ahdheogk5l.xn--p1ai
xmronta.com xnxx699.com
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai xrus.org
xn--oogle-wmc.com xxlargepop.com
xtrafficplus.com yaminecraft.ru
xxxtubesafari.com yaoguangdj.com
yaderenergy.ru
yadro.ru
yhit.press
yogamatsexpert.com yogamatsexpert.com
yourothersite.com yopoint.in
yoursearch.me youradulthosting.com
yourserverisdown.com youtoner.it
zdesformula.ru zakazvzloma.com
zebramart.ru zaobao.com.sg
zeleznobeton.ru zarabotok--doma.ru
zigarettenonlinekaufen1.bloog.pl zixizop.net.ru
zigarettenonlinekaufen2.drupalgardens.com zooggames.com
znaturaloriginal.com zoomovies.org
zoolubimets.ru zoosexart.com
zoominfo.com zootravel.com
zvetki.ru zynax.ua
zzlgxh.com

474
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00author.com 03e.info
01apple.com 0912701309f8ce.com
11235813.webzdarma.cz 100searchengines.com
3th.co.in 13tabs.com
3xforum.ro 1webmaster.ml
7search.com 3w24.com
8kisses.eu a14download.com
9med.net aa8780bb28a1de4eb5bff33c28a218a930.com
aa8b68101d388c446389283820863176e7.com acmebtn.ml
aa9bd78f328a6a41279d0fad0a88df1901.com actulite.com
acc.eu.org adultmeetonline.info
adrunnr.com adultnet.in
adtech.de adviceforum.info
adultfullhd.com affrh2015.com
albamargroup.com aihelen.net
alldezservice.kz alldubai.biz
alloysteel.ru alvtank.se
apartment.ru android-systems.ru
apibatbrowsecom-a.akamaihd.net anlimebel.kiev.ua
apiluckyleapnet-a.akamaihd.net apisecretsaucebi-a.akamaihd.net
apioutoboxnet-a.akamaihd.net apptester.tk
arendatora.ru arquivo.pt
ausmepa.org.au ascat.porn
autolombard-krasnodar.ru asrv-a.akamoihd.net
bankofthewext.com atdedinotuho.tk
bannerconnect.net avek.ru
bestmobilityscooterstoday.com aviva-limoux.com
bildsuche.ru ayanaspa.com
biznesschat.net backuperwebcam.weebly.com
blogqpot.com bankmib.ru
brainxs.us bannerswap.com
bryansk.zrus.org bestcoin.cc
bugshoot.cn bfz.biz
bum.com.ru bharatdefencekavach.com
buzz-porno.info bioinnovate.co
cdnnetwok.xyz bizru.info
chastnoeporno.com book-bets.com
chatroulette.world bronzeaid-a.akamaihd.net
cheapmoncler.xyz bupropion-sr-150-mg.us
chlooe.com burger-tycoon.com
chocolateslim-u-srbiji.com buy-loft.ru
cialischstgerts.com cartierreplica.xyz
cjs.com.ru cascadelink.org
clickzzs.nl cheapmoncler.pw
compliance-brian.top citizenclsdriveri7.pen.io
connectionstrenth.com clickbank.net
contentlook.co compliance-brian.xyz
coolwallpapers-hd.com compliance-nelson.top
counterbot.com cs-passion.pl
cowlmash.com cutt.us
cubs-jerseys.us cvety24.by
dairyindia.in daneshetabiat.com
darodar.com dating-time-now.com
datract.com
day-news.info day-news.info
dekorkeramik.ru
deshevo-nedorogo.ru deshevo-nedorogo.ru
dev78.dioniqlabb.se dipstar.org
dikqlyremy.info disruptingdinnerparties.com
dogoimage.com dlya-android.org
domznaniy.ru draniki.org
downloadme.life e-commerce-seo.com
doxyporno.com easytuningshop.ru
driving.kiev.ua editmedios.com
earthmagic.info ekspresihati.info
ekosmetyki.net.pl election.interferencer.ru
en.home-task.com elitedollars.com
enews.tech elitesportsadvisor.com
eropho.net epicbrogaming.com
expdom.com evehemming.blogspot.com.au
ezigarettenkaufen2.dreamwidth.org evening-dating-club.info
f-loaded.de everflora.ru
f00kclan.de evidencecleanergold.com
failingmarriege.blogspot.com ezigarettekaufen1.hpage.com
famix.xyz ezigaretteshop2.vefblog.net
fanoboi.com familyphysician.ru
fapgon.com filesdatabase.net
favornews.com film-one.ru
ferretsoft.com fishingwholesale.us
filmi-onlain.info florida-tourism.net
financepoints.eu forodvd.com
findpics.pw forumprofi.de
flagcounter.me freeseedsonline.com
flooringinstallation-edmonton.com frustrated-favorable.gq
flytourisme.org fuck-paid-share-buttons.xyz
forsex.info fuckbuddybestgilf.info
free-laptop-reward.com gagrasector.ru
free-video-tool.com galaxy-family.ru
freefoto.ca gaypornmovie.net
froggytube.com gazobeton-p.com.ua
gameplexcity.com gdcentre.ru
gay-file.com gemara.com
gcup.ru gemgrab-a.akamaihd.net
given2.com genericviagrasildenafiled.net
gloverid.site germetiki.com.ua
good-mummy.ru get-your-social-buttons.info
gourcy.altervista.org gg-arena.ru
grand-chlen.ru gifspics.com
growmyfunds.ca glastecfilms.com.my
gyffu.com go2mike.ru
hard-porn.mobi goodhumor24.com
healing-dysplasia.ru gorabagrata.ru
healthcarestore.info greenidesign.co
help.tpu.ru gsou.cf
hkladys.com hasshe.com
hoverboard360.se healbio.ru
iflycapetown.co.za highstairs-a.akamaihd.net
igru-xbox.net hitcpm.com
indetiske.ya.ru hitmuzik.ru
inet-traffic.com homegardenlova.com
idealtits.net
ido3.com
iinstalll-fii1leis.jus0wil.pp.ua
imicrovision.com
indiasourcemart.in
intermesh.net intermesh.net
itag.pw investingclub.ru
itsdp3.com irunfar.com
jennyfire.ru istanbulit.com
kaidalibor.de iwantmyfreecash.com
kaz.kz ixora.pro
kleine-titten.biz izismile.com
klitimg.pw javatex.co.id
konkursowo-24.pl jimmychoosale.online
kopihijau.info kino-ecran.ru
kostenloser-sex.com klumba55.ru
kunstaktien.info kmd-pto.ru
lafourmiliaire.com knowsitall.info
languagecode.com lacasamorett.com
law-check-two.xyz law-check-two.xyz
lerporn.info lego4x4.ru
lenpipet.ru
lesbian.xyz lesbian.xyz
likesdesign.com linksharingt.com
livejasmin.com live-cam6.info
loadopia.com locksmith.jp
manifestyourmillion.com lolnex.us
manimpotence.com lostaruban.com
manve.info m3gadownload.pl
mecash.ru mailemedicinals.com
medicine-4u.org mejoresfotos.eu
medicovi.com memberty.com
mobile-appster.ru mobioffertrck.com
moesen-ficken.com monthlywinners.com
moscow-region.ru mouselink.co
musicpro.monster mtmtv.info
mustat.com myfreecams.com
my-bc.ru
myonigroup.com
myshopmatemac.com myshopmatemac.com
myxdate.info
net-profits.xyz
networkcheck.xyz networkcheck.xyz
noclegonline.info ntic.fr
nordvpn.com o-o-8-o-o.ru
nottyu.xyz
novgorod.xrus.org
odywpjtw.bloger.index.hr odywpjtw.bloger.index.hr
onko-24.com onemactrckr.com
onlainbesplatno.ru ontargetseo.us
online-sbank.ru openfrost.net
onlinedomains.ru optibuymac.com
oranga.host.sk papasdelivery.ru
origin-my.ru paradontozanet.ru
perfectpracticeweb.com peugeot-club.org
perm.xrus.org picsforkeywordsuggestion.com
pestomou.info pinstake.com
picswe.com
piluli.info
pinup-casino1.ru
pinwallpaper.xyz pinwallpaper.xyz
pornogratisdiario.com plastweb.ru
presleycollectibles.com pornonik.com
privetsochi.ru pornoplen.com
prizestohandle.club pornstartits.xblog.in
promgirldresses.xyz positive2b.ru
pukaporn.com pozvonim.com
princeadvantagesales.com
princevc.com
prod2016.com
projectforte.ru
promoforum.ru
pulse33.ru pulse33.ru
raisedseo.com qor360.com
rapidokbrain.com rabot.host.sk
rapidsites.pro rainbowice.ru
remedyotc.com ranksonic.info
remontbiz.ru reversing.cc
rep-am.com ricorsogiustizia.org
rogervivierforsale.com russintv.fr
samoiedo.it sanjosestartups.com
sbwealthsolutions.ca sankt-peterburg.nodup.ru
scanner-jess.top santechnik.jimdo.com
schoolfiles.net sbf441.com
scmor.ilxc.cc scrapy.org
screpy.com se.bnt-team.com
search.xtconnect.com seoanalyses.com
secret.xn--oogle-wmc.com seryeznie-znakomstva.ru
semalt.com sexkvartal.com
seomarketings.online sexyali.com
sexiporno.net sexyteens.hol.es
siteaero.com shisha-swag.de
sladkoevideo.com shoppingmiracles.co.uk
slayerlife.com simplepooltips.com
slkrm.ru sispe.com.br
social-buttons.com smokewithrabbits.com
sockshares.tv sockshares.tv
soochi.co softlinesolutions.me
spywarebegone.com solution4u.com
stopnarco.ru sophang8.com
sudexpert66.ru spaceshipad.com
ssconstruction.co
stairliftstrue.com
strana-krasoty.ru
suhanpacktech.com
sunflowerdrawingpaintings.blogspot.com sunflowerdrawingpaintings.blogspot.com
surintech.ac.th suture.co
template-kid.com svarbit.com
the-usa-games.blogspot.com sygraem.com
thecounter.com td-33.ru
thedownloadfreeonlinegames.blogspot.com telefonsexsofort.tk
thegioixekhach.com terraclicks.com
tizanidinepills.blogspot.com timeallnews.ru
tizanidinevssoma.blogspot.com titelhelden.eu
tmtrck.com tizanidinehcl2mgsideeffects.blogspot.com
top250movies.ru tizanidineorflexeril.blogspot.com
topmira.com tnctrx.com
toproadrunner5.info tradedeals.biz
torrents.life traffictrade.life
totu.us trahic.ru
truemfilelj.gq trailer.cinemaflix.website
trustedmaccleaner.com tvnewsclips.info
tt-ipd.info twinderbella.com
uhdtv.website twodollarshows.com
uhodzalijami.ru udsgame.online
umityangin.net urdoot.win
universals.com.ua vchulkax.com
vbtracker.net verymes.xyz
videochat.cafe vkonche.com
vip-file.com vkontarkte.com
virus-respirators.com vmnmvzsmn.over-blog.com
voprosotvet24.ru vseuznaem.com
vpnhowto.info
vrotike.ru
vzlomfb.com vzlomfb.com
weblibrary.win warning.or.kr
wedding0venues.tk webcamdevochka.com
wejdz-tu.pl webcamtalk.net
wemarketing.se weekes.biz.tc
westermarkanjou.se wholinkstome.com
whatsupinfoley.com witclub.info
wholesalejerseyscheapjerseys.us.com woodyguthrie.se
wjgony.com www.solartek.ru
womama.ru xedserver.com
woman-orgasm.ru xjlottery.com
xaijo.com xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
xbaboon.com xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xn--24-glceagatoq7c2a6ioc.xn--p1ai xn--b1addnj3cah.xn--p1ai
xtube.com xolodremont.ru
xxxkaz.org youdao.com
youradulthosting.com yourdesires.ru
zakon-ob-obrazovanii.ru zastenchivosti.net
zero1.it zeroredirect10.com
zeroredirect8.com
zophim.me

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.35 0; 10.1.0.148 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2641 ### Version: V4.2021.10.2642
### Updated: Sat Oct 9 22:02:46 UTC 2021 ### Updated: Sun Oct 10 10:11:57 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2641 ### Version: V4.2021.10.2642
### Updated: Sat Oct 9 22:02:46 UTC 2021 ### Updated: Sun Oct 10 10:11:57 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##