vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.04.3029 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-04-07 10:09:12 +00:00
parent 8f2b94f952
commit 41a9fed131
47 changed files with 90250 additions and 90240 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3028 #### Version: V4.2022.04.3029
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3028 #### Version: V4.2022.04.3029
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.04.3028 #### Version: V4.2022.04.3029
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11146
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.04.3028 #### Version: V4.2022.04.3029
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11150
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11150
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11150
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11150
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

11150
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

11150
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

11150
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3028 3029

11150
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11150
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.45.218.19 0; 40.88.124.26 0;

11150
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

11150
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

11150
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

11150
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.45.218.19 0; 40.88.124.26 0;

11150
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.45.218.19 0; 40.88.124.26 0;

11150
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
80legs Bolt
Calculon Brandwatch
Crescent CazoodleBot
Extreme Picture Finder CopyRightCheck
Jorgee CyotekWebCopy
Majestic SEO Firefox/7.0
SeobilityBot HonoluluBot
SputnikBot Nmap
SuperHTTP WebEnhancer
magpie-crawler WinHTTrack

306
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404checker 404checker
80legs 80legs
ASPSeek ADmantX
Abonti AIBOT
Acunetix ALittle Client
Aboundexbot
AfD-Verbotsverfahren AfD-Verbotsverfahren
AiHitBot
Alexibot Alexibot
AlphaBot
Anarchie Anarchie
Anthill Apexoo
Aspiegel Aspiegel
Attach
BBBike
BLEXBot BLEXBot
Backlink-Ceck BackWeb
Badass BatchFTP
BetaBot Bigfoot
Bolt Bolt
BotALot
Brandwatch
Buck Buck
Buddy
BunnySlippers BunnySlippers
CATExplorador BuzzSumo
CODE87
CSHttp
Calculon Calculon
Cegbfeieh CazoodleBot
CheTeam CherryPicker
CheeseBot ChinaClaw
Cliqzbot Citoid
Cocolyzebot Cocolyzebot
Cogentbot Copier
CopyRightCheck
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
CrunchBot Crescent
Custo
DBLBot
DIIbot DIIbot
DTS Agent DTS Agent
DigitalPebble DatabaseDriverMysqli
Dispatch Digincore
DittoSpyder Disco
Discobot
Discoverybot
DnBCrawler-Analytics
DomCopBot DomCopBot
DomainCrawler DomainCrawler
Domains Project Dotbot
Dragonfly Dragonfly
EMail Siphon ECCP/1.0
EMail Wolf
Ebingbong Ebingbong
EroCrawler Ecxi
Exabot EirGrabber
Express WebPictures Evil
FDM Extractor
ExtractorPro
FemtosearchBot FemtosearchBot
Flunky Firefox/7.0
FlashGet
Foobot
Freeuploader
FrontPage
Fuzz Fuzz
G-i-g-a-b-o-t FyberSpider
Fyrebot
GT::WWW
GalaxyBot GalaxyBot
Genieo GermCrawler
Go!Zilla GetWeb
Gigabot
Gotit
GrabNet GrabNet
GrapeFX GrapeFX
GridBot GrapeshotCrawler
HMView HMView
HTTP::Lite
HTTrack
Haansoft Haansoft
Havij
Heritrix Heritrix
Hloader Humanlinks
HybridBot IRLbot
IDBTE4M
Iblog Iblog
IlseBot IlseBot
IndeedBot
InfoTekies
InterGET
Iria Iria
JOC Web Spider IstellaBot
JamesBOT JamesBOT
Jbrofuzz
JennyBot
JetCar JetCar
JikeSpider
Joomla Joomla
JustView Jorgee
Kenjin Spider Kenjin Spider
LWP::Simple Keybot Translation-Search-Machine
Leap
LeechFTP
LexiBot
Lftp
Libwhisker Libwhisker
LieBaoFast
Lightspeedsystems Lightspeedsystems
Likse LinkWalker
LinkScan
Linkbot Linkbot
LinkextractorPro
LinkpadBot LinkpadBot
Lipperhey LinksManager
Lipperhey Spider LinqiaMetadataDownloaderBot
Litemage_walker LinqiaScrapeBot
Ltx71
MIIxpc MIIxpc
MJ12bot
MSFrontPage
Mag-Net Mag-Net
Magnet Majestic SEO
Majestic-SEO
MarkMonitor MarkMonitor
MarkWatch MarkWatch
Mata Hari MauiBot
Mb2345Browser
MeanPath Bot MeanPath Bot
Metauri Meanpathbot
Microsoft URL Control MegaIndex.ru
Minefield MicroMessenger
Moblie Safari Moblie Safari
Morfeus Fucking Scanner
Mozlila
Msrabot
NPbot
Nameprotect Nameprotect
NearSite Navroad
Needle
Nessus Nessus
NetZIP
Netcraft Netcraft
Nettrack
NextGenSearchBot NextGenSearchBot
Niki-bot Nibbler
Nimbostratus Nimbostratus
Nmap
Not
Nuclei
Octopus
Offline Explorer Offline Explorer
Offline Navigator Offline Navigator
OnCrawl OnCrawl
Openfind OpenVAS
OutclicksBot OrangeSpider
OutfoxBot
PECL::HTTP PECL::HTTP
PHPCrawl PageAnalyzer
PageGrabber PageScorer
Papa Foto
Pavuk Pavuk
PeoplePal Picscout
Pi-Monster Picsearch
Pimonster Pockey
Pixray ProWebWalker
PleaseCrawl PxBroker
ProPowerBot QueryN Metasearch
RankingBot RankFlex
RankurBot
Re-re Re-re
ReGet
RealDownload RealDownload
Reaper Reaper
Recorder
RepoMonkey RepoMonkey
Rogerbot
SBIder
SEOkicks SEOkicks
SEOlyticsCrawler SEOkicks-Robot
SISTRIX SMTBot
Screaming ScanAlert
ScreenerBot
SemanticJuice SemanticJuice
Semrush
SemrushBot
SentiBot
SeoSiteCheckup SeoSiteCheckup
Seomoz SeobilityBot
Siphon SiteSucker
Site Sucker Sitebeam
SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider
Siteimprove Siteimprove
Sitevigil
SlySearch SlySearch
SmartDownload Sosospider
Sociscraper SpankBot
Sogou web spider Spanner
Sottopop
SpaceBison
Spammen
Spinn3r Spinn3r
SputnikBot Sqlworm
Sqlmap
Sqworm
Steeler
Stripper
Sucuri
SuperBot SuperBot
SuperHTTP SuperHTTP
Suzuran Suzuran
T8Abot Swiftbot
TeleportPro
Telesoft
Telesphorep
The Intraformant The Intraformant
TheNomad
Thumbor Thumbor
Titan
Tracemyfile
Trendiction
Trendictionbot
Turingos
Turnitin Turnitin
Twice TwengaBot
Typhoeus Typhoeus
URLy.Warning URLy.Warning
Upflow VB Project
Vacuum VCI
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
Voil VidibleScraper
WASALive-Bot Virusdie
WBSearchBot Voltron
WPScan
WWW-Collector-E
WWW::Mechanize WWW::Mechanize
WWWOFFLE WeSEE
Wallpapers/3.0
WallpapersHD
Web Auto
Web Fetch
Web Pix Web Pix
Web Sucker Web Sauger
WebBandit WebBandit
WebCollage
WebFuck
WebGo IS WebGo IS
WebImageCollector WebImageCollector
WebLeacher WebLeacher
WebPix WebZIP
WebReaper Webshag
WebSauger WebsiteExtractor
WebStripper WebsiteQuester
WebmasterWorldForumBot
Whack Whack
Whatweb Whatweb
Who.is Bot
WinHTTrack
WiseGuys Robot
Wprecon Wprecon
Xaldon_WebSpider Xaldon WebSpider
Xenu
YoudaoBot
Zade
Zauba
Zeus Zeus
Zitebot Zitebot
ZoominfoBot
ZyBorg ZyBorg
arquivo-web-crawler adscanner
autoemailspider archive.org_bot
cah.io.community
clark-crawler clark-crawler
coccocbot cognitiveseo
crawler.feedback demandbase-bot
dataforseo.com domainsproject.org
evc-batch evc-batch
gopher
heritrix
internetVista monitor
iubenda-radar
linkdexbot linkdexbot
lwp-request lwp-request
lwp-trivial lwp-trivial
magpie-crawler
meanpathbot
mediawords mediawords
netEstate NE Crawler muhstik-scan
oBot pcBrowser
polaris version scalaj-http
probe-image-size scan.lol
satoristudio.net
seobility seobility
seostar seostar
sitechecker.pro serpstatbot
spyfu sexsearcher
trendiction.de sysscan
tAkeOut
voyagerx.com voyagerx.com
webgains-bot
webpros.com webpros.com
x09Mozilla
x22Mozilla
xpymep1.exe xpymep1.exe
zauba.io

280
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 80legs
404enemy
ADmantX ADmantX
AIBOT AIBOT
Aboundex Aboundexbot
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot
Alexibot Alexibot
AlphaBot AllSubmitter
Anarchy Alligator
Anarchie
Anarchy99
Ankit
Anthill Anthill
Aspiegel Apexoo
Attach
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BackDoorBot BBBike
BackStreet BLEXBot
Backlink-Ceck
Badass
Bandit
Barkrowler Barkrowler
Bigfoot
Black Hole Black Hole
BlackWidow Blackboard
Blow Bolt
BotALot
Brandprotect Brandprotect
Brandwatch Bullseye
Buck BunnySlippers
CATExplorador CATExplorador
CCBot CCBot
CODE87
CSHttp CSHttp
CensysInspect CazoodleBot
Cliqzbot Cegbfeieh
Cocolyzebot Citoid
Claritybot
Cogentbot Cogentbot
Collector Copier
CopyRightCheck CopyRightCheck
Copyscape Copyscape
Cosmos Cosmos
Craftbot Craftbot
Crescent Crawling at Home Project
Curious Custo
CyotekWebCopy DBLBot
DSearch DIIbot
DatabaseDriverMysqli
Digincore Digincore
DigitalPebble DigitalPebble
Discoverybot Dirbuster
Disco
DnyzBot DnyzBot
DomainAppender
DomainStatsBot DomainStatsBot
Download Wonder Download Wonder
Dragonfly Dragonfly
Drip
EMail Siphon
EMail Wolf EMail Wolf
Ecxi Ecxi
ExtLinksBot EirGrabber
EroCrawler
Exabot
Express WebPictures
Extractor Extractor
ExtractorPro ExtractorPro
Ezooms
FDM
Fimap
Firefox/7.0 Firefox/7.0
FlashGet
Freeuploader
Fuzz Fuzz
FyberSpider GermCrawler
Fyrebot
GT::WWW
GalaxyBot
Genieo
GetRight GetRight
GetWeb GetWeb
Getintent
Gigabot
Go-Ahead-Got-It
GrabNet
Grabber Grabber
GrapeFX
GridBot GridBot
HTTrack HEADMasterSEO
HTMLparser
HTTP::Lite
Haansoft Haansoft
HaosouSpider HaosouSpider
Harvest Harvest
Havij Havij
Heritrix
Hloader
HonoluluBot HonoluluBot
Humanlinks HybridBot
IDBTE4M IDBTE4M
IRLbot IRLbot
Iblog
IlseBot
Indy Library
InfoNaviRobot
InfoTekies InfoTekies
Intelliseek InternetSeer
InterGET Iskanie
Internet Ninja Jbrofuzz
JennyBot JikeSpider
JetCar
Jetty
Joomla Joomla
JustView JustView
Keybot Translation-Search-Machine
LNSpiderguy LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot
Larbin Larbin
LeechFTP LeechFTP
LexiBot
Lftp Lftp
Libwhisker
Lightspeedsystems Lightspeedsystems
LinkScan LinkScan
Linkbot
LinkpadBot LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
Ltx71 Ltx71
MFC_Tear_Sample
MIDown tool
MIIxpc MIIxpc
MTRobot MQQBrowser
Mag-Net MSIECrawler
Mail.RU_Bot Majestic SEO
MarkWatch Majestic-SEO
Majestic12
MarkMonitor
Mass Downloader
Masscan
Mata Hari Mata Hari
Mb2345Browser Microsoft Data Access
MeanPath Bot Microsoft URL Control
MegaIndex.ru
Minefield
Mister PiX
Moblie Safari Moblie Safari
Mr.4x3 Mozlila
Msrabot
Musobot Musobot
NearSite NPbot
Nameprotect
Navroad
Net Vampire Net Vampire
NetAnts NetZIP
Netcraft
Nettrack
Netvibes Netvibes
NextGenSearchBot
Nibbler Nibbler
NimbleCrawler NimbleCrawler
Ninja
Nuclei
Nutch Nutch
Octopus
Offline Explorer Offline Explorer
Openfind
Openvas Openvas
OrangeBot OrangeBot
OutfoxBot PECL::HTTP
POE-Component-Client-HTTP
Page Analyzer
Pandalytics Pandalytics
Papa Foto Papa Foto
Pavuk
PeoplePal PeoplePal
Petalbot Petalbot
Pi-Monster PictureFinder
Picscout Pixray
Pimonster PleaseCrawl
Pockey
ProWebWalker
Probethenet
Psbot Psbot
Pump
PyCurl PyCurl
Quick-Crawler QueryN Metasearch
RSSingBot RankingBot
Rankivabot Rankivabot
Re-re
ReGet ReGet
RealDownload
RebelMouse RebelMouse
Ripper Recorder
RocketCrawler SEOlyticsCrawler
SBIder
SEOkicks-Robot
SEOprofiler SEOprofiler
SEOstats Scrapy
Screaming Searchestate
ScreenerBot SemanticJuice
SeoSiteCheckup
SeobilityBot
Seomoz
SiteCheckerBotCrawler
SiteLockSpider SiteLockSpider
SiteSnagger
SiteSucker SiteSucker
Siteimprove Sitevigil
Snapbot SlySearch
Snoopy Snoopy
SocialRankIOBot Sottopop
Sociscraper Spammen
Sogou web spider SpankBot
Sosospider SputnikBot
Spanner Sqlworm
Steeler Sqworm
Stripper
Sucker Sucker
SuperBot SuperBot
SuperHTTP SuperHTTP
SurveyBot Teleport
Swiftbot Thumbor
T0PHackTeam
TeleportPro
Telesphorep
The Intraformant
TightTwatBot
Toata Toata
Toweyabot
Tracemyfile Tracemyfile
True_Robot
TurnitinBot TurnitinBot
URLy Warning TwengaBot
URLy.Warning
Upflow
VB Project VB Project
VCI
Vacuum Vacuum
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VoidEYE Virusdie
Voil Voil
Voltron WASALive-Bot
WBSearchBot WBSearchBot
WISENutbot
WPScan WPScan
WWW-Collector-E WWW-Collector-E
WWW-Mechanize WWW-Mechanize
WWW::Mechanize Wallpapers/3.0
Web Auto Web Auto
Web Fetch Web Collage
Web Fuck Web Sauger
Web Pix
Web Sucker
WebAuto
WebBandit WebBandit
WebCopier WebCollage
WebEnhancer
WebFetch
WebImageCollector WebImageCollector
WebPix WebWhacker
WebReaper WebmasterWorldForumBot
WebZIP Webshag
Website Quester Website Quester
WebsiteExtractor WebsiteExtractor
Whack
Widow
WinHTTrack WinHTTrack
WiseGuys Robot
Wonderbot Wonderbot
Woobot Woobot
Wotbox
Wprecon Wprecon
Xaldon WebSpider Zade
Xenu Zermelo
ZoomBot
ZoominfoBot ZoominfoBot
ZyBorg autoemailspider
clark-crawler
crawl.sogou.com crawl.sogou.com
eCatch gopher
magpie-crawler instabid
muhstik-scan internetVista monitor
linkdexbot
lwp-request
lwp-trivial
oBot
page scorer page scorer
plumanalytics plumanalytics
s1z.ru ripz
satoristudio.net
scalaj-http scalaj-http
seobility seobility
seoscanners seoscanners
seostar serpstatbot
sexsearcher spyfu
webmeup-crawler sysscan
webprosbot tAkeOut
x22Mozilla trendiction.de
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
2015god.org apiluckyleapnet-a.akamaihd.net
aa08daf7e13b6345e09e92f771507fa5f4.com design-lands.ru
gribokstop.com foto-telok.net
larose.jb2c.me getmiro.com
ros-ctm.ru kino-filmi.com
social-fun.ru law-check-eight.xyz
topsy.com m1media.net
vroze.com orsonet.ru
yoluxuryevents.com rasteniya-vs-zombi.ru
you-shall-not-pass.is74.ru soietvousmaime.fr

920
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com 1-99seo.com
10bet.com 14b.info
11pikachu.ru 1hwy.com
123locker.com 214jaluwobapef.cf
1kdailyprofit.me 273-fz.ru
1kinobig.ru 3w24.com
20pascals.nl 888.com
300richmond.co.nz aa14ab57a3339c4064bd9ae6fad7495b5f.com
4ur.click aa8780bb28a1de4eb5bff33c28a218a930.com
6128786.com aac500b7a15b2646968f6bd8c6305869d7.com
8lufu.com abbanreddy.com
aad1f4acb0a373420d9b0c4202d38d94fa.com abcdefh.xyz
acgs.tk abwa.tk
acunetix-referrer.com
addblueoff.com.ua addblueoff.com.ua
adktrailmap.com adman.se
admeasures.com admanaerofoil.com
admitad.com
adsfresh.men
adtology.com
adultactioncam.com
adultmeetonline.info
adultnet.in
advokateg.ru advokateg.ru
agecheckadult.com afora.ru
agroeconom.kz aibolita.com
aimayangzhi.com albumsuper.info
akama.com ald2014.org
alert-jdh.xyz alldownload.pw
alessandraleone.com alltheviews.com
alfa-medosmotr.ru alyeskaresort.com
allwomen.info amateurgalls.com
animal-drawings.com animali.deagostinipassion.it
anti-virus-removal.info apiitsthirteende-a.akamaihd.net
apilinkswiftco-a.akamaihd.net appfastplay.com
apparel-offer.com architecturebest.com
apple.com-webbrowsing-security.review arenda-avtoprokat-krasnodar.ru
apptester.tk arquapetrarca.info
ardimobileinfo.ml atlant-auto.info
arvut.org ausergrubhof.info
asrvrep-a.akamaihd.net autolombard-krasnodar.ru
astana.xxxkaz.org autoua.top
audiobangout.com aviav.co
aviapanda.ru aviav.ru
avto-oligarh.ru
avtocredit-legko.ru avtocredit-legko.ru
azbukadiets.ru avtointeres.ru
azbukafree.com balois.worldbreak.com
baditri.com bard-real.com.ua
bdsmgalls.net bbtec.net
beauty-bracelet.com belinka.com.ua
beauty-things.com beremenyashka.com
begalka.xyz best-seo-solution.com
belsetirehafi.tk bestofpicture.com
berdasovivan.ru betshuckclean.com
besthomemadepornsites.com bidbuy.co.kr
bestsexyblog.com bigcities.org
bet365.com binomo.kz
betterhdporn.com biographiya.com
betterscooter.com bloggedporn.com
bigpenisguide.com boazpower.com
blavia.00author.com bonky.biz
bloglag.com bracketsmackdown.com
bmwhighperformers.com bradleylive.xyz
boom-celebs.com brandbucket.com
brimstonehillfortress.org breastaugmentation.co.za
briomotor.co brianhenry.co
btnativenav.com brothers-smaller.ru
btvn.ru brus-vsem.ru
buckspar.gq bumble.cheapwebsitehoster.com
budpost.com.ua buqyxa.rincian.info
buntube.net buypuppies.ca
businesn.men buzzurl.jp
business-made-fun.com candcstuccoandstone.com
buy-cheap-online.info candiceloves.us
buyviagraa.blogspot.com cannibalcheerleader.com
candlewooddentalcentre.com.au capacitacionyaprendizaje.com
candycrushshop.com carabela.com.do
canrioloadm.gq
carapics.com carapics.com
cardul.ru cardsdumps.com
cartechnic.ru cartechnic.ru
cartierbraceletsreplica.pw cenokos.ru
cartierlove2u.xyz cheapjerseysap.com
casinorewards.info
cayado.snn.gr
cheapjerseysa.com
check-this-out-now.online
checktext.org checktext.org
chiblackhawks-jerseys.com chlooe.com
chimiver.info choosecuisine.com
chinese-amezon.com christianlouboutinoutlet.win
christianlouboutinreplica.win cleanallvirus.com
cibpenonptib.flu.cc
cicaki.net
ciceros.co
citysecurity.nu
civilwartheater.com
classicakuhni.ru
cleanersoft.com
clickso.com clickso.com
clicksor.net clksite.com
co.lumb.co codysbbq.com
coinsspb.com conboy.us
colehaanoutlet.store coolwallpapers4k.info
com-onlinesupport.host copro.pw
compliance-alex.top corta.co
compliance-fred.xyz cpajunkies.com
computernetworksonline.com crazyprotocol.com
constantaservice.net culmimg.pw
cookie-law-enforcement-ff.xyz cupday.com
cool-wedding.net danceuniverse.ru
counter.yadro.ru daptravel.com
country-chic.ru datadepths.com
covid-schutzmasken.de dentalpearls.com.au
cowlmash.com destinationrealestate.com
crazyboost.pro dev33.dioniqlabb.se
credit.co.ua diffbot.com
crystalslot.com digilander.libero.it
cscwtalkto.site dirtpics.pw
custom-electric-guitar.com discovertreasure-a.akamaihd.net
customsua.com.ua distver.ru
cv.wallhade.co dkvorota.ru
cyse.tk documentsite.net
ddpills.com dokumentalkino.net
dfwu1019.info domain-submit.info
dgfitness.co domaincdn.xyz
digitalfaq.com dostavimvdom.ru
diminishedvalueoforegon.com dotnetdotcom.org
dmmspy.com drunkenstepfather.com
docstoc.com dwomlink.info
dokfilms.net e-commerce-seo.com
domainroam.win
dostavka-v-ukrainu.ru
dotmass.top
doxysexy.com
draniki.org
dtm-spain.com
e-avon.ru
e-poker-2005.com
e2click.com e2click.com
earncash.com.ua eas-seo.com
easytuningshop.ru ekn-art.se
edge.sharethis.com el-nation.com
elektroniksigaraankara.info election.interferencer.ru
elmacho.xyz elektrischezi.canalblog.com
en.altezza.travel elektrozigarettekaufen1.devhub.com
elektrozigaretten2.yn.lt
elmifarhangi.com
emediate.eu
empathica.com
enbersoft.com enbersoft.com
enpolis.ru eralph.tk
enskedesquashclub.se
epngo.bz
eredijovon.com
eropho.com
eropornosex.ru
erotikstories.ru
es5.com
euroskat.ru
everypony.ru everypony.ru
exci.ru exchangeit.gq
eyessurgery.ru eyes-on-you.ga
ezigarettekaufen1.hpage.com ezigarettekaufen.myblog.de
ezigarettekaufen2.yolasite.com ezigarettenkaufen2.dreamwidth.org
f-loaded.de ezigaretteshop2.mywapblog.com
f012.de fastcrawl.com
falcoware.com fastfixing.tech
falllow.gq fbdownloader.com
fashionindeed.ml feorina.ru
ferrotodo.com fettefrauen.net
findpics.pw fickkontaktehobbyhuren.com
findsexguide.com fireads.men
findthe.pet fm-upgrade.ru
fingerprintjs.com foodgid.net
fivedwld.cf forum20.smailik.org
flprog.com foxweber.com
food.dtu.dk freelinkbuilding.website.tk
foodcrafts.website frivgame250.com
forex-indextop20.ru fuel-gas.com
forminecrafters.ru funnymama.com
forms-mtm.ru fx-brokers-review.com
formulaantiuban.com fxtips.ru
forsex.info fz139.ttk.ru
forum.tvmir.org gagrasector.ru
foto-telok.net gallictures.com
foxydeal.com games.kolossale.ru
franecki.net gembird.com
frequiry.com gentamicineyedrops.blogspot.com
frustrated-favorable.gq get-free-traffic-now.com
fuckbuddybestgilf.info getlamborghini.ga
fungamelands.com gheus.altervista.org
futbolkisales.ru golden-catalog.pro
gamebackyard.com goodwriterssales.com
gazobeton-p.com.ua googlepositions.com
ge0ip.org grand-chlen.ru
geileweiber.tk green-tea.tv
ghostvisitor.com growshop.es
gk-atlant.info guidefs.ru
golmau.host.sk hamilton.ca
growmyfunds.ca hanwei.us
guigyverpo.cf hausfrauensex18.com
guod.me hdhc.ru
hamptonoaks.ca
hdpixion.com
healgastro.com healgastro.com
health-medical-portal.info help.tpu.ru
heartofbeijing.blogspot.com heygidday.biz
helicalpile.us hidemyass.com
hentaiheroes.com hledejvshopech.cz
hesteel.pl hobbyhuren24.net
highspeed5.net home.myplaycity.com
hit-men.men
hmmm.cz
homemade.gq
hop.clickbank.net
hornymatches.com hornymatches.com
hotel-mkad.ru host-tracker.com
hotloans.ru hoverboard360.es
hotsocialz.com howtoclean.club
hptwaakw.blog.fc2.com hyhj.info
hspline.com hystersister.com
ideashome.id ideashome.id
ilikevitaly.com imabase.com
imgarcade.com imitex-plus.ru
imguramx.pw immigrational.info
imperia31.ru incep.imagine-ex.co
infogame.name integritylandscapeservices.com
insomniagamingfestival.com
intellektmedia.at
interior-stickers.ru interior-stickers.ru
invitefashion.com internet-apteka.ru
invivo.hu iptool.xyz
iqoption.pro irkutsk.zrus.org
ixora.pro ispaniya-costa-blanca.ru
izismile.com ix20.ru
jav-fetish.site j-times.ru
javatex.co.id jackpotchances.com
jerseywholesalebizchina.com japfm.com
jetsli.de javrip.net
johnrobertsoninc.com javspace.net
joyceblog.top jerseychinabizwholesale.com
jumptap.com jerseychinabizwholesale.us
justucalling32211123456789.tk jerseywholesaleelitestore.com
karusel-market.ru jewelryandfiligree.com
kazan.zrus.org journeydownthescale.info
keenoutlet.online jwss.cc
kinky-fetishes.com kaac.ru
kinocccp.net kalandranis.gr
kinoduh.ru kamen-e.ru
konoplisemena.com kasino-money.pw
koptims.tiu.ru keki.info
krasivoe-hd.net keywordblocks.com
kinofak.net
kinotorka.ru
kinozapas.com
kletkimehan.ru
konica.kz
konkursowo-24.pl
kostenloser-sex.com
kotaku.com
krafte.ru
krassh.ru krassh.ru
krasula.pp.ua kupiproday.com.ua
lakomka.com.ua kursy-ege.ru
laptoper.net lacasamorett.com
lasvegaslockandsafe.com laexotic.com
laudit.ru lafriore.ru
laulini.soclog.se landmania.ru
ledis.top lechenie-gemorroya.com
li-er.ru
lineshops.biz lineshops.biz
liveinternet.ru link.ac
localmatchbook.com live-sexcam.tk
lookingglassemb.com live-sexchat.ru
lotto6888.com livetsomudvekslingstudent.bloggersdelight.dk
ltvperf.com loftdigital.eu
lumb.co lollypopgaming.com
lyrster.com lonely-mature.com
m0r0zk0-krava.ru louboutinshoes.xyz
macrotek.ru lowephotos.info
madot.onlinewebshop.net lrac.info
magazin-pics.ru macdamaged.tech
magnetpress.sk mahnwachen-helfen.info
mainhp.net manimpotence.com
make-money-online.com marketingvici.com
mararoom.ru masserect.com
marcoislandvacations.net master-muznachas.ru
marcowebonyodziezowe.pl masterseek.com
marmitaco.cat mastershef.club
mbiologi.ru maximpartnerspr.com
meble-bogart.info mcar.in.ua
medi-fitt.hu me-ke.com
megahdporno.net med-zdorovie.com.ua
metallosajding.ru medicine-4u.org
metaxalonevstizanidine.blogspot.com meduza-consult.ru
minharevisao.com megavolt.net.ua
misswell.net meget.co.za
mixx.com member-quiz.com
mmofreegames.online midst.eu
moesonce.com minet.club
monetizer.com-01.site mirmedinfo.ru
morlat.altervista.org missis.top
movies-in-theaters.net misterjtbarbers.com
msk-diplomat.com mlvc4zzw.space
mmostrike.ru
modenamebel.ru
monclercoats.xyz
moncleronline.xyz
moncleroutletonline.xyz
moneyzzz.ru
moonci.ru
mrlmedia.net
mturkcontent.com
musicdaddy.net
mustat.com mustat.com
myyour.eu mypornfree.ru
naval.jislaaik.com myseoconsultant.com
nelc.edu.eg najaden.no
nalogovyy-kodeks.ru
naperehresti.info
nardulan.com
nationalbreakdown.com
nero-us.com nero-us.com
nfljerseyscheapbiz.us nfljerseyscheapbiz.us
nippon-bearings.ru noscrapleftbehind.co
noumeda.com notebook-pro.ru
npoet.ru nuker.com
nuit-artisanale.com nullrefer.com
obiavo.by o-o-6-o-o.ru
ochistka-stokov.ru o-o-8-o-o.com
offtopic.biz o.light.d0t.ru
onlinemeetingnow.com old-rock.com
onlinetvseries.me one-gear.com
ooo-olni.ru online-podarki.com
opinionreelle.com online247.ml
otbelivanie-zubov.com onlyporno.ru
outshop.ru ontargetseo.us
oynat.info oren-cats.ru
paccohichetoti.ml outclicks.net
paclitor.com pandarastore.top
pathwhelp.org parajumpersstore.online
paydayonlinecom.com partner-cdn.men
perso.wanadoo.es partner-pop.men
pestomou.info pattersonsweb.com
photochki.com pc-services.ru
physfunc.ru pechikamini.ru
piccdata.com perederni.net
picturesfrom.com php-market.ru
picturesify.com picphotos.net
pinsdaddy.com picscout.com
pinup-casino1.ru pictures-and-images.net
pixell.club pila.pl
pizza-imperia.com piski.top
plugingeorgia.com pivka.xyz
pochemychka.net pix24x7.com
podshipniki-nsk.ru pl.aasoldes.fr
podshipniki-ntn.ru platezhka.net
poligon.com play-movie.pl
pornosok.ru play.leadzupc.com
pornstartits.xblog.in poems.com.ua
pornzone.tv pogruztehnik.ru
preconnubial.usuby.site pojdelo.weebly.com
printdirectforless.com pokemon-go-play.online
pro-okis.ru polska-poezja.com
popserve.adscpm.net
pornoklad.net
pozdrawleniya.com
pr0fit-b0x.com
priscilarodrigues.com.br
prize44.com
prodess.ru prodess.ru
prointer.net.ua professionaldieselcare.com
prostitutki-odessa.org pronekut.com
purchasepillsnorx.com propranolol40mg.blogspot.com
pussysaga.com prostitutki-almata.org
qor360.com prostitutki-tolyatti.org
rangoman.date proxyradar.com
rank-checker.online putitin.me
realizmobi.com pyramidlitho.webs.com
rebelmouse.com rankia.com
reelheroes.net rapevideosmovies.com
registrationdomainsite.com recinziireale.com
respublica-otel.ru regdefense.com
restorator-msk.ru responsive-test.net
reversing.cc richinvestmonitor.com
rewardpoll.com ritlweb.com
reyel1985.webnode.fr rn-to-bsn.com
rospromtest.ru robot-forex.biz
runtnc.net rossmark.ru
rutor.vip ru-mediaget.ru
rvi.biz sa-rewards.co.za
sad-torg.com.ua sad-torg.com.ua
saddiechoua.com saddiechoua.com
santasgift.ml sady-urala.ru
scmor.ilxc.cc saitevpatorii.com
sdi-pme.com salut-camp.ru
senger.atspace.co.uk samolet.fr
serptehnika.ru saneyes.com
sape.top
saratov.xrus.org
savetubevideo.com
scanner-george.top
scanner-jess.top
scanner-josh.top
scanner-walter.top
scanner-willy.top
scanspyware.net
scrapy.org
sdelatmebel.ru
seasonvar.ru
seoboxes.com
service-core.ru
servingnotice.com
sethrollins.net
sex-watch.com
sexblog.pw
sexkvartal.com
sexreliz.com sexreliz.com
sexs-foto.com
sexstream.pl
sextracker.be
sexvideo-sex.com sexvideo-sex.com
sexxdate.net shama-rc.net
sherlock.se sharebutton.org
ship-marvel.co.ua sheerseo.com
shopfishing.com.ua shemale-sex.net
shymkent.xkaz.org shitting.pro
silvermature.net simpoed.ufop.br
simple-share-buttons.com sizeplus.work
simplepooltips.com skyway24.ru
sisiynas.ru slkrm.ru
sixcooler.de sloopyjoes.com
slonechka.ru small-game.com
sluganarodu.ru snow.nvr163.com
slujbauborki.ru
smadihome.com
smokewithrabbits.com
sms2x2.ru
smsactivator.ru
snomer1.ru
snowplanes.com
soaksoak.ru
soblaznu.net
socas.pluto.ro
social-buttons.xyz
social-s-hhh.xyz
socialbuttons.xyz
soda.media
softlinesolutions.me softlinesolutions.me
space2019.top softomix.net
space4updating.win spabali.org
ssconstruction.co spywarenuker.com
stackthatbucks.com stairliftstrue.com
startwp.org stathat.com
staticfs.host stjamesschool.info
streetfooduncovered.com sumo.com
streha-metalko.si sunflowerdrawingpaintings.blogspot.com
stroicol.net
stroyhelp-dv.ru
stroyplus.ru
subj.ukr-lit.com
surffoundation.nl
svetlotorg.ru
szamponrevita.pl szamponrevita.pl
taaaak.com
tamada69.com tamada69.com
teesdaleflyballclub.co.uk taraz.xkaz.org
tasteidea.com
teguh.info
tehngr.ru tehngr.ru
the-trader.net telegraf.by
thebluenoodle.com thedownloadfromwarez.blogspot.com
thelottosecrets.com thelottosecrets.com
thepantonpractice.co.uk timer4web.com
theprofitsmaker.net titan-ads.life
thetardistimes.ovh tizanidine4mgstreetprice.blogspot.com
titslove.yopoint.in
tizanidinecost.blogspot.com tizanidinecost.blogspot.com
tizanidinedosageforsleep.blogspot.com tizanidinedosageforsleep.blogspot.com
tksn.ru tizanidineduringpregnancy.blogspot.com
toon-families.com tizanidinehcl4mgtabinfo.blogspot.com
toondinsey.com tizanidineonline.blogspot.com
topsiteminecraft.com tn811.us
torrentgamer.net toloka.hurtom.com
torrents.cd tomck.com
tour-line.net tonivedu.it
toursmaps.com tootoo.to
traffictrade.life topappspro.com
tri-slona.org toys.erolove.in
triumf-realty.ru trackmedia101.com
trustaffs.com trafficfactory.biz
tryrating.com trafficnetzwerk.de
tula.howotorg.ru tsyndicate.com
tvand.ru turbodsp.com
tversvet.ru tversvet.ru
uggsale.online twinderbella.com
ugogo.info tyumen.xrus.org
ukr-lit.com uchetunet.su
ultimateclassicrock.com ugguk.online
unece.org uk-zheu20.ru
unpredictable.ga urengoy.pro
uprour.com
uptime-delta.net
ustion.ru ustion.ru
utrolive.ru vbikse.com
uzporno.mobi vclicks.net
v24s.net vedomstvo.net
v720hd.ru versaut.xxx-cam.webcam
vacuumscleaner.com viagengrarx.com
validdomain.xyz
vandrie-ict.nl
vashsvet.com
vduplo.ru
venta-prom.ru
veopornogratis.xxx
via-energy-order.com
videochat.ph videochat.ph
videositename.com videochat.tv.br
vikistars.com videooko.weebly.com
vilingstore.net videtubs.pl
vip2ch.com viel.su
vip-dom.in
viven.host.sk
viveresaniesnelli.it viveresaniesnelli.it
vk-mus.ru
vkontarkte.com vkontarkte.com
vladhistory.com vtc.pw
vodkoved.ru vykupavto-krasnodar.ru
waterefficiency.co vysigy.su
waterpurifier.club watchinf.com
watsonrealtycorp.com web-betting.ru
we-are-gamers.com welck.octopis.com
webinstantservice.com westermarkanjou.se
webix.me wieseversa.no
webpromotion.ae wikes.20fr.com
website-datenbank.de withstandingheartwarming.com
website-speed-up.site wordpresscore.com
websiteaccountant.de
wechatdownload10.com
weprik.ru
wetgames.ru
whatsappbot.flyland.ru
white-truck.life
whiteproduct.com
wildnatureimages.com
wineration.com
worldhistory.biz
woweb.com.ua woweb.com.ua
ww2awards.info www.jbetting.com
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai www.timer4web.com
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai wygraj-teraz.com
xblognetwork.com
xdoza.com
xmlinde.com
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xn----7sbahjd3btneuw1joc.xn--p1ai
xn----9sbebi2bvzr7h.xn--p1ai
xn--24-glceagatoq7c2a6ioc.xn--p1ai
xn--80aanaardaperhcem4a6i.com xn--80aanaardaperhcem4a6i.com
xn--80aikhbrhr.net xuki.us
xnxx-n.com xxart.ru
yachts-cruise.info yachts-cruise.info
yaoguangdj.com
yellowstonevisitortours.com yellowstonevisitortours.com
yoshkarola.zrus.org youandcredit.ru
za-music.mymobiplanet.com yoursite.com
zaloro.com youtubologia.it
zarajbuilders.com yurgorod.ru
zebradudka.com zaimhelp.ru
zverokruh-shop.cz zajm-zalog-krasnodar.ru
zvezdagedon.ru zatjmuzu.info
zx6.ru zdesformula.ru
zdesoboi.com
zeroredirect5.com
zeroredirect9.com
zoomovies.org
zootoplist.com

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
0daymusic.org 123locker.com
1-99seo.com 1688.com
11pikachu.ru 4k-player.pl
1314dh.com 4pp13.com
1xbet4you.com 4senses.co
24videos.tv 57883.net
2itech.ru above.com
2rich4bitches.com abusora.com
6hopping.com adclickservice.com
7minuteworkout.com adelly.bg
DomainStatsBot adultgalls.com
academiacsmendoza.org
aerogo.com
afftrack001.com afftrack001.com
amazingpic.net afslankpillen2017nl.eu
an-donut.com alessandraleone.com
android4fun.org alfa-img.com
apiappenableinfo-a.akamaihd.net alienwheels.de
apimountainbikei-a.akamaihd.net all4invest.info
apiqualitinknet-a.akamaihd.net allnews24.in
appmsr.org aloofly.com
analnoeporno.tv
anaseracresar.tk
animalrank.com
apiluckyleapnet-a.akamaihd.net
apishops.ru
app-ready.xyz
aramaicmedia.org aramaicmedia.org
artdestshop.eu aviav.ru.com
asdfz.pro avramstroy.ru
asrv-a.akamoihd.net bbtec.net
autoua.top bcmp.org
avon-novosib.ru bear.gotcher.us
beatifullandscape.co best-seo-offer.com
bestmobilityscooterstoday.com bestbrainprod.win
bestofferswalkmydogouteveryday.gq bg6s0.com
besttorrentknifta.weebly.com bildsuche.ru
bidbuy.co.kr boazpower.com
bitporno.sx bobba.dzaba.com
bonkers.name bobinoz.com
breastaugmentation.co.za braip.com.br
bronzeaid-a.akamaihd.net brillianty.info
bus-offer.com business-made-fun.com
buybest1.biz cakesplus.com.au
buyparajumpers.online candypeople.se
buypillsorderonline.com cascadelink.org
buyscabiescream.com cercacamion.it
cbox.ws cete.ru
cheapbelstaff.online cheapwebsitehoster.com
cherubinimobili.it chinajerseyswholesalecoupons.com
children-learningreading.info choosecuisine.com
cleanmypc.com cineacademy.ru
compliance-george.xyz cleanersoft.com
compliance-mark.top club-samodelkin.ru
coolwallpapers4k.info commoncrawl.org
covetnica.com compliance-alex.xyz
dailyrank.net compliance-ivan.top
datract.com continent-e.tv
demenageur.com cougfan.info
detective01.ru countbertwistdisp26.soup.io
digital-sale.su cupday.com
dikx.gdn custom-product-labels.com
dkvorota.ru db.speedup.gdn
doctormakarova.ru demon-tweeks.com
domainroam.win detsky-nabytek.info
dostavimvdom.ru dvervmoskvu.ru
doublepimp.com ekspresihati.info
download-of-the-warez.blogspot.com election.interferencer.ru
drillsaw.ru elektrischezi.canalblog.com
drugstoreforyou.com emergencyneeds.org
eaplay.ru erotikstories.ru
easyshoppermac.com este-line.com.ua
eets.net europages.com.ru
efkt.jp ezigarettekaufen2.mpbloggar.se
ekaterinburg.xrus.org fable.in.ua
elidelcream.weebly.com farm26.ru
emailaccountlogin.co fastfixing.tech
empis.magix.net feorina.ru
errorfixing.space ferretsoft.com
essay-edu.biz filmci.pro
etenininrade.ga fodelsedagspresenter.nu
eyelike.com.ua free-today.com
f1nder.org fullfileaccess.com
facebook-mobile.xyz gallerylisting.com
fakehandbags.xyz geckoandfly.com
fashion.stellaconstance.co gigixo.com
fetroshok.ru go2jump.org
finstroy.net gratis-sexkontakte.com
flauntyoursite.com guardlink.org
flirt4free.com guidefs.ru
forensicpsychiatry.ru heygidday.biz
foxjuegos.com historichometeam.com
free-share-buttons.com hkdiiohi.skyrock.com
freshberry.com.ua homik.pw
funtoonez.com hotdl.in
gagrasector.ru hotelcrocenzi.sm
gameonasia.com hotenergy.ru
gamewrath.com howopen.ru
gateway.zscalertwo.net hubbble.com
ghazel.ru i-service.kz
global-ics.co.za imperia31.ru
gomusix.com incanto.in.ua
hatedriveapart.com informatiecentro.be
hawaiielectriclight.com intelhdgraphicsgtdrive6w.metroblog.com
helicalpile.us interferencer.ru
hikesearch.net interfucks.net
howmuchdoestizanidinecost.blogspot.com iqoption-bin.com
hplaserjetpdriver8y.pen.io isitpaleo.info
hystersister.com islandminingsupply.wordpress.com
ideawheel.com it-max.com.ua
ideoworld.org japfm.com
impisr.ru
increasewwwtraffic.info
infodocsportal.com
investpamm.ru
ipchicken.com
iqbazar.ru
izoll.ru
javrip.net javrip.net
jpcycles.com javstock.com
kakablog.net justprofit.xyz
kamorel.com kalandranis.gr
kenaba.su keyhantercume.com
ketoanhanoi.info keywords-monitoring-success.com
keywordbasket.com kino-rating.ru
keywordteam.net kinogonew.ru
kimcurlrvsms.com komputernaya-pomosh-moscow.ru
kinoplen.ru kskjco.club
konturkrasoty.ru kupiproday.com.ua
kosynka-games.ru kuzinsp.ru
krasnodar.zrus.org kwzf.net
lafourmiliaire.com larchik.net
lampokrat.ws law-check-seven.xyz
law-check-nine.xyz link.ac
lietaer.com luckyshop.net.ua
lion.conboy.us mainlinehobby.net
liveu.infoteka.hu marinetraffic.com
louisvuittonoutletstore.net mastershef.club
m5home.ru matchpal-a.akamaihd.net
marcowebonyodziezowe.pl meccadumps.net
maximilitary.ru missis.top
mediawhirl.net mmostrike.ru
mega-polis.biz.ua money-for-placing-articles.com
mikrobiologies.ru moneymaster.ru
mir-business-24.ru morlat.altervista.org
mojpregled.com mountainstream.ms
moncleronline.xyz mrinsidesales.com
moncleroutletonline.win mrmoneymustache.com
mp3s.club mrsdalloways.com
multgo.ru musicdaddy.net
myhydros.org musicpro.monster
myonigroup.com mxgetcode.com
myshopmatemac.com mydownloadengine.com
najaden.no myliveblog.ru
narkologiya-orel.ru myphotopipe.com
news-speaker.com mypornfree.ru
niki-mlt.ru nfljerseysforsalewholesaler.com
nordstar.pro nofreezingmac.click
notfastfood.ru nomuos.it
onefilms.net norththeface.store
onlyporno.ru ntic.fr
oops-cinema.ru o-o-11-o-o.com
optom-deshevo.ru o-o-6-o-o.com
ostrovtaxi.ru oakleyglassesonline.us
pc-virus-d0l92j2.pw obiavo.in
pepperstyle.ru openlibrary.org
perfectpracticeweb.com paparazzistudios.com.au
photostudiolightings.com par-fallen.ga
picscout.com partner-trustworthy.men
pixgood.com pastaleads.com
platezhka.net poiskzakona.ru
playfortuna-play.ru polska-poezja.com
poems.com.ua pourvous.info
pojdelo.weebly.com preventheadacheguide.info
pokemongooo.ml prize44.com
polimga.pw promgirldresses.xyz
pons-presse.com purchasepillsnorx.com
pony-business.com putitin.me
porndroids.com
privacylocationforloc.com
privatamateure.com
prizefestival.mobi
professionaldieselcare.com
prombudpostach.com.ua
prostitutki-novgoroda.org
puzzleweb.ru
qpypcx.com qpypcx.com
rangoman.date quickbuck.com
rewardit.com ragecash.com
rozalli.com rankscanner.com
ruscoininvest.company rarbg.to
russkoe-zdorovie.ru rating-bestcasino.com
savetubevideo.com realresultslist.com
scanner-george.top remont-komputerov-notebook.ru
scanner-julia.top rennlist.com
scrapinghub.com romhacking.ru
secretscook.ru roznica.com.ua
seofirmreviewsus.info rvi.biz
seorank.info sarahmilne.top
serw.clicksor.com savefrom.com
sethrollins.net scanner-brian.top
sexsaoy.com scanner-julianna.top
sexystrippe.info scanner-mark.top
shisha-swag.de scanner-willy.top
sibtest.ru seansonline24.pl
sinestesia.host.sk seasaltwithfood.com
soft-terminal.ru sildenafilcitratemed.com
soietvousmaime.fr sims-sims.ru
spabali.org slkrm.ru
sport-video-obzor.ru slomm.ru
slonechka.ru
smailik.org
small-games.biz
smart-balancewheel.com
smstraf.ru
softomix.net
sohoindia.net
sport7777.net
spywarespy.com spywarespy.com
staff.prairiesouth.ca startufa.ru
stardevine.com staticfs.host
studiokamyk.com.pl steelmaster.lv
subj.ukr-lit.com stillmiracle.com
theporndude.com stroiminsk.com
tizanidineshowupondrugtest.blogspot.com svensk-poesi.com
tizanidinesideeffectsweightloss.blogspot.com swimpool.ca
tnctrx.com swinger-mobil.net
torrents-tracker.com takeflyte.com
takeprofitsystem.com
takethatad.com
tandvardshuset.net
teesdaleflyballclub.co.uk
telegramdownload10.com
telemetryverification.net
tengohydar.tk
thefarmergame.com
themeforest.net
thetattoohut.com
tizanidinehcl4mgtab.blogspot.com
tizanidinehydrochloride2mg.blogspot.com
topanasex.com
topbestgames.com
toys.erolove.in toys.erolove.in
trackzapper.com ucsol.ru
trafficnetzwerk.de uptime-as.net
truck-addzilla.life utrolive.ru
tzritel.tk vbikse.com
ugguk.online vduplo.ru
usacasino.com venerologiya.com
vavilone.com via-energy-cumpara.com
vesnatehno.com videochat.mx
videochat.ph vintontech.info
videooko.weebly.com visionwell.com.cn
videtubs.pl visitcambridge.org
vtc.pw vsesubwaysurfers.com
vysigy.su vtcdns.com
walkme.com webextract.profound.net
walpaperlist.com womensterritory.ru
website-stealer.nufaq.com worldtraveler.world
websiteaccountant.de writingservices17.blogspot.ru
woman-orgasm.ru www.888.com
wygraj-skiny.win xkaz.org
xbaboon.com yaaknaa.info
xclicks.net yellowstonesafaritours.com
xcombear.ru yhit.press
xn----7sbifcamovvfggw9d.xn--p1ai
xn--80aodinpgi.xn--p1ai
xxxnatelefon.ru
yoopsie.com
yoursearch.me
yoursite.com yoursite.com
yuarra.pluto.ro zarepta.com
yun56.co
zambini.ru
zazagames.org
zdesformula.ru
zeroredirect11.com
zooggames.com

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.71 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3028 ### Version: V4.2022.04.3029
### Updated: Thu Apr 7 07:12:39 UTC 2022 ### Updated: Thu Apr 7 10:03:28 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3028 ### Version: V4.2022.04.3029
### Updated: Thu Apr 7 07:12:39 UTC 2022 ### Updated: Thu Apr 7 10:03:28 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##