vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.10.2663 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-10-20 10:18:14 +00:00
parent 0f95ebc3a1
commit 2eff051c69
47 changed files with 71575 additions and 71565 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2662 #### Version: V4.2021.10.2663
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2662 #### Version: V4.2021.10.2663
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.10.2662 #### Version: V4.2021.10.2663
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8810
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.10.2662 #### Version: V4.2021.10.2663
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8816
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8814
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8816
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8814
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.169 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.169 0;
127.0.0.1 0; 127.0.0.1 0;

8814
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.169 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.169 0;
127.0.0.1 0; 127.0.0.1 0;

8814
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.169 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.169 0;
127.0.0.1 0; 127.0.0.1 0;

8814
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2662 2663

8814
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8816
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.92.117.253 0; 40.121.59.79 0;

8816
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.169 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.169 0;
127.0.0.1 0; 127.0.0.1 0;

8816
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.169 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.169 0;
127.0.0.1 0; 127.0.0.1 0;

8816
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.169 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.169 0;
127.0.0.1 0; 127.0.0.1 0;

8816
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.92.117.253 0; 40.121.59.79 0;

8816
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.92.117.253 0; 40.121.59.79 0;

8816
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.169 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Download Wonder Anarchy99
GT::WWW BackStreet
Magnet Blackboard
Nessus CrazyWebCrawler
Openfind GalaxyBot
RankingBot2 JennyBot
RebelMouse PxBroker
SeobilityBot Surfbot
Tracemyfile WebImageCollector
Upflow x22Mozilla

286
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404checker
404enemy 404enemy
80legs ADmantX
AIBOT
ALittle Client ALittle Client
Abonti Aboundex
Aboundexbot AfD-Verbotsverfahren
Acunetix
AhrefsBot AhrefsBot
AiHitBot
Aipbot Aipbot
Alexibot
AllSubmitter
Anarchie Anarchie
Anarchy
Anarchy99 Anarchy99
Aspiegel Apexoo
AwarioRssBot Asterias
BBBike AwarioSmartBot
BDFetch BLEXBot
BackWeb BackDoorBot
BacklinkCrawler Backlink-Ceck
Bandit Bandit
BetaBot BatchFTP
Bigfoot Battleztar Bazinga
Bitacle Black Hole
Blackboard Blackboard
BlowFish Blow
Boardreader Buck
Bolt BuiltBotTough
Brandwatch BuiltWith
Buddy
Bullseye
CATExplorador CATExplorador
Calculon CCBot
CODE87
CSHttp
CazoodleBot
CherryPicker CherryPicker
ChinaClaw
Chlooe Chlooe
Claritybot
Cloud mapping Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot Cogentbot
Collector CopyRightCheck
Copier Cosmos
Craftbot Craftbot
CrazyWebCrawler
CrunchBot CrunchBot
Custo
CyotekWebCopy CyotekWebCopy
DIIbot DTS Agent
DSearch Demon
DataCha0s DigitalPebble
DatabaseDriverMysqli Disco
Deusu Dispatch
Discobot
DittoSpyder
DomainAppender DomainAppender
DomainCrawler
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project
Dotbot
Drip Drip
ECCP/1.0
EMail Siphon EMail Siphon
EMail Wolf EMail Wolf
EirGrabber EasyDL
Evil Ebingbong
Exabot EroCrawler
ExtLinksBot
EyeNetIE
FHscan FHscan
FemtosearchBot Firefox/7.0
Foobot Freeuploader
FrontPage FrontPage
Genieo Fuzz
GermCrawler GalaxyBot
GetWeb
Getintent
Go!Zilla Go!Zilla
Go-Ahead-Got-It Gotit
Grabber Grabber
GrapeFX
GridBot GridBot
HMView
HTTP::Lite HTTP::Lite
HTTrack Hloader
Haansoft
HaosouSpider
HybridBot
IDBot IDBot
IRLbot
Iblog Iblog
Id-search
IlseBot IlseBot
Image Fetch Image Sucker
IndeedBot IndeedBot
Indy Library InfoNaviRobot
Intelliseek
InterGET InterGET
Iria Iskanie
JamesBOT JOC Web Spider
Keyword Density Jbrofuzz
JetCar
Joomla
Kinza
LNSpiderguy LNSpiderguy
Leap LieBaoFast
LibWeb
Libwhisker
Likse
LinkScan LinkScan
LinkWalker
Linkbot Linkbot
Linkdexbot Linkdexbot
LinksManager LinqiaRSSBot
LinqiaScrapeBot MQQBrowser
Lipperhey Spider
Lmspider
MFC_Tear_Sample
MJ12bot
MSFrontPage MSFrontPage
MSIECrawler MTRobot
Magnet
Majestic SEO Majestic SEO
Majestic12 Majestic12
MarkMonitor MarkMonitor
Mass Downloader MarkWatch
Mata Hari Masscan
MauiBot
MeanPath Bot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri Metauri
MicroMessenger
Mister PiX Mister PiX
Moblie Safari
Mojeek Mojeek
Mojolicious Mojolicious
Mozlila
Mr.4x3 Mr.4x3
NPbot
Name Intelligence Name Intelligence
NetAnts NearSite
NetZIP Net Vampire
Netcraft NetLyzer
NetSpider
Netvibes Netvibes
NextGenSearchBot
Niki-bot Niki-bot
Nikto Nikto
Ninja Ninja
Octopus Nmap
Offline Navigator Offline Explorer
OnCrawl OpenLinkProfiler
OpenVAS OpenVAS
OrangeSpider Openvas
PHPCrawl OrangeBot
POE-Component-Client-HTTP OutfoxBot
Page Analyzer PageAnalyzer
PageGrabber Papa Foto
Pandalytics Pavuk
PeoplePal PeoplePal
Pi-Monster Petalbot
Pixray PictureFinder
Pimonster
ProWebWalker
Pump
PyCurl PyCurl
QueryN Metasearch QueryN Metasearch
RSSingBot RankActiveLinkBot
RankFlex RankFlex
RankingBot2
Rankivabot Rankivabot
ReGet RankurBot
RealDownload Re-re
Reaper Reaper
Recorder
RedesScrapy RedesScrapy
RepoMonkey SBIder
Ripper
Rogerbot
SEOkicks-Robot SEOkicks-Robot
SEOstats SEOlyticsCrawler
Scanbot Scanbot
Scrapy
Screaming
ScrepyBot
SearchmetricsBot
Semrush
SeoSiteCheckup
SeobilityBot
Seomoz Seomoz
Shodan
Siphon Siphon
SiteExplorer
SiteSnagger
SiteSucker SiteSucker
Sitebeam Sitebeam
Siteimprove SlySearch
SmartDownload
Snake Snake
Snapbot
Snoopy Snoopy
Sogou web spider
Sosospider Sosospider
Sottopop Sottopop
SpaceBison Spanner
Spammen
Spinn3r
SputnikBot SputnikBot
Sqlworm Sqlworm
Sqworm Sqworm
Steeler Stripper
Sucker
Suzuran
TeleportPro
Telesoft Telesoft
Telesphoreo
Telesphorep
The Intraformant The Intraformant
Thumbor
Toweyabot
Turingos
TwengaBot TwengaBot
Twice
Typhoeus Typhoeus
URLy.Warning UnisterBot
VB Project V-BOT
Vagabondo VelenPublicWebCrawler
VidibleScraper VoidEYE
Virusdie Voil
Voltron Voltron
WASALive-Bot WEBDAV
WISENutbot
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE
Wallpapers/3.0
Web Collage
Web Fuck
Web Pix Web Pix
WebBandit Web Sucker
WebCopier WebAuto
WebEnhancer WebEnhancer
WebFetch
WebFuck WebFuck
WebGo IS WebSauger
WebImageCollector WebWhacker
WebLeacher WebmasterWorldForumBot
WebPix
WebReaper
WebsiteExtractor WebsiteExtractor
WebsiteQuester
Webster Webster
Whack Whack
Whatweb Whatweb
Widow
WiseGuys Robot WiseGuys Robot
Wprecon Wprecon
Xaldon_WebSpider Xaldon_WebSpider
Zauba
Zermelo
Zeus Zeus
Zitebot
ZmEu ZmEu
ZoomBot
ZoominfoBot ZoominfoBot
ZumBot ZumBot
ZyBorg ZyBorg
archive.org_bot archive.org_bot
arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider backlink-check
check1.exe check1.exe
com.plumanalytics crawler.feedback
crawl.sogou.com crawler4j
dataforseo.com
evc-batch evc-batch
gopher lwp-request
instabid
meanpathbot meanpathbot
oBot muhstik-scan
page scorer page scorer
probe-image-size
ripz ripz
s1z.ru
scalaj-http scalaj-http
seobility sitechecker.pro
serpstatbot siteripz
sp_auditbot tAkeOut
spyfu
sysscan
trendiction.de trendiction.de
ubermetrics-technologies.com webmeup-crawler
voyagerx.com
webpros.com webpros.com
webprosbot x09Mozilla
xpymep1.exe zauba.io
zgrab zgrab

310
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider Aboundexbot
404enemy Acunetix
80legs AfD-Verbotsverfahren
ADmantX AhrefsBot
ALittle Client
ASPSeek
AiHitBot AiHitBot
Aipbot
Alexibot
AllSubmitter
Alligator Alligator
Anarchie Anarchy99
Ankit Ankit
Anthill Anthill
Aspiegel
Attach
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike
BDCbot BDCbot
BackStreet BDFetch
BacklinkCrawler BLEXBot
Bandit
BatchFTP BatchFTP
Battleztar Bazinga
BetaBot BetaBot
Bigfoot
Black Hole Black Hole
BlackWidow
Blackboard Blackboard
Boardreader BlowFish
Bolt Brandprotect
BotALot Buck
Brandwatch Buddy
BuiltBotTough CCBot
CATExplorador CODE87
CSHttp Calculon
CensysInspect CazoodleBot
CheTeam
CheeseBot CheeseBot
Chlooe
Cliqzbot
Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot
Collector Collector
Cosmos CopyRightCheck
Craftbot
Crawling at Home Project
Curious
Custo
CyotekWebCopy CyotekWebCopy
Demon DSearch
DTS Agent
DatabaseDriverMysqli
Dirbuster Dirbuster
Discoverybot Discobot
DomainSigmaCrawler Dispatch
Domains Project DnyzBot
Dragonfly DomCopBot
DomainCrawler
Dotbot
Drip Drip
EasyDL EMail Siphon
EirGrabber EMail Wolf
Exabot Ebingbong
ExtractorPro Evil
EyeNetIE
Ezooms
Fimap Fimap
Flunky Firefox/7.0
Foobot FrontPage
Fuzz
FyberSpider FyberSpider
Genieo Fyrebot
GetRight G-i-g-a-b-o-t
GT::WWW
GalaxyBot
GermCrawler
GetWeb
Getintent Getintent
Gigabot
Go!Zilla Go!Zilla
Gotit GoZilla
GrabNet GrabNet
Grabber
Grafula Grafula
GrapeFX
GrapeshotCrawler GrapeshotCrawler
HEADMasterSEO
HTMLparser
HTTP::Lite HTTP::Lite
HaosouSpider
Havij Havij
Heritrix Heritrix
Humanlinks Humanlinks
HybridBot Iblog
IDBTE4M
IRLbot
Image Fetch Image Fetch
Indy Library Image Sucker
IstellaBot Internet Ninja
JOC Web Spider JetCar
Jbrofuzz Jetty
JennyBot
JikeSpider
Joomla Joomla
JustView
Jyxobot Jyxobot
Keyword Density
Kinza
Kozmosbot Kozmosbot
Lanshanbot LWP::Simple
Leap LeechFTP
LexiBot
Lftp Lftp
LibWeb
Libwhisker
LieBaoFast LieBaoFast
Lightspeedsystems
LinkScan
LinkWalker LinkWalker
Linkbot
LinksManager LinksManager
Ltx71 LinqiaRSSBot
MFC_Tear_Sample LinqiaScrapeBot
MSFrontPage Lipperhey
MIDown tool
MJ12bot
MSIECrawler MSIECrawler
MTRobot MTRobot
Mag-Net Mag-Net
Magnet Mail.RU_Bot
MarkWatch Majestic12
Mata Hari
MauiBot
Mb2345Browser Mb2345Browser
MeanPath Bot MeanPath Bot
Mediatoolkitbot Mediatoolkitbot
Mister PiX MicroMessenger
Moblie Safari Mojolicious
Morfeus Fucking Scanner Mr.4x3
Msrabot
Musobot Musobot
NearSite Needle
Net Vampire Net Vampire
NetAnts NetAnts
NetMechanic Nikto
Netvibes
Nimbostratus Nimbostratus
Octopus Ninja
Openfind Nutch
Openvas Offline Explorer
OnCrawl
OpenLinkProfiler
OutclicksBot
OutfoxBot OutfoxBot
POE-Component-Client-HTTP PECL::HTTP
Page Analyzer PageAnalyzer
PageGrabber PageGrabber
PageScorer PageThing.com
Panscient Pavuk
Papa Foto
Petalbot
Pi-Monster Pi-Monster
Picscout Piepmatz
PictureFinder
Pixray Pixray
Pockey
ProWebWalker
Probethenet
Psbot
Pu_iN
Pump
PxBroker PxBroker
QueryN Metasearch PyCurl
Quick-Crawler
RankActive
RankActiveLinkBot RankActiveLinkBot
RankFlex
RankingBot RankingBot
RankingBot2
Rankivabot
Re-re
ReGet
RealDownload RealDownload
Reaper
RebelMouse RebelMouse
RedesScrapy
RepoMonkey
Ripper
SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOprofiler SalesIntelligent
SISTRIX
ScanAlert ScanAlert
Scanbot
ScoutJet ScoutJet
ScreenerBot Searchestate
SearchmetricsBot SemrushBot
SentiBot
SeobilityBot SeobilityBot
Siphon Seomoz
Shodan
Site Sucker Site Sucker
SiteSnagger SiteCheckerBotCrawler
SiteExplorer
SiteSucker
Sitebeam Sitebeam
Siteimprove Siteimprove
Snoopy SlySearch
SmartDownload
Sociscraper Sociscraper
Sogou web spider Sogou web spider
SpaceBison Sosospider
Spammen Sottopop
SpankBot SpankBot
Sqworm Spanner
SuperBot Sqlmap
Sucker
SuperHTTP SuperHTTP
Swiftbot Surfbot
Szukacz SurveyBot
T0PHackTeam T0PHackTeam
Teleport
TeleportPro TeleportPro
Telesoft Telesoft
Telesphoreo TheNomad
Thumbor TightTwatBot
Titan Toata
Tracemyfile Toweyabot
Trendiction True_Robot
Trendictionbot
Turnitin Turnitin
TurnitinBot
TwengaBot TwengaBot
Twice
Typhoeus Typhoeus
URLy.Warning URLy Warning
UnisterBot
Upflow Upflow
V-BOT
VB Project VB Project
VCI VCI
VeriCiteCrawler VelenPublicWebCrawler
VidibleScraper VidibleScraper
Voil WASALive-Bot
WPScan WPScan
WWW-Mechanize WWW::Mechanize
Wallpapers/3.0 Wallpapers/3.0
Web Collage WallpapersHD
Web Enhancer Web Auto
Web Fetch
Web Pix
Web Sauger Web Sauger
WebBandit WebAuto
WebCopier WebCollage
WebFetch WebFuck
WebImageCollector
WebSauger
WebStripper WebStripper
WebWhacker WebmasterWorldForumBot
Webshag Website Quester
WebsiteQuester WebsiteQuester
Whack Whack
Whacker
Whatweb Whatweb
Who.is Bot Woobot
Widow
WinHTTrack
Wotbox Wotbox
Xaldon_WebSpider Wprecon
Xenu YoudaoBot
Zade
Zermelo
Zeus Zeus
Zitebot Zitebot
ZmEu ZoomBot
ZoominfoBot
ZyBorg
archive.org_bot archive.org_bot
autoemailspider arquivo-web-crawler
cah.io.community
coccocbot-web coccocbot-web
cognitiveseo
crawl.sogou.com crawl.sogou.com
crawler.feedback crawler.feedback
demandbase-bot domainsproject.org
facebookscraper eCatch
heritrix heritrix
instabid internetVista monitor
isitwp.com isitwp.com
lwp-request magpie-crawler
lwp-trivial plumanalytics
meanpathbot probe-image-size
mediawords s1z.ru
satoristudio.net satoristudio.net
scalaj-http scan.lol
sitechecker.pro sitechecker.pro
spyfu siteripz
sysscan
trendiction.de trendiction.de
voyagerx.com voyagerx.com
webpros.com
webprosbot
x22Mozilla
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
alienwheels.de adamoads.com
focalink.com delta-line.men
lost-alpha.ru dynainbox.com
mamasuper.prom.ua education-cz.ru
myyour.eu predmety.in.ua
news-speaker.com prostitutki-kharkova.org
picmoonco.pw sady-urala.ru
r-e-f-e-r-e-r.com strongsignal-a.akamaihd.net
uptime-gamma.net trion.od.ua
visitcambridge.org tytoona.com

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1008.su 1001desert.com
123cha.com 101flag.ru
12masterov.com 1kdailyprofit.me
1bet.com 1kinobig.ru
1xbet4you.com 24videos.tv
2728fb936f0.com 24x7-server-support.site
5i2.net 300richmond.co.nz
7ooo.ru 40cg.com
888.com 4rent.ru
91abcw.com 51unlim.ru
a1security.com.ua a14download.com
aac52006ec82a24e08b665f4db2b5013f7.com aab94f698f36684c5a852a2ef272e031bb.com
acads.net aaronabel.com
ad-words.ru actionnooz.com
adcash.com acunetix-referrer.com
adprotect.net adbetclickin.pink
ads.gold adktrailmap.com
advancedcleaner.com adobereader-free.ru
advertex.info adultgalls.com
advokateg.ru afslankpillen2017nl.eu
agadelha.com.br air-edem.ru
aktobe.xkaz.org albumsuper.info
akuhni.by alldezservice.kz
alborzan.com allproblog.com
alfa-medosmotr.ru alveris.ru
allwidewallpapers.com amazingninja.com
amazon-seo-service.com ameblo.top
ample-awards-today.us amyfoxfitness.com
ananumous.ru anabolics.shop
apiallgeniusinfo-a.akamaihd.net animaltoplist.com
apimegabrowsebiz-a.akamaihd.net anonymizeme.pro
anonymous-redirect.com
apartmentratings.com
apessay.com
apiadanaknet-a.akamaihd.net
apisecretsaucebi-a.akamaihd.net
apispringsmartne-a.akamaihd.net apispringsmartne-a.akamaihd.net
appfastplay.com approvedlocal.co.za
appfixing.space arius.tech
apple.com-cleaner.systems atmagroup.ru
asociatia-tipografilor-transilvania.ro audiobangout.com
auto.rusvile.lt
autodo.info autodo.info
automobile-spec.com
autonew.biz autonew.biz
aviav.co autoua.top
avek.ru
aviav.ru
aviav.ru.com aviav.ru.com
avtointeres.ru avon-ukraine.com
axbocz.net avtochehli.by
babosas.com ayeartoforget.com
backlink4u.net azarttoto.com
bannerpower.com back.dog
bard-real.com.ua bag77.ru
beauby.ru balkanfarma.ru
beauty-things.com beachdriveblog.com
berdasovivan.ru belinka.com.ua
bestwaystogetpaid.us best-deals-products.com
bharatdefencekavach.com bestdooz.com
bigames.online billiard-classic.com.ua
bigpictures.club bimatoprost-careprost.com.ua
bimatoprost-careprost.com bio.trade-jp.net
bingoporno.com bioinnovate.co
binomo.kz blobar.org
bird1.ru
bitcoin-ua.top
bizfly.info
blackbabesporn.com
blagovest-med.ru
blockety.co
blog.axant.it blog.axant.it
bloglag.com bolitgorlo.net
bluerobot.info bonanza-fortune.men
bnt-team.com boostmyppc.com
boltalko.xyz braip.com.br
brainzpod.win
brandbucket.com brandbucket.com
breget74.ru break-the-chains.com
brendbutik.ru bronzeaid-a.akamaihd.net
buckspar.gq bumskontakte.org
burger-tycoon.com burger-imperia.com
buy-loft.ru buyk.host.sk
buy-meds24.com buyscabiescream.com
buybest1.biz bvlgariring.xyz
buyclomidonlaine.com cablecar.us
buyparajumpers.online candcstuccoandstone.com
cacheimages.com candycrushshop.com
cah.io.community carsplate.com
cartierbracelet.xyz cartierlove2u.xyz
cartierlovebracelet.xyz cartierloveringreplica.xyz
cartiertrinity.xyz cashonads.com
cartujano-pre.de ccbill.com
cementaresearch.se cejewelry.xyz
cenokos.ru cercacamion.it
cfsrating.sonicwall.com
cheapmarmot.online
cheapmoncler.pw cheapmoncler.pw
cheatcode-lita12.rhcloud.com chimiver.info
check-host.net chocolateslim-en-espana.com
choice-credit.ru chorus.terakeet.com
chuckguilford.com cialischstgerts.com
citizenclsdriveri7.pen.io cicaki.net
clarithromycin500mg.com cl.s7.exct.net
cocolyze.com classiquebijoux.ru
codysbbq.com cleanmypc.com
clevernt.com
clubfashionista.com
cobaltpro.ru
cognitiveseo.com cognitiveseo.com
compliance-ivan.top com-onlinesupport.host
coolingoods.com compliance-elena.xyz
counter.bloke.com compliance-jess.top
cowblog.fr compliance-olga.top
cracksplanet.com computernetworksonline.com
crest-poloski.ru cookie-law-enforcement-ff.xyz
custodieva.ru coop-gamers.ru
darkbooks.org corta.co
ddsoldes.fr coslab.club
demon-tweeks.com cowlmash.com
deniven.1bb.ru crystalslot.com
deshevo-nedorogo.ru cs-passion.pl
diegolopezcastan.com damedingel.ya.ru
digital-video-processing.com davebestdeals.com
dawlenie.com
decoratinghomes.ga
deplim.com
destinationrealestate.com
detective01.ru
dev33.dioniqlabb.se
devochki.top
diamond-necklace.info
dickssportinggoods.com
digest-project.ru
dignitasdata.se dignitasdata.se
div.as diminishedvalueoforegon.com
dkvorota.ru dirtpics.pw
doctormakarova.ru dividendo.ru
documentsite.net djstools.com
dodlive.mil dktr.ru
domaincheck.io dojki-hd.com
domaincrawler.com dojki365.online
domashneeruporno.com donna7753191.ru
dood.live downloadmefiranaratb1972.xpg.com.br
dostavka-v-krym.com drpornogratisx.xxx
drhomes.biz drugspowerstore.com
drivotracker.com earl-brown.info
drugs-no-rx.info editors.choice6912650.hulfingtonpost.com
dtm-spain.com edwinkonijn.com.au
dustyorate.com ege-russian.ru
dvervmoskvu.ru electricwheelchairsarea.com
e-c.al elektrischezigarette2.onsugar.com
earncash.com.ua elektronischezigarette2.mex.tl
edge.sharethis.com elektronischezigarettekaufen2.tumblr.com
ekaterinburg.xrus.org emailaccountlogin.co
elektrozigarette2.webs.com empirepoker.com
elektrozigaretten1.postbit.com en.altezza.travel
elexies.info enskedesquashclub.se
elidelcream.weebly.com eralph.tk
eloxal.ru errorfixing.space
elvenmachine.com etenininrade.ga
emediate.eu euronis-free.com
englate.com ewebarticle.info
estelight.ru exlarseva.webblog.es
etm-consult.de expdom.com
euromasterclass.ru eywords-monitoring-your-success.com
europages.com.ru factorynightclub.com
europeanwatches.ru failingmarriege.blogspot.com
evogarage.com faithe.top
eyelike.com.ua falcoware.com
ezigarettekaufen2.yolasite.com
falllow.gq falllow.gq
familyholiday.ml
familyphysician.ru
fashion.stellaconstance.co
fast-torrent.ru
fatmaelgarny.com
femdom.twiclub.in
feorina.ru
ferieboligkbh.dk ferieboligkbh.dk
fickkontaktehobbyhuren.com ferrotodo.com
filesdatabase.net financeloan.us
finejewelryshop.ru finstroy.net
firstsiteguide.com
flash4fun.com flash4fun.com
forexunion.net
formseo.com formseo.com
forodvd.com foto-sisek.porngalleries.top
foto-weinberger.at fototravel.eu
free-deals.faith fotzen-ficken.com
free-fbook-traffic.com frankofficial.ru
freejabs.com free-share-buttons.blogspot.com
frustrated-favorable.gq freecamdollars.com
fullfileaccess.com freesitetest.com
funnypica.com gael-s.ru
galaxyflowers.ru gamesprite.me
game300.ru gelezki.com
gamedayhouse.com gembird.com
gay-site.store getdot.ru
generic-pills-online.com glastecfilms.com.my
genericviagrasildenafiled.net go2album.com
gentamicineyedrops.blogspot.com goforexvps.com
get-free-social-traffic.com golmau.host.sk
getoutofdebtfree.org
given2.com
goodly.pro goodly.pro
goodprotein.ru googlefeud.com
gorodservis.ru gratuitbaise.com
gpms.org.my greatdealshop.com
grand-chlen.ru
greenidesign.co greenidesign.co
growshop.es gsasearchengineranker.pw
gsasearchengineranker.site gta-club.ru
gsmlab.pl handicapvantoday.com
guardlink.org hauleddes.com
guitar-master.org heygidday.biz
hdfreeporno.net high-speed1.net
headpress.ru holidaypics.org
hentai-manga.porn homeart.space
hermesbelts.xyz hopeonthestreet.co.uk
hermesbracelets.xyz hornymatches.com
hermesreplica.pw hothor.se
himazin.info hothot.ru
hiwibyh.bugs3.com hotshoppymac.com
hjaoopoa.top hotsocialz.com
hobild.net houseofgaga.ru
hop.clickbank.net hoverboard360.at
hoporno.com
hoverboard360.es
https-legalrc.biz https-legalrc.biz
hully.altervista.org hubbble.com
ideoworld.org ibb.com.ua
ihc2015.info ibmdatamanagement.co
imfamous.info ifirestarter.ru
indetiske.ya.ru igrovyeavtomaty777.ru
infoupdate.org ikritikimou.gr
internetartfair.com iloveitaly.ru
invitefashion.com impotentik.com
iradiology.ru infostatsvc.com
istripper.com innoslicon.com
itag.pw insomniagamingfestival.com
ivanovo.zrus.org interfucks.net
izhevsk.zrus.org irkutsk.online-podarki.com
izoll.ru irunfar.com
javcoast.com isistaylorporn.info
javlibrary.cc iskalko.ru
javrip.net isotoner.com
juliaworld.net ispac.org
junketjuice.blogspot.com istmira.ru
jurajskie.info iweblist.info
jyrxd.com jacago.com
karachev-city.ru jerseywholesalebizchina.com
jikoman.info
johnrobertsoninc.com
jpcycles.com
kaliningrad.zrus.org
kazrent.com
kerwinandcariza.com kerwinandcariza.com
keywordhut.com keywordsuggest.org
keywordsuggests.com kicknights.gq
kino-fun.ru kino-filmi.com
kirov.zrus.org konoplisemena.com
kleine-titten.biz kostenlos-sexvideos.com
kol-energo.ru kredit-pod-zalog-krasnodar.ru
komp-pomosch.ru
koopilka.com
krasivye-devushki.net
kskjco.club kskjco.club
kung-fu-ru.com kumuk.info
kupit-adenu.ru kursy-ege.ru
la-fa.ru lanadelreyfans.us
lacloop.info ledpolice.ru
laserpen.club lennatin.info
laulini.soclog.se lernur.net
law-check-nine.xyz letslowbefast.site
laxob.com lflash.ru
letolove.ru linkrr.com
lexaprogeneric.link liquimondo.com
lightinghomes.net live-cam6.info
lineavabit.it liveshoppersmac.com
lineshops.biz louboutinreplica.xyz
linkbolic.com lovi-moment.com.ua
linkbux.ru lulea-auktionsverk.se
linkdebrideur.xyz lyrics.home-task.com
locimge.pw mackeeper-land-672695126.us-east-1.elb.amazonaws.com
lombia.co macrotek.ru
lonely-mature.com magnetpress.sk
lottospring.com managerpak204.weebly.com
lovelycraftyhome.com manve.info
low-format.ru matb3aa.com
lunamedia.co meccadumps.net
m1media.net meetingrainstorm.bid
m3gadownload.pl meget.co.za
m4ever.net
mactechinfo.info
magento-crew.net
manifestyourmillion.com
manualterap.roleforum.ru
mapquestz.us
marwer.info
masserect.com
masthopehomes.com
meds-online24.com
meendo-free-traffic.ga
meteostate.com meteostate.com
methodsmarketing.com michaelkorssaleoutletonline.net
minecraft-rus.org militarysale.pro
mmofreegames.online mir-betting.ru
mobioffertrck.com missvietnam.org
mobplayer.ru mister-shop.com
modenamebel.ru mistr-x.org
mojpreskumanie.com mobot.site
money-for-placing-articles.com moesonce.com
moneymaster.ru moneytop.ru
morf.snn.gr
morlat.altervista.org
morpicert.pw morpicert.pw
moyaterapiya.ru moscow.xrus.org
mozello.ru motherboard.vice.com
mttwtrack.com mountainstream.ms
musezone.ru moz.com
musicspire.online mrvideospornogratis.xxx
mydeathspace.com musicktab.com
myftpupload.com mwtpludn.review
mygameplus.ru myonlinepayday.co
myindospace.com mypornfree.ru
mypets.by mystats.xyz
naperehresti.info nbsproject.ru
narosty.com netcheckcdn.xyz
newstraveller.ru newtechspb.ru
nfljerseyscheapbiz.us newyorkhotelsmotels.info
nhl17coins.exblog.jp nextlnk12.com
niroo.info nicefloor.co.uk
njpalletremoval.com nonameread45.live
nootrino.com novatech.vn
norththeface.store novodigs.com
nuit-artisanale.com nuker.com
nw-servis.ru nw-servis.ru
nyfinance.ml o-o-6-o-o.com
o-o-8-o-o.com obiavo.uz
obuv-kupit.ru ochistka-stokov.ru
oddamzadarmo.eu oivcvx.website
oil-td.ru ok.ru
onlainbesplatno.ru
online-zaymy.ru online-zaymy.ru
openfrost.com onlythegames.com
ops.picscout.com ourville.info
oralsexfilme.net oyster-green.com
ossmalta.com panamaforbeginners.com
ovirus.ru panicatack.com
pammik.ru paparazzistudios.com.au
parajumpersstore.online partner-host.men
pathwhelp.org partner-print.men
pcgroup.com.uy partners-ship.pro
pepperstyle.ru perl.dp.ua
photostudiolightings.com perm-profnastil.ru
pictures-and-images.com petsblogroll.com
pijoto.net pio.polytopesexempt.com
piuminiita.com
pixgood.com pixgood.com
plusnetwork.com pizza-imperia.com
pochemychka.net plastweb.ru
pogodnyyeavarii.gq playboyfiles.xblog.in
pokemon-go-play.online popup-jdh.xyz
polcin.de pornhub-ru.com
pomoc-drogowa.cba.pl
poquoson.org
porno-home365.com
porno.simple-image.com.ua
pornodojd.ru pornodojd.ru
pornophoto.xyz pornoreino.com
pornoplen.com pornstartits.xblog.in
pornosok.ru portside.xyz
pornozhara.com pozdravleniya-c.ru
porto.abuilder.net predmety.in.ua
portside.cc primedice.com
predominant-invent.tk privacyassistant.net
privhosting.com profitwithalex.info
pro-tec.kz purchasepillsnorx.com
profit-opportunity.com pussysaga.com
proprostatit.com puteshestvennik.com
prostitutki-almata.org qpypcx.com
prostitutki-tolyatti.org
qualitymarketzone.com qualitymarketzone.com
quickloanbank.com quangcaons.com
rankia.com radiodigital.co
redirectingat.com rankexperience.com
relax.ru rankings-analytics.com
rankscanner.com
raschtextil.com.ua
rating-bestcasino.com
rczhan.com
realitykings.com
realtytimes.com
redirlock.com
regdefense.com
registrationdomainsite.com
registry-cleaner.net
registrysweeper.com
remote-dba.de remote-dba.de
resultshub-a.akamaihd.net resultshub-a.akamaihd.net
retailwith.com
ribieiendom.no ribieiendom.no
rimedia.org rollercoin.com
ring4rhino.com rome2rio.com
rniaeba.ga
rospromtest.ru
royal-investments.net
rp9.ru rp9.ru
rusenvironmental.net rutor.group
s-forum.biz s1z.ru
sarf3omlat.com sack.net
satoristudio.net sandhillsonline.com
scanner-alexa.top sardinie.us
scanner-nelson.top sawin.beth.webd.pl
semalt.com sax-sex.com
seokicks.de sbetodiodnye-lampy.ru
sbtdesign.co.uk
scalerite.co.za
scanmyphones.com
scanner-jess.top
scanner-john.top
scanner-mary.top
school-diplomat.ru
screentoolkit.com
scrnet.biz.ua
se-welding.ru
se.bnt-team.com
searchaddis.com
searchlock.com
searchmywindow-a.akamaihd.net
sell-fb-group-here.com
seminarykansas904.ml
seolab.top
seorussian.ru seorussian.ru
serpstat.com servisural.ru
service-core.ru seryeznie-znakomstva.ru
sex-sex-sex5.com sexy.babes.frontend-stack.top
sexad.net
sexcamamateurchat.com
sexgalleries.top
sexkvartal.com
shama-rc.net shama-rc.net
share-buttons-for-free.com shemale-sex.net
shariki-zuma-lines.ru shitmovs.com
sharpchallenge.com shmetall.com.ua
shillyourcoins.com shodanhq.com
ship-marvel.co.ua shop.acim.org
shoesonlinebuy.xyz siha.de
shoppingmiracles.co.uk silktide.com
sierraapps.com sindragosa.comxa.com
signx.info site-speed-check.site
simplesite.com skincrate.net
site3.free-share-buttons.com slowmacfaster.trade
social-button.xyz socialtrade.biz
soda.media soheavyblog.com
soft1.ru solinf.co
softxaker.ru solitaire-game.ru
soserfis.com solmarket.by
spravka-medosmotr.ru sortthemesitesby.com
sps-shop.com souvenir.cc
star61.de spammen.de
stathat.com
steamoff.net steamoff.net
stickers-market.ru stpolice.com
stoki.ru streha-metalko.si
studiofmp.com stroyplus.ru
sumo.com strv.se
superstarfloraluk.com
support.nopeas.sk support.nopeas.sk
svetlotorg.ru suture.co
svolze.com tabletkinaodchudzanie.com.pl
swinginwithme.ru tagil.zrus.org
tecnoteakviareggio.it teastory.co
telemetryverification.net telesvoboda.ru
template-kid.com thegameriders.com
texbaza.by tivolibasket.it
tgtclick.com
thepokertimer.com
thesmartsearch.net
titan-ads.life
tizanidine4mgtablets.blogspot.com
tizanidine6mg.blogspot.com
tizanidineformigraines.blogspot.com tizanidineformigraines.blogspot.com
tizanidinehcl2mgtablet.blogspot.com
tizanidinemusclerelaxant.blogspot.com
tizanidinestreetprice.blogspot.com tizanidinestreetprice.blogspot.com
tnctrx.com tizanidinevsvalium.blogspot.com
token-lab.org tizanidinewithdrawalsymptoms.blogspot.com
toondinsey.com
topbestgames.com
topkarkas.com topkarkas.com
torrentgamer.net tovaroboom.vast.ru
torrents.cd tozup.com
torrents.life track-rankings.online
torrnada.ru traffpartners.com
tour-line.net traxdom.ru
tracklead.net tri-slona.org
trahvid.com tryrating.com
tramadolandtizanidine.blogspot.com tsan.net
twitlinks.com tupper-posuda.ru
uchetunet.su twittrading.com
ukkelberg.no typer.one
undergroundcityphoto.com udayavani.com
uptimebot.net univerfiles.com
us-america.ru urccvfmc.bloger.index.hr
utiblog.fr urengoy.pro
urll.eu
urlopener.com
uzporno.mobi uzporno.mobi
vchulkah.net vapeface.club
vbikse.com
vbtracker.net
vegan-foods.us vegan-foods.us
viddyoze.com vekzdorov.ru
video-camer.com via-energy-acquistare.com
video-hollywood.ru viagraneggrx.com
videochat.bz videos-for-your-business.com
vinsit.ru videtubs.pl
vkak.ru visa-pasport.ru
vostoktrade.info vitoriacabos.com
vpnhowto.info vksaver-all.ru
vseuznaem.com wahicbefa31.soup.io
w-journal.ru wallpapersdesk.info
waycash.net wallpapersist.com
web.cvut.cz waterpurifier.club
webinstantservice.com website-analyzer.info
website-speed-up.top wechatdownload10.com
welck.octopis.com
welcomeauto.ru welcomeauto.ru
while.cheapwebsitehoster.com whengirlsgowild.com
win-spy.com wholesalejerseysgaa.com
winner7777.net whos.amung.us
wleuaprpxuvr.ga womama.ru
wordkeyhelper.com woman-h.ru
works.if.ua workle.website
worldhistory.biz wsgames.ru
wovis.site wttavern.com
wufak.com www.jbetting.com
wyniki-lista.pl xitjw.info
xkaz.org xmlinde.com
xlolitka.com xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xn----btbdvdh4aafrfciljm6k.xn--p1ai xn----btbdvdh4aafrfciljm6k.xn--p1ai
xn--1-8sbcpb0bdm8k6a.xn--p1ai xn----ctbbcjd3dbsehgi.xn--p1ai
xn--d1aifoe0a9a.top xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai
xn--h1ahbi.com.ua xn--80ahvj9e.xn--p1ai
xpresscare.ru xpresscare.ru
xxxhdvideo.site xxxdatinglocal.us
xzlive.com xxxkaz.org
yginekologa.com yogamatsexpert.com
yoluxuryevents.com youandcredit.ru
yoursite.com your-bearings.com
yunque.pluto.ro zaapplesales.blogspot.com
zebradudka.com zaimhelp.ru
zed21.net zapatosenventa.info
zeikopay.com zigarettenonlinekaufen1.bloog.pl
zeroredirect12.com zkjovpdgxivg.ga
zigarettenonlinekaufen.tumblr.com zone-kev717.info
zolotoy-lis.ru zoogdisany.com
zona-aqua.ru zoosexart.com
zoodrawings.com zreloeporno.tv
zoolubimets.ru
zootoplist.com
zscalerone.net

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
007angels.com 100dollars-seo.com
101flag.ru 178evakuator178.ru
101lesbian.xyz 24videos.tv
123cha.com 2drittel.de
1qingdao.com 300richmond.co.nz
24-ak.ru 7fon.ru
4kpics.rocks aanapa.ru
4kplayer.pl addbin.men
7ooo.ru adultfriendfinder.com
8lufu.com algarveglobal.com
albuteroli.com allbizne.co.ua
alekseevec.ru amateurgalls.com
alert-hgd.xyz animal-drawings.com
alive-ua.com apper.de
apps-analytics.net appfixing.space
bala.getenjoyment.net apple.com-webbrowsing-security.review
balkanfarma.org asophoto.com
bannerspace.com atleticpharm.org
basedecor.ru availit.weebly.com
bcmp.org avon-ukraine.com
beachdriveblog.com bannerpower.com
bestcurs.org basisches-wasser.net
bharatdefencekavach.com beachtoday.ru
biketank.ga beremenyashka.com
bioscorp.ru berrymall.ru
biznesluxe.ru bigames.online
bitcoinremote.com
blockety.co blockety.co
bluesalt.co bolitgorlo.net
bracketsmackdown.com book-bets.com
callawaygolfstore.online bosefux.esy.es
capacitacionyaprendizaje.com brk-rti.ru
cashkitten-a.akamaihd.net bscodecs.com
cashonads.com burger-imperia.com
cbozhe.com buy-cheap-pills-order-online.com
cercacamion.it callawaygolfoutlet.online
cfacarrosserie74.com candy-glam-hp.com
chcu.net cardul.ru
com-onlinesupport.host cascadelink.org
compiko.info cdnnetwok.xyz
concealthyself.com cellfun.mobi
converse.ddsoldes.fr chatroulette.si
copblock.org chatroulette.video
cupday.com chatroulette.world
customchocolate.business-for-home.com cheapcigarettesc.info
daydream-studio.ru cheapmoncler.xyz
de.zapmeta.com chiptuninger.com
dealitright.click clickcash.com
demian.kz clickhype.com
domini.cat cobaltpro.ru
downloadmefiranaratb1972.xpg.com.br cocolyze.com
drillsaw.ru compliance-alex.xyz
duawitchrarato.tk compliance-walter.top
dwomlink.info continent-e.tv
counciltally.com
credit-online.ws
culmimg.pw
culturevie.info
cutalltheshit.com
cvety24.by
dairyindia.in
datract.com
davebestdeals.com
deadlinkchecker.com
dear-diary.ru
detki-opt.ru
deutschland123.de
dev78.dioniqlabb.se
dpihatinh.gov.vn
drunkenstepfather.com
dyt.net dyt.net
e-buyeasy.com elektir.ru
ec-file.info embedle.com
edudocs.net exportshop.us
elmacho.xyz extractorandburner.com
energydiet24.ru extremepornos.net
enginebay.ru ezigarettekaufen2.yolasite.com
ereko.ru falcon-images.blogspot.com
exdocsfiles.com faracontrol.ir
eywords-monitoring-your-success.com femmesdenudees.com
fearcrow.com feorina.ru
filesvine.com filesvine.com
flashbannernow.com finstroy.net
fodelsedagspresenter.nu fireads.men
forpostlock.ru fordsonmajbor.cf
foto-sisek.porngalleries.top forex.osobye.ru
freefoto.ca free-social-buttons.com
frequiry.com galleryawesome.com
freshmac.space game-top.su
fugarif.ga games.kolossale.ru
gallerylisting.com gdebestkupit.ru
gamedayassist.com generalporn.org
ge0ip.com gilsonchiro.xyz
ge0ip.org gojiberriess.apishops.ru
gentamicineyedrops.blogspot.com goodwriterssales.com
germetiki.com.ua grizzlysgrill.com
gettpromos.com grupografico-pilar.com.ar
getyourimage.club gsasearchengineranker.pw
glcomputers.ru guarrasdelporno.xxx
go2album.com guitar-master.org
goforexvps.com gwhwpxbw.bloger.index.hr
goodnightjournal.com home-task.com
gsasearchengineranker.top home.myplaycity.com
hdpixent.com horoshieokna.com
hdwalls.xyz ie.57883.net
headpress.ru
healgastro.com
healmytrauma.info
hit-kino.ru
hmmm.cz
holidaypics.org
hollywoodactress.info
homeinns.com
hoste.octopis.com
hostingclub.lk
hotcore.info
hotshoppymac.com
hoverboard360.nl
humanelydrew.com
i-midias.net.br
id-forex.com
ilmexico.com
imitex-plus.ru
import-sales.com import-sales.com
incep.imagine-ex.co infektsii.com
iradiology.ru intim-uslugi.info
iyasimasennka.com izismile.com
jeremyeaton.co jbl-charge.info
jewelryandfiligree.com journeydownthescale.info
jwcialislrt.com kaliningrad.zrus.org
karaganda.xkaz.org kamalsinha.com
kidd.reunionwatch.com kinobest.pl
kino-filmi.com kirov.zrus.org
kino-fun.ru klejonka.info
kino2018.cc koleso24.com.ua
kinofak.net krasivoe-hd.com
krassh.ru krasivoe-hd.net
law-check-nine.xyz krasnodar-avtolombards.ru
legionalpha.com krasnodar.ru
lernur.net letmacwork.world
libertybilisim.com letslowbefast.today
luchshie-topcasino.ru lexaprogeneric.link
magiadinamibia.blogspot.com lflash.ru
matrixalchemy.com liupis.com
maxxtor.eu lob.com.ru
mediaoffers.click lolnex.us
megakino.net long-beach-air-conditioning.com
minyetki.ru lutherstable.org
mixed-wrestling.ru luxmagazine.cf
mobioffertrck.com mabdoola.blogspot.com
mojpreskumanie.com megaindex.ru
mokrayakiska.com mensandals.xyz
moncleroutletonline.pw mesto-x.com
mosrif.ru meteocast.net
musflashtv.com mikrobiologies.ru
musicdaddy.net mirzonru.net
muznachas-service.ru mobstarr.com
myra.top mrmoneymustache.com
muz-tracker.net
mydirtyhobby.com
mykings.pw
nastroyke.net nastroyke.net
news-speaker.com naval.jislaaik.com
nhl17coins.exblog.jp nibbler.silktide.com
o-dachnik.ru nomuos.it
oecnhs.info nrv.co.za
okout.ru o00.in
onlineporno.site ons-add.men
partners-ship.pro openfrost.com
partybunny.ru osagonline.ru
parvezmia.xyz ownshop.cf
pcimforum.com ownshop.win
pflexads.com pacificair.com
pills24h.com paramountmarble.co.uk
pinwallpaper.top pay2me.pl
pony-business.com pensplan4u.com
portadd.men pharmacyincity.com
pr-ten.de phrcialiled.com
purchasepillsnorx.com pictures-and-images.com
pussyfleet.com poquoson.org
putitin.me porno-play.net
rebrand.ly pornokorol.com
reklama-i-rabota.ru pornophoto.xyz
remote-dba.de positive2b.ru
rent2spb.ru printie.com
rewardit.com productarium.com
robot-forex.biz prokotov.com
rockingclicks.com psoriasis-file.trade
rogervivierforsale.com puzo2arbuza.ru
rrutw.com r-e-f-e-r-e-r.com
rubanners.com rarbg.to
rusexy.xyz rcpmda.ikan1080.xyz
sabizonline.com referencemoi.com
saneyes.com rep-am.com
scanner-alex.top research.ifmo.ru
scmor.ilxc.cc russintv.fr
scopich.com sa-live.com
se.bnt-team.com samoiedo.it
sexyteens.hol.es sarafangel.ru
shinikiev.com.ua savememoney.co.za
signoredom.com scanner-fred.top
simple-image.com.ua scanner-walt.top
siteripz.net sdelai-prosto.ru
skachat-besplatno-obrazcy.ru search.alot.com
slonechka.ru sei80.com
slowmacfaster.trade seminarykansas904.ml
soblaznu.net seofied.com
sodexo.com sexsaoy.com
soft-program.com sheerseo.com
sortthemesitesby.com sibecoprom.ru
spasswelt.net sirpornogratis.xxx
speeddream.xyz sizeplus.work
spl63.fr skypasss.com
stair.registrydomainservices.com smichovbike.cz
stanthonyscatholicchurch.org softomix.com
stop-gepatit.te.ua srecorder.com
strangeduckfilms.com startwp.org
studiofmp.com streetfooduncovered.com
studiokamyk.com.pl stretchmate.net
sugarlyflex.pw stroilka.info
symphonyintegratedhealthcare.com sukirgenk.dvrlists.com
tandvardshuset.net suralink.com
tech4master.com svnuppsalaorebro.se
teencastingporn.com swinon.site
telefonsex-ohne0900.net tampabaywatch.org
the-trader.net teenfuck.tv
thetoiletpaper.com themeforest.net
threecolumnblogger.com thesmartsearch.net
tjkckpytpnje.com tizanidinehcl2mg.blogspot.com
tmearegion26.com tkanorganizma.ru
toolsky.com tmm-kurs.ru
topanasex.com toon-families.com
torrents.cd totu.us
trafficshaper.com tozup.com
trafficstars.com tracfone.com
tuberkuleznik.ru traderzplanet.in
tubo360.com trafficjunky.net
unece.org trustedhealthtips.com
ttsq.fr
tuckpointingmasonrysystems.com
uggbootsoutletsale.us
univerfiles.com univerfiles.com
unmaroll.ya.ru uvozdeckych.info
uptime-delta.net vapeface.club
url-extractor.xyz vchulkah.net
usadacha.net ventelnos.com
vacuumscleaner.com virtuagirl.com
vapomnoncri.tk vitalads.net
viagroid.ru vizag.kharkov.ua
watchmyfb.pl vzlom-na-zakaz.com
westen-z.life walpaperlist.com
wholesalecheapjerseysfree.com watchinf.com
wholesalejerseysgaa.com westum.se
williamrobsonproperty.com winner7777.net
wowcasinoonline.ooo wladimirpayen.com
x5market.ru wma-x.com
xn----7sbaphztdjeboffeiof6c.xn--p1ai xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn--hxazdsfy.blogspot.com xsfetish.org
xn--oogle-wmc.com xxxrus.org
xnxx-n.com you-shall-not-pass.is74.ru
xnxx699.com youdesigner.kz
xsion.net youghbould.wordpress.com
xxxguitars.com youjizz.vc
yougetsignal.com your-bearings.com
zakonobosago.ru zastenchivosti.net
zapiszto.pl zhacker.net
zebramart.ru zoogdisany.com
zhuravlev.info
zigzog.ru
zionstar.net
zolotoy-lis.ru
zoomovies.org zoomovies.org
zzlgxh.com

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.169 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2662 ### Version: V4.2021.10.2663
### Updated: Wed Oct 20 05:20:25 UTC 2021 ### Updated: Wed Oct 20 10:12:27 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2662 ### Version: V4.2021.10.2663
### Updated: Wed Oct 20 05:20:25 UTC 2021 ### Updated: Wed Oct 20 10:12:27 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##