vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-05 12:09:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.10.2648 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-10-13 10:18:08 +00:00
parent 33a63201f7
commit b086278b48
47 changed files with 70482 additions and 70492 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2647 #### Version: V4.2021.10.2648
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2647 #### Version: V4.2021.10.2648
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.10.2647 #### Version: V4.2021.10.2648
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8676
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.10.2647 #### Version: V4.2021.10.2648
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8680
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8680
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8680
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8680
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 1; 10.1.1.181 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 0; 10.1.1.181 0;
127.0.0.1 0; 127.0.0.1 0;

8680
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 1; 10.1.1.181 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 0; 10.1.1.181 0;
127.0.0.1 0; 127.0.0.1 0;

8680
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 1; 10.1.1.181 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 0; 10.1.1.181 0;
127.0.0.1 0; 127.0.0.1 0;

8680
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2647 2648

8680
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8680
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.86.31.68 0; 20.69.94.2 0;

8680
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 1; 10.1.1.181 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 0; 10.1.1.181 0;
127.0.0.1 0; 127.0.0.1 0;

8680
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 1; 10.1.1.181 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 0; 10.1.1.181 0;
127.0.0.1 0; 127.0.0.1 0;

8680
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 1; 10.1.1.181 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 0; 10.1.1.181 0;
127.0.0.1 0; 127.0.0.1 0;

8680
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.86.31.68 0; 20.69.94.2 0;

8680
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.86.31.68 0; 20.69.94.2 0;

8680
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 1; 10.1.1.181 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
ADmantX AfD-Verbotsverfahren
FDM AwarioSmartBot
Navroad Custo
Needle DIIbot
Pockey IlseBot
Sosospider Iskanie
Trendictionbot IstellaBot
WebStripper JOC Web Spider
mediawords LinkScan
scan.lol MarkWatch

296
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 404checker
404enemy AIBOT
80legs ALittle Client
ADmantX Aboundex
ASPSeek AhrefsBot
Aboundexbot AiHitBot
Alligator Alexibot
Anarchie Anarchie
Anarchy
Anarchy99
Anthill Anthill
Aspiegel Aspiegel
Asterias Asterias
Attach AwarioSmartBot
BDCbot BBBike
Backlink-Ceck BDFetch
BacklinkCrawler BackStreet
BackWeb
Bandit
Barkrowler
Battleztar Bazinga Battleztar Bazinga
Bitacle
Black Hole Black Hole
Blackboard
Blow Blow
BlowFish BlowFish
Brandprotect BotALot
Buck Brandwatch
CazoodleBot BuiltBotTough
Cegbfeieh BuzzSumo
CensysInspect CensysInspect
CheTeam CheeseBot
CherryPicker
Chlooe Chlooe
Cloud mapping Cloud mapping
Cocolyzebot
Cogentbot Cogentbot
Copier Copier
CopyRightCheck Cosmos
Copyscape
Crawling at Home Project Crawling at Home Project
Crescent CrazyWebCrawler
CrunchBot
Curious
Custo Custo
DBLBot DBLBot
DIIbot DIIbot
DSearch
DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Dirbuster Demon
DnyzBot
DomCopBot DomCopBot
DomainStatsBot DomainAppender
Domains Project Domains Project
Download Wonder Dotbot
Dragonfly Dragonfly
Drip EMail Wolf
EasyDL EasyDL
EirGrabber Ebingbong
EroCrawler
Evil
Exabot Exabot
Express WebPictures ExtLinksBot
Extreme Picture Finder Extractor
EyeNetIE EyeNetIE
FDM Ezooms
FHscan FemtosearchBot
Fimap
Firefox/7.0 Firefox/7.0
Flunky FlashGet
Freeuploader Foobot
FrontPage
Fuzz Fuzz
FyberSpider FyberSpider
Fyrebot G-i-g-a-b-o-t
GetWeb
Getintent
Go!Zilla
Go-Ahead-Got-It
GrabNet GrabNet
Grabber Grabber
Grafula
GrapeFX GrapeFX
GridBot GrapeshotCrawler
HMView HMView
HTMLparser HTMLparser
Haansoft HTTrack
Havij Harvest
Heritrix Hloader
Humanlinks
HybridBot HybridBot
IDBot IDBot
IRLbot
Iblog Iblog
Id-search Image Sucker
Image Fetch InfoTekies
IndeedBot
InternetSeer
Iskanie
JOC Web Spider JOC Web Spider
JamesBOT JikeSpider
Jbrofuzz JustView
Jorgee Kozmosbot
LWP::Simple LNSpiderguy
Lanshanbot LeechFTP
Lftp LexiBot
Libwhisker
LinkScan LinkScan
LinkextractorPro Linkbot
LinkpadBot LinkpadBot
LinksManager LinqiaRSSBot
LinqiaScrapeBot
Lipperhey
Lipperhey Spider Lipperhey Spider
Litemage_walker Litemage_walker
Ltx71 MFC_Tear_Sample
MQQBrowser MQQBrowser
MSFrontPage MSFrontPage
MTRobot
Mag-Net
Majestic-SEO Majestic-SEO
MarkMonitor MarkWatch
Masscan Mata Hari
Mediatoolkitbot Mb2345Browser
Metauri MegaIndex.ru
Microsoft URL Control Microsoft Data Access
Mister PiX
Mojolicious Mojolicious
Morfeus Fucking Scanner NPbot
Mr.4x3
Name Intelligence Name Intelligence
Nameprotect
NearSite NearSite
Net Vampire
NetLyzer
NetMechanic
NetSpider NetSpider
Netvibes
NextGenSearchBot NextGenSearchBot
Niki-bot Nibbler
Nuclei Nimbostratus
Ninja
Nmap
Nutch
Offline Explorer Offline Explorer
OnCrawl
OpenVAS OpenVAS
OutfoxBot OrangeBot
PECL::HTTP PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
Panscient Page Analyzer
PageAnalyzer
PageGrabber
Papa Foto Papa Foto
PeoplePal Pavuk
Pi-Monster Petalbot
Picsearch Picsearch
PictureFinder
Pixray Pixray
Pockey PxBroker
Psbot
QueryN Metasearch QueryN Metasearch
RSSingBot Quick-Crawler
RankFlex
RankingBot2
Rankivabot
RankurBot
Re-re Re-re
RebelMouse ReGet
RedesScrapy Recorder
RepoMonkey RepoMonkey
SBIder Ripper
Rogerbot
SEOkicks-Robot SEOkicks-Robot
SEOprofiler SMTBot
SalesIntelligent
ScanAlert ScanAlert
Scanbot Scanbot
Scrapy ScoutJet
Screaming
Searchestate Searchestate
SearchmetricsBot SeobilityBot
SentiBot
SeoSiteCheckup
Seomoz
Siphon Siphon
Site Sucker
SiteExplorer
SiteSnagger SiteSnagger
Sitebeam
Siteimprove
SmartDownload
Snapbot Snapbot
Snoopy Snoopy
SocialRankIOBot SocialRankIOBot
Sociscraper Sogou web spider
Spammen Sottopop
SpaceBison
SpankBot SpankBot
Spanner Spbot
Spinn3r SputnikBot
Sqlmap
Sqworm Sqworm
Sucker
Sucuri Sucuri
Surfbot Suzuran
SurveyBot Swiftbot
Teleport T8Abot
TeleportPro
Telesoft Telesoft
Telesphorep Telesphoreo
TheNomad TheNomad
Titan Toata
Tracemyfile Toweyabot
Trendictionbot Trendiction
True_Robot TwengaBot
Turnitin
TurnitinBot
Twice
Typhoeus
Upflow
V-BOT
VB Project
VCI
Vagabondo
VeriCiteCrawler VeriCiteCrawler
VidibleScraper Virusdie
VoidEYE Voil
Voltron
WISENutbot WISENutbot
WPScan
WWW-Collector-E WWW-Collector-E
WWW-Mechanize WWW::Mechanize
Web Collage WallpapersHD
Web Fuck
Web Pix Web Pix
Web Sauger Web Sauger
WebAuto WebAuto
WebBandit WebBandit
WebCollage
WebEnhancer WebEnhancer
WebFetch WebReaper
WebFuck WebStripper
WebGo IS
WebPix
WebZIP WebZIP
Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
WebsiteQuester WebsiteExtractor
Whack
Whacker
Whatweb Whatweb
WiseGuys Robot Widow
WinHTTrack
Wonderbot Wonderbot
Woobot Woobot
Wprecon
Xaldon WebSpider
YoudaoBot YoudaoBot
Zauba Zauba
Zermelo
Zitebot Zitebot
ZoominfoBot ZumBot
ZyBorg
archive.org_bot archive.org_bot
arquivo-web-crawler arquivo-web-crawler
autoemailspider autoemailspider
backlink-check clark-crawler
cah.io.community
check1.exe
coccocbot-web
cognitiveseo
demandbase-bot demandbase-bot
eCatch evc-batch
heritrix
instabid instabid
internetVista monitor ips-agent
isitwp.com lwp-request
magpie-crawler
meanpathbot meanpathbot
netEstate NE Crawler mediawords
muhstik-scan
oBot oBot
page scorer plumanalytics
polaris version probe-image-size
ripz s1z.ru
scalaj-http
seobility
seoscanners seoscanners
serpstatbot
sexsearcher
sitechecker.pro sitechecker.pro
sogouspider
spyfu
sysscan sysscan
tAkeOut trendiction.com
voyagerx.com trendiction.de
webpros.com webpros.com
xpymep1.exe xpymep1.exe
zgrab zauba.io

288
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 404checker
80legs 80legs
Aboundexbot ADmantX
AIBOT
Abonti
Acunetix
AiHitBot AiHitBot
Alexibot
Anarchie
Anarchy99 Anarchy99
Ankit Anthill
Asterias Aspiegel
Attach AwarioRssBot
BBBike BDFetch
BDCbot
BackDoorBot BackDoorBot
BackStreet
BackWeb BackWeb
Backlink-Ceck Badass
BatchFTP Black Hole
BetaBot
Blackboard
Blow
Bolt
Brandprotect
Buck Buck
BuiltBotTough BuiltBotTough
Bullseye BunnySlippers
BuzzSumo
CATExplorador
CCBot CCBot
CazoodleBot CazoodleBot
Cegbfeieh ChinaClaw
CheTeam Chlooe
CherryPicker Cliqzbot
Cosmos Cloud mapping
Crescent Cocolyzebot
Collector
CopyRightCheck
Crawling at Home Project
Curious
Custo Custo
CyotekWebCopy
DIIbot DIIbot
DTS Agent DSearch
DataCha0s
Devil Devil
Disco Digincore
Discobot Dirbuster
Discoverybot
Dispatch Dispatch
DittoSpyder
DnyzBot DnyzBot
DomainAppender DomCopBot
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
Domains Project
Dotbot Dotbot
Download Wonder Download Wonder
Dragonfly Drip
ECCP/1.0 EMail Siphon
EroCrawler EMail Wolf
Ebingbong
EirGrabber
Evil
Express WebPictures Express WebPictures
ExtLinksBot
ExtractorPro ExtractorPro
Extreme Picture Finder Ezooms
EyeNetIE
FDM FDM
FemtosearchBot FHscan
Fimap Flunky
Foobot
Freeuploader
FrontPage
FyberSpider
Fyrebot Fyrebot
GalaxyBot GT::WWW
GermCrawler Genieo
Getintent GetWeb
Go!Zilla GrabNet
GoZilla GrapeFX
Grafula
GrapeshotCrawler GrapeshotCrawler
GridBot
HEADMasterSEO
Haansoft Haansoft
Harvest Harvest
Havij Heritrix
IDBTE4M Humanlinks
IlseBot Iblog
Id-search
Image Fetch Image Fetch
IndeedBot
Indy Library Indy Library
InfoTekies InfoTekies
Intelliseek Intelliseek
InterGET InternetSeer
Internet Ninja JamesBOT
Iria Jbrofuzz
IstellaBot JennyBot
JetCar JetCar
Jetty
Jorgee Jorgee
JustView
Jyxobot
Kenjin Spider Kenjin Spider
Keyword Density
Kinza
Kozmosbot Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple
Lanshanbot Lanshanbot
Larbin
Leap Leap
LeechFTP LeechGet
Lftp Lftp
LieBaoFast Lightspeedsystems
LinkScan LinkScan
LinkWalker
Linkdexbot Linkdexbot
Lipperhey LinkextractorPro
MFC_Tear_Sample LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Litemage_walker
Lmspider
MIIxpc MIIxpc
Mag-Net MSFrontPage
MSIECrawler
MTRobot
Mail.RU_Bot Mail.RU_Bot
Majestic SEO
MarkMonitor MarkMonitor
Masscan Mass Downloader
Mata Hari Mata Hari
MauiBot MeanPath Bot
Moblie Safari Meanpathbot
Mediatoolkitbot
MicroMessenger
Mojolicious Mojolicious
Mozlila Morfeus Fucking Scanner
Mr.4x3
Musobot Musobot
NICErsPRO NICErsPRO
NPbot Net Vampire
Nessus NetZIP
NetMechanic
Netcraft
Nettrack
NextGenSearchBot NextGenSearchBot
Nibbler Nibbler
Nikto Nikto
NimbleCrawler NimbleCrawler
Nimbostratus Nimbostratus
Nmap
Nuclei
Offline Explorer Offline Explorer
OrangeSpider Offline Navigator
OutfoxBot OpenLinkProfiler
PageAnalyzer OpenVAS
PageThing.com PHPCrawl
PageScorer
Pandalytics Pandalytics
Papa Foto Pavuk
Petalbot PeoplePal
Pi-Monster
Picscout
Picsearch Picsearch
Piepmatz PictureFinder
Pixray PleaseCrawl
Pockey
ProPowerBot
ProWebWalker
Probethenet Probethenet
Pu_iN
PxBroker
PyCurl
QueryN Metasearch
RankActive
RankActiveLinkBot
RankingBot RankingBot
RankingBot2
Rankivabot
Re-re Re-re
RebelMouse ReGet
RealDownload
Recorder Recorder
RedesScrapy RedesScrapy
RocketCrawler
Rogerbot
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler
SEOstats SEOstats
SISTRIX SISTRIX
SMTBot SMTBot
SalesIntelligent SalesIntelligent
ScoutJet Scanbot
Scrapy
ScreenerBot
Searchestate Searchestate
SearchmetricsBot
SeoSiteCheckup
SeobilityBot SeobilityBot
Shodan Siphon
SiteLockSpider
Site Sucker Site Sucker
Sitevigil Sitebeam
SocialRankIOBot Siteimprove
SmartDownload
Snoopy
Sociscraper Sociscraper
Sosospider
Sottopop Sottopop
SpankBot Spammen
Spbot Spbot
Spinn3r
Sqlmap Sqlmap
Sqlworm Sqlworm
Sqworm Steeler
Stripper Szukacz
SurveyBot
T0PHackTeam T0PHackTeam
Teleport T8Abot
Telesoft Telesoft
Telesphoreo
Telesphorep Telesphorep
TightTwatBot
Trendiction
Trendictionbot
True_Robot
Turnitin Turnitin
Twice URLy Warning
Typhoeus
UnisterBot UnisterBot
V-BOT
Vacuum Vacuum
Vagabondo Vagabondo
VeriCiteCrawler VidibleScraper
VoidEYE VoidEYE
Voil Voil
Voltron Voltron
WEBDAV WEBDAV
WISENutbot
WPScan WPScan
WWW-Collector-E
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WallpapersHD WWWOFFLE
WeSEE Web Auto
Web Enhancer
Web Fetch Web Fetch
Web Sauger
WebCopier
WebFetch
WebLeacher
Web Pix Web Pix
WebWhacker WebGo IS
WebLeacher
WebStripper
WebSucker
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Webshag
Website Quester Wprecon
WebsiteExtractor
WebsiteQuester
Webster
WiseGuys Robot
Xaldon WebSpider
Xaldon_WebSpider
Xenu Xenu
Zade
Zauba
Zermelo Zermelo
Zeus ZoominfoBot
Zitebot ZumBot
ZmEu
autoemailspider autoemailspider
backlink-check backlink-check
cah.io.community cah.io.community
check1.exe cognitiveseo
coccocbot-web
com.plumanalytics com.plumanalytics
dataforseo.com dataforseo.com
domainsproject.org heritrix
gopher
instabid instabid
internetVista monitor
lwp-request
magpie-crawler
muhstik-scan
netEstate NE Crawler netEstate NE Crawler
oBot oBot
plumanalytics page scorer
pcBrowser
polaris version polaris version
s1z.ru probe-image-size
ripz
satoristudio.net satoristudio.net
scalaj-http scalaj-http
scan.lol scan.lol
seostar sexsearcher
sitechecker.pro sogouspider
siteripz sysscan
spyfu
tAkeOut
trendiction.com trendiction.com
trendiction.de webmeup-crawler
x22Mozilla x09Mozilla
xpymep1.exe
zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
adconscious.com budilneg.xyz
dostavimvdom.ru citizenclsdriveri7.pen.io
ferretsoft.com dataloading.net
housemilan.ru invitefashion.com
oneshotdate.com iwanttodeliver.com
sexvporno.ru nsatc.net
sucsesofinspiration.com replicaclub.ru
tootoo.to turkeyreport.tk
vysigy.su winwotgold.pl
zawyna.ua yogamatsexpert.com

926
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com 00go.com
0lovespells0.blogspot.com 00webcams.com
108shot.com 1000n1.ru
123cha.com 100dollars-seo.com
123locker.com 178evakuator178.ru
1bet.com 1adult.com
214jaluwobapef.cf 5tey7463.weebly.com
2pxg8bcf.top 7a2rnuey1tw9ar.ru
40cg.com 7wind.ru
4rent.ru 8xv8.com
7makemoneyonline.com absolutelyanalog.com
7xc4n.com academia-nsk.org
883zy.com actulite.com
aa625d84f1587749c1ab011d6f269f7d64.com adrunnr.com
aarbur.com adtech.us
adclickservice.com afmuseum.com
addtoadd.men all4invest.ru
adminshop.com
admitad.com
adserver-e7.com
advokateg.ru
affrh2015.com
afslankpillen2017nl.eu
ald2014.org
alfa9.com
all4invest.info
allcredits.su allcredits.su
alldownload.pw alpinism.ru
allwidewallpapers.com am-se.com
alpharma.net android-vsem.org
alveris.ru angigreene.com
amazingninja.com animalia-life.club
ample-awards-today.us anjalika.co.in
ampower.me apiappenableinfo-a.akamaihd.net
anapa-inns.ru apibrowsesmartne-a.akamaihd.net
aosheng-tech.com
apiadanaknet-a.akamaihd.net
apiitsthirteende-a.akamaihd.net apiitsthirteende-a.akamaihd.net
apioasisspacenet-a.akamaihd.net appfastplay.com
applicationg29.com apple.com-webbrowsing-security.science
applyneedy.xyz
appsaurus.com
appsecurityr.com appsecurityr.com
aquarium-pro.ru arendas.net
arenda-avtoprokat-krasnodar.ru articlesdirectoryme.info
asiavirtualsolutions.com ascat.porn
atelielembrancaqueencanta.com.br aucoinhomes.com
atley.eu.pn autovideobroadcast.com
audiofree.ru aviapanda.ru
autobudpostach.club avitocars.ru
azlex.uz avon-novosib.ru
babyguns.ru azbaseimages.net
bang-hotties.com backiee.com
bankmib.ru baditri.com
bbsoldes.fr balkanfarma.ru
beauty-things.com battleforupdating.site
best-mam.ru beachfix.co
bestofferswalkmydogouteveryday.gq beachtoday.ru
besttorrentknifta.weebly.com belinvestdom.by
bg6s0.com best-games.today
bio.trade-jp.net bestimagecoollibrary.com
bkns.vn betteroffers.review
bonvillan.com beyan.host.sk
brandimensions.com bidr.trellian.com
buchananshardware.com biketank.ga
billiard-classic.com.ua
bitnote.co
board.f00d.de
bobba.dzaba.com
boo-arts.com
boobsimge.com
bookmaker-bet.com
boole.onlinewebshop.net
boom-celebs.com
bristolhostel.com
buckspar.gq
burger-tycoon.com
businesxxl.com
buttons-for-your-website.com
buyantiviralwp.com
buybest1.biz buybest1.biz
buyparajumpers.online buzzonclick.com
bvlgariwallet.xyz bvlgariring.xyz
bzero1jewelry.net callcafe.info
cardiosport.com.ua candycrushshop.com
carfax.com.ua canyougethighofftizanidine.blogspot.com
celebsopics.com capsderfudd.tk
carrauterie.be
cartierlove2u.xyz
cartierlovebracelet.xyz
cartierreplica.xyz
casinopinup-wins.com
casinorewards.info
catalogs-parts.com
changde.58.com
cheap-pills-norx.com
cheapbelstaff.online cheapbelstaff.online
cheapjerseysfootballshop.com cheatcode-lita12.rhcloud.com
chelyabinsk.xrus.org
cherubinimobili.it
chiptuninger.com
chlooe.com chlooe.com
cinemaenergy-hd.ru christianlouboutinreplica.win
citetick.com claytransformations.info
clash-clans.ru
cleansearch.net cleansearch.net
clickintext.net clodo.ru
clickso.com cmd.kz
clicksor.com cntravelre.com
clips.ua.ac.be cocyq.inwtrade.com
cmhomestayagency.com codq.info
com-onlinesupport.host cometorussia.net
compliance-josh.top compliance-alexa.top
contentsexpress.com compliance-barak.xyz
conocer-sanabria.com
coolwallpapers4k.info
cornomase.win
cosmetic.donna7753191.ru
cosmeticswomens-womensports.rhcloud.com cosmeticswomens-womensports.rhcloud.com
counter.yadro.ru cpajunkies.com
covetnica.com crackguru.tk
crafthubs.com creativehutindia.com
crest3d.ru creditnation.ru
culturevie.info crest-poloski.ru
customboxes4less.com cryptoswap.biz
cyclobenzaprinevstizanidine.blogspot.com customerguru.in
cyprusbuyproperties.com dame-ns.kz
danashop.ru
data1.scopich.com
davebestdeals.com davebestdeals.com
depositfiles-porn.ga dawlenie.com
detalizaciya-tut.biz daydream-studio.ru
deda-moroza-zakaz.ru
delayreferat.ru
derevesendeco.com
diegolopezcastan.com diegolopezcastan.com
digital-sale.su discovertreasure-a.akamaihd.net
digitalfaq.com disruptingdinnerparties.com
dimaka.info dkvorota.ru
dirtpics.pw dnepr-avtospar.com.ua
discovertreasurenow.com
dmmspy.com
documentsite.net documentsite.net
dominateforex.ml dokumentalkino.net
dostavka-v-krym.com donna7753191.ru
dprtb.com dood.live
drugs-no-rx.info doreenblog.online
drugstoreforyou.com doxyporno.com
dvrlists.com duawitchrarato.tk
e-commerce-seo1.com e-commerce-seo.com
eandsgallery.com
ebooknovel.club ebooknovel.club
edwinkonijn.com.au educatemetv.com
elvenar.com educontest.net
erectile.bid eets.net
eredijovon.com ekaterinburg.xrus.org
eropho.net electronicadirect.com
erotik-kostenlos.net elektrischezigarettekaufen2.cowblog.fr
ertelecom.ru elektronischezigarettekaufen2.tumblr.com
esfchat.tk elmifarhangi.com
essaypro.com en.home-task.com
eu-cookie-law.info enskedesquashclub.se
eurosamodelki.ru erot.co
evidencecleanergold.com eshop4u.jp
extremal-blog.com essay-edu.biz
ezofest.sk estelight.ru
evening-dating-club.info
ezaz.info
ezigarettekaufen1.hpage.com
ezigarettekaufen2.blox.pl
ezigarettekaufen2.yolasite.com
faithe.top
fanoboi.com
farmprofi.net
fashionavenuegame.com fashionavenuegame.com
fast-wordpress-start.com fellowshipoftheminds.com
favornews.com fertilitetsradgivningen.se
fickkontakte.org fickblock18.com
fifa-coins.online fishtauto.ru
finemanteam.com fitness-video.net
fitfloponline.store flavors.me
fitnesspiks.com formularz-konkurs.tk
fok.nl
food.dtu.dk
foodgid.net
formseo.com
forpackningsutveckling.se
forum.tvmir.org
forum69.info forum69.info
fototravel.eu fotzen-ficken.com
foxydeal.com foxinsocks.ru
fr-bearings.ru free-video-tool.com
free-fb-traffic.com freelotto.com
free-laptop-reward.com gambarkataku.co
free-share-buttons.blogspot.com gamevalue7.weebly.com
freenode.info gaming-journal.com
freewlan.info genericviagrasildenafiled.net
frustrated-favorable.gq genetworx.com
fuckmill.com gentamicineyedrops.blogspot.com
fungamelands.com getoutofdebtfree.org
galleryawesome.com glicol.kz
gamerextra.com global-ics.co.za
ge0ip.com goodly.pro
generic-pills-online.com googglet.com
get-free-social-traffic.com googlepositions.com
get-seo-domain.com gourcy.altervista.org
getridofstretchmarks.org greatzip.com
gktt.ru grtyi.com
glass-msk.ru gsasearchengineranker.site
globalsurfari.com gsasearchenginerankerhelp.com
gne8.com gta-club.ru
gojiberriess.apishops.ru guigyverpo.cf
golaya.pw gwebtools.com
goo.ne.jp h2monline.com
goodhumor24.com hamilton.ca
gopixdatabase.com hccoder.info
gosmeb.ru
greatidea.marketing
gribkovye-zabolevaniya.com
gribokstop.com
gsasearchengineranker.space
guardlink.org
handicapvansarea.com
havepussy.com
hchha.com
hd-film.pl hd-film.pl
hdwallpapers-free.com
headpress.ru headpress.ru
healgastro.com help.tpu.ru
health-medical-portal.info hentaiheroes.com
helicalpile.us hidefiles.org
helpmymacfaster.trade highstairs-a.akamaihd.net
heroz.fr hobbyhuren-datenbank.com
hitmuzik.ru homegardenlova.com
hoporno.com homemature.net
hoste.octopis.com hopeonthestreet.co.uk
hotsocialz.com hostingclub.lk
hotxnights.info hotenergy.ru
houdom.net hoterika.com
houtings.xyz howtoclean.club
hoverboard360.at hustoon.over-blog.com
hundejo.com ideibiznesa2015.ru
i-service.kz ikritikimou.gr
ibmdatamanagement.co imagine-ex.co
imagez.co inet-traffic.com
imfamous.info insider.pro
imgarcade.com instasexyblog.com
impisr.edunsk.ru invest-pamm.ru
import-sales.com invivo.hu
indiasourcemart.in iomoio.net
insta-add.pro iopeninghours.co.uk
internet-apteka.ru iradiology.ru
investyb.com irkutsk.online-podarki.com
irkutsk.zrus.org iron-age.info
isoveti.ru iskalko.ru
islamtoday.co.za
it-max.com.ua
itis4you.com
iusstf.org iusstf.org
jaxcube.info ivearchenceinflu.cf
jerseyswholesalechinalimited.com iwanttodeliver.com
kaidalibor.de izhevsk.xrus.org
kaliningrad.zrus.org jackpotchances.com
kareliatobacco.ru japan-bearings.ru
jongose.ninja
joy-penguin.com
jrcigars.com
juliadiets.com
jurajskie.info
jwcialislrt.com
kazka.ru kazka.ru
keyword-suggestions.com kerei.ru
kino-ecran.ru khadastoafarde.tk
kino2018.cc kicknights.gq
kinoduh.ru kinobest.pl
kinohall.ru kinomaniatv.pl
kinozapas.com kletkimehan.ru
kinozapas.org klosetkitten.com
kndxbkdx.bloger.index.hr kmd-pto.ru
koptims.tiu.ru koronirealestate.gr
kreativperlen.ch kosynka-games.ru
kredit-blog.ru ku6.com
kursy-ege.ru kw21.org
landscaping.center lafriore.ru
languagecode.com landoftracking.com
lapitec.eu
lapolis.it
larose.jb2c.me larose.jb2c.me
larutti.ru legionalpha.com
lashstudia.ru lion.misba.us
laulini.soclog.se littlecity.ch
lernur.net livefixer.com
lexixxx.com
lida-ru.com
light.ifmo.ru
limtu.ifmo.ru
lionking-1994.blogspot.com
lirunet.ru
live-cam6.info
liveinternet.ro
llastbuy.ru llastbuy.ru
luxup.ru loadingpages.me
macfix.life locatellicorretor.com.br
mactechinfo.info loftdigital.eu
lolnex.us
lostfilm-online.ru
lumb.co
lunchrock.co
luxmagazine.cf
m292.info
madot.onlinewebshop.net madot.onlinewebshop.net
makenahartlin.com makemoneyonline.com
mansiondelrio.co manuscript.su
marblestyle.ru masterseek.com
masturbate.co.uk medanestesia.ru
mebeldekor.com.ua medicineseasybuy.com
member-quiz.com migente.com
members.ghanaweb.com miniature.io
mesbuta.info mirzonru.net
miniads.ca moscow.online-podarki.com
missis.top moz.com
misswell.net mttwtrack.com
mjchamonix.org mug-na-chas-moscow.ru
mompussy.net mulberryoutletonlineeu.com
monsterdivx.tv musezone.ru
morepoweronmac.trade music7s.me
mostantikor.ru mustat.com
moxo.com mwtpludn.review
mp3ringtone.info myanyone.net
mrcsa.com.au mylesosibirsk.ru
mturkcontent.com mymobilemoneypages.com
musicspire.online naj-filmy24.pl
my-bc.ru narkologiya-penza.ru
my-big-family.com nash-krym.info
my-cash-bot.co net-radar.com
mydeathspace.com nhl17coins.exblog.jp
mydirtystuff.com nibbler.silktide.com
myhealthcare.com nisuturnetdgu.tk
mylida.org o-o-8-o-o.ru
na-telefon.biz oddamzadarmo.eu
nailsimg.com office-windows.ru
nakrutka.cc office2web.com
nardulan.com ohmyrings.com
narosty.com ok.ru
news-readers.ru online-hit.info
next-dentists.tk onlinefilmz.net
nfljerseyscheapchinabiz.com onlinemegax.com
nikhilbahl.com onlineporno.site
notaria-desalas.com owohho.com
noumeda.com paramountmarble.co.uk
novosibirsk.xrus.org partnerline.men
nrjmobile.fr pcboa.se
o-dachnik.ru pdns.download
o.light.d0t.ru pendelprognos.se
offer.camp
okout.ru
onlinadverts.com
onlineserialy.ru
onlinewritingjobs17.blogspot.ru
onlyporno.ru
opinionreelle.com
orel-reshka.net
paighambot.com
parajumpersjakkesalgnorge.info
pastaleads.com
pawli.eu
paydayonlinecom.com
perm.xrus.org perm.xrus.org
perosan.com
photosaga.info
piccdata.com
picphotos.net picphotos.net
pigrafix.at pihl.se
plastweb.ru pinwallpaper.top
pogruztehnik.ru piuminiita.com
poisk-zakona.ru pixelrz.com
poppen-nw.net pixgood.com
popunder.ru pizza-imperia.com
porn555.com placid-rounded-coast.glitch.me
pornoforadult.com plastgranar.nu
pornohd1080.online plastjulgranar.se
pornoinn.com poems.com.ua
pornoplen.com pokemongooo.ml
pornosmola.info pony-business.com
pornotubs.com pornhive.org
powenlite24.ru porno-chaman.info
predmety.in.ua pornogig.com
pregnant.guru pornoklad.net
preparevideosafesystem4unow.site posible.net
printie.com pp-budpostach.com.ua
privat-girl.net pr0fit-b0x.com
prlog.ru preconnubial.usuby.site
profit-opportunity.com predominant-invent.tk
profitsport.club priora-2.com
progress-upakovka.ru productarium.com
promoforum.ru proekt-mos.ru
prosmibank.ru prointer.net.ua
pshare.biz prostitutki-kharkova.org
psoriasis-file.trade psoriasis-file.trade
pxhdwsm.com pufip.com
py100.ru r-control.ru
qcstrtvt.bloger.index.hr rada.ru
qwertty.net raisedseo.com
ranking2017.ga rapidgator-porn.ga
rasteniya-vs-zombi.ru raschtextil.com.ua
razorweb-a.akamaihd.net rating-bestcasino.com
realtytimes.com rebelmouse.com
recipedays.ru redhotfreebies.co.uk
remontgruzovik.ru registrysweeper.com
retailwith.com reklama1.ru
retreatia.com renecaovilla.online
reward-survey.net renewablewealth.com
rff-cfal.info responsinator.com
ric.info reyel1985.webnode.fr
ricorsogiustizia.org rezeptiblud.ru
rique.host.sk rn-to-bsn.com
robotixix.com rockma.se
rockingclicks.com rockprogblog.com
ruex.org.ua ruex.org.ua
ruf777.com running-line.ru
russkie-gorki.ru runofilms.ru
s1z.ru rvzr-a.akamaihd.net
scanner-brian.top sakhboard.ru
scanner-julia.top sawin.beth.webd.pl
scanner-olga.top sbricur.com
search.pch.com sbt-aqua.ru
searchengineranker.email scanner-alex.top
searchimpression.com scanner-elena.top
searchkut.com scanner-willy.top
searchinquire.com
securesmrt-dt.com securesmrt-dt.com
service.adtech.fr sel-hoz.com
serviporno.com seminarygeorgia59.ga
sex-dating.co seolab.top
sex-foto.pw servicecenter.co.ua
sexreliz.com serving.adbetclickin.pink
sexspornotub.com servingnotice.com
sexuria.net sex-watch.com
sexkontakte-seite.com
sexkvartal.com
sexy.babes.frontend-stack.top
sexystrippe.info
sexyteens.hol.es
shariki-zuma-lines.ru shariki-zuma-lines.ru
silktide.com shikiso.info
simon3.ru signoredom.com
simplepooltips.com site-speed-check.site
siteexpress.co.il sixcooler.de
sitevaluation.com sladkoevideo.com
sizeplus.work slim.sellany.ru
smokewithrabbits.com slujbauborki.ru
snowplanes.com small-games.biz
snsdeainavi.info snts.shell-pmr.ru
snyatie-lomki-v-stacionare.ru sogimlecal.tk
soblaznu.net spb.ru
soc-proof.su sr-rekneskap.no
social-s-hhh.xyz stefanbakosab.se
socialbutton.xyz stickers-market.ru
soda.media stop-zavisimost.com
soochi.co storehouse.ua
sovetogorod.ru
special-porn.com
speechfoodie.com
spitfiremusic.com
stairliftstrue.com
statoutlook.info
stats-collector.org
strag-invest.ru strag-invest.ru
strangeduckfilms.com stretchmate.net
stroicol.net strongholdsb.ru
sucsesofinspiration.com stroyhelp-dv.ru
sunflowerdrawingpaintings.blogspot.com stroymonolit.su
supermesta.ru studiofmp.com
svetlotorg.ru superinterstitial.com
svetodiodoff.ru supermama.top
swimpool.ca svnuppsalaorebro.se
tamada69.com swinginwithme.ru
syndicate.fun
tandvardshuset.net
tattoo33.ru tattoo33.ru
taxi-v-eisk.ru
teastory.co
tedy.su tedy.su
testbotprocessor44.com templates.radiodigital.co
terrafootwear.us
testingads.pro
tfxiq.com tfxiq.com
theallgirlarcade.com thaismartloan.com
theendivechronicles.com thebestphotos.eu
thefotosgratis.eu thedownloadfromwarez.blogspot.com
thenews-today.info theporndude.com
theprofitsmaker.net titangel-vietnam.com
tizanidinehydrochloride4mgstreetvalue.blogspot.com tizanidineandhydrocodone.blogspot.com
tizanidinesideeffectsweightloss.blogspot.com toolsky.com
tn811.us trafficgenius.xyz
tnaionline.org tsan.net
top-study.work tubeline.biz
top1-seo-service.com turbo-suslik.org
top10registrycleaners.com
topads.men
topappspro.com
torrents-tracker.com
totu.info
toyota.7zap.com
tracklead.net
traffique.net
trustaffs.com
tubeoffline.com
tula.mdverey.ru
tupper-shop.ru
turizm.bz turizm.bz
u17795.netangels.ru tvorozhnaja-zapekanka-recept.ru
uac.net.au twelvevisionspartyofcolorado.com
udayavani.com typimga.pw
ugogo.info ufa.xrus.org
uhod-za-sobakoj.ru uggbootsoutletsale.us
ukrtextbook.com uhdtv.website
upstore.me unitexindia.com
upupa.net usadacha.net
utiblog.fr uvozdeckych.info
valueclick.com vadimkravtcov.ru
vektorpress.ru validccseller.com
venta-prom.ru valmetrundan.se
via-energy-order.com vclicks.net
via-gra.webstarts.com veerotech.com
viagra-soft.ru ventelnos.com
viagroid.ru verymes.xyz
videogamesecrets.com video-production.com
virus-respirators.com vinylvault.co.uk
virus-schutzmasken.de vkak.ru
vkontaktemusic.ru vod.com.ua
vltai.com voprosotvet24.ru
vodaodessa.com vseigru.one
voronezh.xrus.org vseigry.fun
vucms.com vzlomsn.org
walkme.com w7s.ru
wallpapersist.com walpaperlist.com
warningzscaler.heraeus.com wanker.us
webcamtalk.net watchmyfb.pl
webix.me web-betting.ru
webmasterhome.cn web.cvut.cz
webix.biz
webscouter.net
westen-v.life westen-v.life
whipme.yopoint.in wheelchairliftsarea.com
wholesalejerseysgaa.com winner7777.net
wineitudes.wordpress.com
winterclassichockeyjerseys.com winterclassichockeyjerseys.com
withstandingheartwarming.com
womama.ru womama.ru
woodyguthrie.se works.if.ua
wordkeyhelper.com world-mmo.com
wrona.it wtsindia.in
ww1943.ru www.timer4web.com
wyniki-lista.pl wygraj-teraz.com
x-diesel.org xmladserver.com
x-rates.ru xn----9sbubg3ambdfl1j.xn--p1ai
xn--80abgj3a5acid6ghs.top xn--80aanaardaperhcem4a6i.com
xn--80atua3d.xn--p1ai xn--80aodinpgi.xn--p1ai
xn--90acenikpebbdd4f6d.xn--p1ai xn--hxazdsfy.blogspot.com
xportvusbdriver8i.snack.ws xn--oogle-wmc.com
xtrafficplus.com xolodremont.ru
xxxrus.org xpresscare.ru
youdesigner.kz xxlargepop.com
youghbould.wordpress.com yaaknaa.info
your-bearings.com yadro.ru
youtoner.it
youtubologia.it
ytmnd.com ytmnd.com
zajm-pod-zalog-nedvizhimosti.ru zaloadi.ru
zakonobosago.ru zebramart.ru
zed21.net zeroredirect12.com
zeroredirect7.com zeroredirect9.com
zigarettenonlinekaufen1.blox.pl zhacker.net
zoodrawings.com zoompegs.com
zrelaya.pw zygophyceous.womanstars.site
zrus.org zynax.ua
zryydi.com

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
08800.top 01apple.com
202ch.com 123any.com
3xforum.ro 51unlim.ru
72-news.com 7zap.com
777-club.ru
abcguru.xyz
absugars.com
abtasty.com
abusora.com abusora.com
adexprts.com acads.net
admitad.com adloads.com
adultnet.in advokat-grodno.by
advertex.info affordablewebsitesandmobileapps.com
advokateg.xyz affrh2015.com
ahrntal.verymes.xyz agroeconom.kz
akusherok.ru alert.scansafe.net
alf-img.com alkoravto.ru
allforminecraft.ru allboard.xobor.de
aosheng-tech.com amyfoxfitness.com
apiitsthirteende-a.akamaihd.net apibetweenlinesn-a.akamaihd.net
aproposde.com app-ready.xyz
arkartex.ru appfixing.space
automatic-seo.com appleid-verification.com
autoseo-traffic.com applicationg29.com
azartmix.com apxeo.info
backlinkwatch.com arendovalka.xyz
banksrf.ru as5000.com
besttorrentknifta.weebly.com asseenontv.ru
bildsuche.ru auto.rusvile.lt
bocoarchives.org automobile-spec.com
bouda.kvalitne.cz aviav.eu
bscodecs.com bannerswap.com
buehne-fuer-menschenrechte.de basedecor.ru
bumskontakte.org bbsoldes.fr
bus-offer.com beauty-b0x.pl
businesn.men berlininsl.com
californianews.cf bestprofits-there.com
calvet.altervista.org betshuckclean.com
cartierbraceletsreplica.pw bg6s0.com
cartierreplica.top bget.ru
casablancamanor.co.za biznesluxe.ru
cgi2.nintendo.co.jp bluejays-jerseys.us
chatroulette.world briomotor.co
chinajerseyswholesalecoupons.com burger-tycoon.com
christianlouboutinoutlet.win businescoop.men
cleanmypc.com bvlgaribracelet.xyz
clips.ua.ac.be callawaygolfstore.online
cmhomestayagency.com canrioloadm.gq
com-secure.download cartierlovestore.com
commoncrawl.org certifywebsite.win
creditwell.ru cheapmoncler.win
customboxes4less.com chiptuninger.com
dame-ns.kz claytransformations.info
daydream-studio.ru club-samodelkin.ru
dealighted.com coolgramgoods.com
designdevise.com cowblog.fr
deutschland123.de culad.com
diusyawiga.tk ddpills.com
doctormakarova.ru dentalpearls.com.au
djihispano.com
docs4all.com
domain-tracker.com domain-tracker.com
download-walpaperhd.blogspot.com doreenblog.online
doyouknowtheword-flummox.ml drugs-no-rx.info
ecxtracking.com
edelstahlschornstein-123.de edelstahlschornstein-123.de
ekto.ee educontest.net
election.interferencer.ru egovaleo.it
electrik-avenue.com elektronischezigarettekaufen1.myblog.de
elektrozigarette2.webs.com european-torches.ru
enskedesquashclub.se exchanges-bet.com
escortslet.net f-online.de
eupornstar.info fapgon.com
ez8motelseaworldsandiego.com fast-wordpress-start.com
fanpagerobot.com femdom.twiclub.in
fashion.stellaconstance.co flowersbazar.com
fbdownloader.com flprog.com
feeriaclub.ru fok.nl
fefo.gdn forex.osobye.ru
fellowshipoftheminds.com free-social-buttons.com
fikasound.tk freegamesplay.online
findsexguide.com freenom.link
footbalive.org freesitetest.com
fotosfotos.eu freetangodownload.com
frankofficial.ru fuel-gas.com
freewhatsappload.com gamebackyard.com
freza-sverlo.ru geile-lelly.eu
geilehausfrauen.net gentamicineyedrops.blogspot.com
gemara.com getoutofdebtfree.org
go2jump.org gg.zzyjxs.com
golfresa.lucania.se glicol.kz
gopixdatabase.com globalscam.ga
graphicwe.org goldenggames.com
guardlink.org goodnightjournal.com
guidefs.ru gotwebsite1.com
hamptonoaks.ca guardlink.com
hazardky.net highland-homes.com
hdwalls.xyz hkladys.com
hildinghr.se home.myplaycity.com
hotkeys.com hully.altervista.org
hotloans.ru husky-shop.cz
imadedinner.net hut1.ru
imicrovision.com hydropump.su
infobabki.ru idegenvezeto.eu
innoslicon.com ilo134uloh.com
intelhdgraphicsgtdrive6w.metroblog.com images-graphics-pics.com
interior-stickers.ru imfamous.info
imgarit.pw
imitex-plus.ru
infodocsportal.com
intim-uslugi.info
io9.com io9.com
it-max.com.ua iopeninghours.co.uk
itrevolution.cf iqbazar.ru
ixora.pro istizanidineanarcoticdrug.blogspot.com
izhevsk.xrus.org jacago.com
journalhome.com javlibrary.cc
karaganda.xkaz.org jbl-charge.info
kino2018.cc juliaworld.net
kinoflux.net karpun-iris.ru
kinogolos.ru katadhin.co
kinohit1.ru katjimej.blog.fc2.com
kiwi237au.tk kerwinandcariza.com
kumuk.info keywordblocks.com
ladsblue.com kimcurlrvsms.com
lflash.ru konkursowo-24.pl
lifehacĸer.com labvis.host.sk
live-sexchat.ru lechenie-gemorroya.com
lkbennettstore.online live-cam6.info
macfix.life lollypopgaming.com
magicalfind-a.akamaihd.net louboutinreplica.xyz
mansparskats.com lyrics.home-task.com
meetingrainstorm.bid mabdoola.blogspot.com
mhi-systems.ru mac-shield.com
midst.eu magnetpress.sk
modabutik.ru marcoislandvacations.net
moncleronline.xyz mega-bony-2017.pl
more-letom.ru memberty.com
morf.snn.gr moesen-ficken.com
morocco-nomad-excursions.com moetomnenie.com
moroccosurfadventures.com
moscow.online-podarki.com
mountainstream.ms
mouselink.co
mowser.com mowser.com
mrpornogratis.xxx moxo.com
nacap.ru mriyadh.com
narosty.com musezone.su
negral.pluto.ro musirc.com
netoil.no muztops.ru
newtechspb.ru nalogovyy-kodeks.ru
nhl17coinsforps3.gratisblog.biz naphukete.ru
niroo.info neodownload.webcam
nodup.ru newstaffadsshop.club
novodigs.com oda.as
nufaq.com olympescort.com
nw-servis.ru online7777.com
o-o-6-o-o.com oralsexfilme.net
oecnhs.info paclitor.com
oivcvx.website
onclkads.com
onlyforemont.ru
openfrost.com
optikremont.ru
paparazzistudios.com.au paparazzistudios.com.au
partners-ship.pro paradontozanet.ru
pekori.to patterntrader-en.com
perl.dp.ua
picscout.com picscout.com
pills24h.com pixshark.com
podshipniki-ntn.ru playlott.com
porno-raskazy.ru polimga.pw
pornokajf.com popunder.ru
pornoklad.net pornosemki.info
pozdravleniya-c.ru pornozhara.com
pozvonim.com porodasobak.net
professionalwritingservices15.blogspot.ru powenlite24.ru
pskcijdc.bloger.index.hr prizestohandle.club
purplesphere.in productarium.com
qld10000.net professionaldieselcare.com
quangcaons.com profitfx.online
qwertty.net pulse33.ru
qwesa.ru realmonte.net
reckonstat.info rednise.com
remont-komputerov-notebook.ru reversing.cc
remontvsamare.su sad-torg.com.ua
resant.ru sadaholding.com
reyel1985.webnode.fr satoristudio.net
rubanners.com sayyoethe.blogspot.co.za
rukino.org sbt-aqua.ru
rus-pornuha.com scanner-don.top
safe-app.net scanner-fred.top
salutmontreal.com scanner-julianna.top
sanyuprojects.com schalke04fc.info
scanner-george.top scripted.com
scanspyware.net searchinterneat-a.akamaihd.net
sex-tracker.de security60-e.com
sextracker.be seminarygeorgia59.ga
sexvporno.ru seofirmreviewsus.info
shopvilleroyboch.com.ua seorank.info
site.ru servethis.com
skanninge.se serving.adbetclickin.pink
skatestick.bid servisural.ru
slimcdn.com sexy-pings.com
slkrm.ru slomm.ru
smart-balancewheel.com smartbalanceworld.com
snsdeainavi.info snworks.com
sps-shop.com social-button.xyz
spywareit.com sptslmtrafms.com
stariy-baku.com staticfs.host
streha-metalko.si supermesta.ru
studentguide.ru supermodni.com.ua
susanholtphotography.com sweepstakes.rewardit.com
svolze.com syvertsen-da.no
tamada69.com szamponrevita.pl
taxi-v-eisk.ru tech4master.com
thiegs.reco.ws thebestweightlosspills.ovh
tizanidinedosage.blogspot.com theendivechronicles.com
tizanidineingredients.blogspot.com tizanidine4mgstreetvalue.blogspot.com
tizanidinerestlesslegsyndrome.blogspot.com tizanidinenarcotic.blogspot.com
tnctrx.com tk-assortiment.ru
toyota.7zap.com totu.info
traffmonster.info tovaroboom.vast.ru
trkdf.com tradgardspartner.se
trion.od.ua
trucri.me
truebeauty.cc
ttsq.fr ttsq.fr
tvorozhnaja-zapekanka-recept.ru tupper-posuda.ru
uac.net.au uasb.ru
underthesite.com unblocksit.es
unionmarkt.de unimodemhalfduplefw.pen.io
uogonline.com unrealcommander.com
upproar.com utrolive.ru
vapsy.com vasileostrovsky-rayon.ru
virus-respirators.com via-energy-acquistare.com
volume-pills.biz viagra.pp.ua
vzlom-na-zakaz.com videotuber.ru
wait3sec.org vip.51.la
wallpapers-best.com vitalads.net
wallpapersdesk.info vkmusics.ru
warningzscaler.heraeus.com webadvance.club
wdrake.com webjam.com
webmasterhome.cn webmonetizer.net
while.cheapwebsitehoster.com wesharepics.info
wildnatureimages.com williamrobsonproperty.com
windowssearch-exp.com win-spy.com
wormix-cheats.ru wtsindia.in
xn----9sbubg3ambdfl1j.xn--p1ai www.wohnkabinen-shop.de
xn--80aanaardaperhcem4a6i.com wygraj-skiny.win
xnxx-n.com x-lime.com
xpresscare.ru xev.ru
yatrk.xyz xn--80aikhbrhr.net
yougame.biz xn--80aodinpgi.xn--p1ai
youghbould.wordpress.com xtremeeagles.net
youjizz.vc xxxguitars.com
yoursite.com yaaknaa.info
youtoner.it yopoint.in
zarenica.net youradulthosting.com
zvetki.ru zarabotat-na-sajte.ru
zigzog.ru
znakomstvaonlain.ru
zophim.me
zvezdagedon.ru

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.90 0; 10.1.1.181 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2647 ### Version: V4.2021.10.2648
### Updated: Tue Oct 12 22:02:59 UTC 2021 ### Updated: Wed Oct 13 10:12:13 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2647 ### Version: V4.2021.10.2648
### Updated: Tue Oct 12 22:02:59 UTC 2021 ### Updated: Wed Oct 13 10:12:13 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##