vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-09 05:54:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.05.3079 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-05-01 22:07:08 +00:00
parent ec1bf7b3ed
commit 08463f5e77
47 changed files with 96809 additions and 96799 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3078 #### Version: V4.2022.05.3079
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3078 #### Version: V4.2022.05.3079
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.05.3078 #### Version: V4.2022.05.3079
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11964
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.05.3078 #### Version: V4.2022.05.3079
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11968
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11968
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11968
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11968
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 1; 10.1.0.77 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 0; 10.1.0.77 0;
127.0.0.1 0; 127.0.0.1 0;

11968
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 1; 10.1.0.77 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 0; 10.1.0.77 0;
127.0.0.1 0; 127.0.0.1 0;

11968
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 1; 10.1.0.77 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 0; 10.1.0.77 0;
127.0.0.1 0; 127.0.0.1 0;

11968
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3078 3079

11968
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11968
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.245.49.2 0; 20.245.105.176 0;

11968
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 1; 10.1.0.77 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 0; 10.1.0.77 0;
127.0.0.1 0; 127.0.0.1 0;

11968
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 1; 10.1.0.77 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 0; 10.1.0.77 0;
127.0.0.1 0; 127.0.0.1 0;

11968
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 1; 10.1.0.77 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 0; 10.1.0.77 0;
127.0.0.1 0; 127.0.0.1 0;

11968
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.245.49.2 0; 20.245.105.176 0;

11968
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.245.49.2 0; 20.245.105.176 0;

11968
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 1; 10.1.0.77 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Black Hole Bitacle
Cosmos Bullseye
DnyzBot Harvest
Gotit IndeedBot
Moblie Safari InfoNaviRobot
NetLyzer Jorgee
Openvas Majestic12
Woobot Nettrack
Zeus Webalta
trendiction.de ripz

292
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider
404checker
80legs 80legs
AIBOT AIBOT
ALittle Client Aboundex
ASPSeek Aboundexbot
Acunetix Acunetix
AhrefsBot Alligator
Aipbot AlphaBot
Alexibot Anarchy
Anarchie
Ankit
Apexoo Apexoo
Aspiegel Aspiegel
Asterias AwarioRssBot
Atomseobot BatchFTP
Attach
BDFetch
BLEXBot
BacklinkCrawler
Bandit
Barkrowler
Battleztar Bazinga
BetaBot BetaBot
Bigfoot Bigfoot
Bitacle BlackWidow
Blackboard Blackboard
Boardreader
Bolt Bolt
BotALot
Brandprotect Brandprotect
Buck Buddy
BuiltBotTough BuiltWith
BunnySlippers
CATExplorador
CCBot CCBot
CODE87 CazoodleBot
Calculon
Cegbfeieh
CensysInspect
CheeseBot
ChinaClaw ChinaClaw
Chlooe
Claritybot Claritybot
Cocolyzebot Cocolyzebot
Cogentbot Collector
Cosmos Cosmos
Craftbot Crawling at Home Project
Crescent CrazyWebCrawler
CrunchBot
Custo Custo
CyotekWebCopy DBLBot
DSearch DataCha0s
Demon
Deusu
DigitalPebble
Dirbuster Dirbuster
Discoverybot Discobot
Dispatch DittoSpyder
DnyzBot
DomCopBot DomCopBot
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Dotbot
Download Wonder
ECCP/1.0 ECCP/1.0
EMail Wolf
EasyDL
Evil
Exabot Exabot
Express WebPictures
ExtLinksBot ExtLinksBot
Extractor ExtractorPro
EyeNetIE Ezooms
FDM
FHscan
Fimap
FlashGet FlashGet
Flunky Foobot
Freeuploader
FrontPage FrontPage
FyberSpider FyberSpider
Fyrebot G-i-g-a-b-o-t
GT::WWW GT::WWW
GalaxyBot GetRight
Genieo Go-Ahead-Got-It
GetWeb
Go!Zilla
GoZilla GoZilla
Gotit
GrabNet GrabNet
GrapeFX Grafula
GrapeshotCrawler
HMView HMView
HaosouSpider HTTrack
Harvest Haansoft
Heritrix
HonoluluBot HonoluluBot
Humanlinks Humanlinks
HybridBot
IDBTE4M IDBTE4M
IDBot
Iblog
Id-search
Image Fetch
Image Sucker Image Sucker
InternetSeer IndeedBot
Iria InfoNaviRobot
Iskanie InterGET
JetCar Internet Ninja
IstellaBot
Joomla
Jorgee Jorgee
JustView
Keyword Density Keyword Density
Kozmosbot Kozmosbot
LNSpiderguy
Lanshanbot Lanshanbot
Lftp Leap
LeechFTP
Libwhisker
LinkScan LinkScan
Linkbot LinkpadBot
LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Spider Litemage_walker
Lmspider
MIIxpc MIIxpc
MJ12bot MJ12bot
MSIECrawler MQQBrowser
MSFrontPage
MTRobot MTRobot
Mag-Net
Magnet Magnet
Mail.RU_Bot Majestic-SEO
MarkMonitor MarkMonitor
MarkWatch MarkWatch
Mass Downloader Mata Hari
MauiBot MegaIndex.ru
Mb2345Browser MicroMessenger
Meanpathbot Microsoft URL Control
Mediatoolkitbot Mister PiX
Mozlila Moblie Safari
Mojolicious
MolokaiBot
Mr.4x3 Mr.4x3
Musobot Musobot
NPbot Name Intelligence
Nameprotect
Navroad Navroad
NearSite NearSite
NetAnts Needle
NetZIP Net Vampire
NetSpider
Netcraft Netcraft
Nettrack Nibbler
Netvibes Nmap
Nikto
Not
Octopus Octopus
OpenVAS OpenVAS
Openfind
OrangeBot OrangeBot
OrangeSpider OutfoxBot
PECL::HTTP
PHPCrawl
Page Analyzer Page Analyzer
PageScorer PageScorer
PageThing.com
Panscient
Papa Foto Papa Foto
Petalbot
Picscout
Picsearch Picsearch
PictureFinder ProPowerBot
Piepmatz Pu_iN
Pimonster Pump
Pixray QueryN Metasearch
Psbot
Quick-Crawler
RSSingBot RSSingBot
RankFlex RankingBot2
Rankivabot
RankurBot RankurBot
RebelMouse Re-re
RepoMonkey RedesScrapy
Ripper Ripper
RocketCrawler
SBIder SBIder
SEOkicks
SEOlyticsCrawler SEOlyticsCrawler
SEOstats SMTBot
SISTRIX
SalesIntelligent SalesIntelligent
Scanbot Scanbot
Screaming ScoutJet
Scrapy
ScrepyBot ScrepyBot
SearchmetricsBot
Seekport
SemanticJuice SemanticJuice
Semrush
SeobilityBot SeobilityBot
Siphon
SiteExplorer
SiteSucker
Siteimprove Siteimprove
SmartDownload
Snake
Snoopy
Sociscraper Sociscraper
SpaceBison Spanner
Spbot Spbot
Sqlworm Sqlworm
Sqworm Steeler
Sucuri Sucker
SuperBot
SuperHTTP SuperHTTP
Surfbot Surfbot
TeleportPro Teleport
Telesoft
Telesphoreo Telesphoreo
Telesphorep
The Intraformant
Thumbor Thumbor
TightTwatBot TightTwatBot
Titan
Toata
Toweyabot
Trendictionbot Trendictionbot
True_Robot
Turnitin Turnitin
Twice Twice
Upflow Typhoeus
VB Project UnisterBot
VCI V-BOT
Vagabondo
VeriCiteCrawler VeriCiteCrawler
Virusdie VidibleScraper
VoidEYE
Voil Voil
Voltron
WISENutbot WISENutbot
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
Web Auto Web Fetch
Web Collage Web Sauger
Web Enhancer Web Sucker
Web Fuck
WebAuto WebAuto
WebCollage
WebCopier WebCopier
WebImageCollector WebFetch
WebSauger WebFuck
WebWhacker WebLeacher
WebZIP WebReaper
Webalta WebStripper
Webshag WebSucker
WebmasterWorldForumBot
WebsiteExtractor WebsiteExtractor
Webster Webster
Who.is Bot Who.is Bot
WiseGuys Robot WinHTTrack
Wotbox Wotbox
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade Zade
Zeus Zermelo
Zitebot ZoomBot
ZumBot ZumBot
ZyBorg
archive.org_bot archive.org_bot
arquivo-web-crawler check1.exe
arquivo.pt
autoemailspider
coccocbot coccocbot
com.plumanalytics cognitiveseo
crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j crawler4j
domainsproject.org evc-batch
gopher facebookscraper
heritrix meanpathbot
isitwp.com netEstate NE Crawler
iubenda-radar
lwp-trivial
mediawords
muhstik-scan
page scorer page scorer
plumanalytics polaris version
seocompany.store probe-image-size
satoristudio.net
seobility
seostar
sexsearcher sexsearcher
sitechecker.pro
spyfu spyfu
sysscan voyagerx.com
tAkeOut webgains-bot
trendiction.com
ubermetrics-technologies.com
webprosbot webprosbot
x09Mozilla x22Mozilla
zgrab

298
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 80legs
404checker AIBOT
ADmantX
ALittle Client ALittle Client
ASPSeek Abonti
Aboundexbot Aboundexbot
Acunetix
AiHitBot AiHitBot
Aipbot
Alexibot Alexibot
AllSubmitter AllSubmitter
Alligator Alligator
AlphaBot Anarchy
Anarchy99
Anthill Anthill
Aspiegel
Atomseobot Atomseobot
BBBike
BDCbot BDCbot
BDFetch
Backlink-Ceck Backlink-Ceck
BacklinkCrawler
Badass Badass
Bandit
BatchFTP BatchFTP
BetaBot BetaBot
Bigfoot Bigfoot
Black Hole BlackWidow
Brandprotect Boardreader
BunnySlippers Brandwatch
CATExplorador Buddy
Calculon BuiltBotTough
CheTeam Bullseye
Claritybot CCBot
Cliqzbot CODE87
CSHttp
CazoodleBot
Cegbfeieh
CensysInspect
Cloud mapping Cloud mapping
Copier Copier
CopyRightCheck
Copyscape Copyscape
DIIbot Craftbot
CrazyWebCrawler
CrunchBot
Curious
Custo
DSearch DSearch
DTS Agent
DataCha0s DataCha0s
Deusu Demon
Dispatch Dirbuster
DnBCrawler-Analytics DnyzBot
DomCopBot
DomainAppender DomainAppender
DomainCrawler ECCP/1.0
Dragonfly EasyDL
EMail Wolf Ecxi
Ebingbong Exabot
EirGrabber FDM
EroCrawler
Express WebPictures
ExtLinksBot
Extreme Picture Finder
FHscan FHscan
Fimap Fimap
Flunky
Foobot
Fuzz Fuzz
GermCrawler GermCrawler
GetRight GetRight
GetWeb
Getintent
Go-Ahead-Got-It Go-Ahead-Got-It
GrapeFX Grabber
GridBot HMView
HTTrack Haansoft
Heritrix Heritrix
Hloader
HonoluluBot HonoluluBot
Humanlinks Humanlinks
HybridBot
IDBot
IlseBot IlseBot
Image Fetch Image Sucker
IndeedBot
Indy Library Indy Library
InfoNaviRobot Intelliseek
Internet Ninja
Iria
Iskanie
IstellaBot IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz
JennyBot JennyBot
Jetty Jetty
JikeSpider JikeSpider
Joomla Joomla
Jorgee Keyword Density
Kenjin Spider LeechGet
LWP::Simple LexiBot
LeechFTP Libwhisker
Lftp
LieBaoFast LieBaoFast
LinkScan Likse
LinkWalker LinkWalker
Linkbot LinksManager
LinkextractorPro LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
Litemage_walker Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc MIIxpc
MQQBrowser
MSFrontPage
MSIECrawler MSIECrawler
Magnet Majestic SEO
Majestic12
MarkMonitor MarkMonitor
MarkWatch Mass Downloader
Masscan
Mata Hari
Mb2345Browser
Meanpathbot
Mediatoolkitbot Mediatoolkitbot
Metauri MegaIndex.ru
Minefield MicroMessenger
Mister PiX Microsoft URL Control
Moblie Safari
MolokaiBot MolokaiBot
Morfeus Fucking Scanner
Msrabot
Musobot Musobot
NICErsPRO
Name Intelligence Name Intelligence
Nessus Nameprotect
NetAnts NetAnts
NetSpider NetMechanic
NetZIP NetZIP
Netcraft
Nettrack Nettrack
Netvibes
Nibbler
Niki-bot
Nimbostratus Nimbostratus
Nutch Not
Octopus
Offline Explorer Offline Explorer
Offline Navigator
OrangeBot OrangeBot
OutfoxBot OrangeSpider
PECL::HTTP
PHPCrawl PHPCrawl
PageAnalyzer Page Analyzer
PageGrabber PageScorer
Pi-Monster Papa Foto
PeoplePal
Petalbot
Picscout Picscout
PictureFinder Picsearch
Piepmatz PleaseCrawl
Pockey ProWebWalker
Psbot Pu_iN
Pump
PxBroker PxBroker
RSSingBot RankFlex
RankActive RankingBot
RankingBot2
RankurBot RankurBot
Ripper Re-re
ReGet
RedesScrapy
RepoMonkey
RocketCrawler RocketCrawler
SEOkicks-Robot Rogerbot
SEOstats SEOlyticsCrawler
SMTBot ScoutJet
ScanAlert Scrapy
Scanbot
Screaming
ScrepyBot
Searchestate Searchestate
SentiBot SearchmetricsBot
SeobilityBot SemrushBot
Seomoz
Siphon
Site Sucker Site Sucker
SlySearch SiteLockSpider
Snoopy SiteSucker
Sosospider Siteimprove
Sitevigil
Snake
SocialRankIOBot
SpaceBison SpaceBison
Spanner SpankBot
Spbot Sqlmap
Sqlworm Sqlworm
Sucker Sucuri
SuperBot SuperHTTP
Surfbot Surfbot
Suzuran
Swiftbot Swiftbot
T0PHackTeam T0PHackTeam
T8Abot
Teleport
Telesoft Telesoft
Telesphorep
The Intraformant The Intraformant
Thumbor
TightTwatBot
Toata
Toweyabot
Tracemyfile Tracemyfile
Trendiction
Trendictionbot
True_Robot True_Robot
Turingos
Turnitin Turnitin
TurnitinBot VCI
TwengaBot
Typhoeus
UnisterBot
Vacuum
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
Virusdie Virusdie
VoidEYE Voltron
Voil WASALive-Bot
WEBDAV
WPScan WPScan
WWW-Mechanize WWW-Collector-E
Wallpapers
WallpapersHD
WeSEE WeSEE
Web Auto
Web Collage Web Collage
Web Enhancer
Web Sauger
Web Sucker Web Sucker
WebBandit
WebCollage
WebCopier WebCopier
WebEnhancer
WebFetch WebFetch
WebPix WebFuck
WebReaper
WebSauger
WebStripper WebStripper
WebSucker WebWhacker
WebZIP
Webalta Webalta
WebmasterWorldForumBot
WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster Whack
Whacker
Who.is Bot Who.is Bot
Widow Widow
WiseGuys Robot WiseGuys Robot
Wonderbot
Woobot Woobot
Wprecon
Xaldon WebSpider Xaldon WebSpider
Xenu YoudaoBot
Zauba Zauba
ZumBot ZoominfoBot
adscanner adscanner
archive.org_bot
arquivo.pt
cah.io.community cah.io.community
check1.exe
clark-crawler
coccocbot
cognitiveseo cognitiveseo
com.plumanalytics
crawl.sogou.com crawl.sogou.com
crawler4j dataforseo.com
demandbase-bot domainsproject.org
eCatch
facebookscraper
gopher
heritrix heritrix
instabid instabid
internetVista monitor
iubenda-radar
linkdexbot linkdexbot
lwp-trivial lwp-request
magpie-crawler magpie-crawler
meanpathbot page scorer
mediawords plumanalytics
muhstik-scan probe-image-size
netEstate NE Crawler s1z.ru
oBot seoscanners
ripz
scalaj-http
scan.lol
seostar seostar
sexsearcher serpstatbot
sitechecker.pro sogouspider
tAkeOut sp_auditbot
trendiction.com
trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webprosbot webprosbot
xpymep1.exe x09Mozilla
zauba.io x22Mozilla
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
germes-trans.com agysya.ru
impresagaia.it inzn.ru
keyhantercume.com seorankinglinks.us
keywordsuggest.org sientalyric.co
kostenlos-sexvideos.com surintech.ac.th
labelwater.se textads.men
lider-zhaluzi.kiev.ua thepornsex.org
segol.tv traffic2money.com
tiandeural.ru yunque.pluto.ro
znakomstvaonlain.ru zvuker.net

938
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
007angels.com 0daymusic.org
00it.com 101lesbian.xyz
01apple.com 24videos.tv
123kuma.com 256bit.by
1688.com 4istoshop.com
1bet.com 4k-player.pl
1pamm.ru 4rent.ru
202ch.com 5i2.net
273-fz.ru DomainStatsBot
2drittel.de aa14ab57a3339c4064bd9ae6fad7495b5f.com
4ureyesonly.com aa8780bb28a1de4eb5bff33c28a218a930.com
7search.com abcdeg.xyz
a.frcls.fr abcguru.xyz
aac52006ec82a24e08b665f4db2b5013f7.com abiente.ru
abctoppictures.net academiacsmendoza.org
abwa.tk acortarurl.es
activecampaign.dreamhosters.com actices.com
adclickservice.com adclickservice.com
adimmix.com adconscious.com
admatic.com.tr adman.se
adminshop.com adtech.us
adrenalin-stk.ru adviceforum.info
ads-seo.men advokateg.xyz
adultfriendfinder.com agapovdg.ru
adverster.com aghanyna.com
air-edem.ru airlandsea.info
akama.com alienwheel.es
alldubai.biz alveris.ru
allkrim.com amazingninja.com
amigobulls.com amazon-adsystem.com
amoi.tn android-vsem.org
amung.us angkortours.vn
analnoeporno.tv anmysite.com
anatomiy.com
anniemation.com
apartamentwroclaw.eu apartamentwroclaw.eu
appiq.mobi apibrowsesmartne-a.akamaihd.net
arabsexxxtube.com apiitsthirteende-a.akamaihd.net
arquapetrarca.info apps-analytics.net
aruplighting.com aproposde.com
asophoto.com aramaicmedia.org
asrv-a.akamaihd.net as5000.com
asrv-a.akamoihd.net atlasvkusov.ru
atout-energie-69.com autobrennero.it
autolombard-krasnodar.ru autolombard-krasnodar.ru
availit.weebly.com avon-novosib.ru
aversis.net azbukadiets.ru
azazu.ru azte.ch
b-styles.xyz bagsonsale.online
babs.com.ua baixar-musicas-gratis.com
bala.getenjoyment.net bbsoldes.fr
banan.tv bear.gotcher.us
bankiem.pl bestmarriages.com
bannerconnect.net bfz.biz
basedecor.ru bidbuy.co.kr
bavsac.com bimatoprost-careprost.com
best-businessman.ru bird1.ru
best-way.men birzha-truda.eu
bestchoice.cf biznesschat.net
bestpriceninja.com bjorkbacken.nu
binomo.com blackcurranthumidifiers.site
bio-market.kz bloglag.com
bitnote.co blogos.kz
bjanshee.ru blogsfunky672.weebly.com
blackwitchcraft.ru blpmovies.com
boanonihaca.tk bluesman.nu
boeuklad.com bmw.afora.ru
bolezniorganov.ru bolegapakistan.com
bonky.biz boobsimge.com
bonzibuddi.com bookmark4you.com
bosman.pluto.ro brakehawk.com
brandbucket.com brianhenry.co
brus-vsem.ru brillianty.info
buckspar.gq budpost.com.ua
buigas.00it.com business-made-fun.com
builtwith.com buytizanidineonlinenoprescription.blogspot.com
bumascloset.com buyviagraa.blogspot.com
bus-offer.com byme.se
businescoop.men cactussoft.biz
buttons-for-website.com calvet.altervista.org
buyfriend.ru
buynorxx.com
buytizanidineonline.blogspot.com
c1.onedmp.com
calcularpagerank.com
californianews.cf
callawaygolfstore.online
cam-kontakte.org cam-kontakte.org
cannibalcheerleader.com canyougethighofftizanidine.blogspot.com
carapics.com
cartechnic.ru
cartierlovebracelet.xyz cartierlovebracelet.xyz
casablancamanor.co.za cartujano-pre.de
cashonads.com cercacamion.it
cdncash.net cerev.info
cenoval.ru cezartabac.ro
cgi2.nintendo.co.jp chatroulette.world
cialischmrx.com cheapjerseysfootballshop.com
clayvasedesigns.tk checkm8.com
chelyabinsk.xrus.org
ci.ua
cialischstgerts.com
cleanpcnow.com cleanpcnow.com
coin-hive.com clickhype.com
colobit.biz collegeessay19.blogspot.ru
compliance-ivan.xyz compliance-george.top
compliance-willy.top compliance-viktor.top
connectionstrenth.com conocer-sanabria.com
construmac.com.mx contentlook.co
coolgamechannel.com cordstrap.cc
coolyarddecorations.com creditnation.ru
cosmeticswomens-womensports.rhcloud.com creditwell.ru
crirussian.ru cryptoswap.biz
culad.com culmimg.pw
custom-electric-guitar.com custom-electric-guitar.com
customchocolate.business-for-home.com cyprusbuyproperties.com
cymbaltaandtizanidine.blogspot.com d2jsp.org
daneshetabiat.com dailyfinancefix.com
damedingel.ya.ru
daretodonate.co daretodonate.co
de.zapmeta.com daydream-studio.ru
deanmoore.ie daymusam.com
dekoration.us db.speedup.gdn
dengi-pod-zalog-nedvizhimosti.ru dbmkfhqk.bloger.index.hr
detalizaciya-tut.biz demenageur.com
dfiles.me depositfiles-porn.ga
dieswaene.com deshevo-nedorogo.ru
digitalassetmanagement.site detoxmed24.ru
diminishedvalueoforegon.com deutschehobbyhuren.net
dipstar.org dickssportinggoods.com
documentbase.net disruptingdinnerparties.com
domain-submit.info divku.ru
domaincdn.xyz dmmspy.com
dnsrsearch.com
domaincrawler.com domaincrawler.com
domainsatcost.com domik-derevne.ru
drugs-no-rx.info doublepimp.com
dvrlists.com downloadmefiranaratb1972.xpg.com.br
e-poker-2005.com dtm-spain.com
emailaccountlogin.co dtnlyss.com
eonpal.com dvd-famille.com
eropho.com dverimegapolis.ru
event-tracking.com earncash.com.ua
easync.io
ecommerce-seo.org
ecxtracking.com
electro-prom.com
elektrischezigarettekaufen2.cowblog.fr
elektronischezigarettekaufen2.tumblr.com
eloxal.ru
empis.magix.net
englishdictionaryfree.com
ereko.ru
essayassist.com
european-torches.ru
evidencecleanergold.com evidencecleanergold.com
exchangeit.gq exchangeit.gq
exchanges-bet.com expolicenciaslatam.co
exmasters.com extlabs.io
expertblog.info eyessurgery.ru
eyes-on-you.ga eywords-monitoring-your-success.com
ezigarettekaufen2.mpbloggar.se f5mtrack.com
falcoware.com fanrto.com
fatfasts-4tmz.com fashion-mk.net
fefo.gdn fashion-stickers.ru
fetishinside.com fealq.com
fifa-coins.online fellowshipoftheminds.com
findpics.pw fettefrauen.net
firstsiteguide.com fillmewithhappiness.com
fishtauto.ru fingerprintjs.com
fiuxy.com floating-share-buttons.com
flashbannernow.com flowersbazar.com
florida-tourism.net foodgid.net
fm-upgrade.ru fordsonmajbor.cf
forum69.info formulaantiuban.com
fototravel.eu foto-basa.com
fr.netlog.com free-floating-buttons.com
free411games.com
freetangodownload.com freetangodownload.com
fres-news.com freshsuperbloop.com
frvo.alptandem.ru frivgame250.com
fuck-paid-share-buttons.xyz fuckbuddybestgilf.info
fuckingawesome.com fungirlsgames.net
funnel.co.za funnypica.com
gamedayassist.com fx-brokers-review.com
gameonasia.com g33.org
gamezblox.com gael-s.ru
gaypornmovie.net gallictures.com
gelezki.com gardene.ru
gerhardhealer.com gay-file.com
gay.adultgalls.com
genericlowlatencyasiodriverhq.aircus.com
getdot.ru
ghostvisitor.com
gilsonchiro.xyz gilsonchiro.xyz
girlsatgames.ru goldenggames.com
given2.com greatzip.com
glastecfilms.com.my gsasearchenginerankerhelp.com
glavprofit.ru guod.me
go2jump.org habermetre.com
googst2.ru hdxnxxtube.mobi
greatdealshop.com headpharmacy.com
grtyi.com heartofbeijing.blogspot.com
gsasearchengineranker.space hidefiles.org
gsou.cf historichometeam.com
guidefs.ru hiwibyh.bugs3.com
guildebzh.info homedo.fabpage.com
gwebtools.com.br hotelcrocenzi.sm
haikuware.com housemilan.ru
handicapbathtubarea.com hscsscotland.com
harmonyglen.us hunthillfarmtrust.org
hdpixent.com hyhj.info
hdwallpapers-free.com
headpress.ru
healbio.ru
help.tpu.ru
hexpilot.com
himazin.info
hobbyhuren-datenbank.com
hotloans.ru
hspline.com
hybrid.ru
ihc2015.info
iideaidekonkatu.info
iinstalll-fii1leis.jus0wil.pp.ua iinstalll-fii1leis.jus0wil.pp.ua
ikritikimou.gr ilbe.club
imagui.eu imgarcade.com
imgarit.pw imitex-plus.ru
inboundlinks.win imperia31.ru
indo-export.ru indo-export.ru
inet-traffic.com infobabki.ru
integritylandscapeservices.com infodocsportal.com
intellekt21.ru infoupdate.org
interesnie-faktu.ru inmate-locator.us
ipostroika.ru isotoner.com
iqupdatetmz.win istanbulit.com
istizanidineanarcoticdrug.blogspot.com jagg.info
ivanovo.zrus.org jav-idol.com
ivearchenceinflu.cf jaxcube.info
jabimgo.pw jimmychoosale.online
jacago.com joy-penguin.com
jav-fetish.site
jav-way.site
javrip.net
javstock.com
jeremyeaton.co
jetsli.de
journeydownthescale.info
juliadiets.com juliadiets.com
jurajskie.info
justprofit.xyz justprofit.xyz
jwss.cc karaganda.xkaz.org
kambasoft.com
kamen-e.ru
karusel-market.ru karusel-market.ru
kazinogames.lv kasino-money.pw
keenoutlet.online keywordblocks.com
kellyonline.xyz kiwe-analytics.com
kinoflux.net klejonka.info
kinoplen.ru konoplisemena.com
kiprinform.com kosynka-games.ru
klin3952.ru
kollesa.ru
krasnodar.ru
kreativperlen.ch
kriokomora.info
ktotut.net
kurbappeal.info kurbappeal.info
kvartir-remont.biz la-fa.ru
landscaping.center lacapilla.info
legalrc.biz lafriore.ru
lernur.net laparfumotec.com
lexiangwan.com lasvegaslockandsafe.com
lightinghomes.net laulini.soclog.se
lignofix.ua law-check-eight.xyz
lineshops.biz letslowbefast.site
linkarena.com lezbiyanki.net
lifeinsurancekb.com
linkrr.com
lion.misba.us
liquimondo.com liquimondo.com
livejasmin.com liveinternet.ro
lombardfinder.ru lombardfinder.ru
lonely-mature.com longlifelomilomi.info
lostaruban.com lovi-moment.com.ua
lsp-awak-perikanan.com lsex.xyz
luckyday.world lubetube.com
macdamaged.tech lyrics.home-task.com
magicalfind-a.akamaihd.net m0r0zk0-krava.ru
mall.uk mabdoola.blogspot.com
mansparskats.com mafcards.ru
mantramusic.ru magazin-pics.ru
matb3aa.com magnetpress.sk
matsdale.com mail.allnews24.in
mbiologi.ru manimpotence.com
mecash.ru matpre.top
meccadumps.net medicineseasybuy.com
med-zdorovie.com.ua meds-online24.com
medispainstitute.com.au
megavolt.net.ua megavolt.net.ua
misswell.net mejoresfotos.eu
mixx.com menhealed.net
mncrftpcs.com mesbuta.info
mobilemedia.md metarip.ru
mobstarr.com minecraft-rus.org
modnie-futbolki.net miniads.ca
mole.pluto.ro minyetki.ru
monclerjacketsoutlet.pw mnimmigrantrights.net
mostantikor.ru moi-glazki.ru
mozello.ru monarhs.info
mulberryoutletonlineeu.com moncleronline.xyz
muschisexbilder.com moneyzzz.ru
musicspire.online mosdverka.ru
muz-baza.net moz.com
my-floor.in.ua mrpornogratis.xxx
myliveblog.ru mug-na-chas-moscow.ru
music.utrolive.ru
muz-tracker.net
muzaporn.com
mymercy.info mymercy.info
myperiod.club nacap.ru
namenectar.com namecrumilchlet.tk
naperehresti.info naughtyconnect.com
net-radar.com negociosdasha.com
netallergy.ru netallergy.ru
networkcheck.xyz news-readers.ru
newyorkhotelsmotels.info nextrent-crimea.ru
nonews.co nfljerseysforsalewholesaler.com
nvssf.com nicovideo.jp
oakleyglassesonline.us nofreezingmac.click
obiavo.in notfastfood.ru
odoratus.net nottyu.xyz
of-ireland.info novosti-hi-tech.ru
ofanda.com nw-servis.ru
offers.bycontext.com o-o-6-o-o.com
ohmyrings.com obiavo.uz
okroshki.ru obuv-kupit.ru
okuos.com oddamzadarmo.eu
online-podarki.com office-windows.ru
online-zaymy.ru oklogistic.ru
onlineku.com olgacvetmet.com
onthemarch.co oneclickfiles.com
ops.picscout.com oneminutesite.it
orion-code-access.net onlinefilmz.net
optitrade24.com
osnova3.ru osnova3.ru
ossmalta.com palocco.it
page2rss.com panicatack.com
paphoselectricianandplumber.com parajumpersoutlet.online
paydayloanslocal.com partner-trustworthy.men
pc-test.net pc-services.ru
pcboa.se pertlocogasilk.tk
pictures-and-images.net petsblogroll.com
picturesboss.com phuketscreen.com
piluli.info piccshare.com
pintattoos.com picturesfrom.com
piski.top pills24h.com
pix24x7.com pillscheap24h.com
pl-vouchers.com pizda.lol
platesauto.com pkr1hand.com
poem-paying.gq pl-top.pl
porno-dojki.net play-mp3.com
pornosee.info pogosh.com
pricheskaonline.ru pons-presse.com
privatbank46.ru popads.net
privetsochi.ru popup.matchmaker.com
porndairy.in
pornogig.com
pornokorol.com
pornoplen.com
portside.cc
pp-budpostach.com.ua
pregnant.guru
preventheadacheguide.info
privacylocationforloc.com
privhosting.com
prlog.ru
pro-okis.ru
prodess.ru prodess.ru
productarium.com proftests.net
prosmibank.ru prostitutki-kiev.org
prospekt-st.ru proxyelite.biz
prostitutki-almata.org purplestats.com
push-ad.com puteshestvennik.com
pussyfleet.com putevka24.ru
qor360.com pyramidlitho.webs.com
quickcashlimited.com quickloanbank.com
r-e-f-e-r-e-r.com quit-smoking.ga
rapidsites.pro ranksays.com
razamicroelectronics.com rapidgator-porn.ga
realitykings.com refererx.com
reeyanaturopathy.com reining.lovasszovetseg.hu
regdefense.com remont-fridge-tv.ru
relax.ru remont-ustanovka-tehniki.ru
remedyotc.com remybutler.fr
remontgruzovik.ru restorator-msk.ru
rennlist.com riders.ro
richinvestmonitor.com rock-cafe.info
robertefuller.com roma-kukareku.livejournal.com
rockma.se ros-ctm.ru
rome2rio.com rospromtest.ru
royal-betting.net ru-mediaget.ru
ruscopybook.com russia-tao.ru
sa-live.com sape.top
sabaapress.com scalerite.co.za
salutmontreal.com scanner-brian.top
sayyoethe.blogspot.co.za scanner-ivan.top
scanhub.ru scanner-nelson.top
scanner-andrew.top scenarii-1-sentyabrya.uroki.org.ua
scanner-viktor.top sdelatmebel.ru
school-diplomat.ru search-goo.com
searchwebknow-a.akamaihd.net searchimage.co
seinterface.com searchtooknow-a.akamaihd.net
seoheap.com seasonvar.ru
seorankinglinks.us seksvideoonlain.com
seorankinglinks.xyz seo-platform.com
service.adtech.us seo-tools-optimizing.com
serviporno.com seo-traffic-ranking.info
sexspornotub.com seocheki.net
shlyahten.ru seoprofiler.com
si-unique.com servethis.com
sibvitr.ru sextracker.com
simple-image.com.ua sextracker.de
sitebeam.net sexyteens.hol.es
siteimprove.com shijian.ac.cn
skutecznetabletkinaporostwlosow.pl shillyourcoins.com
smartadserver.com shohanb.com
snowplanes.com shymkent.xkaz.org
snworks.com sierraapps.com
social-s-ggg.xyz simplesite.com
social-s-hhh.xyz sinel.info
socialmonkee.com sirpornogratis.xxx
sortthemesitesby.com sispe.com.br
sosdepotdebilan.com skincrate.net
sovinsteel.ru skuteczna-dieta.co.pl
spinazdrav.ru slkrm.ru
spitfiremusic.com social-fun.ru
spravka130.ru sockshare.net
spy-sts.com sport-video-obzor.ru
spywarenuker.com spywarenuker.com
stairliftsarea.com
stariy-baku.com
starpages.net
stathat.com
stats-collector.org stats-collector.org
sterva.cc stats-public.grammarly.io
stevemonsen.com stmassage.ru
stoki.ru storehouse.ua
stopnarco.ru strigkaomsk.ru
store-rx.com stroiminsk.com
susanholtphotography.com stroydetali.ru
suture.co stroymonolit.su
svarbit.com stroyplus.ru
swagbucks.com students-cheapskate.ml
swingerseiten.com suggest-keywords.com
swsociety.se sunflowerdrawingpaintings.blogspot.com
sygraem.com superkanpo.com
symphonyintegratedhealthcare.com surcentro.com
takeflyte.com syvertsen-da.no
tandvardshuset.net tampabaywatch.org
taranerymagesswa.blogspot.com tarad.com
tds-advert005.info tasteidea.com
telefonsex-ohne0900.net tattooha.com
templates.franklinfire.co techart24.com
theendivechronicles.com tehngr.ru
thegamerznetwork.com telefonsexsofort.tk
tizanidine6mg.blogspot.com terrafootwear.us
tizanidineforopiatewithdrawal.blogspot.com thebestweightlosspills.ovh
todohr.com thelottosecrets.com
tovaroboom.vast.ru thepokertimer.com
toxicwap.com timer4web.com
toyota.7zap.com titan-ads.life
tradgardspartner.se tizanidinehcl4mgtablet.blogspot.com
traffic2cash.org tizanidineonline.blogspot.com
traffictrade.life tizanidinepills.blogspot.com
tri-slona.org toondinsey.com
triumf-realty.ru toonfamilies.net
ukkala.xyz toproadrunner5.info
ukkelberg.no totu.info
unisexjewelry.org trafficfactory.biz
urlopener.com trafficgenius.xyz
usswrite.com truck-addzilla.life
vabasa.inwtrade.com tuckermktg.com
valmetrundan.se turbo-suslik.org
video-chat.cn turvgori.ru
videochat.bz tvoystartup.ru
videochat.life tx41tclega.ru
videochat.tv.br uac.net.au
visa-pasport.ru uhod-za-sobakoj.ru
vitoriacabos.com ukrtextbook.com
vkontaktemusic.ru um-razum.ru
vksaver-all.ru umekana.ru
voditeltrezviy.ru uptimechecker.com
voluumtrk.com urll.eu
vsdshnik.com valoresito.com
vzube.com vapsy.com
w3javascript.com vchulkah.net
wallpaperaccess.com vertaform.com
warningzscaler.heraeus.com veselokloun.ru
webadvance.club viagroid.ru
webtherapy.ru viktoria-center.ru
weburlopener.com virtuagirl.com
wedding-salon.net vonradio.com
wejdz-tu.pl vtcdns.com
wemarketing.se vzglyadriv.kg
whengirlsgowild.com vzlom-na-zakaz.com
while.cheapwebsitehoster.com walkme.com
whipme.yopoint.in wcb.su
wikes.20fr.com webjam.com
winner7777.net webnode.me
womama.ru webshoppermac.com
www.souvenirua.com website-speed-checker.site
x-diesel.com websites-reviews.com
wesharepics.com
whatistizanidinehclusedfor.blogspot.com
wholesalejerseyscheapjerseys.us.com
windowssearch-exp.com
wnoz.de
wordpresscore.com
worldis.me
wpsecurity.website
wufak.com
x-lime.net
x-rates.ru
x-stars.ru x-stars.ru
xchat26.myfreecams.com xjlottery.com
xedserver.com xmnb.net
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
xn--80aanaardaperhcem4a6i.com xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
xn--80ajbshivpvn2i.xn--p1ai
xn--80ak6aa92e.com
xn--e1afanlbnfckd7c3d.xn--p1ai
xn--l1aengat.xn--p1ai xn--l1aengat.xn--p1ai
xn--oogle-wmc.com xnxxandxvideos.com
xxx-treker.ru xxxasianporn.net
xxxsiterips.xyz yaaknaa.info
xxxtube69.com yachts-cruise.info
yes-com.com yebocasino.co.za
youbrainboost.asia yottos.com
yourothersite.com yougame.biz
youtuhe.com yourmovies.pl
yuweng.info ypmuseum.ru
zacreditom.ru yuarra.pluto.ro
zaobao.com.sg
zapiszto.pl
zarabotat-na-sajte.ru
zeroredirect1.com
zeroredirect6.com zeroredirect6.com
zona-aqua.ru zhacker.net
zrus.org zhongwenlink.com
zverokruh-shop.cz znaturaloriginal.com
zojirushi-products.ru

494
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
3-letter-domains.net 03p.info
3weekdiet.com 2020iscoming.info
40cg.com 4webmasters.com
a.frcls.fr 70casino.online
aac500b7a15b2646968f6bd8c6305869d7.com 991mostfm.co.id
acgs.tk 9med.net
activepr.ru abusora.com
adidas.frwebs.fr acads.net
adsfresh.men addbin.men
adultfriendfinder.com ads-seo.men
advancedsoftwaresupport.com adtology.com
aibolita.com airlandsea.info
akvamaster.dp.ua alveris.ru
alert.scansafe.net android-style.com
angigreene.com android-vsem.org
apiadanaknet-a.akamaihd.net anlimebel.kiev.ua
atlant-auto.info apper.de
audiobangout.com apple.com-cleaner.systems
ayanaspa.com asociatia-tipografilor-transilvania.ro
b-styles.xyz atmagroup.ru
baikaleminer.com autorn.ru
bangla-chat-uk-paralud.ga babosas.com
bannerspace.com bambi.ck.ua
berdasovivan.ru banan.tv
best-games.today bank.uz
billiard-classic.com.ua best-deals-products.com
bmw-ark.ru besthoro.ru
bolezniorganov.ru bezcmexa.ru
bonkers.name bioinnovate.co
bpro1.top blagovest-med.ru
brandedleadgeneration.com bloglag.com
buchananshardware.com bnt-team.com
businesn.men bodybuilding-shop.biz
buzzonclick.com brk-rti.ru
byme.se butstrap.space
cannibalcheerleader.com buzzsumo.com
cartierjusteunclou.xyz bvlgariwallet.xyz
cartierloveringreplica.xyz camel-beach.com
cartierlovestore.xyz carina-sy.de
cejewelry.xyz cartierreplica.top
chocolateslim-en-espana.com censys.io
christianlouboutinsaleoutletonline.us cerev.info
clarithromycin500mg.com cheapcigarettesc.info
clash-clans.ru christianlouboutinoutlet.win
classicakuhni.ru cialisovercounteratwalmartusa.com
clips.ua.ac.be clayvasedesigns.tk
co.lumb.co clean-virus-mac.com
coin-hive.com clickintext.net
compliance-don.top com-secure.download
cookie-law-enforcement-aa.xyz compliance-jack.top
copyrightclaims.org contentlook.co
crynet.cc contextualyield.com
cs-passion.pl craftinsta.ru
cupday.com crazyprotocol.com
dengi-pod-zalog-nedvizhimosti.ru cyclobenzaprinevstizanidine.blogspot.com
deshevo-nedorogo.ru damedingel.ya.ru
deutschland123.de dekorkeramik.ru
dinkolove.ya.ru delta-line.men
domik-derevne.ru dikx.gdn
e-biznes.info divci-hry.info
election.interferencer.ru doeco.ru
eloconcream.blogspot.com duplicashapp.com
expolicenciaslatam.co e90post.com
fanpagerobot.com ec-file.info
filerockstar298.weebly.com ekobata.ru
filmi-onlain.info elektrischezigarettekaufen2.cowblog.fr
findacheaplawyers.com enginebay.ru
finemanteam.com englishgamer.com
fireads.men eralph.tk
flavors.me erotische-geschichten-xxl.com
flex4launch.ru escortslet.net
flowersbazar.com esfchat.tk
forex.osobye.ru eugenevaultstorage.com
forexgb.ru exchanges-bet.com
forum.poker4life.ru exportshop.us
foxtechfpv.com eyelike.com.ua
freecamdollars.com ezigarettekaufen1.hpage.com
friendflnder.com f0815.de
front.to f1nder.org
gameprimary.com fanoboi.com
gateway.zscalertwo.net ferieboligkbh.dk
gigapeta.com ferretsoft.com
goatse.ru findthe.pet
golaya.pw forum.doctissimo.fr
gowreckdiving.com forum69.info
guigyverpo.cf free-share-buttons.top
guruofcasino.com free-stock-illustration.com
hdxnxxtube.mobi fuck-paid-share-buttons.xyz
headpress.ru gamewrath.com
healing-dysplasia.ru garciniaxt.us
hentaiheroes.com gdzkurokam.ru
highland-homes.com gelezki.com
historichometeam.com go2album.com
homeart.space gorodservis.ru
huimin764128.com gsasearchengineranker.site
husky-shop.cz gsmlab.pl
iboss.com handicapbathtubarea.com
icaseclub.ru healgastro.com
ideoworld.org hetmanship.xyz
ilovevitaly.com himazin.info
investingclub.ru hostsshop.ru
isitwp.com images.gyffu.com
istizanidineacontrolledsubstance.blogspot.com imagui.eu
istizanidineanopiate.blogspot.com in-tandem.co
javcoast.com inboundlinks.win
javitas.info incep.imagine-ex.co
justucalling32211123456789.tk indianmedicaltourismshop.com
kakablog.net inmate-locator.us
investpamm.ru
invivo.hu
io9.com
jerseychinabizwholesale.com
jmat.cn
jpcycles.com
jwss.cc
jyrxd.com
kanimage.com kanimage.com
kazrent.com kino2018.cc
keenoutlet.online kountrylife.com
keywords-monitoring-your-success.com kozhniebolezni.com
keywordsdoctor.com kreativperlen.ch
kino-rf.ru kw21.org
kino2018.club ldrtrack.com
kinozapas.org lifehacĸer.com
kiwe-analytics.com lineshops.biz
kmgamex.cf louboutinreplica.pw
kostenloser-sex.com lovelycraftyhome.com
kskjco.club lovi-moment.com.ua
lakomka.com.ua lyngdalhudterapi.no
laparfumotec.com
laptoper.net
lashstudia.ru
lasvegaslockandsafe.com
lecbter-relationships.ga
lida-ru.com
lietaer.com
likesuccess.com
limtu.ifmo.ru
linkarena.com
linkpulse.com
livefixer.com
longadventure.com
longlifelomilomi.info
losangeles-ads.com
magazin-pics.ru
magicplayer-s.acestream.net magicplayer-s.acestream.net
mailemedicinals.com mamasuper.prom.ua
medicines-choice.com maxxtor.eu
medizinreisen.de med-zdorovie.com.ua
meendo-free-traffic.ga medosmotr-ufa.ru
megakino.net mere.host.sk
metafilter.com michellblog.online
mini-modus.ru mlvc4zzw.space
mirobuvi.com.ua mmostrike.ru
monclercoats.xyz mojpregled.com
montazhnic.ru more-letom.ru
mrinsidesales.com moscow-region.ru
msk.afora.ru nardulan.com
muizre.ru net-radar.com
musezone.ru nextrent-crimea.ru
musicktab.com nodding-passion.tk
musirc.com o.light.d0t.ru
mydirtyhobby.com offside2.5v.pl
mylovelibrabry.com onko-24.com
nikitsyringedrivelg.pen.io orgasmus-virtual.com
northfacestore.online paramountmarble.co.uk
npoet.ru partner-high.men
o333o.com perl.dp.ua
ochistka-stokov.ru photokitchendesign.com
office2web.com picturesify.com
openmediasoft.com pipki.r.acdnpro.com
owohho.com pk-services.ru
pandarastore.top ploenjitmedia.azurewebsites.net
paphoselectricianandplumber.com pokemongooo.ml
pc-test.net popander.mobi
perso.wanadoo.es porno.simple-image.com.ua
petsblogroll.com pornoinn.com
picquery.com pornoklad.net
popugaychiki.com pornosmola.info
pornogig.com privetsochi.ru
pornotubexxx.name pwwysydh.com
prostitutki-astana.org qwesa.ru
prostitutki-odessa.org rank-checker.online
pshare.biz rankscanner.com
push-ad.com ranksonic.com
registrationdomainsite.com ratemodels.net
reimageplus.com realresultslist.com
rep-am.com registratciya-v-moskve.ru
resultshub-a.akamaihd.net remedyotc.com
retailwith.com rff-cfal.info
rightenergysolutions.com.au
roll123.com roll123.com
rossanasaavedra.net santechnik.jimdo.com
royalcar-ufa.ru scoopquest.com
royalvegascasino.com sdelatmebel.ru
rumamba.com searchimage.co
rus-pornuha.com secret.xn--oogle-wmc.com
rusexy.xyz seo-prof1.xyz
rutor.vip seoexperimenty.ru
sankt-peterburg.nodup.ru serving.adbetclickin.pink
sc-specialhost.com sexkvartal.com
schoolfiles.net silvercash.com
seadragonherbery.com skincrate.net
search.pch.com social-button.xyz
searchencrypt.com socialbookmarksubmission.org
searchlock.com sophang8.com
seoanalyses.com spl63.fr
servingnotice.com sps-shop.com
sevendays.com.ua spywarebegone.com
si-unique.com spywarenuker.com
site3.free-share-buttons.com sribno.net
siteonomy.com stathat.com
skanninge.se stopnarco.ru
slomm.ru sudexpert66.ru
soda.media supermodni.com.ua
softomix.com surfbuyermac.com
sortthemesitesby.com suture.co
spywareit.com tandvardshuset.net
stackthatbucks.com technopellet.gr
stairliftstrue.com thexart.club
startufa.ru thiegs.reco.ws
streha-metalko.si tilido.com
strfls.com tobeyouday.win
stroimajor.ru toondinsey.com
studiofaca.com top10-way.com
szucs.ru topdownloads.ru
taqplayer.info traffic2money.com
tattooha.com traffpartners.com
telegramdownload10.com tubo360.com
thefotosgratis.eu twelvevisionspartyofcolorado.com
threecolumnblogger.com ucoz.ru
tizanidineandhydrocodone.blogspot.com uhod-za-sobakoj.ru
tizanidinestreetvalue.blogspot.com unrealcommander.biz
top-karkas.ru upproar.com
topsy.com upupa.net
tour-line.net usbggettwku.ga
track2.shop v24s.net
tracksz.co vapomnoncri.tk
triumf-realty.ru vbikse.com
u17795.netangels.ru videochat.cafe
ugogo.info vilingstore.net
urccvfmc.bloger.index.hr visionwell.com.cn
vasileostrovsky-rayon.ru viteonlusarezzo.it
vekzdorov.ru vladimir.zrus.org
videochat.life vodaodessa.com
vladhistory.com vremya.eu
wieseversa.no watracker.net
x-musics.com webextract.profound.net
xep.info webix.me
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai westum.se
xn--d1acah0c.xn--p1ai whosonmyserver.com
yebocasino.com worldoffiles.ru
yellocloud.be wurr.voila.net
yellowstonevisitortours.com www.souvenirua.com
youradulthosting.com x-diesel.org
zakazfutbolki.com xboxster.ru
zarepta.com xgftnlrt.bloger.index.hr
zigarettenonlinekaufen1.bloog.pl xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai
xnxxandxvideos.com
xxxrus.org
youtoner.it
zarabotat-na-sajte.ru
znakomstva-piter78.ru
zoogdinsney.com
zs2vm.top

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.21 0; 10.1.0.77 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3078 ### Version: V4.2022.05.3079
### Updated: Sun May 1 10:03:30 UTC 2022 ### Updated: Sun May 1 22:01:14 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3078 ### Version: V4.2022.05.3079
### Updated: Sun May 1 10:03:30 UTC 2022 ### Updated: Sun May 1 22:01:14 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##