vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.05.3089 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-05-06 22:07:28 +00:00
parent 759c0e8e00
commit f340190ade
47 changed files with 96342 additions and 96332 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3088 #### Version: V4.2022.05.3089
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3088 #### Version: V4.2022.05.3089
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.05.3088 #### Version: V4.2022.05.3089
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11906
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.05.3088 #### Version: V4.2022.05.3089
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11910
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11910
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11910
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11910
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.0.106 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.0.106 0;
127.0.0.1 0; 127.0.0.1 0;

11910
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.0.106 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.0.106 0;
127.0.0.1 0; 127.0.0.1 0;

11910
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.0.106 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.0.106 0;
127.0.0.1 0; 127.0.0.1 0;

11910
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3088 3089

11910
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11910
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.66.89.32 0; 20.66.122.102 0;

11910
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.0.106 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.0.106 0;
127.0.0.1 0; 127.0.0.1 0;

11910
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.0.106 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.0.106 0;
127.0.0.1 0; 127.0.0.1 0;

11910
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.0.106 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.0.106 0;
127.0.0.1 0; 127.0.0.1 0;

11910
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.66.89.32 0; 20.66.122.102 0;

11910
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.66.89.32 0; 20.66.122.102 0;

11910
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.0.106 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Asterias 360Spider
BackWeb Id-search
Bolt IndeedBot
CopyRightCheck Leap
Freeuploader Mediatoolkitbot
MQQBrowser Nameprotect
RankFlex Sqworm
Surfbot Toweyabot
Thumbor Upflow
sp_auditbot VidibleScraper

302
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 360Spider
ASPSeek 404enemy
Aboundex 80legs
AfD-Verbotsverfahren Acunetix
AhrefsBot AhrefsBot
AiHitBot Aipbot
Alexibot
AlphaBot
Anarchy Anarchy
Apexoo Ankit
Atomseobot Asterias
BDFetch AwarioSmartBot
BacklinkCrawler BDCbot
Bitacle BLEXBot
BlackWidow BackWeb
Badass
BetaBot
Bigfoot
Black Hole
Blow Blow
Buck BotALot
BuiltBotTough
Bullseye
BunnySlippers
CATExplorador CATExplorador
Calculon CCBot
CheTeam CODE87
CazoodleBot
CheeseBot CheeseBot
ChinaClaw ChinaClaw
Claritybot Chlooe
Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot Copyscape
Cogentbot Cosmos
Collector Craftbot
Copier Crawling at Home Project
CrazyWebCrawler
Crescent
Curious Curious
Custo Custo
CyotekWebCopy
DIIbot DIIbot
DSearch DSearch
DTS Agent
Demon
Devil Devil
DigitalPebble
Dirbuster Dirbuster
Discoverybot DittoSpyder
DnyzBot
DomCopBot
DomainAppender DomainAppender
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project
Dragonfly Dragonfly
EMail Wolf Drip
EMail Siphon
EasyDL EasyDL
Extreme Picture Finder Ecxi
EyeNetIE EroCrawler
Evil
ExtLinksBot
FDM
FHscan FHscan
Fimap Fimap
Firefox/7.0
Flunky
Foobot Foobot
Freeuploader Freeuploader
FrontPage FrontPage
FyberSpider FyberSpider
G-i-g-a-b-o-t GalaxyBot
Genieo Genieo
GetRight
GetWeb
Gigabot
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
Gotit Grabber
Grafula GrapeFX
HEADMasterSEO GrapeshotCrawler
HMView GridBot
HTMLparser Harvest
HTTP::Lite Hloader
HaosouSpider Humanlinks
HybridBot
IDBTE4M IDBTE4M
IDBot Id-search
IlseBot IlseBot
Image Sucker
IndeedBot
Indy Library Indy Library
InfoNaviRobot InfoTekies
Internet Ninja
Iskanie Iskanie
IstellaBot IstellaBot
JetCar JOC Web Spider
Jbrofuzz
JennyBot
Jetty
JustView JustView
Kenjin Spider Keyword Density
Kinza
Kozmosbot Kozmosbot
LNSpiderguy
LWP::Simple LWP::Simple
Leap Leap
LeechFTP LeechGet
Lftp LexiBot
Libwhisker
LieBaoFast LieBaoFast
Lightspeedsystems LinkScan
LinkWalker
LinkextractorPro
LinkpadBot
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot Lipperhey Spider
Litemage_walker
Lmspider Lmspider
Ltx71 MIDown tool
MFC_Tear_Sample
MSFrontPage
MTRobot
Magnet Magnet
Majestic SEO Majestic SEO
MarkMonitor Majestic12
MarkWatch
Mass Downloader
Masscan Masscan
Mata Hari Mata Hari
MeanPath Bot MeanPath Bot
MegaIndex.ru MegaIndex.ru
Minefield
Mister PiX
Moblie Safari Moblie Safari
Mojeek
Mojolicious
MolokaiBot
Mozlila
Mr.4x3 Mr.4x3
Msrabot Name Intelligence
Nameprotect Navroad
NearSite
Needle Needle
Nessus
Net Vampire Net Vampire
Netcraft NetMechanic
NetSpider
NetZIP
Nettrack Nettrack
Netvibes Netvibes
Nikto Nutch
Nuclei Offline Navigator
Offline Explorer
OnCrawl OnCrawl
OpenLinkProfiler Openfind
Openvas OrangeSpider
OutclicksBot OutfoxBot
PECL::HTTP PageScorer
POE-Component-Client-HTTP Papa Foto
PageAnalyzer Pavuk
PageGrabber Petalbot
PageThing.com Picscout
PeoplePal
Picsearch Picsearch
PictureFinder
Piepmatz Piepmatz
Pimonster
Pixray Pixray
PleaseCrawl Pockey
ProWebWalker
Probethenet
Psbot Psbot
PyCurl Pump
QueryN Metasearch Quick-Crawler
RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankingBot2
Rankivabot
Re-re
ReGet
Reaper
RebelMouse RebelMouse
RepoMonkey SEOkicks-Robot
Ripper SEOlyticsCrawler
RocketCrawler SEOprofiler
Rogerbot SISTRIX
SEOstats SMTBot
SalesIntelligent
ScanAlert
Screaming Screaming
Searchestate ScrepyBot
SearchmetricsBot SearchmetricsBot
SemanticJuice SemanticJuice
Seomoz SentiBot
SeobilityBot
Shodan
Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler
SiteSnagger SiteSnagger
Siteimprove SiteSucker
Sitevigil Sitebeam
SlySearch Snoopy
SmartDownload Sociscraper
Snake
SocialRankIOBot
Sogou web spider Sogou web spider
Sosospider Sosospider
SpankBot SpankBot
SputnikBot Spanner
Sqlworm Spinn3r
Sqworm
Stripper
Sucuri
SuperHTTP SuperHTTP
SurveyBot Surfbot
Swiftbot Suzuran
Szukacz Szukacz
Teleport
TeleportPro TeleportPro
Telesoft Telesoft
Telesphoreo
The Intraformant
TheNomad TheNomad
Thumbor Thumbor
Trendiction
Trendictionbot
Turingos
Turnitin Turnitin
TwengaBot TurnitinBot
Twice
Typhoeus Typhoeus
URLy.Warning URLy.Warning
UnisterBot
Upflow Upflow
V-BOT VidibleScraper
Vagabondo Voil
Virusdie
Voltron
WBSearchBot WBSearchBot
WEBDAV
WISENutbot
WPScan WPScan
WWW::Mechanize WWW-Collector-E
WWW-Mechanize
WWWOFFLE
Wallpapers Wallpapers
Web Collage Wallpapers/3.0
Web Auto
Web Enhancer Web Enhancer
WebBandit Web Sauger
WebFetch WebFetch
WebWhacker WebGo IS
WebZIP WebImageCollector
WebmasterWorldForumBot WebLeacher
WebReaper
WebSauger
Website Quester Website Quester
WebsiteQuester WebsiteQuester
Whacker Whacker
Who.is Bot Widow
WiseGuys Robot WinHTTrack
Wotbox Woobot
Wprecon Wprecon
Xaldon_WebSpider Xaldon_WebSpider
Zade Xenu
Zitebot Zermelo
ZoomBot
ZoominfoBot
ZumBot ZumBot
ZyBorg
archive.org_bot archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt
backlink-check backlink-check
cah.io.community
com.plumanalytics
crawl.sogou.com
crawler.feedback
crawler4j crawler4j
demandbase-bot
domainsproject.org domainsproject.org
facebookscraper
gopher
heritrix heritrix
instabid instabid
isitwp.com
linkdexbot
lwp-request
magpie-crawler magpie-crawler
mediawords meanpathbot
muhstik-scan muhstik-scan
netEstate NE Crawler netEstate NE Crawler
oBot s1z.ru
plumanalytics satoristudio.net
scan.lol seocompany.store
seobility
sexsearcher sexsearcher
sitechecker.pro
sp_auditbot sp_auditbot
tAkeOut tAkeOut
trendiction.com voyagerx.com
webgains-bot x09Mozilla
webpros.com x22Mozilla
zgrab zgrab

304
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
80legs 404checker
404enemy
ADmantX ADmantX
ASPSeek AIBOT
Abonti Abonti
Acunetix Aboundexbot
AfD-Verbotsverfahren AfD-Verbotsverfahren
AiHitBot AhrefsBot
Alexibot
Alligator Alligator
AlphaBot AlphaBot
Anarchie
Anarchy Anarchy
Anarchy99 Anarchy99
Anthill
Apexoo
Aspiegel Aspiegel
Asterias Asterias
Atomseobot AwarioRssBot
BDFetch
BLEXBot BLEXBot
BackDoorBot
BackStreet
BackWeb
Backlink-Ceck Backlink-Ceck
Battleztar Bazinga BacklinkCrawler
BetaBot Badass
Blackboard CATExplorador
BlowFish CSHttp
Brandprotect
Brandwatch
Buck
Buddy
BuzzSumo
CCBot
CODE87
CensysInspect CensysInspect
CherryPicker CheTeam
Citoid
Cliqzbot Cliqzbot
Cocolyzebot Crawling at Home Project
Copier
Craftbot
CrazyWebCrawler
Crescent Crescent
CrunchBot DIIbot
Custo DSearch
Devil DatabaseDriverMysqli
Demon
Deusu
Dirbuster Dirbuster
Disco DnyzBot
DittoSpyder DomCopBot
DnBCrawler-Analytics
DomainCrawler DomainCrawler
DomainStatsBot DomainStatsBot
Dotbot
Dragonfly Dragonfly
Drip ECCP/1.0
EasyDL Ecxi
EirGrabber
Exabot Exabot
Extractor Express WebPictures
ExtLinksBot
ExtractorPro ExtractorPro
EyeNetIE EyeNetIE
FDM
FHscan
FemtosearchBot
Firefox/7.0
Flunky Flunky
FyberSpider FrontPage
Fuzz
GT::WWW
Genieo Genieo
Getintent Gotit
Gigabot
Go!Zilla
GoZilla
GrabNet
GrapeshotCrawler GrapeshotCrawler
HTMLparser GridBot
Haansoft HMView
HybridBot HaosouSpider
Harvest
Heritrix
Hloader
HonoluluBot
Humanlinks
IDBTE4M
IDBot
IRLbot
Iblog Iblog
IlseBot IlseBot
Image Fetch Image Fetch
Image Sucker InfoNaviRobot
IndeedBot
InterGET InterGET
Internet Ninja Internet Ninja
InternetSeer
Iskanie Iskanie
JennyBot Jbrofuzz
Jetty
JikeSpider JikeSpider
Joomla Keybot Translation-Search-Machine
Jorgee Keyword Density
Kinza Kinza
Kozmosbot Kozmosbot
LNSpiderguy Lanshanbot
LWP::Simple
Larbin
Leap Leap
LeechGet LexiBot
Lftp LibWeb
Libwhisker LieBaoFast
Likse LinkScan
LinkpadBot Linkbot
LinkextractorPro
LinksManager LinksManager
LinqiaRSSBot LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
Litemage_walker Lmspider
MFC_Tear_Sample Ltx71
MIDown tool MIDown tool
MIIxpc
MSIECrawler MSIECrawler
MTRobot Magnet
Majestic SEO Mail.RU_Bot
Majestic12
MarkMonitor
MarkWatch MarkWatch
Mass Downloader Mass Downloader
Mata Hari Masscan
Meanpathbot MeanPath Bot
MegaIndex.ru
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Minefield
Mister PiX Mister PiX
Moblie Safari Mojolicious
Morfeus Fucking Scanner NICErsPRO
Mr.4x3 Nameprotect
Musobot
NearSite
Needle Needle
Nessus NetAnts
NetMechanic
NetSpider NetSpider
NetZIP
Netcraft
NextGenSearchBot NextGenSearchBot
Nikto Nibbler
Nmap NimbleCrawler
Not Octopus
OnCrawl OnCrawl
OpenLinkProfiler
Openvas
OrangeBot OrangeBot
OrangeSpider
OutclicksBot
OutfoxBot OutfoxBot
PHPCrawl Page Analyzer
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageGrabber
Pavuk
PeoplePal PeoplePal
Pi-Monster
Picscout Picscout
Picsearch
PictureFinder PictureFinder
Pimonster Pimonster
Pixray Pixray
ProWebWalker ProPowerBot
Psbot Psbot
QueryN Metasearch QueryN Metasearch
RankActiveLinkBot Quick-Crawler
RankActive
RankFlex
RankingBot RankingBot
Re-re Rankivabot
RealDownload ReGet
Reaper
RebelMouse RebelMouse
RedesScrapy RedesScrapy
RepoMonkey Rogerbot
RocketCrawler SBIder
SEOlyticsCrawler SEOkicks
SISTRIX SEOprofiler
SEOstats
ScanAlert
Scanbot Scanbot
ScoutJet ScoutJet
Scrapy
ScreenerBot ScreenerBot
SearchmetricsBot Seekport
SemanticJuice
Semrush Semrush
SemrushBot
Seomoz
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteExplorer
SiteSnagger Sitebeam
Siteimprove
SlySearch
Snapbot Snapbot
SocialRankIOBot SocialRankIOBot
Sociscraper Sociscraper
Spammen Sosospider
Spbot Spanner
Sqlmap Spinn3r
Sqlworm SputnikBot
Steeler Steeler
Stripper
SuperBot
SuperHTTP SuperHTTP
SurveyBot Surfbot
T8Abot Suzuran
Telesphoreo Teleport
Telesoft
Telesphorep Telesphorep
TheNomad TightTwatBot
Toweyabot Titan
Tracemyfile
Trendictionbot Trendictionbot
True_Robot True_Robot
Turingos
TurnitinBot TurnitinBot
Typhoeus Twice
URLy.Warning URLy.Warning
VB Project VB Project
VCI Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VidibleScraper VeriCiteCrawler
VoidEYE
Voltron Voltron
WASALive-Bot Web Collage
WISENutbot Web Pix
WPScan
WWW-Collector-E
WWW::Mechanize
WWWOFFLE
Web Fetch
Web Fuck
Web Sauger Web Sauger
WebBandit WebBandit
WebFetch WebCollage
WebFuck WebEnhancer
WebGo IS
WebImageCollector WebImageCollector
WebSauger WebPix
WebSucker WebSucker
WebWhacker WebWhacker
Website Quester WebZIP
WebsiteExtractor WebsiteQuester
Webster Whacker
Whatweb Widow
Who.is Bot
Wonderbot Wonderbot
Wprecon Wprecon
Xaldon WebSpider Zade
Zermelo Zauba
Zitebot ZoomBot
ZmEu ZoominfoBot
ZumBot ZumBot
adscanner adscanner
archive.org_bot archive.org_bot
arquivo.pt arquivo.pt
backlink-check cah.io.community
check1.exe coccocbot
cognitiveseo cognitiveseo
crawler.feedback crawl.sogou.com
crawler4j crawler4j
dataforseo.com
domainsproject.org domainsproject.org
gopher iubenda-radar
heritrix linkdexbot
mediawords lwp-request
meanpathbot
muhstik-scan muhstik-scan
netEstate NE Crawler
oBot oBot
pcBrowser polaris version
ripz
s1z.ru
satoristudio.net satoristudio.net
scalaj-http
scan.lol
seocompany.store seocompany.store
seoscanners serpstatbot
sitechecker.pro
siteripz siteripz
sogouspider
sp_auditbot
spyfu spyfu
sysscan trendiction.de
tAkeOut webpros.com
voyagerx.com webprosbot
webgains-bot x09Mozilla
webmeup-crawler x22Mozilla
xpymep1.exe zgrab
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
acheterviagrafr24.com atout-energie-69.com
biznesluxe.ru findacheaplawyers.com
bumskontakte.org integritylandscapeservices.com
kit-opt.ru kellyonline.xyz
raschtextil.com.ua krassh.ru
stevemonsen.com manualterap.roleforum.ru
telesvoboda.ru otbelivanie-zubov.com
templates.franklinfire.co rating-casino2021.ru
topmira.com taihouse.ru
yourads.website viven.host.sk

926
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0daymusic.org 1000n1.ru
1flag.co.za 101raccoon.ru
24-ak.ru 123locker.com
24videos.tv 1kinobig.ru
24x7-server-support.site 1webmaster.ml
2rich4bitches.com 2pxg8bcf.top
3-letter-domains.net 2x2fan.ru
4inn.ru 3weekdiet.com
4pp13.com 4replicawatch.net
5-steps-to-start-business.com 4webmasters.org
5000-cotydzien.com 6128786.com
6hopping.com 7makemoneyonline.com
777-club.ru aa8b68101d388c446389283820863176e7.com
883zy.com aab94f698f36684c5a852a2ef272e031bb.com
91abcw.com abbanreddy.com
aa08daf7e13b6345e09e92f771507fa5f4.com abcguru.xyz
aad1f4acb0a373420d9b0c4202d38d94fa.com absolutelyanalog.com
abcdeg.xyz
acunetix-referrer.com
adanih.com adanih.com
advokateg.ru adhome.biz
affrh2015.com adpremium.org
aktivator-windows10.blogspot.com adtech.us
alfapro.ru adviceforum.info
all-streaming-media.com alevinefesleri.com
all4wap.ru allblogroll.com
allcredits.su allkrim.com
alldezservice.kz allornamenti.com
alldubai.biz allproblog.com
allnews24.in alphahoverboards.com
allwidewallpapers.com amung.us
alphavisions.net android4fun.org
ampower.me apimountainbikei-a.akamaihd.net
anonymousfox.co apioasisspacenet-a.akamaihd.net
apiappenableinfo-a.akamaihd.net app-ready.xyz
apibrowsesmartne-a.akamaihd.net
apidiamondatanet-a.akamaihd.net
apple.com-cleaner.systems apple.com-cleaner.systems
applyneedy.xyz apxeo.info
arate.ru articlesdirectoryme.info
arenanews.com.ua
artavenuegardenstudios.com
aruplighting.com
arvut.org
as5000.com
asociatia-tipografilor-transilvania.ro
asrvrep-a.akamaihd.net asrvrep-a.akamaihd.net
auto-moto-elektronika.cz asrvvv-a.akamaihd.net
autobudpostach.club auto4style.ru
autodo.info autoblogger24.info
avek.ru autobrennero.it
azte.ch aviav.ru.com
b2b-lounge.com avkzarabotok.com
backlinkwatch.com avon-novosib.ru
baladur.ru avon-severozapad.ru
azazaporn.com
azazu.ru
azbukafree.com
backgroundpictures.net
bag77.ru
balois.worldbreak.com
baltgem.com baltgem.com
bazaronline24.ru battle.net
beachpics.com beauby.ru
belinvestdom.by beauty-bracelet.com
bestmarriages.com belstaffstore.online
bestsexyblog.com best-seo-solution.com
bestadbid.com
bestbuy.ca
bestwebsiteawards.com bestwebsiteawards.com
bestwrinklecreamnow.com beta.hotkeys.com
bharatdefencekavach.com bgviagrachrx.com
binomo.kz big-cash.net
bjgugu.net.ua bikini-image.com
blogs.rediff.com bitcoins-live.ru
bluesman.nu bitporno.sx
bmw-ark.ru bjetjt.com
bombla.org blacktwhite.com
bongiornos.info blobar.org
boo-arts.com bloggen.be
bpro1.top bloggerads.net
bradleylive.xyz boeuklad.com
brandedleadgeneration.com bonky.biz
bravegirlsclub.com boobsimge.com
brewdom.ru brianhenry.co
browserprotecter.com btc4u.ru
buckspar.gq buigas.00it.com
burger-imperia.com buqayy0.livejournal.com
business-made-fun.com businesn.men
bytimedance.ru buttons-for-website.com
c-english.ru buzz-porno.info
cam-kontakte.org bvlgariwallet.xyz
carapics.com call-of-duty.info
cartierlovestore.com callejondelpozo.es
cartiertrinity.xyz callmd5map.com
callstevens.com
cand.jp
capacitacionyaprendizaje.com
carina-sy.de
cartierlovebracelet.xyz
cbb1smartlist12.click
cbcseward.com cbcseward.com
cdnnetwok.xyz cbozhe.com
cdncash.com
cementaresearch.se
cenoval.ru cenoval.ru
cheapjerseysfootballshop.com censys.io
cheapsergiorossi.online centre-indigo.org.ua
cheapwebsitehoster.com ceotrk.com
cheatcode-lita12.rhcloud.com cheapbarbour.online
cialischstgerts.com cheapestjerseysonwholesale.com
cicaki.net cheapjerseysap.com
clickbank.net checktext.org
clips.ua.ac.be chimiver.info
chocolateslim-en-espana.com
christianlouboutinreplica.win
cialischsrx.com
cialisovercounteratwalmartusa.com
clash-clans.ru
classicakuhni.ru
clickcash.com
clicksor.net
clipartnew.com
clothesforcash.com
coaching-netz.info coaching-netz.info
coinsspb.com collegerentals.com
cometorussia.net compliance-barak.xyz
compliance-don.top compliance-mary.top
compliance-fred.top
compliance-ivan.top
conboy.us conboy.us
conocer-sanabria.com
construmac.com.mx
converse.ddsoldes.fr converse.ddsoldes.fr
cool-wedding.net cookielawblog.wordpress.com
cordstrap.cc coolwallpapers4k.info
counter.bloke.com coolyarddecorations.com
counter.yadro.ru corta.co
creams.makeforum.eu costablue.xyz
cvety24.by cracksplanet.com
dairyindia.in credit.co.ua
daretodonate.co crest3d.ru
crystalslot.com
customcatchcan.com
darmebel.com.ua darmebel.com.ua
data-ox.com dataloading.net
depositfiles-porn.ga destinationrealestate.com
dirtpics.pw dieta-personalna.pl
divku.ru diffbot.com
docs4all.com dispo.de
dojki365.online djonwatch.ru
domain-submit.info dkvorota.ru
downvids.net doctissimo.fr
e-biznes.info doctormakarova.ru
eandsgallery.com documentsite.net
earl-brown.info domainsatcost.com
dominateforex.ml
dostavka-v-ukrainu.ru
download-walpaperhd.blogspot.com
druzhbany.ru
eachdayisagift.review
edge.sharethis.com
editmedios.com
ekn-art.se
ekobata.ru ekobata.ru
emergencyneeds.org ekspertmed.com
en.altezza.travel electrik-avenue.com
eshop4u.jp elektrozigarette2.webs.com
espaceinventoristes.com elektrozigaretten1.postbit.com
evogarage.com empis.magix.net
executehosting.com engines-usa.com
extremez.net enhand.se
ezigarettekaufen2.yolasite.com erectile.bid
faithe.top eredijovon.com
fandlr.com eu-cookie-law.blogspot.com
faptitans.com europages.com.ru
farmprofi.net exoneration-project.us
ferieboligkbh.dk ezaz.info
financeloan.us ezigarettenkaufen2.dreamwidth.org
financetip.eu ezigaretteshop2.mywapblog.com
findacheaplawyers.com ezigaretteshop2.vefblog.net
finemanteam.com f07.de
flowwwers.com facecup.top
folowsite.com factorynightclub.com
forminecrafters.ru falllow.gq
franecki.net familyholiday.ml
farm26.ru
fashionindeed.ml
fba-mexico.com
femmesdenudees.com
fenoyl.batcave.net
ferrotodo.com
filesvine.com
filmidivx.com
filunika.com.ru
findpik.com
fireads.men
fitfloponline.store
footbalive.org
forexgb.ru
forum.poker4life.ru
forum20.smailik.org
foxinsocks.ru
free-floating-buttons.com free-floating-buttons.com
free-laptop-reward.com free-laptop-reward.com
freejabs.com free-video-tool.com
freeuploader.ml
freevpn.space freevpn.space
frequiry.com fxund.us
futbolkisales.ru galleryawesome.com
fxtips.ru gambarxkata.co
g33.org gamedayassist.com
gameonasia.com gamedayhouse.com
gaytube.com gamerextra.com
gazoblok.net.ua gaypornmovie.net
geckoandfly.com geckoandfly.com
gelezki.com
genetworx.com
gepezz.info
gidonline.one
gigapeta.com gigapeta.com
girlgamerdaily.com gk-atlant.info
girls-ufa.ru
gkvector.ru
glall.ru glall.ru
globatur.ru glass-msk.ru
go2jump.org glcomputers.ru
goldandcard.ru glicol.kz
golden-catalog.pro globalscam.ga
goodwriterssales.com go2mike.ru
gosreg.amchs.ru goldadpremium.com
gotwebsite1.com goldpanningtools.com
gowreckdiving.com goodhousekeeping.com
gz2.bbsoldes.fr gopro-online.info
hitcpm.com grizzlysgrill.com
holidaypics.org grtyi.com
homeart.space grupografico-pilar.com.ar
hotel-mkad.ru gsasearchenginerankerdiscount.com
hotenergy.ru guardlink.com
houseofgaga.ru gunsvicceadadebt.tk
hoverboard360.nl hdwalls.xyz
hem.passagen.se
hermesbracelets.xyz
histock.info
homeinns.com
homemade.gq
homik.pw
host-protection.com
hotblognetwork.com
hotloans.ru
hotsocialz.com
howtowhitenteethfast.xyz howtowhitenteethfast.xyz
hoztorg-opt.ru hyiphunter.org
hundejo.com iamsport.org
hybrid.ru iboss.com
hystersister.com imgata.com
iklysha.ml inmate-locator.us
imgarit.pw investmac.com
imperiafilm.ru investsuccess.org
inboxdollars.com
infobanks.ru
integritylandscapeservices.com
io9.com io9.com
iomoio.net iopeninghours.co.uk
iplogger.org iplusbit.blogspot.co.za
istizanidinelikexanax.blogspot.com irkutsk.online-podarki.com
itis4you.com
itronics.ca
ixora.pro ixora.pro
jagg.info jimmychoosale.online
jav-fetish.com jjbabskoe.ru
jerseysbizwholesalecheap.com job.icivil.ir
jerseyschinabizwholesale.us juliaworld.net
jerseywholesaleelitestore.com jus0wil.pp.ua
jofucipiku.tk karachev-city.ru
johannesburgsingles.co.za kashubadesign.ru
jose.mulinohouse.co kfon.eu
joyceblog.top kinky-fetishes.com
junglenet-a.akamaihd.net
kamen-e.ru
kartiniresto.com
kazrent.com
kimcurlrvsms.com
kino-filmi.com kino-filmi.com
kino-rating.ru kinoflux.net
kinohall.ru kinogolos.ru
kupiproday.com.ua kinohit1.ru
kvartira-sutochno.com kol-energo.ru
kvartiry-remont.ucoz.ru kolotiloff.ru
landreferat.ru krasivye-devushki.net
lankarns.com krasnodar.zrus.org
larger.io kredit-pod-zalog-krasnodar.ru
leboard.ru ktotut.net
legalrc.biz kvartir-remont.biz
lescinq.com
letolove.ru letolove.ru
lezbiyanki.net lexiangwan.com
lflash.ru linerdrilling.com
livesex-amateure.info lomb.co
loftdigital.eu lonely-mature.com
lol-smurfs.com luchshie-topcasino.ru
lonerangergames.com
lostaruban.com
lsp-awak-perikanan.com
luckyshop.net.ua luckyshop.net.ua
m3gadownload.pl makemoneyonline.com
mabdoola.blogspot.com mantramusic.ru
mackeeper-center.club mapquestz.us
mafa-free.com matematikus.info
makis.nu matsdale.com
marcowebonyodziezowe.pl me-ke.com
martlinker.com medosmotr-ufa.ru
mathgym.com.au mesbuta.info
med-zdorovie.com.ua metafilter.com
medmajor.ru mhi-systems.ru
medtherapy.ru
meduza-consult.ru
mhtr.be
michellblog.online michellblog.online
microstatic.pl microsoftportal.net
militarysale.pro middlerush-a.akamaihd.net
mindeyegames.com milblueprint.com
mirobuvi.com.ua mindbox.co.za
mister-shop.com miniature.io
mistr-x.org mir-business-24.ru
mmofreegames.online mnogabukaff.net
mmog-play.ru mockupui.com
mobile-appster.ru morpicert.pw
mobile.ok.ru moskva.nodup.ru
mobilemedia.md mossmesi.com
mojpreskumanie.com mrwhite.biz
monclerjacketsoutlet.win myblogregistercm.tk
monetizationking.net mygameplus.ru
moneyteam24.com
morlat.altervista.org
mriyadh.com
mrpornogratis.xxx
msk-diplomat.com
mulberryoutletonlineeu.com
my-aladin.com
my-floor.in.ua
mycouponizemac.com
mydownloadengine.com
myindospace.com
mylesosibirsk.ru mylesosibirsk.ru
myshopmatemac.com naphukete.ru
mzdish.site national-today-winning-winner.club
nakozhe.com netanalytics.xyz
namecrumilchlet.tk nevansk.ru
narkologiya-peterburg.ru newfilmsonline.ru
neks.info nextbackgroundcheck.gq
ningessaybe.me nextconseil.com
niroo.info nordvpn.com
novatech.vn noumeda.com
o-o-11-o-o.com nucia.biz.ly
oballergiya.ru o-o-6-o-o.com
obiavo.su oconto.ru
obuv-kupit.ru odywpjtw.bloger.index.hr
okout.ru okonich.com.ua
olympescort.com oneclickfiles.com
onescreen.cc online-hit.info
online-zaymy.ru onlinefilmz.net
onlinewritingjobs17.blogspot.ru onlinemeetingnow.com
onlineserialy.ru
onload.pw onload.pw
onthemarch.co openstat.com
orion-code-access.net oxford-book.com.ua
ororodnik.goodbb.ru pathwhelp.org
ovirus.ru pcads.ru
oz-offers.com phobia.us
ozas.net photo.houseofgaga.ru
paperwritingservice17.blogspot.ru picmoonco.pw
par-fallen.ga picscout.com
pastaleads.com picturesfrom.com
pavlodar.xkaz.org pk-services.ru
paydayonlinecom.com pkr1hand.com
pcboa.se pl.aasoldes.fr
petsblogroll.com plastgranar.nu
pirateday.ru play-movie.pl
pk-pomosch.ru pons-presse.com
plastjulgranar.se popads.net
platesauto.com
polcin.de
pomoc-drogowa.cba.pl
poppen-nw.net
porn-w.org
porn555.com porn555.com
pornobrazzers.biz pornoblood.com
pornogad.com pornoelita.info
pornogig.com
pornosee.info pornosee.info
pornosmola.info pornzone.tv
pozdrawleniya.ru pospr.waw.pl
predmety.in.ua potoideas.us
predominant-invent.tk privatamateure.com
pro-poly.ru privatov-zapisi.ru
producm.ru privhosting.com
prohoster.info profitfx.online
projefrio.com.br propranolol40mg.blogspot.com
promalp-universal.ru prosmibank.ru
promgirldresses.xyz prospekt-st.ru
prostitutki-kharkova.org prostitutki-novgoroda.org
prostitutki-odessa.org pssucai.info
prpops.com ptr.ruvds.com
pts163.ru pussyfleet.com
pulse33.ru puteshestvennik.com
purplesphere.in qor360.com
pxhdwsm.com quangcaons.com
raisedseo.com quickloanbank.com
realting-moscow.ru quit-smoking.ga
recipedays.com quizzitch.net
recordpage-a.akamaihd.net qwarckoine.com
reklamuss.ru ragecash.com
relayblog.com rating-bestcasino.com
replica-watch.ru razvratnoe.org
replicalouboutin.xyz rcb101.ru
retreatia.com realmonte.net
rewardit.com renewablewealth.com
richinvestmonitor.com rep-am.com
ricorsogiustizia.org
rightenergysolutions.com.au
ring4rhino.com
robotixix.com robotixix.com
rockprogblog.com rockingclicks.com
roofers.org.uk rogervivierforsale.com
rootandroid.org russia-today-video.ru
rosbalt.com.ua rvtv.ru
rossmark.ru rvzr-a.akamaihd.net
runofilms.ru s-forum.biz
rvi.biz sady-urala.ru
sajatvelemeny.com sankt-peterburg.nodup.ru
sammlungfotos.online saratov.xrus.org
samo-soznanie.ru scanner-irvin.top
samoiedo.it scanner-julia.top
samolet.fr scanner-walter.top
sbprabooks.com scansafe.net
scanhub.ru scottbywater.com
scanner-barak.top semaltmedia.com
scanner-willy.top
searchimage.co
searchmywindow-a.akamaihd.net
secret.xn--oogle-wmc.com
semprofile.com semprofile.com
semxiu.com
senger.atspace.co.uk senger.atspace.co.uk
seotoolsagency.com seo-prof1.xyz
servicecenter.co.ua seoanalyses.com
sevendays.com.ua seocheckupx.com
sevenstars7.com serviporno.com
sextracker.com sex.hotblog.top
silvermature.net sexfreepornoxxx.com
simple-share-buttons.com share-buttons-for-free.com
sisiynas.ru shohanb.com
sispe.com.br shopperifymac.com
site-speed-check.site shoppingjequiti.com.br
skuteczna-dieta.co.pl shoppingmiracles.co.uk
slimcdn.com signx.info
smailik.org site5.com
smallseotools.com siteaero.com
smartbalanceworld.com snworks.com
smichovbike.cz soaksoak.ru
soblaznu.net soblaznu.net
social-button.xyz socialbookmarksubmission.org
softonicads.com solnplast.ru
sophang8.com sortthemesitesby.com
sotechco.co spaceshipad.com
souvenirua.com speeddream.xyz
space-worry.ml sprttrack.com
spb-plitka.ru squidoo.com
spb.ru stackthatbucks.com
specialfinanceoffers.com startwp.org
sportbetfair.com staticfs.host
srecorder.com sticken.co
srgwebmail.nl stopnarco.ru
stard.shop strag-invest.ru
starwars.wikia.com streamin.to
statoutlook.info superstarfloraluk.com
steame.ru swagbucks.com
stromerrealty.com sygraem.com
suggest-keywords.com tandvardshuset.net
sugvant.ru taqywu51.soup.io
sunflowerdrawingpaintings.blogspot.com
superoboi.com.ua
supers.com.ua
svtrd.com
swimpool.ca
swinger-mobil.net
swinon.site
symbaloo.com
taboola.com
tam-gde-more.ru
taranerymagesswa.blogspot.com
tech4master.com
tedy.su tedy.su
testingads.pro teencastingporn.com
thegolfclub.info teenforporn.com
theguardlan.com thaismartloan.com
theprofitsmaker.net thebitcoincode.com
tizanidineandhydrocodone.blogspot.com thegameriders.com
tizanidinecost.blogspot.com theheroes.ru
tizanidinenarcotic.blogspot.com tivolibasket.it
tizanidineoral.blogspot.com tizanidinehcl2mg.blogspot.com
tizanidinevsvalium.blogspot.com tizanidinehcl2mgsideeffects.blogspot.com
tmearegion26.com tizanidinestreetprice.blogspot.com
tmm-kurs.ru tizanidinestreetvalue.blogspot.com
todohr.com torrentdownloadhub.com
token-lab.org tourcroatia.co.uk
top-karkas.ru tovaroboom.vast.ru
top250movies.ru trafficgenius.xyz
topads.men traffixer.com
tpu.ru triumf-realty.ru
trackzapper.com truck-land.life
tracxn.com trustedmaccleaner.com
traffic2cash.org tupper-posuda.ru
truck-rece.life twelvevisionspartyofcolorado.com
trucri.me ua-company.ru
tubeoffline.com ua.tc
tuberkuleznik.ru ultimateclassicrock.com
turist-strani.ru ultimatesetnewfreeallsoftupgradesystems.pw
turizm.bz urlopener.com
twittrading.com us-america.ru
uamtrk.com uzporno.mobi
ugogo.info v-doc.co
unimodemhalfduplefw.pen.io vchulkah.net
urll.eu vduplo.ru
uroki.net velobikestock.com
v24s.net viagraneggrx.com
vadimkravtcov.ru videtubs.pl
vepad.com viel.su
via-gra.webstarts.com vipromoffers.com
video-hollywood.ru vitanail.ru
videosbox.ru vizitki.net
videositename.com vkgaleria.com
vitalads.net vsesubwaysurfers.com
viteonlusarezzo.it watchmygf.net
vk-mus.ru web.cvut.cz
vkontarkte.com webcamdevochka.com
vremya.eu
vse-pesni.com
vseigry.fun
vtc.pw
vut.com.ru
vzubah.com
wallpaperstock.net
wdrake.com
webmasters.stackexchange.com webmasters.stackexchange.com
wejdz-tu.pl weboptimizes.com
webpromotion.ae
website-analyzer.info
wetgames.ru wetgames.ru
whatsupinfoley.com whatsappbot.flyland.ru
wineration.com wheelchairliftsarea.com
witclub.info wikes.20fr.com
xjrul.com williamrobsonproperty.com
xmronta.com womens-journal.net
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai world-mmo.com
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai wosik-dach.service-for-web.de
xn--80ajbshivpvn2i.xn--p1ai wurr.voila.net
xn--e1afanlbnfckd7c3d.xn--p1ai www.kabbalah-red-bracelets.com
xolodremont.ru www.souvenirua.com
xportvusbdriver8i.snack.ws www.wohnkabinen-shop.de
yeartwit.com xcombear.ru
youradexchange.com xmladserver.com
yourdesires.ru xn--24-glceagatoq7c2a6ioc.xn--p1ai
youtube-downloader.savetubevideo.com xn--c1acygb.xn--p1ai
yugo-star.ru xn--e1aggki3c.xn--80adxhks
zapiszto.pl xxxguitars.com
ynymnwbm.bloger.index.hr
youbrainboost.asia
youjizz.com
zaapplesales.blogspot.com
zakonobosago.ru
zarajbuilders.com
zarenica.net zarenica.net
zeroredirect8.com zeroredirect2.com
zigarettenonlinekaufen1.bloog.pl znakomstvaonlain.ru
znakom.sibtest.ru zooggames.com
zvetki.ru

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
007angels.com 000free.us
100dollars-seo.com 08800.top
123locker.com 0912701309f8ce.com
3-letter-domains.net 123movies.love
404.mn 1314dh.com
4inn.ru 1kdailyprofit.me
abtasty.com 24x7-server-support.site
admitad.com 66cpwgln.space
adverster.com 72-news.com
ageofclones.com 76brighton.co.uk
alvtank.se 883zy.com
apartmentratings.com a1security.com.ua
arquapetrarca.info acortarurl.es
artel116.ru addblueoff.com.ua
asiengirls.net admatic.com.tr
ausmepa.org.au afftrack001.com
autolombard-krasnodar.ru agecheckadult.com
automobile-spec.com agysya.ru
avon-severozapad.ru alert.scansafe.net
backlinkwatch.com alevinefesleri.com
berdasovivan.ru androids-store.com
betterscooter.com apiwebwebgetcom-a.akamaihd.net
bin-brokers.com applicationg29.com
bio-japan.net arendovalka.xyz
bio.trade-jp.net arvut.org
blackplanet.com audiofree.ru
blagovest-med.ru ausergrubhof.info
bolitgorlo.net automatic-seo.com
bonzbuddy.com avek.ru
bookhome.info badavit.com.ua
bradleylive.xyz bag77.ru
browserprotecter.com baladur.ru
businesxxl.com battlecarnival.su
buyantiviralwp.com beatifullandscape.co
buypanicdisorderpill.com bestprofits-there.com
buzz-porno.info bestwebsiteawards.com
buzzonclick.com bet365.com
bvlgariring.xyz blue-square.biz
calc-for-credit.ru bmwhighperformers.com
calvet.altervista.org bolezniorganov.ru
carfax.com.ua brillianty.info
cartujano-pre.de buckspar.gq
censys.io budilneg.xyz
chatroulette.life business-made-fun.com
cheapbarbour.online buzzurl.jp
cheapmoncler.pw bwlx.prepedu.cn
chee-by.biz callawaygolfoutlet.online
cherrypointplace.ca ccjp.eu
citysecurity.nu chatroulette.video
classiquebijoux.ru choosecuisine.com
clickzzs.nl clodo.ru
clippingphotoindia.com compliance-brian.xyz
coinsspb.com contentlook.co
coldfilm.ru contentsexpress.com
compliance-checker-7.info contextualyield.com
compliance-fred.xyz crackguru.tk
compliance-willy.top crest-poloski.ru
coolgamechannel.com culturevie.info
counter.bloke.com cyclobenzaprinevstizanidine.blogspot.com
creams.makeforum.eu datadepths.com
creditwell.ru dataloading.net
customcatchcan.com degerlund.net
dellalimov.com delitime.info
depositfiles-porn.ga depositfiles-porn.ga
deshevo-nedorogo.ru detoxmed24.ru
dgfitness.co diffbot.com
dividendo.ru dimfour.com
dogoimage.com dipstar.org
domaincdn.xyz diusyawiga.tk
dvervmoskvu.ru doctissimo.fr
easyshoppermac.com doxyporno.com
educontest.net droidlook.net
ekosmetyki.net.pl drunkmoms.net
elektrozigarettekaufen1.devhub.com eandsgallery.com
erectile.bid ednorxmed.com
estelight.ru educatemetv.com
etm-consult.de efkt.jp
exct.net ekspertmed.com
executehosting.com electronicadirect.com
ezigarettekaufen2.blox.pl eleimgo.pw
ezigarettenkaufen1.pagina.gr elektrischezigarette2.onsugar.com
ezigarettenshop2.postbit.com emerson-rus.ru
fba-mexico.com en.altezza.travel
fickluder69.com eshop.md
filerockstar298.weebly.com etotupo.ru
fishingwholesale.us eyes-on-you.ga
fm-upgrade.ru falllow.gq
fok.nl fastcrawl.com
forum.doctissimo.fr flashbannernow.com
foxinsocks.ru flirt4free.com
gamerscorps.com folowsite.com
food.dtu.dk
footbalive.org
fordsonmajbor.cf
foto-telok.net
foxydeal.com
fullgirl.ru
games.kolossale.ru
gay-site.store
getprismatic.com
go2mike.ru
gok-kasten.net gok-kasten.net
gorodservis.ru graphicwe.org
greenshop.su heliko.no
growmyfunds.ca hexpilot.com
hamacapty.com homemade.gq
hawaiielectriclight.com horoshieokna.com
hchha.com
hdseriale.pl
heartofpayne.xyz
hebr.myddns-flir.com
helicalpile.us
hobild.net
hostnow.men
hreade.com
huimin764128.com
hully.altervista.org hully.altervista.org
hybrid.ru incomekey.net
i4track.net inet-traffic.com
iccornacircri.cf infazavr.ru
ico.re informatiecentro.be
innodgfdriverhm.aircus.com int.search.tb.ask.com
insomniagamingfestival.com ipornox.xxx
instasexyblog.com jeremyeaton.co
isitpaleo.info kamalsinha.com
istizanidinelikexanax.blogspot.com keyword-suggestions.com
iwanttodeliver.com keywordsking.com
jasonpartington.com kinogolos.ru
joy-penguin.com kinozapas.com
junketjuice.blogspot.com kolotiloff.ru
justkillingti.me konkursowo-24.pl
karachev-city.ru kumuk.info
karusel-market.ru lacloop.info
keki.info lafriore.ru
keywordblocks.com landinez.co
kino-fun.ru leadwayau.com
krafte.ru lecbter-relationships.ga
krasnodar.xrus.org lkbennettoutlet.online
laapp.com loadopia.com
lafourmiliaire.com locustdesign.co
larose.jb2c.me lonfon.xyz
letslowbefast.site long-beach-air-conditioning.com
lineavabit.it
linkbuddies.com
liupis.com
longgreen.info
longlifelomilomi.info
losangeles-ads.com
m.b00kmarks.com m.b00kmarks.com
marcowebonyodziezowe.pl m0r0zk0-krava.ru
marsgatan.com macnewtech.com
mcnamaratech.com magento-crew.net
mesbuta.info mailemedicinals.com
mir-business-24.ru malls.com
mirmedinfo.ru mathgym.com.au
mirobuvi.com.ua mbiologi.ru
mobplayer.ru mediafresh.online
muztops.ru mega-bony-2017.pl
mxgetcode.com mega-bony2017.pl
my-bc.ru megaindex.ru
nac-bearings.ru member-quiz.com
namenectar.com minneapoliscopiers.com
newhairstylesformen2014.com mnogabukaff.net
normalegal.ru moneyteam24.com
noscrapleftbehind.co montazhnic.ru
notaria-desalas.com musezone.ru
notasprensa.info mybackgroundlandscape.blogspot.com
o-o-11-o-o.com myghillie.info
of-ireland.info myplaycity.com
online.ktc45.ru naj-filmy24.pl
ons-add.men net-radar.com
orion-code-access.net new-apps.ru
photochki.com nextlnk12.com
pinsdaddy.com ngps1.ru
piski.top nodding-passion.tk
plastgran.com nodup.ru
platesauto.com nofreezingmac.click
pogodnyyeavarii.gq nomuos.it
pomoc-drogowa.cba.pl nonews.co
pornoklad.net obiavo.uz
poshiv-chehol.ru pammik.ru
pozdravleniya-c.ru partner-cdn.men
profitwithalex.info partner-host.men
prombudpostach.com.ua picswe.com
propranolol40mg.blogspot.com pl-top.pl
putevka24.ru pladform.ru
razamicroelectronics.com pokemon-go-play.online
registry-cleaner.net popmarker.com
rucrypt.com popunder.net
saddiechoua.com pornmania.pl
saitevpatorii.com porno-dojki.net
sampleletters.net porno-home365.com
satellite.maps.ilovevitaly.com pornoplen.com
sayyoethe.blogspot.co.za pravoholding.ru
sbornik-zakonov.ru privatamateure.com
seolab.top prlog.ru
sevenstars7.com ptr.ruvds.com
sex-znakomstva.online pxhdwsm.com
shodanhq.com rankscanner.com
shopsellcardsdumps.com ranksonic.info
site-speed-check.site razleton.com
site.ru redbottomheels.xyz
siteaero.com registry-clean-up.net
sitebeam.net relayblog.com
ring4rhino.com
roleforum.ru
rusexy.xyz
ruspdd.com
russia-today-video.ru
russian-postindex.ru
s8-nowy-wygraj.comli.com
scanmyphones.com
seo-prof1.xyz
seogadget.ru
seorankinglinks.xyz
sitopreferito.it
sivs.ru sivs.ru
sledstvie-veli.net slkrm.ru
smartpet.ru slowmacfaster.trade
snsdeainavi.info smadihome.com
social-buttons.com smallseotools.com
space4updating.win sms2x2.ru
spb-plitka.ru softxaker.ru
sr-rekneskap.no solnplast.ru
stackthatbucks.com speedup-my.site
stmassage.ru
stpicks.com stpicks.com
success-seo.com superoboi.com.ua
tabakur77.com technika-remont.ru
telefonsexsofort.tk telefonsex-ohne0900.net
tetracsaudi.com telegraf.by
the-torrent-tracker.blogspot.com thaismartloan.com
theallgirlarcade.com theautoprofit.ml
thebitcoincode.com themestotal.com
ticketsys.inetwd.com thepornsex.org
tivolibasket.it tiandeural.ru
tizanidinedosage.blogspot.com tizanidinehcl4mgtab.blogspot.com
tizanidinehcl4mgtablet.blogspot.com tizanidinehcl4mgtablet.blogspot.com
tizanidinerestlesslegsyndrome.blogspot.com toursmaps.com
toloka.hurtom.com uac.net.au
torrents-tracker.com ultramart.biz
track2.shop vbabule.net
tracxn.com verymes.xyz
traffpartners.com video-production.com
trainoffend.ml videosbox.ru
triumf-realty.ru vinylvault.co.uk
twu.com.ua vodaodessa.com
txxx.com vseigru.one
uchil.net wallpaperaccess.com
ucoz.ru website-analytics.online
ufa.xrus.org whatistizanidine2mg.blogspot.com
uptime-gamma.net while.cheapwebsitehoster.com
urccvfmc.bloger.index.hr
usbggettwku.ga
vadimkravtcov.ru
varbergsvind.se
vellings.info
viddyoze.com
vksaver-all.ru
wedding0venues.tk
wineration.com wineration.com
winx-play.ru xaijo.com
wpsecurity.website xn----ctbbcjd3dbsehgi.xn--p1ai
www.wohnkabinen-shop.de xn--lifehacer-1rb.com
x-stars.ru xvideosbay.com
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xxxkaz.org
zapiszto.pl yaderenergy.ru
zebradudka.com yebocasino.co.za
zeleznobeton.ru yellowstonesafaritours.com
zigarettenonlinekaufen1.blox.pl yopoint.in
zooggames.com zarabiaj-dzis.pl
zrelaya.pw zarabotat-na-sajte.ru
zytpirwai.net zvooq.eu

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.0.106 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3088 ### Version: V4.2022.05.3089
### Updated: Fri May 6 10:03:39 UTC 2022 ### Updated: Fri May 6 22:01:36 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3088 ### Version: V4.2022.05.3089
### Updated: Fri May 6 10:03:39 UTC 2022 ### Updated: Fri May 6 22:01:36 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##