vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-05 12:09:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.02.3578 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-02-13 10:09:34 +00:00
parent f3e6ee8636
commit b15ba30bf6
47 changed files with 102573 additions and 102566 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.02.3577 #### Version: V4.2023.02.3578
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.02.3577 #### Version: V4.2023.02.3578
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2023.02.3577 #### Version: V4.2023.02.3578
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12682
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2023.02.3577 #### Version: V4.2023.02.3578
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12686
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12686
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12686
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12686
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.0.20 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.0.20 0;
127.0.0.1 0; 127.0.0.1 0;

12686
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.0.20 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.0.20 0;
127.0.0.1 0; 127.0.0.1 0;

12686
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.0.20 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.0.20 0;
127.0.0.1 0; 127.0.0.1 0;

12686
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3577 3578

12686
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12686
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.171.141.224 0; 40.77.93.0 0;

12686
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.0.20 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.0.20 0;
127.0.0.1 0; 127.0.0.1 0;

12686
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.0.20 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.0.20 0;
127.0.0.1 0; 127.0.0.1 0;

12686
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.0.20 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.0.20 0;
127.0.0.1 0; 127.0.0.1 0;

12686
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.171.141.224 0; 40.77.93.0 0;

12686
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.171.141.224 0; 40.77.93.0 0;

12686
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.0.20 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BuiltWith ALittle Client
BunnySlippers GridBot
CheeseBot Keyword Density
Copyscape Netvibes
Craftbot Offline Explorer
LinksManager OrangeBot
Siteimprove Page Analyzer
WebImageCollector SemrushBot
Zermelo WinHTTrack
ZmEu coccocbot

300
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider
404checker
AIBOT
ALittle Client ALittle Client
ASPSeek ASPSeek
Abonti Aboundex
AfD-Verbotsverfahren Acunetix
AhrefsBot
AiHitBot
Aipbot
Alexibot Alexibot
Alligator AllSubmitter
AlphaBot
Anarchie
Anarchy
Anthill Anthill
Aspiegel
Atomseobot Atomseobot
AwarioRssBot Attach
AwarioSmartBot AwarioSmartBot
BBBike BBBike
BackDoorBot BDCbot
BLEXBot
BackStreet BackStreet
Backlink-Ceck
Bandit
Barkrowler Barkrowler
BlackWidow BatchFTP
BlowFish Black Hole
Bolt Blow
Brandprotect BotALot
Buck Brandwatch
BuiltBotTough
BuiltWith
Bullseye Bullseye
CATExplorador
CCBot CCBot
CazoodleBot CheeseBot
Cegbfeieh
CensysInspect
CheTeam
CherryPicker
Chlooe Chlooe
Citoid
Claritybot
Cloud mapping
Cocolyzebot
CopyRightCheck CopyRightCheck
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
CrunchBot CrunchBot
DSearch DIIbot
DatabaseDriverMysqli Devil
DnBCrawler-Analytics DigitalPebble
Dispatch
DnyzBot DnyzBot
DomainCrawler DomainSigmaCrawler
Domains Project Domains Project
Download Wonder Download Wonder
Dragonfly
Drip Drip
EMail Siphon ECCP/1.0
Ebingbong Ebingbong
EirGrabber
EroCrawler
Evil Evil
ExtractorPro ExtractorPro
FHscan Fimap
FlashGet
Freeuploader Freeuploader
FrontPage FrontPage
G-i-g-a-b-o-t FyberSpider
GetWeb Fyrebot
GT::WWW
GermCrawler
Getintent Getintent
Gigabot GrapeFX
GrapeshotCrawler
GridBot GridBot
HEADMasterSEO HTMLparser
HMView HTTrack
HTTP::Lite HybridBot
Haansoft IRLbot
HaosouSpider
Harvest
Heritrix
IDBTE4M
IDBot
Id-search
IlseBot IlseBot
Image Sucker Image Sucker
IndeedBot
InfoNaviRobot
InfoTekies
Intelliseek Intelliseek
InternetSeer
Iskanie Iskanie
IstellaBot JOC Web Spider
Jbrofuzz Jyxobot
JikeSpider
Kenjin Spider
LWP::Simple LWP::Simple
Lanshanbot Larbin
LeechGet LeechFTP
LexiBot Lightspeedsystems
LieBaoFast LinkScan
Likse LinkpadBot
LinkWalker
Linkbot
LinksManager LinksManager
Lipperhey
Lipperhey Spider
Litemage_walker Litemage_walker
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc MIIxpc
MQQBrowser
MSFrontPage MSFrontPage
Magnet MSIECrawler
MTRobot
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic-SEO
Majestic12 Majestic12
MarkMonitor
MarkWatch MarkWatch
Masscan Mass Downloader
MauiBot MauiBot
MeanPath Bot Meanpathbot
MegaIndex.ru MegaIndex.ru
Metauri Metauri
MicroMessenger
Microsoft URL Control Microsoft URL Control
Minefield Mister PiX
Mojeek Mojeek
Mojolicious Mojolicious
MolokaiBot Morfeus Fucking Scanner
Mr.4x3
Msrabot
Musobot
NICErsPRO NICErsPRO
NPbot NPbot
Name Intelligence
Nameprotect Nameprotect
Navroad NearSite
Nessus Needle
NetAnts NetLyzer
NetMechanic
NetSpider
NetZIP NetZIP
Netvibes Netcraft
Nibbler Nibbler
NimbleCrawler Nmap
Offline Explorer Not
OpenVAS Nutch
OutclicksBot Octopus
PECL::HTTP Openfind
Page Analyzer OrangeBot
PageGrabber POE-Component-Client-HTTP
PageThing.com PageScorer
Panscient Papa Foto
Pavuk Pavuk
PeoplePal Petalbot
Pi-Monster
Picscout Picscout
PictureFinder Pimonster
Piepmatz PleaseCrawl
Pockey Pockey
ProPowerBot ProPowerBot
Probethenet ProWebWalker
PyCurl PyCurl
QueryN Metasearch QueryN Metasearch
Quick-Crawler Rankivabot
Re-re RocketCrawler
ReGet
RebelMouse
RedesScrapy
Rogerbot Rogerbot
SBIder
SEOkicks SEOkicks
SEOprofiler SEOprofiler
SMTBot SEOstats
SalesIntelligent SISTRIX
Scanbot
ScoutJet ScoutJet
Screaming Scrapy
Searchestate Searchestate
SearchmetricsBot Seekport
SemanticJuice
Semrush
SentiBot SentiBot
SiteCheckerBotCrawler SeobilityBot
Shodan
Site Sucker
SiteExplorer
SiteSnagger SiteSnagger
Sitebeam SiteSucker
SmartDownload
Snake
Snapbot
Snoopy
Sociscraper
Sogou web spider Sogou web spider
Sosospider Sosospider
SpaceBison SpaceBison
Spammen
SpankBot
Spanner Spanner
Spbot
Spinn3r
SputnikBot SputnikBot
Sqlmap
Sqworm
Steeler Steeler
Stripper T0PHackTeam
Sucker Teleport
Sucuri TeleportPro
SuperHTTP Telesphoreo
Szukacz TheNomad
The Intraformant Toata
Toweyabot Tracemyfile
Trendiction
Trendictionbot
True_Robot
Turingos Turingos
Turnitin Turnitin
TurnitinBot
TwengaBot
URLy Warning
URLy.Warning URLy.Warning
Upflow Upflow
V-BOT
VCI
Vagabondo Vagabondo
VelenPublicWebCrawler Voltron
VeriCiteCrawler WBSearchBot
VidibleScraper WWW-Collector-E
VoidEYE
WEBDAV
WISENutbot
WPScan
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWWOFFLE
Wallpapers Wallpapers
WeSEE Wallpapers/3.0
Web Auto WallpapersHD
Web Pix Web Pix
WebAuto
WebBandit
WebEnhancer
WebFuck
WebGo IS WebGo IS
WebImageCollector WebImageCollector
WebPix
WebReaper WebReaper
WebStripper WebSauger
WebSucker WebSucker
WebmasterWorldForumBot
Webshag Webshag
Website Quester Website Quester
WebsiteExtractor WebsiteExtractor
Webster WebsiteQuester
Whack
Whatweb Whatweb
Widow WinHTTrack
WiseGuys Robot WiseGuys Robot
Woobot Wprecon
Wotbox Zauba
Xaldon WebSpider Zeus
Xaldon_WebSpider adscanner
Xenu archive.org_bot
YoudaoBot
ZoomBot
arquivo-web-crawler arquivo-web-crawler
autoemailspider backlink-check
cah.io.community cah.io.community
domainsproject.org check1.exe
coccocbot
com.plumanalytics
demandbase-bot
eCatch eCatch
heritrix evc-batch
instabid facebookscraper
gopher
internetVista monitor internetVista monitor
iubenda-radar ips-agent
lwp-request isitwp.com
linkdexbot
lwp-trivial lwp-trivial
magpie-crawler magpie-crawler
mediawords mediawords
muhstik-scan muhstik-scan
oBot
pcBrowser pcBrowser
probe-image-size
ripz ripz
satoristudio.net satoristudio.net
seobility scan.lol
seocompany.store
seostar
serpstatbot
sexsearcher
siteripz siteripz
tAkeOut trendiction.de
ubermetrics-technologies.com
webmeup-crawler webmeup-crawler
webprosbot webprosbot
x09Mozilla x22Mozilla
xpymep1.exe

290
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404enemy ASPSeek
ADmantX Aboundexbot
Abonti
Acunetix
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot Aipbot
AllSubmitter Anarchy99
AlphaBot
Anarchie
Ankit Ankit
Anthill Apexoo
Aspiegel
Asterias
Atomseobot Atomseobot
AwarioSmartBot AwarioSmartBot
BDFetch BBBike
BLEXBot BLEXBot
BackStreet
BackWeb BackWeb
Badass Bigfoot
Battleztar Bazinga
Bitacle Bitacle
Black Hole Black Hole
Blackboard BlackWidow
Blow Blow
Bolt Bolt
Buck BotALot
Buddy BuiltBotTough
Bullseye Bullseye
BunnySlippers
BuzzSumo BuzzSumo
CCBot CATExplorador
CODE87 CODE87
CSHttp CSHttp
Calculon Calculon
CazoodleBot
CheeseBot CheeseBot
CherryPicker
ChinaClaw ChinaClaw
Citoid Cloud mapping
Cliqzbot
Cogentbot
Collector Collector
Cosmos Copier
Craftbot CopyRightCheck
Crawling at Home Project
Crescent Crescent
Custo Custo
CyotekWebCopy
DBLBot
DIIbot DIIbot
DTS Agent DSearch
Deusu DatabaseDriverMysqli
Demon
Devil Devil
Dirbuster Digincore
Disco Discobot
Discoverybot Discoverybot
Dispatch
DittoSpyder DittoSpyder
DomainAppender DnyzBot
Domains Project Download Wonder
Dragonfly Dragonfly
Drip Drip
EMail Siphon
EasyDL EasyDL
Express WebPictures Ebingbong
ExtractorPro EirGrabber
Extreme Picture Finder Evil
Fimap Extractor
EyeNetIE
FDM
Firefox/7.0 Firefox/7.0
FlashGet Flunky
Foobot Foobot
Freeuploader Freeuploader
FrontPage
FyberSpider
Fyrebot
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW GalaxyBot
Genieo GetWeb
Go-Ahead-Got-It Getintent
Gotit
Grabber Grabber
GrapeshotCrawler HMView
HTMLparser HTTrack
Hloader Hloader
HybridBot HybridBot
IDBTE4M
IDBot
IlseBot IlseBot
Image Sucker Indy Library
InfoNaviRobot
InfoTekies
Intelliseek Intelliseek
InterGET InterGET
Internet Ninja JOC Web Spider
InternetSeer
Jbrofuzz Jbrofuzz
JustView JennyBot
Jetty
Joomla
Jorgee
Jyxobot
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Kinza Kozmosbot
LNSpiderguy
LWP::Simple
Leap Leap
LeechGet LeechFTP
LibWeb
Libwhisker Libwhisker
Likse
LinkScan
LinkextractorPro LinkextractorPro
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey LinqiaScrapeBot
MIDown tool Lmspider
MIIxpc Ltx71
MJ12bot MQQBrowser
Mag-Net
Mail.RU_Bot Mail.RU_Bot
Majestic SEO
Majestic-SEO
Majestic12
MarkMonitor MarkMonitor
Mass Downloader Masscan
Mata Hari Mata Hari
MauiBot Mb2345Browser
Microsoft Data Access MeanPath Bot
Microsoft URL Control MicroMessenger
Mister PiX MolokaiBot
Name Intelligence NICErsPRO
Nameprotect
Net Vampire
NetAnts
NetLyzer NetLyzer
NetSpider
Netcraft
Nettrack Nettrack
Netvibes
NextGenSearchBot NextGenSearchBot
Nibbler NimbleCrawler
Nikto Nutch
Nmap Offline Explorer
Nuclei
Octopus
Offline Navigator Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS OpenVAS
Openfind
OrangeBot OrangeBot
OutclicksBot OutclicksBot
OutfoxBot OutfoxBot
PHPCrawl PECL::HTTP
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageScorer
PageThing.com PageThing.com
Pandalytics Pandalytics
Pi-Monster
Picsearch Picsearch
Piepmatz Pockey
Pixray Pu_iN
PleaseCrawl RSSingBot
ProPowerBot RankActiveLinkBot
ProWebWalker
Probethenet
Psbot
Pump
QueryN Metasearch
RankActive
RankFlex RankFlex
ReGet RankingBot
Reaper RankingBot2
RebelMouse RealDownload
RedesScrapy
Ripper Ripper
RocketCrawler RocketCrawler
Rogerbot
SBIder
SEOkicks-Robot SEOkicks-Robot
SISTRIX
SalesIntelligent SalesIntelligent
Scanbot ScanAlert
ScoutJet ScoutJet
Searchestate Screaming
SentiBot ScrepyBot
SearchmetricsBot
Seekport
SeoSiteCheckup
SeobilityBot SeobilityBot
SiteCheckerBotCrawler Shodan
SiteExplorer
SiteLockSpider
SiteSucker
Sitebeam
Sitevigil
SmartDownload
Snapbot Snapbot
Snoopy SpaceBison
Sottopop Spinn3r
Spammen Sqlmap
Spbot Sqlworm
Steeler Steeler
SuperBot Sucker
Sucuri
Surfbot
SurveyBot SurveyBot
Suzuran
Swiftbot
Szukacz
T0PHackTeam T0PHackTeam
T8Abot Teleport
Telesoft Telesoft
Telesphoreo Telesphoreo
Thumbor Telesphorep
TightTwatBot
Titan Titan
Toata Toata
Trendiction Trendiction
Turnitin Turnitin
Typhoeus Typhoeus
URLy Warning URLy Warning
UnisterBot
V-BOT
Vacuum Vacuum
VelenPublicWebCrawler Vagabondo
VeriCiteCrawler VeriCiteCrawler
WASALive-Bot Virusdie
WWW-Collector-E Voltron
WWW::Mechanize WBSearchBot
WeSEE WEBDAV
WPScan
WWW-Mechanize
Wallpapers
Web Auto Web Auto
Web Enhancer Web Collage
WebBandit WebCollage
WebCopier WebCopier
WebEnhancer
WebGo IS
WebLeacher WebLeacher
WebPix WebPix
WebReaper WebSucker
WebWhacker WebWhacker
Webalta WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag
Website Quester
WebsiteQuester WebsiteQuester
Webster Webster
Whack
Whatweb Whatweb
WiseGuys Robot Who.is Bot
Wonderbot Woobot
Zade YoudaoBot
Zeus Zeus
ZoominfoBot Zitebot
archive.org_bot ZmEu
autoemailspider ZoomBot
backlink-check adscanner
check1.exe check1.exe
crawler4j cognitiveseo
crawl.sogou.com
crawler.feedback
demandbase-bot demandbase-bot
domainsproject.org
eCatch eCatch
facebookscraper evc-batch
ips-agent ips-agent
isitwp.com magpie-crawler
linkdexbot
lwp-request
lwp-trivial
meanpathbot meanpathbot
mediawords mediawords
netEstate NE Crawler oBot
pcBrowser pcBrowser
polaris version polaris version
s1z.ru seobility
satoristudio.net
scalaj-http
scan.lol
seoscanners seoscanners
serpstatbot seostar
sexsearcher
siteripz siteripz
sogouspider spyfu
sysscan
trendiction.com
trendiction.de trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
webpros.com webgains-bot
x22Mozilla webprosbot
zauba.io xpymep1.exe
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
adnotbad.com appfastplay.com
buyclomidonlaine.com epicbrogaming.com
impisr.edunsk.ru free-gluten.ru
koversite.info idc.com.ua
megaapteka.ru moetomnenie.com
new7ob.com negociosdasha.com
no-fuel.org snegozaderzhatel.ru
rarbg.to softlinesolutions.me
tilido.com uhodzalijami.ru
wygraj-skiny.win zoominfo.com

916
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
123movies.love 1000n1.ru
12u.info 1314dh.com
14b.info 1j7740kd.website
1millionusd.xyz 2728fb936f0.com
3dracergames.com 4ur.click
4ureyesonly.com 98oi.ru
51unlim.ru a96527gi.beget.tech
8gold.com aa81bf391151884adfa3dd677e41f94be1.com
8xv8.com
aaa38852e886ac4af1a3cff9b47cab6272.com
aad1f4acb0a373420d9b0c4202d38d94fa.com
absugars.com
academia-nsk.org academia-nsk.org
actices.com adidas.frwebs.fr
adclickservice.com adsref.men
adinterax.com
ads.gold
adsland.men
adultgalls.com
affordablewebsitesandmobileapps.com
agreda.pluto.ro
ahmedabadwebs.com
ahrefs.com ahrefs.com
alert-fdm.xyz akita.kz
allboard.xobor.de alf-img.com
allwomen.info alfa-medosmotr.ru
alpharma.net alfapro.ru
altamayoztourism.com alienwheels.de
amateurlivechat.org allsilver925.co.il
amung.us alvtank.se
anonymous-redirect.com amazon-seo-service.com
aosheng-tech.com anapa-inns.ru
apartmentratings.com anlimebel.kiev.ua
apiitsthirteende-a.akamaihd.net apessay.com
apiqualitinknet-a.akamaihd.net apibetweenlinesn-a.akamaihd.net
app-ready.xyz apibrowsesmartne-a.akamaihd.net
apispringsmartne-a.akamaihd.net
appiq.mobi
apple.com-webbrowsing-security.review apple.com-webbrowsing-security.review
arate.ru approved.su
arraty.altervista.org arcteryxstore.online
ascat.porn arendakvartir.kz
astrochicks.com arendovalka.xyz
atelielembrancaqueencanta.com.br asrv-a.akamoihd.net
aviav.ru.com atovh.local-finders.com
aviva-limoux.com ausergrubhof.info
ayodhya.co automatic-seo.com
azte.ch autotrends.today
back.dog availit.weebly.com
bankmib.ru aviav.org
avtointeres.ru
azadnegar.com
azartniy-bonus.com
azazaporn.com
babespcs.com
bablonow.ru
backlinkwatch.com
baersaratov.ru
balla-boo.se
bankiem.pl
battle.net
battlecarnival.su battlecarnival.su
bestfortraders.com bazaronline24.ru
bfz.biz beachdriveblog.com
bio-optomarket.ru beachtoday.ru
bioscorp.ru beauty-things.com
bjanshee.ru bestofferswalkmydogouteveryday.gq
besttorrentknifta.weebly.com
bestwebsiteawards.com
bezsporno.ru
bin-brokers.com
bitporno.sx
biznesluxe.ru
blackle.com blackle.com
blockety.co blagovest-med.ru
blog.f00kclan.de blog.koreadaily.com
bluerobot.info blogs.rediff.com
bobinoz.com bobinoz.com
bookhome.info boeuklad.com
bookmaker-bet.com boleznikogi.com
brainboostingsupplements.org bonanza-fortune.men
braindaily.xyz bonzybuddy.com
breastaugmentation.co.za braip.com.br
brimstonehillfortress.org
bronzeaid-a.akamaihd.net bronzeaid-a.akamaihd.net
browsepulse-a.akamaihd.net bryansk.zrus.org
browserprotecter.com
budilneg.xyz budilneg.xyz
buqayy0.livejournal.com buttons-for-your-website.com
burkesales.com buzzsumo.com
buscarfoto.com bvlgariring.xyz
buynorxx.com camsex.xxx-cam.webcam
carloans.com cardul.ru
carsplate.com cartierring.xyz
cartechnic.ru ceotrk.com
casinopinup-wins.com cercacamion.it
check-this-out-now.online cerev.info
chimiver.info cheapmoncler.win
christianlouboutinshoes.xyz cheatcode-lita12.rhcloud.com
cjmc.info chiblackhawks-jerseys.com
classicakuhni.ru cialis-samples.com
clickso.com cialischsrx.com
clmforexeu.com ciproandtizanidine.blogspot.com
compliance-alexa.xyz communisave.co.za
compliance-brian.top compliance-andrew.top
compliance-brian.xyz compliance-don.top
compliance-ivan.top compliance-jane.top
compliance-viktor.top compliance-josh.top
connectionstrenth.com comsysnet.com
constantaservice.net conocer-sanabria.com
counciltally.com cornerstone-countertops.com
counter.bloke.com cougfan.info
courtshipgift.com covetnica.com
cowblog.fr crackzplanet.com
creatives.adbetclickin.pink cranly.net
cubs-jerseys.us cs-passion.pl
cyprusbuyproperties.com culturevie.info
czat.wp.pl customsua.com.ua
daydream-studio.ru danceuniverse.ru
deanmoore.ie daptravel.com
decoratinghomes.ga darodar.com
dengi-pod-zalog-nedvizhimosti.ru daymusam.com
deniven.1bb.ru deadlinkchecker.com
deplim.com detmebel.su
detoxmed24.ru detoxmed24.ru
detskie-zabavi.ru detskie-zabavi.ru
dev33.dioniqlabb.se dfiles.me
dnepropetrovsk.mistr-x.org dieta-personalna.pl
domaincrawler.com dildofotzen.net
domik-derevne.ru dinosaurus.site
donna7753191.ru discountliv.com
dotmass.top disruptingdinnerparties.com
downvids.net dnepr-avtospar.com.ua
drhomes.biz dnsrsearch.com
driving.kiev.ua dodlive.mil
drpornogratisx.xxx domaincheck.io
dtm-spain.com downloadeer.net
drugs-no-rx.info
dumuelave.xyz
dvd-famille.com
e-buyeasy.com
eaplay.ru
eas-seo.com eas-seo.com
editmedios.com easync.io
ekaterinburg.xrus.org ecookna.com.ua
eldiariodeguadalajara.com eldiariodeguadalajara.com
elektrozigaretten1.webs.com electrik-avenue.com
enbersoft.com elektronischezigarettekaufen1.myblog.de
enginebay.ru enews.tech
enpolis.ru eng-lyrics.com
eorogo.top envaseslotusama.com
epngo.bz ero-advertising.com
escort-russian.com erotiktreff24.info
esopini.com etenininrade.ga
estelight.ru eu-cookie-law.blogspot.com
europeanwatches.ru european-torches.ru
exci.ru eurosamodelki.ru
expolicenciaslatam.co evidencecleanergold.com
extremepornos.net excitacion.info
extremez.net extremez.net
ezigarettekaufen2.yolasite.com ez8motelseaworldsandiego.com
facebook-mobile.xyz ezigarettenkaufen1.pagina.gr
falcoware.com ezigarettenshop2.postbit.com
fastfixing.tech f07.de
filmetricsasia.com femmesdenudees.com
filmi-v.online fenoyl.batcave.net
firesub.pl ferretsoft.com
fitfloponline.store fettefrauen.net
fikasound.tk
film-one.ru
findanysex.com
fishingwholesale.us
fishtauto.ru
flavors.me
flooringinstallation-edmonton.com flooringinstallation-edmonton.com
floridamobilebillboards.com
fm-upgrade.ru
folowsite.com folowsite.com
formulaantiuban.com foxtechfpv.com
forodvd.com franecki.net
forums.toucharcade.com frankofficial.ru
foto-weinberger.at freakycheats.com
franch.info
free-fbook-traffic.com
free-share-buttons.blogspot.com free-share-buttons.blogspot.com
free-stock-illustration.com free-today.com
freejabs.com freelinkbuilding.website.tk
fres-news.com freshdz.com
fuckmill.com frustrated-favorable.gq
fuel-gas.com funcrushgames.com
fx-brokers-review.com gallery.rennlist.com
g33.org gamevalue7.weebly.com
galeria-zdjec.com gavuer.ru
gallerily.com gdcentre.ru
gallerylisting.com get-your-social-buttons.info
gambarkatabaru.com gfaq.ru
gearcraft.us given2.com
gelezki.com glicol.kz
germes-trans.com googlemare.com
globatur.ru
gloverid.site
goodwriterssales.com
googleseo.com.tr googleseo.com.tr
growshop.es gosmeb.ru
guildebzh.info gotorussia.com
gratuitxblcodes.com
greatgrace.ru
growmyfunds.ca
gunsvicceadadebt.tk
gyffu.com gyffu.com
hamilton.ca h2monline.com
help.tpu.ru harmonyglen.us
hentaiheroes.com hdwallpapers-free.com
hesteel.pl hentai-manga.porn
hexpilot.com himazin.info
highspeed5.net
hit-kino.ru
hitcpm.com hitcpm.com
hitsbox.info hjaoopoa.top
honyaku.yahoofs.jp hkdiiohi.skyrock.com
houdom.net homedecoguide.info
howlongdoestizanidinestayinyoursystem.blogspot.com hornymatches.com
host-protection.com
hostsshop.ru
hothor.se
hothot.ru
howmuchdoestizanidinecost.blogspot.com howmuchdoestizanidinecost.blogspot.com
hut1.ru ideawheel.com
i-service.kz ideibiznesa2015.ru
ico.re ikritikimou.gr
idealtits.net imagecoolpub.com
ifirestarter.ru imagerydatabase.com
ilovevitaly.com images.gyffu.com
images-graphics-pics.com imgarcade.com
imagine-ex.co import-sales.com
infowarcraft.ru inboxdollars.com
innoslicon.com infokonkurs.ru
insta-add.pro internetproviderstucson.com
internet-apteka.ru isitwp.com
intervsem.ru istizanidineacontrolledsubstance.blogspot.com
intimshop-fantasy.ru iusstf.org
investzalog.ru iwanttodeliver.com
iweblist.info javitas.info
iyasimasennka.com jo24news.com
jackwolfskinoutlet.online joessmogtestonly.com
je7.us journeydownthescale.info
jerseywholesalechinabiz.com karadene.com
jofucipiku.tk karaganda.xkaz.org
johannesburgsingles.co.za keki.info
kakablog.net kemerovo.zrus.org
kalandranis.gr
karachev-city.ru
kashubadesign.ru
katushka.net
keywordglobal.co.za keywordglobal.co.za
kino-rf.ru kinogolos.ru
klosetkitten.com kosmetyki.tm.pl
knowyournextmove.com laptop-4-less.com
kozhniebolezni.com le-clos-des-alouettes.com
kurwa.win letslowbefast.today
kvartir-remont.biz li-er.ru
labelwater.se libertybilisim.com
landreferat.ru lion.conboy.us
larger.io lionking-1994.blogspot.com
laxdrills.com
lego4x4.ru
letsart.ru
linkdebrideur.xyz
live-sexcam.tk live-sexcam.tk
lyrics.home-task.com liver-chintai.org
mabdoola.blogspot.com lkbennettstore.online
magazintiande.ru locatellicorretor.com.br
magento-crew.net locimge.pw
mainlinehobby.net lubetube.com
marcoislandvacations.net luckyday.world
maridan.com.ua m-google.xyz
masserect.com m292.info
mb140.ru m4ever.net
meduza-consult.ru mac-shield.com
megakino.net macdamaged.space
meteostate.com macresource.co.uk
mhi-systems.ru mactechinfo.info
midst.eu mafcards.ru
mobile.ok.ru magicalfind-a.akamaihd.net
monetizationking.net mamasuper.prom.ua
monsterdivx.tv manimpotence.com
morepoweronmac.trade maximpartnerspr.com
mosrif.ru me-ke.com
moviemail-online.co.uk mebelcomplekt.ru
mrbojikobi4.biz medicineseasybuy.com
mttwtrack.com medispainstitute.com.au
musezone.su mega-bony-2017.pl
mygameplus.ru meltwater.com
mymercy.info mensandals.xyz
mysex21.com mesto-x.com
metafilter.com
metascephe.com
mikozstop.com
militarysale.pro
minegam.com
misterjtbarbers.com
mixed-wrestling.ru
mmgq.ru
mobile-appster.ru
mobplayer.ru
most.gov.iq
moxo.com
mp3ringtone.info
mrsdalloways.com
mural.co
mwtpludn.review
mymobilemoneypages.com
myperiod.club
mypornfree.ru
myprintscreen.com
mysexpics.ru mysexpics.ru
myshopmatemac.com na15.ru
naj-filmy24.pl nastydollars.com
narkologiya-orel.ru neks.info
narosty.com northfacestore.online
naturalpharm.com.ua notaria-desalas.com
netoil.no nudejapan.net
nextconseil.com obiavo.su
nina.az obiavo.uz
njpalletremoval.com oda.as
noclegonline.info offer.wpsecurity.website
nonews.co office-windows.ru
notasprensa.info okroshki.ru
novodigs.com omoikiri-japan.ru
novosibirsk.xrus.org orel-reshka.net
nuit-artisanale.com outpersonals.com
o333o.com
obiavo.kz
offers.bycontext.com
okeinfo.online
one.net.in
oneshotdate.com
onlineslotmaschine.com
openmediasoft.com
orenburg-gsm.ru
orion-code-access.net
ossmalta.com
outclicks.net
outshop.ru outshop.ru
ovirus.ru oxford-book.com.ua
owohho.com ozoz.it
ozas.net panchro.xyz
paphoselectricianandplumber.com perfectplanned.com
parfusale.se
parlament.biz
pdn-4.com
pdns.cz
perso.wanadoo.es
peugeot-club.org peugeot-club.org
pflexads.com phrcialiled.com
photo-clip.ru picmoonco.pw
pinsdaddy.com pinkduck.ga
pintattoos.com pizda.lol
pizdeishn.com pl-top.pl
pizza-imperia.com
pl-vouchers.com
plaff-go.ru
platesauto.com platesauto.com
polcin.de polska-poezja.com
pomoc-drogowa.cba.pl pomoc-drogowa.cba.pl
popup-jdh.xyz pornodojd.ru
pornmania.pl pornofeuer.com
pornofiljmi.com
pornogad.com pornogad.com
pornogig.com pornozhara.com
pornoinn.com portal-eu.ru
pornonik.com
pornosexrolik.com
portside.cc portside.cc
pravoholding.ru pr0fit-b0x.com
princevc.com predmety.in.ua
priora-2.com pricheskaonline.ru
privhosting.com printie.com
prizefestival.mobi prizesbook.online
prombudpostach.com.ua profitkode.com
promgirldresses.xyz profitwithalex.info
prostitutki-belgoroda.org pshare.biz
psoriasis-file.trade purchasepillsnorx.com
quickcashlimited.com pussyspace.net
quickloanbank.com pwwysydh.com
qwertty.net quelle.ru
ranksonic.com rabot.host.sk
registry-cleaner.net rankia.com
reklama-i-rabota.ru ranksays.com
relatodelpresente.com.ar razvratnoe.org
remont-mobile-phones.ru razyboard.com
ricorsogiustizia.org realmonte.net
ringporno.com rebuildermedical.com
risparmiocasa.bz.it recordpage-a.akamaihd.net
royal-investments.net referencemoi.com
rukino.org regdefense.com
russia-today-video.ru reining.lovasszovetseg.hu
sanjosestartups.com remedyotc.com
sarahmilne.top remont-comp-pomosh.ru
sax-sex.com rfid-locker.co
scalerite.co.za ritlweb.com
scanner-jessica.top rixpix.ru
scanner-julia.top rniaeba.ga
scanner-nelson.top robertefuller.com
schlampen-treffen.com rockma.se
school-diplomat.ru roma-kukareku.livejournal.com
sdelai-prosto.ru rossmark.ru
search.pch.com runtnc.net
searchencrypt.com rus-pornuha.com
searchtooknow-a.akamaihd.net rusexy.xyz
see-your-website-here.com ryetaw.com
seorank.info s-iwantyou.com
seorankinglinks.com s8-nowy-wygraj.comli.com
service.adtech.us salutmontreal.com
servingnotice.com sbtdesign.co.uk
sex-tracker.de scanner-don.top
sexpornotales.com schalke04fc.info
sexstream.pl sdsjweb.com
sexy-screen-savers.com search.alot.com
sexy.babes.frontend-stack.top searchinterneat-a.akamaihd.net
sexyebonyteen.com searchmywindow-a.akamaihd.net
seasaltwithfood.com
sei80.com
seocdvig.ru
sergiorossistore.online
serpstat.com
servethis.com
sevendays.com.ua
sex-dating.co
sex-tracker.com
sexfreepornoxxx.com
sexiporno.net
sexreliz.com
sextracker.com
sexytrend.ru sexytrend.ru
share-buttons-for-free.com sfj-ror.no
shop.acim.org shmetall.com.ua
shopvilleroyboch.com.ua shop.xz618.com
sibtest.ru sibtest.ru
sibvitr.ru sildenafil-tadalafil.info
sierraapps.com sisiynas.ru
signal03.ru
similardeals.net
simply.net
siteonomy.com siteonomy.com
siteripz.net siteripz.net
sitevaluation.org sivs.ru
sitiz.club sixcooler.de
skylta.com sklad-24.ru
skypasss.com slujbauborki.ru
slomm.ru smallseotools.com
slowmacfaster.trade snip.tw
soc-econom-problems.ru socialbookmarksubmission.org
soc-proof.su socialmediasuggest.com
social-button.xyz softtor.com
social-s-iii.xyz soochi.co
socialmonkee.com spaceshipad.com
sotkal.lark.ru speedup-my.site
soundfrost.org spywareit.com
spravka130.ru statoutlook.info
spywarenuker.com stauga.altervista.org
squidoo.com steebook.com
stairliftstrue.com strana-krasoty.ru
stefanbakosab.se stroiminsk.org
stockspmb.info sumo.com
storehouse.ua superoboi.com.ua
studentguide.ru supers.com.ua
studiokamyk.com.pl
sukarame.net
summerlinhomes411.info
sunflowerdrawingpaintings.blogspot.com
superstats.com
suture.co
svetlotorg.ru
svolze.com
sygraem.com
tabletkinaodchudzanie.com.pl
taboola.com
takeprofitsystem.com takeprofitsystem.com
tarad.com taqplayer.info
tasteidea.com taraz.xkaz.org
td-33.ru
td-l-market.ru td-l-market.ru
templates.franklinfire.co technopellet.gr
tgtclick.com telegramdownload10.com
thecoolimages.net theheroes.ru
thecoral.com.br thepornsex.org
thepantonpractice.co.uk thetattoohut.com
thepokertimer.com thruport.com
timecrimea.ru timeallnews.ru
tizanidineandhydrocodone.blogspot.com titan-ads.life
tizanidinehcl4mgisitanarcotic.blogspot.com
tizanidinerecreationaluse.blogspot.com tizanidinerecreationaluse.blogspot.com
tizanidineshowupondrugtest.blogspot.com tizanidinesleepaid.blogspot.com
topclickguru.com tmearegion26.com
traffic-club.info todohr.com
trafficcentr.xyz tooplay.com
trafficmp.com top-l2.com
trahvid.com tour-line.net
trailer.cinemaflix.website toys.erolove.in
trumpetedextremes.com track.deriv.com
tvand.ru tradgardspartner.se
twittruth.com traffic2money.com
udayavani.com trafficnetzwerk.de
uggsale.online truck-land.life
unitexindia.com tubeline.biz
uptime-as.net tupper-shop.ru
uptimebot.net turizmus.us
urccvfmc.bloger.index.hr txxx.com
uroffer.link uni.me
v720hd.ru unionmarkt.de
vacuumcleanerguru.com unitygame3d.com
vavilone.com upupa.net
vegan-foods.us urengoy.pro
vekzdorov.ru usbggettwku.ga
velen.io vbabule.net
veopornogratis.xxx viagra-soft.ru
vintontech.info viagraneggrx.com
virus-schutzmasken.de vilingstore.net
vitanail.ru vipsiterip.org
viveresaniesnelli.it visa-pasport.ru
vksex.ru vkontarkte.com
volgograd.xrus.org vriel.batcave.net
vpnmouse.com vsexkontakte.net
vrnelectro.ru vzlom-na-zakaz.com
vtc.pw wallpapersdesk.info
watchinf.com wardreapptokone.tk
waterpurifier.club warezaccess.com
watracker.net watracker.net
webmonetizer.net weburlopener.com
websitevaluebot.com weburok.com
white-truck.life wedding0venues.tk
whosonmyserver.com welck.octopis.com
wleuaprpxuvr.ga worldinternetauthority.com
woman-orgasm.ru worldtraveler.world
word-vorlagen.xyz
wormix-cheats.ru
wovis.site
wowas31.ucoz.ru wowas31.ucoz.ru
woweb.com.ua
wrz0iuebwhp5fg.freeddns.com wrz0iuebwhp5fg.freeddns.com
www.888.com wufak.com
www.jbetting.com ww1943.ru
x-diesel.org x-true.info
x-musics.com xaijo.com
x-rates.ru xblognetwork.com
xerox-douglas.cf xlolitka.com
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xmnb.net
xn----ctbigni3aj4h.xn--p1ai xmronta.com
xn--80aikhbrhr.net xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xn--80aodinpgi.xn--p1ai xn--b1adccaf1bzj.xn--p1ai
xn--d1acah0c.xn--p1ai xn--h1ahbi.com.ua
xn--oogle-wmc.com xnxxandxvideos.com
xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai yaminecraft.ru
xnxx-n.com yellocloud.be
xolodremont.ru
xsion.net
xtremeeagles.net
xxxsiterips.xyz
xxxtubesafari.com
youghbould.wordpress.com youghbould.wordpress.com
youjizz.com your-bearings.com
yourserverisdown.com yur-p.ru
yunque.pluto.ro
yuweng.info yuweng.info
za-music.mymobiplanet.com zaapplesales.blogspot.com
zarabiaj-dzis.pl zarepta.com
zarenica.net zastroyka.org
zeleznobeton.ru zawyna.ua
zeroredirect1.com zhorapankratov7.blogspot.com
zeroredirect9.com znakom.sibtest.ru
znaturaloriginal.com znaturaloriginal.com
zoogdinsney.com zoolubimets.ru
zoogdisany.com zskdla.site
zrelaya.pw zx6.ru

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1001watch.com.ua 007angels.com
100dollars-seo.com 1-free-share-buttons.com
101raccoon.ru 108shot.com
5ws.dating-app.ru 123locker.com
76brighton.co.uk 1bet.com
aasoldes.fr 2itech.ru
acortarurl.es 3rbseyes.com
adktrailmap.com aac500b7a15b2646968f6bd8c6305869d7.com
adman.se aad1f4acb0a373420d9b0c4202d38d94fa.com
adtech.de admitad.com
adultactioncam.com adprotect.net
advertisingtag.net adtech.fr
adzpower.com advancedmassagebysara.com
ahrefs.com advokateg.ru
air-edem.ru anal-acrobats.hol.es
aksonural.ru angry-fermi-7633.arukascloud.io
amazingninja.com anlimebel.kiev.ua
apxeo.info apiitsthirteende-a.akamaihd.net
arcadeplayhouse.com artdeko.info
arenda-yeisk.ru asscenihotosidea.blogspot.com
autoplate.info ausergrubhof.info
availit.weebly.com avtointeres.ru
aviav.co azarttoto.com
baditri.com babs.com.ua
bagsonsale.online backgroundpictures.net
basisches-wasser.net backlinkwatch.com
beauty-bracelet.com baladur.ru
beetpics.pw beauty-b0x.pl
bestssaker.com beremenyashka.com
bezlimitko.xyz bestcalovebracelet.cn
bingoporno.com blog.f00kclan.de
bio-japan.net blogporn.in
biznesschat.net braip.com.br
bonanza-fortune.men breget74.ru
brandimensions.com brus.city
buytizanidineonline.blogspot.com buyessayonline19.blogspot.ru
carapics.com callawaygolfstore.online
cartierlovebracelet.xyz camsex.xxx-cam.webcam
certifywebsite.win candlewooddentalcentre.com.au
cheapcigarettesc.info cartierreplica.xyz
citysecurity.nu checkpagerank.net
chinese-amezon.com
ci.ua
clothesforcash.com
coldfilm.ru
com-onlinesupport.host
compliance-fred.top
compliance-jess.top
compliance-john.top
compliance-julia.top
conocer-sanabria.com conocer-sanabria.com
cosmetic.donna7753191.ru consorzioilmosaico.org
converse.ddsoldes.fr
counter.bloke.com counter.bloke.com
cryptoswap.biz counter.yadro.ru
culturevie.info crafthubs.com
custom-electric-guitar.com csgo4.win
customsua.com.ua czat.wp.pl
danschawbel.com daisye.top
darmebel.com.ua davebestdeals.com
data.vtc.pw deanmoore.ie
dildofotzen.net decorationspcs.com
domznaniy.ru delitime.info
dood.live dnepropetrovsk.mistr-x.org
doyouknowtheword-flummox.ml documentsite.net
dvd-famille.com earncash.com.ua
dyt.net ecookna.com.ua
e-collantes.com eropornosex.ru
easyukraine.com estelight.ru
efnor-ac.com europeanwatches.ru
embedle.com exhibitionplus.eu
erectile.bid expdom.com
erotische-geschichten-xxl.com extrabot.com
esfchat.tk extstat.com
esnm.ru f-online.de
evrotekhservis.ru f5mtrack.com
extremez.net falllow.gq
farm26.ru farm26.ru
favornews.com feeriaclub.ru
fba-mexico.com fefo.gdn
fermersovet.ru forexunion.net
fertilitetsradgivningen.se funtoonez.com
fikasound.tk gamebackyard.com
filmania-x.ru gamezblox.com
find1friend.com gemara.com
flashbannernow.com google-liar.ru
flprog.com googlefeud.com
foodcrafts.website gotorussia.com
footballfarrago.com gov.yanao.ru
forminecrafters.ru gpirate.com
freelifetimefuckbook.com greatdealshop.com
fxgallery.com gta-top.ru
g33.org hentaiheroes.com
galaxyflowers.ru hollywoodactress.info
gambarkatabaru.com hop.clickbank.net
gambarxkata.co horoshieokna.com
gamewrath.com hvd-store.com
gembird.com hyiphunter.org
goldenggames.com images-free.net
gowreckdiving.com imagez.co
graphicwe.org incanto.in.ua
gsmlab.pl intim-uslugi.info
gz2.bbsoldes.fr isistaylorporn.info
handsandlegs.ru istanbulit.com
hannasolution.ru istizanidineanarcoticdrug.blogspot.com
helpmymacfaster.trade itsdp3.com
home-task.com jerseychinabizwholesale.us
homedo.fabpage.com jolic2.com
housediz.com jwss.cc
hut1.ru kamalsinha.com
ilovevitaly.com karusel-market.ru
infodocsportal.com katjimej.blog.fc2.com
iqoption-bin.com kazrent.com
juliaworld.net khadastoafarde.tk
justbcause.com kiinomaniak.pl
kakablog.net kinohall.ru
karadene.com kiskinhouse.com
keki.info kliksaya.com
kino-filmi.com klin3952.ru
kostenloser-sex.com kndxbkdx.bloger.index.hr
krasivye-devushki.net kongoultry.net
layola.biz.tc koversite.info
ledpolice.ru kustanay.kz
lenvred.org kw21.org
letslowbefast.site landscaping.center
lincolntheatre.com liumimgx.pw
localmatchbook.com
locimge.pw locimge.pw
loginduepunti.it ltvperf.com
love-baby.cz max-eclat.men
lowephotos.info maxxtor.eu
m4ever.net meccadumps.net
mackeeper-center.club meget.co.za
magazintiande.ru miniads.ca
maju.bluesalt.co mnogabukaff.net
mall.uk mrmoneymustache.com
manimpotence.com mtmtv.info
marinetraffic.com my-big-family.com
mebelcomplekt.ru mydirtyhobby.com
mediafresh.online narkologiya-orel.ru
meltwater.com negociosdasha.com
mhtr.be nlfjjunb5.ru
migente.com officedocuments.net
modabutik.ru onlinedomains.ru
moxo.com onlinefilmz.net
mp3s.club oops-cinema.ru
mundoaberrante.com ordernorxx.com
nac-bearings.ru outpersonals.com
naperehresti.info owathemes.com
nfljerseysforsalewholesaler.com paceform.com
nmrk.ru panamaforbeginners.com
nudejapan.net papasdelivery.ru
onefilms.net parajumpersoutlet.online
online-templatestore.com pathwhelp.org
onlywoman.org petrovka-online.com
ooo-olni.ru photokitchendesign.com
orion-code-access.net picurams.pw
oyster-green.com pio.polytopesexempt.com
ozas.net poffet.net
parlament.biz pokemon-go-play.online
partner-stop.men popugaychiki.com
pekori.to pornobrazzers.biz
pensplan.com
photo.houseofgaga.ru
picswe.com
pinkduck.ga
pipki.r.acdnpro.com
plaff-go.ru
plastweb.ru
popander.mobi
popunder.ru
poquoson.org
pornofiljmi.com pornofiljmi.com
proftests.net presleycollectibles.com
prom23.ru pro-tec.kz
propranolol40mg.blogspot.com produkto.net
prostitutki-almata.org promodj.com
prostitutki-kiev.org push-ad.com
purplesphere.in pwwysydh.com
pussysaga.com quit-smoking.ga
quick-offer.com
raavidesigns.com
rank-checker.online rank-checker.online
ranksignals.com rasteniya-vs-zombi.ru
rapevideosmovies.com reimageplus.com
razamicroelectronics.com reyel1985.webnode.fr
refudiatethissarah.info rezeptiblud.ru
rfd-split.hr riralmolamsaca.tk
rocis.site rvzr-a.akamaihd.net
rusprostitute.com saneyes.com
russian-postindex.ru satoristudio.net
saltspray.ru scanner-julia.top
salutmontreal.com scenicmissouri.us
samara.rosfirm.ru scripted.com
sanatorrii.ru seasonvar.ru
sarafangel.ru securityallianceservices.com
scoopquest.com seogadget.ru
sdelai-prosto.ru sexreliz.com
searchimage.co sexspornotub.com
secret.xn--oogle-wmc.com sexvideo-sex.com
seeingmeerkat.com silktide.com
servingnotice.com sindragosa.comxa.com
sexs-foto.top site3.free-share-buttons.com
sextracker.be smartpet.ru
skladvaz.ru solitaire-game.ru
smallseotools.com sovetogorod.ru
snomer1.ru
socialbutton.xyz
softwaretrend.net
spywarespy.com spywarespy.com
stats-collector.org sudexpert66.ru
stroydetali.ru sugarlyflex.pw
subj.ukr-lit.com sunflowerdrawingpaintings.blogspot.com
sugvant.ru superiends.org
supermodni.com.ua talant-factory.ru
syvertsen-da.no tecspb.ru
tds-advert005.info teencastingporn.com
technika-remont.ru terrafootwear.us
theallgirlarcade.com threecolumnblogger.com
tizanidineinteractions.blogspot.com tizanidineandhydrocodone.blogspot.com
toondinsey.com tizanidineformigraines.blogspot.com
tsan.net tizanidinevssoma.blogspot.com
tx41tclega.ru topkarkas.com
ublaze.ru torrent-to-magnet.com
ukrobstep.com torrentred.games
um-razum.ru traffic2cash.org
universals.com.ua trafficreceiver.club
us-america.ru traffmonster.info
ustion.ru ttsq.fr
vacances-voyages.info txxx.com
vepad.com tyumen.xrus.org
versaut.xxx-cam.webcam uggbootsoutletsale.us
vozbujdenie.com umekana.ru
vsdshnik.com uptime.com
vuclip.com uroffer.link
vut.com.ru vabasa.inwtrade.com
vzlomtw.com vandrie-ict.nl
wahicbefa31.soup.io vedomstvo.net
wallpaperaccess.com viagengrarx.com
wallpapers-best.com videositename.com
weblibrary.win vids18.site
wetgames.ru vikistars.com
whos.amung.us vipcallsgirls.com
winwotgold.pl vkgaleria.com
wordkeyhelper.com vksaver-all.ru
worldtraveler.world voprosotvet24.ru
x-lime.com vostoktrade.info
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf wardreapptokone.tk
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai website-analytics.online
xn--80ahdheogk5l.xn--p1ai witherrom55.eklablog.fr
xn--90acenikpebbdd4f6d.xn--p1ai wrz0iuebwhp5fg.freeddns.com
xnxx699.com wtsindia.in
youbrainboost.asia xn--24-glceagatoq7c2a6ioc.xn--p1ai
youdao.com xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
youradulthosting.com xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
yoursite.com xn--80ajbshivpvn2i.xn--p1ai
youtoner.it yatrk.xyz
yurgorod.ru yoopsie.com
zarajbuilders.com yoshkarola.zrus.org
zazagames.org youbloodyripper.com
zeroredirect11.com youresponsive.com
za-music.mymobiplanet.com
zakazvzloma.com
zootoplist.com
zvezdagedon.ru

83
dev-tools/test_units/ratelimittest-master.txt
View file

@ -8,44 +8,6 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -494,6 +456,41 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
@ -584,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.0.20 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.02.3577 ### Version: V4.2023.02.3578
### Updated: Sun Feb 12 22:01:12 UTC 2023 ### Updated: Mon Feb 13 10:03:34 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.02.3577 ### Version: V4.2023.02.3578
### Updated: Sun Feb 12 22:01:12 UTC 2023 ### Updated: Mon Feb 13 10:03:34 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##