vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-11 15:54:46 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.08.3264 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-08-01 22:07:33 +00:00
parent 9917d204b4
commit ecb6966d1c
47 changed files with 99872 additions and 99882 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.08.3263 #### Version: V4.2022.08.3264
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.08.3263 #### Version: V4.2022.08.3264
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.08.3263 #### Version: V4.2022.08.3264
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12346
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.08.3263 #### Version: V4.2022.08.3264
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12350
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12350
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12350
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12350
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 1; 10.1.0.239 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 0; 10.1.0.239 0;
127.0.0.1 0; 127.0.0.1 0;

12350
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 1; 10.1.0.239 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 0; 10.1.0.239 0;
127.0.0.1 0; 127.0.0.1 0;

12350
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 1; 10.1.0.239 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 0; 10.1.0.239 0;
127.0.0.1 0; 127.0.0.1 0;

12350
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3263 3264

12350
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12350
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.127.69.63 0; 23.102.108.35 0;

12350
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 1; 10.1.0.239 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 0; 10.1.0.239 0;
127.0.0.1 0; 127.0.0.1 0;

12350
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 1; 10.1.0.239 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 0; 10.1.0.239 0;
127.0.0.1 0; 127.0.0.1 0;

12350
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 1; 10.1.0.239 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 0; 10.1.0.239 0;
127.0.0.1 0; 127.0.0.1 0;

12350
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.127.69.63 0; 23.102.108.35 0;

12350
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.127.69.63 0; 23.102.108.35 0;

12350
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 1; 10.1.0.239 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Crescent AwarioSmartBot
Custo EMail Wolf
Indy Library Ecxi
NetLyzer Firefox/7.0
Page Analyzer Havij
Papa Foto Image Sucker
Pu_iN LNSpiderguy
Swiftbot Lanshanbot
True_Robot Web Sauger
Wallpapers WebWhacker

330
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404checker 404checker
ADmantX 404enemy
80legs
ALittle Client ALittle Client
Abonti
Aboundex Aboundex
AhrefsBot Aboundexbot
Acunetix
AiHitBot AiHitBot
Aipbot
Alexibot Alexibot
AllSubmitter AllSubmitter
Anarchie AlphaBot
Anarchy
Anarchy99 Anarchy99
Anthill Ankit
Apexoo Apexoo
Atomseobot Aspiegel
AwarioRssBot Asterias
AwarioSmartBot AwarioSmartBot
BDCbot BDCbot
BDFetch
BLEXBot
BackWeb
Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass
Bandit Bandit
BatchFTP BatchFTP
Battleztar Bazinga
BetaBot BetaBot
BotALot Bigfoot
Buddy Black Hole
BuiltBotTough BlackWidow
BunnySlippers Blackboard
BuzzSumo BlowFish
Buck
BuiltWith
CCBot CCBot
CODE87
CSHttp CSHttp
CensysInspect Calculon
CherryPicker
ChinaClaw ChinaClaw
Claritybot Claritybot
Cloud mapping Cloud mapping
Cocolyzebot
Copier Copier
CopyRightCheck Copyscape
Craftbot Crawling at Home Project
CrazyWebCrawler Crescent
CrunchBot Curious
CyotekWebCopy DBLBot
Deusu DatabaseDriverMysqli
Digincore Demon
Devil
DigitalPebble DigitalPebble
Dirbuster Discoverybot
Disco DnyzBot
Discobot DomainCrawler
DittoSpyder
DnBCrawler-Analytics
DomCopBot
DomainAppender
DomainStatsBot
Download Wonder
Dragonfly
Drip
EMail Siphon
EMail Wolf EMail Wolf
Exabot Ecxi
ExtractorPro EirGrabber
Extreme Picture Finder Extreme Picture Finder
FHscan Ezooms
FemtosearchBot FDM
Firefox/7.0 Flunky
FrontPage Fyrebot
FyberSpider G-i-g-a-b-o-t
GalaxyBot
Genieo Genieo
Getintent GetRight
Go!Zilla GetWeb
GoZilla
Gotit Gotit
GrabNet
Grabber Grabber
Grafula
HEADMasterSEO HEADMasterSEO
HMView HMView
HTTP::Lite HTMLparser
Haansoft Hloader
Harvest
IDBTE4M IDBTE4M
IRLbot IDBot
IlseBot Id-search
Image Fetch Image Fetch
Image Sucker
Indy Library Indy Library
InfoTekies
Intelliseek Intelliseek
IstellaBot InterGET
Internet Ninja
InternetSeer
Iria
Iskanie
JennyBot JennyBot
JetCar Jetty
JikeSpider Jyxobot
Kinza Keyword Density
LNSpiderguy Lanshanbot
LeechGet
LexiBot
Lftp
Libwhisker Libwhisker
Linkbot LieBaoFast
LinkWalker
LinkextractorPro LinkextractorPro
LinkpadBot
LinksManager
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MSFrontPage
MJ12bot MSIECrawler
MTRobot
Majestic SEO
Majestic-SEO
Majestic12 Majestic12
MarkMonitor Mass Downloader
MarkWatch
Masscan Masscan
Mata Hari
MauiBot MauiBot
Mb2345Browser MeanPath Bot
Meanpathbot
MicroMessenger MicroMessenger
Microsoft URL Control
Mister PiX Mister PiX
Morfeus Fucking Scanner Mojolicious
Mozlila MolokaiBot
Msrabot
Musobot Musobot
NPbot
Name Intelligence Name Intelligence
Nameprotect Nameprotect
NearSite Navroad
Net Vampire Needle
NetLyzer NetAnts
NetMechanic Netvibes
NetZIP NimbleCrawler
Netcraft
Nettrack
NextGenSearchBot
Nimbostratus Nimbostratus
Ninja
Nuclei Nuclei
Octopus Nutch
Offline Explorer
OpenLinkProfiler OpenLinkProfiler
OpenVAS OpenVAS
Openfind
Openvas Openvas
OrangeBot
OrangeSpider OrangeSpider
OutfoxBot OutclicksBot
PHPCrawl PECL::HTTP
PageThing.com PageAnalyzer
Pandalytics PageGrabber
Panscient
Papa Foto Papa Foto
PeoplePal Pavuk
Picscout Picscout
Picsearch
Piepmatz
Pimonster
Pixray Pixray
Pockey
ProPowerBot ProPowerBot
Probethenet
Psbot
Pump
PyCurl
Quick-Crawler Quick-Crawler
RSSingBot
RankActive
RankActiveLinkBot RankActiveLinkBot
RankFlex RankFlex
RankingBot
Rankivabot Rankivabot
RankurBot RealDownload
Re-re
Recorder Recorder
RedesScrapy RedesScrapy
Rogerbot Ripper
SEOkicks Scanbot
SEOlyticsCrawler ScoutJet
ScrepyBot ScreenerBot
Searchestate SemanticJuice
SemrushBot SentiBot
SeoSiteCheckup Siphon
SeobilityBot Site Sucker
Seomoz SiteExplorer
Shodan SiteLockSpider
Sitebeam Siteimprove
Sitevigil SmartDownload
Snapbot Snake
SocialRankIOBot Snoopy
Sosospider Sosospider
Spammen Sottopop
SpankBot Spanner
Spinn3r SputnikBot
Sqlmap
Sqworm Sqworm
Sucuri Steeler
SuperHTTP SuperBot
Surfbot SurveyBot
Szukacz Suzuran
T0PHackTeam T8Abot
Telesoft Telesoft
Telesphoreo Telesphorep
The Intraformant The Intraformant
Toata TheNomad
Trendiction TightTwatBot
True_Robot Tracemyfile
Turnitin Twice
TurnitinBot UnisterBot
TwengaBot Vagabondo
Typhoeus WWWOFFLE
URLy Warning Wallpapers/3.0
VeriCiteCrawler
VidibleScraper
Voil
Voltron
WASALive-Bot
WBSearchBot
WPScan
WWW-Collector-E
Wallpapers
Web Auto Web Auto
Web Collage Web Collage
Web Fetch Web Fuck
Web Pix Web Pix
Web Sauger Web Sauger
WebAuto WebCollage
WebLeacher WebFetch
WebSauger WebSucker
WebStripper WebWhacker
WebZIP Webshag
Webalta
WebmasterWorldForumBot
Website Quester Website Quester
Webster WebsiteExtractor
Whack Whacker
Whatweb Whatweb
WinHTTrack Widow
Wonderbot
Woobot Woobot
Wotbox Wotbox
Xaldon WebSpider Wprecon
Zeus YoudaoBot
Zitebot Zade
Zauba
Zermelo
ZmEu ZmEu
ZoominfoBot ZoomBot
adscanner
archive.org_bot archive.org_bot
eCatch arquivo-web-crawler
arquivo.pt
autoemailspider
check1.exe
coccocbot
com.plumanalytics
domainsproject.org
heritrix heritrix
internetVista monitor internetVista monitor
ips-agent ips-agent
isitwp.com lwp-request
iubenda-radar lwp-trivial
mediawords magpie-crawler
muhstik-scan oBot
pcBrowser polaris version
plumanalytics probe-image-size
satoristudio.net s1z.ru
scan.lol scalaj-http
seobility seobility
seocompany.store seoscanners
serpstatbot serpstatbot
spyfu siteripz
sysscan
tAkeOut tAkeOut
trendiction.com trendiction.com
ubermetrics-technologies.com ubermetrics-technologies.com
x22Mozilla voyagerx.com
xpymep1.exe webprosbot
zgrab

298
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404checker 360Spider
404enemy 80legs
ADmantX ALittle Client
ASPSeek ASPSeek
Acunetix Aboundexbot
AiHitBot AiHitBot
AllSubmitter
Alligator
AlphaBot
Anarchy Anarchy
Anarchy99 Anthill
Attach Attach
AwarioSmartBot AwarioSmartBot
BBBike BBBike
BackStreet BackStreet
BacklinkCrawler BetaBot
Badass Bigfoot
Bandit
BatchFTP
Battleztar Bazinga
Bitacle Bitacle
Black Hole Black Hole
BlackWidow
BlowFish
Boardreader Boardreader
Bolt Bolt
BotALot BotALot
Brandwatch Buck
Buddy
BuiltBotTough
BuiltWith BuiltWith
Bullseye CODE87
BuzzSumo CSHttp
Calculon CensysInspect
CazoodleBot CheTeam
CheeseBot
ChinaClaw ChinaClaw
Citoid Citoid
Claritybot Claritybot
Cloud mapping
Cocolyzebot Cocolyzebot
Copier
CopyRightCheck
Craftbot
Crawling at Home Project
CrazyWebCrawler
Crescent
Curious Curious
Custo Custo
DIIbot DSearch
DTS Agent DTS Agent
DataCha0s
DatabaseDriverMysqli
Deusu
Devil Devil
DnBCrawler-Analytics Dirbuster
Disco
Dispatch
DnyzBot DnyzBot
DomainCrawler
DomainSigmaCrawler
Domains Project Domains Project
Dotbot Dotbot
EMail Siphon EMail Siphon
EasyDL
Ebingbong Ebingbong
EroCrawler EroCrawler
Exabot
ExtLinksBot
Extractor Extractor
Extreme Picture Finder
EyeNetIE EyeNetIE
FDM Ezooms
FHscan
FemtosearchBot
Fimap
Firefox/7.0 Firefox/7.0
FlashGet
Flunky Flunky
Freeuploader
Fuzz Fuzz
Fyrebot GT::WWW
Go-Ahead-Got-It GalaxyBot
Gotit Genieo
Grabber GetRight
GetWeb
Getintent
Gigabot
Go!Zilla
GoZilla
Grafula Grafula
Haansoft GrapeFX
HMView
HTMLparser
HaosouSpider HaosouSpider
Heritrix Heritrix
Hloader
HonoluluBot HonoluluBot
Humanlinks Humanlinks
HybridBot HybridBot
IRLbot IRLbot
Iblog Iblog
Id-search
IlseBot IlseBot
Image Sucker
IndeedBot
InfoNaviRobot InfoNaviRobot
InfoTekies
InterGET InterGET
Internet Ninja InternetSeer
Iria
Iskanie
IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz JetCar
JennyBot
Jetty Jetty
Joomla Keyword Density
Kenjin Spider Kinza
Keybot Translation-Search-Machine
Kozmosbot
LNSpiderguy LNSpiderguy
Larbin LWP::Simple
LeechFTP Leap
LeechGet
LexiBot LexiBot
Lftp Lftp
LinkWalker LibWeb
Lightspeedsystems
Linkbot
LinkextractorPro LinkextractorPro
LinksManager Lmspider
LinqiaMetadataDownloaderBot MIDown tool
LinqiaRSSBot MTRobot
Litemage_walker
MJ12bot
MQQBrowser
MSFrontPage
Mail.RU_Bot Mail.RU_Bot
Majestic-SEO Majestic-SEO
Majestic12
MarkMonitor MarkMonitor
MarkWatch
Mass Downloader
MeanPath Bot MeanPath Bot
MicroMessenger Mediatoolkitbot
Microsoft URL Control Mr.4x3
Moblie Safari Musobot
Morfeus Fucking Scanner Netcraft
NICErsPRO
Name Intelligence
Navroad
Nessus
Net Vampire
NetAnts
NetZIP
Nettrack Nettrack
Netvibes
NextGenSearchBot
Niki-bot Niki-bot
NimbleCrawler NimbleCrawler
Nimbostratus Nmap
Not Not
Nuclei Nutch
Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler Openfind
OpenVAS
Openvas
OrangeBot
OutfoxBot OutfoxBot
PHPCrawl PHPCrawl
PageScorer PageGrabber
Pandalytics
Panscient Panscient
Pavuk
Petalbot Petalbot
Pi-Monster
Picscout Picscout
Picsearch Picsearch
PictureFinder
Piepmatz
Pimonster
Pixray Pixray
ProWebWalker ProPowerBot
Probethenet
Psbot Psbot
Pump Pump
PxBroker QueryN Metasearch
Quick-Crawler
RSSingBot RSSingBot
RankingBot RankActiveLinkBot
RealDownload RankingBot2
Recorder Rankivabot
Ripper RebelMouse
SEOlyticsCrawler SEOkicks
SEOkicks-Robot
SEOprofiler SEOprofiler
SEOstats Scanbot
SMTBot ScoutJet
SalesIntelligent Screaming
ScreenerBot
Searchestate Searchestate
SearchmetricsBot SearchmetricsBot
SemanticJuice Seekport
Semrush
SentiBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot
Shodan
Siphon
Site Sucker
SiteSnagger
SiteSucker
Sitebeam Sitebeam
SlySearch Sitevigil
SmartDownload
Snake
SocialRankIOBot SocialRankIOBot
Sogou web spider Sogou web spider
Sosospider
Sottopop Sottopop
Spammen
Spanner Spanner
Sqlmap
Sqlworm Sqlworm
Sqworm
Steeler Steeler
Stripper
Sucker Sucker
Sucuri Sucuri
Surfbot Telesphorep
Szukacz Titan
T0PHackTeam Toata
T8Abot Toweyabot
Telesphoreo Trendiction
Thumbor
TightTwatBot
True_Robot True_Robot
Turingos
TurnitinBot
Twice
URLy Warning URLy Warning
VelenPublicWebCrawler UnisterBot
VB Project
Vagabondo
VeriCiteCrawler VeriCiteCrawler
VidibleScraper VidibleScraper
Voil VoidEYE
WASALive-Bot
WBSearchBot WBSearchBot
WISENutbot WEBDAV
WPScan Wallpapers
WWW-Collector-E WeSEE
WWWOFFLE
WallpapersHD
Web Collage Web Collage
Web Enhancer WebAuto
Web Fetch WebBandit
Web Pix WebCollage
Web Sucker WebCopier
WebImageCollector
WebLeacher WebLeacher
WebPix WebPix
WebSauger WebStripper
Webalta WebWhacker
WebsiteExtractor WebmasterWorldForumBot
WebsiteQuester Webshag
Webster Wotbox
Whacker
WinHTTrack
WiseGuys Robot
Wonderbot
Wprecon Wprecon
ZmEu Xaldon WebSpider
Xaldon_WebSpider
Zade
Zauba
Zeus
ZyBorg
adscanner adscanner
archive.org_bot cah.io.community
arquivo.pt check1.exe
clark-crawler
cognitiveseo cognitiveseo
com.plumanalytics crawl.sogou.com
crawler.feedback crawler4j
dataforseo.com
domainsproject.org
eCatch eCatch
gopher evc-batch
instabid
iubenda-radar iubenda-radar
linkdexbot linkdexbot
lwp-trivial lwp-request
meanpathbot mediawords
netEstate NE Crawler
oBot oBot
pcBrowser plumanalytics
polaris version
probe-image-size
ripz ripz
s1z.ru scan.lol
satoristudio.net seocompany.store
scalaj-http sitechecker.pro
seoscanners sogouspider
sp_auditbot
sysscan
ubermetrics-technologies.com ubermetrics-technologies.com
webgains-bot
webpros.com webpros.com
zauba.io webprosbot
x22Mozilla
xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
1kinobig.ru apiholdingmypage-a.akamaihd.net
anaseracresar.tk back.dog
forum.poker4life.ru chatroulette.online
graphicwe.org drugs-no-rx.info
hdwallpapers-free.com healgastro.com
topquality.cf hotxnights.info
truebeauty.cc next-dentists.tk
ukrtextbook.com photorepair.ru
virus-respirators.com pornophoto.xyz
wechatdownload10.com zero1.it

934
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com 03p.info
0daymusic.org 101flag.ru
108shot.com 12bet.com
123locker.com 1hwy.com
4webmasters.com 1xbet4you.com
5forex.ru 2020iscoming.info
883zy.com 2345.com
8gold.com 2daytrendingnews.com
abclauncher.com 3w24.com
above.com 4pp13.com
activecampaign.dreamhosters.com 4ureyesonly.com
7yue.org
98oi.ru
aa8b68101d388c446389283820863176e7.com
aac500b7a15b2646968f6bd8c6305869d7.com
absolute-s.ru
addray.pro addray.pro
adpremium.org adidas.frwebs.fr
adrenalin-stk.ru admitad.com
adsland.men adultfullhd.com
adsloads.com adventureparkcostarica.com
adsref.men agapovdg.ru
adultnet.in ahrefs.com
advokat-grodno.by
aero2.ru
aerogo.com
afmuseum.com
air-edem.ru
ald2014.org
alert-hgd.xyz
alfa-img.com alfa-img.com
allbizne.co.ua allnews.md
allstatesugarbowl.org allnews24.in
amazingpic.net alpharma.net
an-donut.com amanda-porn.ga
animali.deagostinipassion.it amazon-adsystem.com
anonymizeme.pro android4fun.org
ap.senai.br androids-store.com
apibetweenlinesn-a.akamaihd.net angigreene.com
apimountainbikei-a.akamaihd.net apibatbrowsecom-a.akamaihd.net
apispringsmartne-a.akamaihd.net apiwebwebgetcom-a.akamaihd.net
app5.letmacworkfaster.world
applicationg29.com
apxeo.info
arcadepages.com
arcteryxsale.online
arendakvartir.kz arendakvartir.kz
aruplighting.com atmovs.com
asacopaco.tk atout-energie-69.com
availit.weebly.com australia-opening-times.com
aviav.ru autoplate.info
avitocars.ru autorn.ru
avon-severozapad.ru autowebmarket.com.ua
avtolombard-krasnodar.ru avon-ukraine.com
badavit.com.ua azadnegar.com
banan.tv azazu.ru
bangla-chat-uk-paralud.ga b-styles.xyz
backuperwebcam.weebly.com
baksman.com
bala.getenjoyment.net
bankcrediti.ru bankcrediti.ru
banki76.ru barbourjackets.online
barnfurnituremart.com battle.net
beachdriveblog.com bayareaaeroclub.org
beachtoday.ru becuo.com
beetpics.pw bestcalovebracelet.cn
begalka.xyz bet-prognoz.com
bensbargains.net
beremenyashka.com
bestdooz.com
bestwaystogetpaid.us
bezlimitko.xyz bezlimitko.xyz
bg6s0.com bigames.online
bingo8888.com bookhome.info
bizcheapjerseyswholesalechina.com bookmark4you.com.biz
blagovest-med.ru browsepulse-a.akamaihd.net
blavia.00author.com bscodecs.com
bmw.afora.ru buntube.net
bnt-team.com bus-offer.com
bonky.biz buyessayonline19.blogspot.ru
boostmyppc.com buyviagraa.blogspot.com
brainboosting.club buzzonclick.com
braintobe.win
brg8.com
business-suggested.tk
buyantiviralwp.com
buyessay3.blogspot.ru
buypanicdisorderpill.com
buzzzg.men buzzzg.men
bycontext.com c1.onedmp.com
callmd5map.com camdenmemorials.com
canzoni.ru canrole.com
carmuffler.net carsplate.com
carrauterie.be charmstroy.info
cars-modification.net cheapbarbour.online
cb.iphantom.com children-learningreading.info
cfcl.co.uk cleanpcnow.com
chatroulette.life cleansearch.net
check-host.net clickhype.com
checkhit.com clickzzs.nl
checkm8.com com-onlinesupport.host
checkpagerank.net com-secure.download
chee-by.biz cosmeticswomens-womensports.rhcloud.com
chidporn.com creditnation.ru
chocolateslim-en-espana.com customcatchcan.com
christianlouboutinsaleoutletonline.us dailyrank.net
citetick.com
classiquebijoux.ru
clicksor.net
coin-hive.com
coinsspb.com
comissionka.net
commoncrawl.org
compliance-andrew.xyz
compliance-george.top
compliance-john.top
compliance-mary.top
comsysnet.com
cookie-law-enforcement-hh.xyz
cookingmeat.ru
coolwallpapers4k.info
copyrightclaims.org
cougfan.info
counciltally.com
cxpromote.com
dandiyabeats.in
datsun-do.com datsun-do.com
davebestdeals.com delta-line.men
ddospanel.com deshevo-nedorogo.ru
dear-diary.ru diffbot.com
detective01.ru digest-project.ru
detmebel.su digital-sale.su
deutschehobbyhuren.net
diarioaconcagua.com
dikqlyremy.info dikqlyremy.info
dikx.gdn dikx.gdn
discountliv.com discountliv.com
documentbase.net dktr.ru
dojki365.online dms-sw.ru
domaincheck.io dnepr-avtospar.com.ua
dominateforex.ml doctormakarova.ru
dostavimvdom.ru dollartree.info
dotmass.top domination.ml
e-buyeasy.com donna7753191.ru
editmedios.com dorratex.tn
ednorxmed.com drillsaw.ru
educatemetv.com e-commerce-seo.com
education-cz.ru earthmagic.info
election.interferencer.ru ege-russian.ru
eleimgo.pw el-nation.com
emerson-rus.ru elektrischezi.canalblog.com
elektrischezigaretten1.blogse.nl
elektrozigaretten1.tumblr.com
elmacho.xyz
empis.magix.net empis.magix.net
enbersoft.com englate.com
eng-lyrics.com enthuse.computernetworksonline.com
eorogo.top eonpal.com
erotische-geschichten-xxl.com eropornosex.ru
es5.com erot.co
esnm.ru erotik0049.com
essay-edu.biz errorfixing.space
essayassist.com ertelecom.ru
eu-cookie-law.info essay-writing.work
evidencecleanergold.com etur.ru
evrotekhservis.ru eupornstar.info
excaliburfilms.com exhibitionplus.eu
excitacion.info exportshop.us
expertblog.info extlabs.io
eyessurgery.ru eyessurgery.ru
ezb.elvenmachine.com failingmarriege.blogspot.com
ezigarettenshop2.postbit.com falcon-images.blogspot.com
ezigaretteshop2.mywapblog.com fashionindeed.ml
facialporntube.com favorcosmetics.com
familyphysician.ru fettefrauen.net
famix.xyz ff30236ddef1465f88547e760973d70a.com
favoritemoney.ru fikasound.tk
fba-mexico.com film-one.ru
felizporno.com financeloan.us
fellowshipoftheminds.com findanysex.com
ferieboligkbh.dk
fertilitetsradgivningen.se
financetip.eu
firesub.pl
flauntyoursite.com flauntyoursite.com
forminecrafters.ru fm-upgrade.ru
forpackningsutveckling.se forpostlock.ru
free-deals.faith forum69.info
freewlan.info frameimage.org
fullfileaccess.com free-today.com
fun-mobi.pl free-traffic.xyz
fun2cell.net freecamdollars.com
funnymama.com freelinkbuilding.website.tk
gambarkataku.co freewhatsappload.com
game-top.su g7m.pl
genericlowlatencyasiodriverhq.aircus.com gambarkatabaru.com
genetworx.com gamedayhouse.com
getpopunder.com gameonasia.com
gettpromos.com gay-site.store
gheus.altervista.org gayxperience.com
given2.com ge0ip.net
golden-catalog.pro genericviagrasildenafiled.net
goldenggames.com geoads.com
gifspics.com
gigapeta.com
gk170.ru
gkvector.ru
globatur.ru
goodly.pro
goodwriterssales.com
gopixdatabase.com gopixdatabase.com
gpms.org.my gorodservis.ru
greenshop.su gowreckdiving.com
guiadeserraazul.com gox.com.ua
hackers-crackers.tk grand-chlen.ru
hacktougroup.ru graphicwe.org
hanink.biz.ly gribokstop.com
harmonyglen.us grupografico-pilar.com.ar
hauleddes.com gsasearchengineranker.pw
hausfrauensex18.com gwagka.com
hdhc.ru hazardky.net
heliko.no headpress.ru
hentaiheroes.com healgastro.com
himazin.info hiwibyh.bugs3.com
hmywwogw.bloger.index.hr hkladys.com
hoholikik.club homik.pw
hoste.octopis.com
hostingclub.lk hostingclub.lk
hotchatdate.com
hotkeys.com
hotloans.ru
housemilan.ru
hoverboard360.nl hoverboard360.nl
howtowhitenteethfast.xyz i-midias.net.br
hplaserjetpdriver8y.pen.io
iboss.com
icaseclub.ru
iflycapetown.co.za iflycapetown.co.za
ilbe.club iideaidekonkatu.info
ilikevitaly.com ilovevitaly.ru
ilmen.net imagecoolpub.com
imgarcade.com images-graphics-pics.com
integritylandscapeservices.com impisr.ru
investingclub.ru in-tandem.co
istock-mebel.ru incest-ru.com
iweblist.info incolors.club
izismile.com investpamm.ru
jabimgo.pw ipchicken.com
james13prix.info irkutsk.zrus.org
jennyfire.ru itbc.kiev.ua
jrcigars.com ivanovo.zrus.org
justdating.online ivearchenceinflu.cf
kalandranis.gr ivoiretechnocom.ci
kambasoft.com izhevsk.zrus.org
kazan.xrus.org jackpotchances.com
kemerovo.zrus.org jav-idol.com
kinocccp.net jeremyeaton.co
jerseychinabizwholesale.com
jerseyschinabizwholesale.us
jo24news.com
jobmarket.com.ua
jofucipiku.tk
journalhome.com
justkillingti.me
justucalling32211123456789.tk
kaliningrad.zrus.org
kamagragelusa.net
karachev-city.ru
ketoanhanoi.info
keywordsuggest.org
kino-filmi.com
kinoduh.ru
kinosed.net kinosed.net
krasula.pp.ua kol-energo.ru
kredit-pod-zalog-krasnodar.ru kolotiloff.ru
kostenlos-sexvideos.com
krasnodar.ru
kreativperlen.ch
kretpicf.pw kretpicf.pw
lacapilla.info ks1234.com
lacave.ntic.fr kung-fu-ru.com
ladsblue.com kw21.org
laexotic.com lacasamorett.com
lancheck.net lafriore.ru
larchik.net lalalove.ru
laxdrills.com lampokrat.ws
le-clos-des-alouettes.com landmania.ru
likesdesign.com laptop-4-less.com
lineshops.biz larutti.ru
live-sexchat.ru layola.biz.tc
ljusihus.se linkbuddies.com
lombia.com lion.conboy.us
lomza.info lisque.batcave.net
lordzfilmz.me liumimgx.pw
lottospring.com lob.com.ru
luckyday.world lodki-pvh.dp.ua
m.b00kmarks.com lolnex.us
macdamaged.space lombia.co
macfix.life lostaruban.com
manimpotence.com ltvperf.com
marblestyle.ru m292.info
marketingtechniques.info macrotek.ru
maslenka.kz mapquestz.us
masturbate.co.uk marinetraffic.com
medanestesia.ru massage-info.nl
medictube.ru meget.co.za
medispainstitute.com.au meteostate.com
medkletki.ru minegam.com
meetingrainstorm.bid mitrasound.ru
mensandals.xyz mixx.com
mexicosleevegastrectomy.com mlf.hordo.win
michellblog.online mmofreegames.online
mikrobiologies.ru mobile-appster.ru
millionare.com monclercheap.xyz
miniads.ca monthlywinners.com
mirobuvi.com.ua morlat.altervista.org
misslike.ru mrbitsandbytes.com
mjchamonix.org mrmoneymustache.com
monarchfind-a.akamaihd.net music7s.me
money-every-day.com musicdaddy.net
moonci.ru muz-shoes.ru
motherboard.vice.com muz-tracker.net
moviezbonkerssk.cf mxgetcode.com
movpod.in
mrpornogratis.xxx
mundoaberrante.com
my-cash-bot.co
mybackgroundlandscape.blogspot.com mybackgroundlandscape.blogspot.com
nakozhe.com myblogregistercm.tk
nakrutka.cc myiptest.com
nalogovyykodeks.ru myseoconsultant.com
naughtyconnect.com na-telefon.biz
net-profits.xyz nbsproject.ru
nextlnk12.com negral.pluto.ro
nhl17coins.exblog.jp netvouz.com
nikitabuch.com newstaffadsshop.club
nmrk.ru nfljerseys.online
nootrino.com nina.az
nottyu.xyz novosibirsk.xrus.org
oakridgemo.com o-o-8-o-o.com
obesidadealgarve.com odesproperty.com
obiavo.su of-ireland.info
oecnhs.info offf.info
offers.bycontext.com ohmyrings.com
okonich.com.ua okonich.com.ua
omsk.xrus.org online-x.ru
onlinebay.ru ooo-gotovie.ru
onlinewritingjobs17.blogspot.ru oops-cinema.ru
ooomeru.ru ops.picscout.com
ordernorxx.com optitrade24.com
orgasmus-virtual.com oranga.host.sk
orion-code-access.net osoznanie-narkotikam.net
ostrovtaxi.ru ostroike.org
ourville.info
outpersonals.com
outrageousdeal-a.akamaihd.net
ownshop.win
oxotl.com oxotl.com
oyster-green.com partner-high.men
park.above.com partner-host.men
pc-services.ru pc-virus-d0l92j2.pw
peugeot-club.org penisvergrotendepillennl.ovh
phormchina.com perederni.net
photokitchendesign.com perm.xrus.org
php-market.ru picphotos.net
picsearch.com pillscheap24h.com
pictures-and-images.net pintattoos.com
pijoto.net piulatte.cz
platesauto.com pod-muzyku.club
platezhka.net poem-paying.gq
playlott.com polimga.pw
pliks.pl popcash.net
plohaya-kreditnaya-istoriya.ru popunder.net
pojdelo.weebly.com pornoelita.info
poquoson.org portside.cc
porno-video-chati.ru powitania.pl
pornoblood.com pr0fit-b0x.com
portnoff.od.ua pravoholding.ru
powenlite24.ru privacylocationforloc.com
preventheadacheguide.info professionalwritingservices15.blogspot.ru
prod2016.com profitwithalex.info
profolan.pl progonrumarket.ru
projectforte.ru prosmibank.ru
prospekt-st.ru purplesphere.in
puzo2arbuza.ru purplestats.com
quality-traffic.com puserving.com
push-ad.com
qor360.com
quickchange.cc quickchange.cc
quizzitch.net rankscanner.com
rainbowice.ru ranksonic.com
rapidgator-porn.ga razyboard.com
razvratnoe.org regdefense.com
redbottomheels.xyz repeatlogo.co.uk
refudiatethissarah.info
registratciya-v-moskve.ru
remmling.de
remote-dba.de
rewardpoll.com rewardpoll.com
rn-to-bsn.com rn-to-bsn.com
rockingclicks.com ruex.org.ua
runtnc.net runofilms.ru
rusenvironmental.net runstocks.com
rusoft-zone.ru ruspdd.com
rvi.biz russintv.fr
samolet.fr russkoe-zdorovie.ru
sape.top rutor.vip
satellite.maps.ilovevitaly.com s1z.ru
scanmarine.info sabaapress.com
scanner-alex.top sammyweaver.com
scanner-nelson.top sarm.tk
scanner-walter.top satoristudio.net
scanner-willy.top sbt-aqua.ru
scat.porn sbtdesign.co.uk
schlampen-treffen.com sbwealthsolutions.ca
scopich.com scanner-brian.top
scrnet.biz.ua scanner-fred.top
search.alot.com scmor.ilxc.cc
selectads.men scottbywater.com
serdcenebolit.com search.pch.com
sexobzor.info searchaddis.com
sexs-foto.com seofirmreviewsus.info
sextracker.de sethrollins.net
shymkent.xkaz.org sex-dating.co
sibdevice.ru sex-sex-sex5.com
sildenafilcitratemed.com sextracker.be
sim-service.net shakhtar-doneck.ru
sispe.com.br shareyards.com
shiksabd.com
shitting.pro
shopcheermakeup.info
shoppytoolmac.com
shopwme.ru
sibecoprom.ru
sibvitr.ru
sientalyric.co
sigmund-freud.co.uk
simul.co
site-speed-check.site site-speed-check.site
siteexpress.co.il siteripz.net
slavic-magic.ru sitevalued.com
slavkokacunko.de slonechka.ru
sluganarodu.ru smokewithrabbits.com
slujbauborki.ru snts.shell-pmr.ru
slutloadlive.com sockshares.tv
smailik.org sohoindia.net
small-games.biz spanking.to
smartbalanceworld.com spasswelt.net
snegozaderzhatel.ru spidtest.org
socas.pluto.ro starpages.net
sockshare.net stevemonsen.com
soda.media stop-zavisimost.com
soietvousmaime.fr strfls.com
solicita.info studiokamyk.com.pl
solmarket.by studworks.org
space-worry.ml success-seo.com
speechfoodie.com sugvant.ru
staff.prairiesouth.ca svetodiodoff.ru
statoutlook.info tamada69.com
steebook.com teastory.co
streetfooduncovered.com terraclicks.com
strongsignal-a.akamaihd.net the-torrent-tracker.blogspot.com
stroymonolit.su thebestphotos.eu
strv.se thepantonpractice.co.uk
stuffpride.com tizanidinecapsules.blogspot.com
styro.ru tizanidinehcl4mgtablet.blogspot.com
sukarame.net tizanidineinfo.blogspot.com
sukirgenk.dvrlists.com tizanidinemusclerelaxant.blogspot.com
szamponrevita.pl tizanidinerecreationaluse.blogspot.com
tabakur77.com tizanidinevsvalium.blogspot.com
teguh.info tkanorganizma.ru
telemetryverification.net tmearegion26.com
thedownloadfreeonlinegames.blogspot.com tnctrx.com
themestotal.com tomatis.gospartner.com
thiegs.reco.ws toproadrunner5.info
tizanidineandhydrocodone.blogspot.com torrentgamer.net
tizanidinevssoma.blogspot.com torture.ml
toloka.hurtom.com trafficjunky.net
topcar-krasnodar.ru trafficstars.com
torontoplumbinggroup.com trahic.ru
torrnada.ru trailer.cinemaflix.website
touchmods.fr truebeauty.cc
tracksurf.daooda.com try-rx.com
tradedeals.biz tryrating.com
trafficcentr.xyz tula.howotorg.ru
traffpartners.com turbo-suslik.org
tvteleport.ru turizmus.us
typer.one u555u.info
u-cheats.ru uac.net.au
udsgame.online uasb.ru
ultimatesetnewfreeallsoftupgradesystems.pw uk-zheu20.ru
umnovocaminho.com unitexindia.com
unmaroll.ya.ru uptimechecker.com
url-extractor.xyz usbggettwku.ga
uvozdeckych.info vacuumcleanerguru.com
validdomain.xyz validccseller.com
valoresito.com vandrie-ict.nl
vapomnoncri.tk vapomnoncri.tk
ventelnos.com vchulkah.net
video-chat.love vduplo.ru
video-woman.com vekzdorov.ru
videochat.life velpanex.ru
videochat.tv.br viagroid.ru
viandpet.com
videogamesecrets.com
videokrik.net videokrik.net
videooko.weebly.com vip-dom.in
vietimgy.pw vip-file.com
vip2ch.com vip2ch.com
vipps.com.my visitcambridge.org
vizitki.net vkmusics.ru
vkonche.com vksaver-all.ru
vkontarkte.com vladimir.xrus.org
voloslove.ru voltrknc1.com
voluumtracker1.com vpnmouse.com
w3data.co vuclip.com
wallpapers-best.com vut.com.ru
wallpapersist.com vzlomsn.org
webadvance.club walpaperlist.com
weblibrary.win wardreapptokone.tk
website-datenbank.de warningwar.ru
website-speed-up.site warningzscaler.heraeus.com
watchmyfb.pl
watchmygf.net
waterpurifier.club
watracker.net
website-speed-check.site
wechatdownload10.com wechatdownload10.com
welck.octopis.com
wesharepics.site wesharepics.site
westen-v.life westvilletowingservices.co.za
whatistizanidine2mg.blogspot.com whatistizanidinehclusedfor.blogspot.com
while.cheapwebsitehoster.com wholesalecheapjerseysfree.com
wholesalejerseychinaoutlet.com wholesalejerseychinaoutlet.com
wieseversa.no wholinkstome.com
wingsofrefuge.net whosonmyserver.com
wma-x.com
word-vorlagen.net word-vorlagen.net
www.timer4web.com worldoffiles.ru
xedserver.com wpsecurity.website
xev.ru writersgroup580.web.fc2.com
xkaz.org ww1943.ru
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai www.ehscloud.cn
xn----itbeirbjbi7bc6bh2d.xn--p1ai www.souvenirua.com
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai wwwadultcheck.com
xn--b1agm2d.net wygraj-skiny.win
y8games-free.com x69ty.ru
yellowads.men xaijo.com
yginekologa.com xchat26.myfreecams.com
youjizz.com xmladserver.com
youradexchange.com xn--80aanaardaperhcem4a6i.com
yoursearch.me xolodremont.ru
youtuhe.com you-shall-not-pass.is74.ru
zajm-zalog-krasnodar.ru youdao.com
zeg-distribution.com yougotanewdomain.com
zetmaster.ru yubikk.info
zhorapankratov7.blogspot.com yugk.net
zigarettenonlinekaufen.tumblr.com yugo-star.ru
zixizop.net.ru zajm-pod-zalog-nedvizhimosti.ru
zona-aqua.ru zeroredirect.com
zeroredirect8.com
zigarettenonlinekaufen1.blox.pl
zone-kev717.info
zoominfo.com
zzbroya.com.ua

494
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1stat.ru 00go.com
28n2gl3wfyb0.ru 10bet.com
3-letter-domains.net 1hwy.com
3weekdiet.com aac500b7a15b2646968f6bd8c6305869d7.com
a.frcls.fr activepr.ru
acheterviagrafr24.com adtiger.tk
addbin.men affiliate-fr.com
adf.ly agysya.ru
aktivator-windows10.blogspot.com alert-fdm.xyz
alfa-img.com all4invest.info
allnews.md allknow.info
aludecor.info allornamenti.com
an-donut.com amanda-porn.ga
animali.deagostinipassion.it apiluckyleapnet-a.akamaihd.net
apioasisspacenet-a.akamaihd.net apiportalmorecom-a.akamaihd.net
apispringsmartne-a.akamaihd.net apishops.ru
arenda-avtoprokat-krasnodar.ru aramaicmedia.org
auto-zapchasti.org asmxsatadriverin.aircus.com
ayerbo.xhost.ro aversis.net
babs.com.ua aviav.eu
baltgem.com azartmix.com
bannerswap.com babespcs.com
bbtec.net backlink4u.net
beachtoday.ru bag77.ru
berlininsl.com best-seo-offer.com
betterhdporn.com bestadbid.com
bezcmexa.ru bezsporno.ru
bigcareer.info bimatoprost-careprost.com.ua
bitcoinpile.com bin-brokers.com
bizfly.info bloggerads.net
blog-hits.com blogs.rediff.com
bluesman.nu bookmark4you.com
bmwhighperformers.com brains2.biz
bochemit.com.ua brillianty.info
bombla.org bronzeaid-a.akamaihd.net
cacheimages.com buqyxa.rincian.info
camdenmemorials.com buyessaynow.biz
candypeople.se buytizanidineonline.blogspot.com
canrole.com camsex.xxx-cam.webcam
cerev.info cartierreplica.pw
chatroulette.world cdnnetwok.xyz
christianlouboutinsaleoutletonline.us celejihad.info
coindirect.io centrdebut.ru
companies-catalog.com cfacarrosserie74.com
compliance-walt.top chee-by.biz
compliance-walter.top cibpenonptib.flu.cc
cordstrap.cc circlesl.com
customerpromos-a.akamaihd.net cobaltpro.ru
cyclobenzaprinevstizanidine.blogspot.com coccoc.com
dear-diary.ru coin-hive.com
compliance-brian.top
compliance-jess.top
conboy.us
contentlook.co
cookie-law-enforcement-hh.xyz
copypaste.traderzplanet.in
covetnica.com
crazyboost.pro
creatives.adbetclickin.pink
cryptoswap.biz
customsua.com.ua
dawlenie.com
defenderxtactical.com defenderxtactical.com
dirtpics.pw deplim.com
dktr.ru dev78.dioniqlabb.se
downloaddy.net docs4all.com
downloadeer.net downloader12.ru
draniki.org downloadkakaotalk.com
e-poker-2005.com eachdayisagift.review
electro-prom.com easycommerce.cf
elektrozigarette2.wordpressy.pl eduinfosite.com
embedle.com ege-essay.ru
eralph.tk el-nation.com
ereko.ru elitedollars.com
essayassist.com esfchat.tk
everflora.ru etenininrade.ga
exdocsfiles.com european-torches.ru
executehosting.com extener.org
extlabs.io f5mtrack.com
extremez.net facebook-mobile.xyz
ezigarettekaufen1.hpage.com fettefrauen.net
ezigaretteshop2.vefblog.net fifa-coins.online
falcon-images.blogspot.com finstroy.net
fbdownloader.com fitfloponline.store
femdom.twiclub.in free-fb-traffic.com
fickblock18.com free-share-buttons.com
findanysex.com frivgame250.com
finejewelryshop.ru fun2cell.net
footballfarrago.com gamerscorps.com
forum-engineering.ru gelezki.com
frighteningremain.cf gemgrab-a.akamaihd.net
gael-s.ru get-free-traffic-now.com
gamezblox.com getlamborghini.ga
gazoblok.net.ua gilsonchiro.xyz
gcup.ru globalsurfari.com
gosreg.amchs.ru googlepositions.com
graphid.com graphicwe.org
gratuitbaise.com gratis-sexkontakte.com
gribkovye-zabolevaniya.com greendream.com.ua
guildebzh.info guitar-master.org
homeinns.com gunsvicceadadebt.tk
hostingclub.lk hatdc.org
houdom.net hccoder.info
icaseclub.ru healbio.ru
iloveitaly.ru helicalpile.us
incep.imagine-ex.co highspeed5.net
increasewwwtraffic.info hop.clickbank.net
inspiring-desperate.tk houston-vikings.com
inzn.ru hptwaakw.blog.fc2.com
ip-guide.com ifirestarter.ru
irkutsk.zrus.org images-graphics-pics.com
iusstf.org infobabki.ru
ivanstroi.ru inome.com.ua
izoll.ru ispac.org
jav-idol.com j33x.com
jerseysbizwholesalecheap.com jav-way.site
joingames.org kandidos.com
joyceblog.top katjimej.blog.fc2.com
kemerovo.zrus.org kinohall.ru
kino2018.cc kinostorm.net
kinogolos.ru kit-opt.ru
kletkimehan.ru kndxbkdx.bloger.index.hr
klikbonus.com kosova.de
komp-pomosch.ru kriokomora.info
konteiner24.com landscapebackgrounds.blogspot.com
kw21.org laxob.com
landscaping.center lecbter-relationships.ga
lankarns.com leftofcentrist.com
law-check-seven.xyz letto.by
legionalpha.com liupis.com
letmacwork.world liver-chintai.org
livejasmin.com lookingglassemb.com
lodki-pvh.dp.ua lost-alpha.ru
madot.onlinewebshop.net lutherstable.org
magiadinamibia.blogspot.com magazintiande.ru
manifestyourmillion.com maridan.com.ua
mapquestz.us matrixalchemy.com
markjaybeefractal.com mcadamssupplyco.com
matematikus.info meble-bogart.info
me-ke.com mir-business-24.ru
metallosajding.ru mobilemedia.md
metarip.ru moxo.com
mexicotravelnet.com moyakuhnia.ru
missvietnam.org mvpicton.co.uk
moesonce.com mylesosibirsk.ru
mojpregled.com mysex21.com
mojpreskumanie.com namenectar.com
moroccosurfadventures.com nationalbreakdown.com
mossmesi.com naturalshair.site
mostantikor.ru nicefloor.co.uk
mturkcontent.com nikitsyringedrivelg.pen.io
multgo.ru ningessaybe.me
muz-shoes.ru njkmznnb.ru
mydirtystuff.com notebook-pro.ru
myfreecams.com nuker.com
napalm51.nut.cc nullrefer.com
naperehresti.info omoikiri-japan.ru
nl.netlog.com openfrost.net
northfacestore.online partnerads.men
o00.in parvezmia.xyz
onemactrckr.com perl.dp.ua
online-podarki.com photo.houseofgaga.ru
osoznanie-narkotikam.net photosaga.info
outpersonals.com pictures-and-images.com
oynat.info polytopesexempt.com
p-business.ru preg.marketingvici.com
paceform.com privat-girl.net
picquery.com privatefx.all4invest.info
pizda.lol prombudpostach.com.ua
podshipniki-ntn.ru pussyfleet.com
pomoc-drogowa.cba.pl pussysaga.com
pons-presse.com pussyspace.net
pornhubforum.tk rabotaetvse.ru
pornoblood.com rangjued.com
pornohub.me razvratnoe.org
portnoff.od.ua realizmobi.com
projectforte.ru realresultslist.com
psa48.ru remontbiz.ru
puzo2arbuza.ru rfserial.net
quality-traffic.com rialp.getenjoyment.net
quickloanbank.com risparmiocasa.bz.it
ranksonic.com rocketchange.ru
responsive-test.net rybalka-opt.ru
respublica-otel.ru saddiechoua.com
ring4rhino.com samoiedo.it
rn-to-bsn.com saveriopiazza.it
salut-camp.ru sc-specialhost.com
sarf3omlat.com scanner-walt.top
semxiu.com scat.porn
seomarketings.online seansonline24.pl
shikiso.info semaltmedia.com
shtora66.ru serdcenebolit.com
sibtest.ru seryeznie-znakomstva.ru
sim-service.net sheerseo.com
sitiz.club shopwme.ru
skatestick.bid site3.free-share-buttons.com
slavkokacunko.de softonicads.com
sluganarodu.ru spywarebegone.com
smokewithrabbits.com stal-rulon.ru
sms2x2.ru start.myplaycity.com
social-fun.ru sticken.co
sofit-dmd.ru
soserfis.com
spasswelt.net
speedup-my.site
stats-collector.org
supermama.top supermama.top
svarkagid.com suture.co
sweepstakes.rewardit.com tabletkinaodchudzanie.com.pl
tattooreligion.ru tako3.com
td-l-market.ru thecounter.com
tds-advert002.info thedownloadfromwarez.blogspot.com
thegolfclub.info tiandeural.ru
tizanidinesideeffects.blogspot.com timhost.ru
top10-online-games.com tizanidinenarcotic.blogspot.com
topanasex.com tizanidineorflexeril.blogspot.com
torrents-tracker.com tizanidinerestlesslegsyndrome.blogspot.com
tradgardspartner.se tomatis.gospartner.com
traffic2money.com topads.men
trainoffend.ml track-rankings.online
tuckpointingmasonrysystems.com ttrraacckkrr.com
turist-strani.ru turbabitload.weebly.com
tvory.predmety.in.ua
ukrtvir.com.ua
unacittaconte.org unacittaconte.org
unrealcommander.com unblocksit.es
urlopener.com uprour.com
usdx.us uzporno.mobi
veles.shop vacuumscleaner.com
viagra.pp.ua validccseller.com
video-production.com vashsvet.com
vigrx-original.ru vertaform.com
vip-parfumeria.ru viagraneggrx.com
vip.51.la viktoria-center.ru
visitcambridge.org vipcallsgirls.com
vita.com.hr vipms.ru
vitoriacabos.com we-are-gamers.com
vodaodessa.com webkeyit.com
voluumtracker1.com welck.octopis.com
vseigry.fun wesharepics.info
vtcdns.com whatzmyip.net
warning.or.kr where-toget.com
webads.co.nz wholesalejerseychinashop.com
webnode.me wholinkstome.com
website-speed-check.site wnoz.de
wemarketing.se woman-tampon.ru
westen-z.life www.souvenirua.com
wfb.hatedriveapart.com xn----7sbifcamovvfggw9d.xn--p1ai
wjgony.com xn--24-glceagatoq7c2a6ioc.xn--p1ai
word-vorlagen.xyz xuki.us
xcc24.pl xxxrus.org
xn----ctbbcjd3dbsehgi.xn--p1ai yellowfootprints.com
xnxx699.com yourporngay.com
xzlive.com yoursearch.me
yougotanewdomain.com zaloadi.ru
zarabotat-na-sajte.ru zooggames.com
zawyna.ua zophim.me
zeroredirect1.com
zeroredirect11.com
zeroredirect8.com
zmoda.hostreo.com
zoogdisany.com

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.168 0; 10.1.0.239 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.08.3263 ### Version: V4.2022.08.3264
### Updated: Mon Aug 1 10:03:23 UTC 2022 ### Updated: Mon Aug 1 22:01:40 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.08.3263 ### Version: V4.2022.08.3264
### Updated: Mon Aug 1 10:03:23 UTC 2022 ### Updated: Mon Aug 1 22:01:40 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##