vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 10:40:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.10.3407 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-10-13 10:10:11 +00:00
parent ee054aaf10
commit 3e02b1dab6
47 changed files with 99253 additions and 99263 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3406 #### Version: V4.2022.10.3407
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3406 #### Version: V4.2022.10.3407
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.10.3406 #### Version: V4.2022.10.3407
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12272
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.10.3406 #### Version: V4.2022.10.3407
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12276
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12276
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12276
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12276
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.1.170 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.1.170 0;
127.0.0.1 0; 127.0.0.1 0;

12276
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.1.170 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.1.170 0;
127.0.0.1 0; 127.0.0.1 0;

12276
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.1.170 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.1.170 0;
127.0.0.1 0; 127.0.0.1 0;

12276
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3406 3407

12276
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12276
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.43.242.168 0; 20.66.29.227 0;

12276
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.1.170 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.1.170 0;
127.0.0.1 0; 127.0.0.1 0;

12276
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.1.170 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.1.170 0;
127.0.0.1 0; 127.0.0.1 0;

12276
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.1.170 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.1.170 0;
127.0.0.1 0; 127.0.0.1 0;

12276
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.43.242.168 0; 20.66.29.227 0;

12276
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.43.242.168 0; 20.66.29.227 0;

12276
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.1.170 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
80legs Aipbot
Cliqzbot CODE87
Dirbuster Cegbfeieh
DnBCrawler-Analytics Getintent
FDM HTTrack
PageAnalyzer Minefield
PageGrabber MolokaiBot
Quick-Crawler Morfeus Fucking Scanner
Sogou web spider Sitebeam
SputnikBot lwp-trivial

300
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com AIBOT
360Spider
404enemy
ADmantX
ASPSeek ASPSeek
Abonti Aboundex
Aboundexbot AfD-Verbotsverfahren
Acunetix
AiHitBot AiHitBot
Anarchie Aipbot
Anarchy99
Apexoo Apexoo
BDCbot Asterias
Atomseobot
Attach
AwarioRssBot
BackWeb BackWeb
Backlink-Ceck
Bandit Bandit
Barkrowler BetaBot
BatchFTP
Bigfoot
Bitacle Bitacle
Black Hole
BlackWidow BlackWidow
Blow Blackboard
Brandwatch Brandprotect
BunnySlippers BuiltWith
BuzzSumo Bullseye
CATExplorador
CCBot
CODE87 CODE87
Cegbfeieh CSHttp
Calculon
CazoodleBot
CheTeam CheTeam
CheeseBot
ChinaClaw
Chlooe
Citoid Citoid
Claritybot
Cliqzbot Cliqzbot
Cogentbot Cloud mapping
Collector Cocolyzebot
Cosmos Copier
CyotekWebCopy CopyRightCheck
DBLBot Craftbot
Crawling at Home Project
Crescent
Curious
DIIbot DIIbot
DSearch DSearch
DTS Agent
DataCha0s
Devil Devil
Digincore Disco
DigitalPebble
Dirbuster
Discobot Discobot
Dispatch
DittoSpyder
DnyzBot DnyzBot
DomCopBot DomCopBot
DomainAppender
DomainCrawler
DomainSigmaCrawler
Domains Project
Dotbot Dotbot
ECCP/1.0 Download Wonder
EMail Siphon
EMail Wolf EMail Wolf
Ecxi EasyDL
EirGrabber EirGrabber
EroCrawler
Exabot
Express WebPictures
ExtLinksBot
Extractor
ExtractorPro
Extreme Picture Finder
Ezooms
FDM FDM
FHscan
FemtosearchBot FemtosearchBot
FyberSpider Fimap
Foobot
FrontPage
G-i-g-a-b-o-t
GT::WWW GT::WWW
Genieo
GetRight GetRight
Getintent
Gigabot Gigabot
Go!Zilla Grabber
Go-Ahead-Got-It GridBot
GrabNet
HEADMasterSEO HEADMasterSEO
HTTP::Lite HTMLparser
Haansoft
HaosouSpider HaosouSpider
Harvest
Havij Havij
HybridBot Hloader
Iblog HonoluluBot
Humanlinks
IDBTE4M
IDBot
IRLbot
IlseBot IlseBot
Image Sucker InfoNaviRobot
Internet Ninja Internet Ninja
Iskanie InternetSeer
Iria
IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT Jbrofuzz
JetCar Jorgee
Jetty Jyxobot
Keybot Translation-Search-Machine Kenjin Spider
LNSpiderguy
Lanshanbot Lanshanbot
Leap Larbin
LeechFTP LeechFTP
Libwhisker LexiBot
LinqiaScrapeBot LibWeb
Lipperhey Lightspeedsystems
LinksManager
LinqiaMetadataDownloaderBot
Lipperhey Spider Lipperhey Spider
Litemage_walker Litemage_walker
Ltx71 MFC_Tear_Sample
MIDown tool MIDown tool
MIIxpc
MQQBrowser MQQBrowser
Mag-Net MSIECrawler
Mail.RU_Bot MTRobot
Majestic SEO Majestic SEO
Majestic-SEO
Majestic12 Majestic12
Mata Hari MarkMonitor
MauiBot Masscan
MeanPath Bot Mister PiX
Mediatoolkitbot Moblie Safari
MicroMessenger
Microsoft Data Access
Minefield
Mojeek
Morfeus Fucking Scanner Morfeus Fucking Scanner
Msrabot
Musobot Musobot
NICErsPRO
NPbot
Nameprotect
Navroad Navroad
Needle NearSite
Nessus Nessus
NetLyzer Net Vampire
NetMechanic NetAnts
NetSpider NetZIP
Nettrack Netvibes
NextGenSearchBot Nibbler
Nikto Nikto
NimbleCrawler
Ninja
Nmap
Not Not
Nuclei
Nutch Nutch
Offline Explorer OpenLinkProfiler
Openfind
Openvas Openvas
PageAnalyzer OrangeBot
PECL::HTTP
POE-Component-Client-HTTP
PageGrabber PageGrabber
Picsearch PageScorer
PictureFinder PageThing.com
Piepmatz Pandalytics
Papa Foto
Petalbot
Picscout
Pimonster Pimonster
Pockey Pixray
PleaseCrawl
ProPowerBot ProPowerBot
Probethenet Probethenet
Psbot
Pu_iN
Pump
QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankingBot
RankingBot2 RankingBot2
Rankivabot Re-re
RealDownload ReGet
Recorder RebelMouse
RocketCrawler RedesScrapy
SISTRIX RepoMonkey
Ripper
SBIder
SEOlyticsCrawler
SEOprofiler
SEOstats
SMTBot
Scanbot Scanbot
Scrapy
Screaming Screaming
SemanticJuice Searchestate
SemrushBot SearchmetricsBot
SeobilityBot Semrush
SentiBot
Seomoz Seomoz
Shodan
Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteSnagger SiteSnagger
SiteSucker SiteSucker
Sitebeam Sitebeam
Siteimprove
Sitevigil
SlySearch SlySearch
Snake
Sogou web spider Sogou web spider
Sosospider Sosospider
Sottopop Spbot
Steeler
Stripper
Sucuri Sucuri
SuperBot SuperBot
SuperHTTP SuperHTTP
Surfbot
Suzuran Suzuran
Swiftbot
Szukacz Szukacz
T0PHackTeam T0PHackTeam
Teleport Telesoft
Telesphoreo
Telesphorep
The Intraformant The Intraformant
TightTwatBot TightTwatBot
Toata Toata
Toweyabot
Tracemyfile Tracemyfile
Trendictionbot Trendictionbot
Turnitin
Twice Twice
Typhoeus
Vacuum Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler
Virusdie Virusdie
Voil
Voltron
WBSearchBot
WEBDAV WEBDAV
WPScan WWW-Collector-E
WWW::Mechanize WWW::Mechanize
Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WeSEE WeSEE
Web Sucker
WebBandit WebBandit
WebCollage
WebCopier
WebFetch
WebGo IS
WebLeacher
WebPix WebPix
WebStripper WebSucker
WebWhacker WebsiteExtractor
WebZIP
Webshag
WebsiteQuester WebsiteQuester
Whack
Whatweb
Widow Widow
WinHTTrack Zauba
Wotbox Zermelo
Xaldon WebSpider
YoudaoBot
Zitebot
ZmEu ZmEu
arquivo-web-crawler ZoomBot
ZoominfoBot
autoemailspider
check1.exe check1.exe
coccocbot clark-crawler
dataforseo.com crawler.feedback
evc-batch crawler4j
demandbase-bot
gopher
heritrix
instabid instabid
ips-agent
linkdexbot
lwp-request
magpie-crawler
meanpathbot
mediawords mediawords
muhstik-scan muhstik-scan
oBot oBot
polaris version probe-image-size
ripz satoristudio.net
s1z.ru
scalaj-http scalaj-http
seoscanners scan.lol
seobility
seocompany.store
serpstatbot
sitechecker.pro sitechecker.pro
siteripz
sp_auditbot sp_auditbot
spyfu spyfu
x09Mozilla trendiction.com
webgains-bot
webpros.com
xpymep1.exe
zauba.io zauba.io
zgrab

310
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404checker 404checker
404enemy 404enemy
80legs ALittle Client
Abonti ASPSeek
AhrefsBot Aboundex
Alexibot Aboundexbot
Anarchie
Anarchy Anarchy
Anthill
Apexoo Apexoo
Asterias Asterias
Atomseobot AwarioSmartBot
AwarioRssBot
BBBike BBBike
BDCbot
BDFetch BDFetch
BLEXBot
Bandit
Barkrowler Barkrowler
BatchFTP BatchFTP
Battleztar Bazinga BetaBot
BlowFish Bigfoot
Bolt Bolt
BuiltBotTough Brandwatch
BuzzSumo Buck
CCBot Buddy
CheTeam Bullseye
Cliqzbot CODE87
Cegbfeieh
CensysInspect
CheeseBot
CherryPicker
ChinaClaw
Citoid
Cloud mapping Cloud mapping
Collector Cocolyzebot
Copyscape
Cosmos Cosmos
Crescent
CrunchBot CrunchBot
Curious Curious
DTS Agent Custo
DataCha0s Demon
Deusu
Digincore Digincore
DigitalPebble Disco
Discobot Discobot
Discoverybot Discoverybot
Dispatch
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot DomainAppender
DomCopBot
DomainCrawler
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project Domains Project
Download Wonder Dragonfly
Drip
EMail Siphon
EMail Wolf EMail Wolf
EasyDL EasyDL
Ecxi Ebingbong
EirGrabber EirGrabber
EroCrawler Exabot
Extreme Picture Finder Extractor
Ezooms Ezooms
Flunky Fimap
Foobot Foobot
Fuzz Freeuploader
FyberSpider
Fyrebot
GT::WWW
GermCrawler GermCrawler
GetWeb
Getintent
Gigabot Gigabot
Go-Ahead-Got-It
GoZilla GoZilla
GrabNet Grafula
GrapeFX GridBot
GrapeshotCrawler HMView
HEADMasterSEO HTTP::Lite
HTMLparser
Haansoft Haansoft
Havij Harvest
Humanlinks Hloader
IRLbot HybridBot
IDBot
IlseBot IlseBot
Indy Library
InfoNaviRobot InfoNaviRobot
InfoTekies Iria
Intelliseek JennyBot
InternetSeer Jetty
JOC Web Spider
Jbrofuzz
JikeSpider
Jorgee Jorgee
Kozmosbot Keyword Density
LNSpiderguy
LWP::Simple
Lanshanbot Lanshanbot
LeechFTP Larbin
Leap
LeechGet LeechGet
LieBaoFast Libwhisker
LinkWalker Likse
Linkbot
LinkextractorPro LinkextractorPro
LinkpadBot LinkpadBot
LinqiaScrapeBot LinksManager
Ltx71 LinqiaRSSBot
Lipperhey Spider
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MIIxpc
MJ12bot MQQBrowser
MSIECrawler
Mag-Net
Mail.RU_Bot Mail.RU_Bot
Majestic12 Majestic-SEO
MarkMonitor MauiBot
Mass Downloader Mb2345Browser
Mata Hari
MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot Mediatoolkitbot
Metauri
MicroMessenger MicroMessenger
Microsoft Data Access
Moblie Safari Moblie Safari
Mojeek Mojeek
MolokaiBot Msrabot
Morfeus Fucking Scanner Nameprotect
Navroad NearSite
Nessus
Net Vampire
NetLyzer
NetMechanic NetMechanic
NetZIP
Netcraft
Nettrack Nettrack
NextGenSearchBot
Nibbler Nibbler
Nikto Nikto
Ninja
Nmap
Not Not
Nuclei Octopus
OnCrawl OrangeBot
Openfind PHPCrawl
Openvas
OrangeSpider
PageGrabber
PageScorer
PageThing.com PageThing.com
Pandalytics
Pavuk Pavuk
Picscout Petalbot
Picsearch PictureFinder
Piepmatz
Pixray
Pockey Pockey
ProPowerBot
ProWebWalker
Probethenet Probethenet
Psbot Psbot
Pu_iN
PxBroker
PyCurl PyCurl
QueryN Metasearch
RankActive RankActive
RankFlex RankActiveLinkBot
RankingBot2 Rankivabot
RankurBot
Re-re Re-re
ReGet ReGet
RealDownload
Recorder Recorder
RedesScrapy
RepoMonkey
Ripper Ripper
RocketCrawler RocketCrawler
SBIder Rogerbot
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SEOprofiler
SEOstats SEOstats
SISTRIX SISTRIX
SMTBot
Scanbot
ScoutJet
Screaming Screaming
Semrush SearchmetricsBot
SemanticJuice
SentiBot
SeoSiteCheckup
Siphon
SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSucker Sitebeam
SlySearch Sitevigil
Snake SmartDownload
Snapbot Snapbot
Sogou web spider SocialRankIOBot
Sociscraper
Sosospider Sosospider
Sottopop Sottopop
Spammen SputnikBot
SpankBot Sqlmap
Spanner Sqlworm
Spbot Steeler
Spinn3r Sucuri
Sqworm SuperBot
Stripper SurveyBot
SuperHTTP
Surfbot
Szukacz Szukacz
T0PHackTeam
T8Abot
Telesphoreo
Telesphorep
The Intraformant
TheNomad TheNomad
Thumbor
TightTwatBot TightTwatBot
Turnitin Toata
Toweyabot
Trendictionbot
TwengaBot
Twice
Typhoeus
URLy Warning URLy Warning
UnisterBot
Upflow Upflow
V-BOT
VB Project VB Project
Vacuum VCI
VeriCiteCrawler VeriCiteCrawler
VidibleScraper VoidEYE
WASALive-Bot Voltron
WBSearchBot
WEBDAV WEBDAV
WWW-Mechanize WISENutbot
WWW::Mechanize WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
Web Auto WallpapersHD
Web Fetch
Web Pix
Web Enhancer Web Enhancer
WebLeacher
Web Pix Web Pix
Web Sauger Web Sauger
WebWhacker WebFetch
WebGo IS
WebLeacher
WebReaper
WebStripper
WebmasterWorldForumBot
Webshag
Website Quester Website Quester
Whatweb Webster
Whack
Whacker
Who.is Bot Who.is Bot
Widow
WinHTTrack
Woobot Woobot
YoudaoBot Wprecon
Zauba Xaldon_WebSpider
ZoominfoBot Zade
adscanner Zitebot
ZoomBot
ZumBot
ZyBorg
archive.org_bot archive.org_bot
arquivo-web-crawler
autoemailspider
backlink-check backlink-check
check1.exe cah.io.community
coccocbot clark-crawler
crawler4j cognitiveseo
crawler.feedback
dataforseo.com dataforseo.com
demandbase-bot
eCatch eCatch
facebookscraper
heritrix heritrix
internetVista monitor ips-agent
isitwp.com
lwp-request
magpie-crawler magpie-crawler
muhstik-scan oBot
pcBrowser page scorer
plumanalytics polaris version
s1z.ru probe-image-size
ripz
satoristudio.net
scalaj-http scalaj-http
scan.lol
seobility seobility
seoscanners seoscanners
seostar seostar
serpstatbot spyfu
sitechecker.pro sysscan
sp_auditbot zgrab
trendiction.de
voyagerx.com
webpros.com
webprosbot
x09Mozilla
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
availit.weebly.com balois.worldbreak.com
jaxcube.info freelifetimefuckbook.com
moonci.ru lonfon.xyz
needtosellmyhousefast.com master-muznachas.ru
ourville.info net-profits.xyz
rniaeba.ga social-buttons.xyz
robotixix.com solution4u.com
royalcar-ufa.ru spb-plitka.ru
shymkent.xkaz.org whatzmyip.net
truemfilelj.gq znakomstva-piter78.ru

916
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00it.com 101billion.com
123any.com 123cha.com
12masterov.com 13tabs.com
1314dh.com 1kinobig.ru
178evakuator178.ru 2020iscoming.info
1millionusd.xyz 24-ak.ru
2015god.org 24x7-server-support.site
202ch.com 2daytrendingnews.com
7ooo.ru 4istoshop.com
7search.com 5ws.dating-app.ru
7zap.com 66cpwgln.space
aa625d84f1587749c1ab011d6f269f7d64.com 6hopping.com
76brighton.co.uk
83net.jp
8558l.jobs.net
8gold.com
9med.net
a.frcls.fr
aa9d046aab36af4ff182f097f840430d51.com
aasoldes.fr
abwa.tk abwa.tk
acgs.tk actices.com
acortarurl.es adbetclickin.pink
add-add.men add-add.men
addray.pro adsfresh.men
adf.ly advokateg.xyz
adloads.net adzpower.com
adserver-e7.com agapovdg.ru
adtech.us agysya.ru
ahrntal.verymes.xyz alevinefesleri.com
alarmobninsk.ru alf-img.com
alert-fdm.xyz alfapro.ru
alessandraleone.com allsilver925.co.il
all-streaming-media.com
alphaforum.ru
alphavisions.net
alpinism.ru alpinism.ru
amateurgalls.com alyeskaresort.com
amospalla.es amazon-adsystem.com
anal-acrobats.hol.es anaseracresar.tk
angkortours.vn apccargo.com
apidigidocketnet-a.akamaihd.net
apiholdingmypage-a.akamaihd.net apiholdingmypage-a.akamaihd.net
appiq.mobi apiwebwebgetcom-a.akamaihd.net
approved.su apparel-offer.com
arcteryxsale.online appfastplay.com
auto-moto-elektronika.cz apple.com-webbrowsing-security.science
autolombard-krasnodar.ru apptester.tk
avon-ukraine.com arendakvartir.kz
avtolombard-krasnodar.com as5000.com
azbuka-mo.ru asiengirls.net
azbukafree.com asociatia-tipografilor-transilvania.ro
b00kmarks.com atdedinotuho.tk
backlink4u.net audiobangout.com
bagcionderlab.com ausergrubhof.info
bard-real.com.ua ausmepa.org.au
basedecor.ru auto.rusvile.lt
beclean-nn.ru aversis.net
best-mam.ru avtoarenda.by
bestdraws.com axbocz.net
besthomemadepornsites.com ayodhya.co
besthoro.ru babespcs.com
bestplacetobuyeriacta.jw.lt bad-stars.net
bidbuy.co.kr balkanfarma.ru
bigcareer.info bankiem.pl
bioca.org bayareaaeroclub.org
biocn.dx.am bedandbreakfast.com
bioscorp.ru best-games.today
bestwebsiteawards.com
big-boards.info
bigpenisguide.com
bin-brokers.com
birzha-truda.eu birzha-truda.eu
bitnote.co bloggers.nl
bjetjt.com bluesman.nu
boanonihaca.tk bolitgorlo.net
bosman.pluto.ro bongiornos.info
brianhenry.co bookmark4you.com
brus-vsem.ru braintobe.win
budilneg.xyz brandimensions.com
bum.com.ru briomotor.co
bumascloset.com
buqyxa.rincian.info buqyxa.rincian.info
buttons-for-website.com buypuppies.ca
buy-forum.ru buzzsumo.com
buyantiviralwp.com
buybest1.biz
buzz-porno.info
buzzurl.jp
byme.se byme.se
bzero1jewelry.net c-gainsbourg.com
cablecar.us cactussoft.biz
callawaygolfoutlet.online carloans.com
camdenmemorials.com carsplate.com
carfax.com.ua
carina-sy.de
carson.getenjoyment.net
carstrends2015.com
cartechnic.ru
cartierlove2u.com cartierlove2u.com
cartierlovebracelet.xyz caulderwoodkennels.com
cartierreplica.xyz cdncash.com
casablancamanor.co.za chatroulette.online
cezartabac.ro
cheapjerseysap.com cheapjerseysap.com
cheapjerseysbizwholesale.us cheapmoncler.win
chocolateslim-u-srbiji.com
chorus.terakeet.com
cjs.com.ru
clarithromycin500mg.com
cleanallvirus.com cleanallvirus.com
clipartnew.com clipartnew.com
clothesforcash.com cognitiveseo.com
com-onlinesupport.site collegerentals.com
compliance-don.xyz compliance-andrew.top
compliance-mark.top compliance-barak.top
compliance-walter.top compliance-checker-7.info
courtshipgift.com compliance-jess.top
compliance-josh.top
compliance-willy.top
cookielawblog.wordpress.com
coolwallpapers4k.info
covetnica.com covetnica.com
covid-schutzmasken.de
cpajunkies.com
crafthubs.com
crazyprotocol.com crazyprotocol.com
creatives.adbetclickin.pink customerpromos-a.akamaihd.net
custom-product-labels.com
customcollegeessays.net
customsua.com.ua customsua.com.ua
cypernhuset.se cypernhuset.se
d0t.ru
dailystrength.org
darkages.info darkages.info
darkbooks.org dataloading.net
datadepths.com davebestdeals.com
detsky-nabytek.info dcj-nn.ru
dieswaene.com dekorkeramik.ru
dmmspy.com den-noch24.ru
docsportal.net dentalpearls.com.au
deplim.com
diegolopezcastan.com
dinosaurus.site
discovertreasure-a.akamaihd.net
dodlive.mil dodlive.mil
doktoronline.no dogclothing.store
domain-tracker.com drhomes.biz
domainsphoto.com dustyorate.com
domination.ml e-avon.ru
dominterior.org e-poker-2005.com
donna7753191.ru eager-nash.188-93-233-196.plesk.page
dostavimvdom.ru easync.io
dostavka-v-krym.com ebonyporn.site
drpornogratisx.xxx ed-shop01.ru
duawitchrarato.tk ekto.ee
dumpsmania24.com electricwheelchairsarea.com
dvrlists.com electro-prom.com
e-buyeasy.com elite-sex-finders.com
ege-essay.ru en.home-task.com
electrik-avenue.com enternet.ee
elektrozigarette2.webs.com epicbrogaming.com
englishdictionaryfree.com escortplius.com
eralph.tk estibot.com
ero-advertising.com etotupo.ru
es5.com everypony.ru
espaceinventoristes.com
essay-writing.work
eupornstar.info
exchanges-bet.com exchanges-bet.com
extractorandburner.com exmasters.com
eyelike.com.ua expresstoplivo.ru
ezigarettenkaufen1.pagina.gr extrabot.com
facebook-mobile.xyz ezaz.info
facialporntube.com ezofest.sk
fapgon.com f00kclan.de
favoritemoney.ru family1st.ca
feel-planet.com farm26.ru
fetishinside.com fetishinside.com
fikasound.tk ff30236ddef1465f88547e760973d70a.com
fil.ru filerockstar298.weebly.com
filmfanatic.com finteks.ru
financepoints.eu
fireads.men
fishtauto.ru fishtauto.ru
flauntyoursite.com flowwwers.com
flowersbazar.com foodgid.net
forex-indextop20.ru forex-indextop20.ru
forpostlock.ru forpostlock.ru
forsex.info forum.tvmir.org
foxjuegos.com fototravel.eu
frankofficial.ru freecamdollars.com
free-social-buttons.xyz freelotto.com
free-stock-illustration.com front.ru
freewareseek.com
freewebs.com
ftns.ru
fuck-paid-share-buttons.xyz
fuckmill.com fuckmill.com
funtoonez.com fugarif.ga
fxund.us fym.com.ru
fz139.ttk.ru gael-s.ru
gallictures.com gamedayassist.com
games.kolossale.ru geile-lelly.eu
gay.adultgalls.com genetworx.com
ge0ip.org gheus.altervista.org
generic-pills-online.com girlgamerdaily.com
gidonline.one globalscam.ga
globetrotting-culture.ru glogow.pl
gobongo.info googlemare.com
gosmeb.ru greatfind-a.akamaihd.net
grand-chlen.ru gsasearchengineranker.pw
growboxbl.ru gsmtlf.ru
habermetre.com guge.io
hawaiisurf.com guigyverpo.cf
hcate.com guildebzh.info
hdimagelib.com h2monline.com
hdseriale.pl hanink.biz.ly
help.tpu.ru hard-porn.mobi
hesteel.pl hatdc.org
hidemyass.com hd720kino.ru
hotdl.in helicalpile.us
hopeonthestreet.co.uk
host-tracker.com
hosting-tracker.com
hostingclub.lk
hotkeys.com
humanorightswatch.org
hustoon.over-blog.com hustoon.over-blog.com
ideashome.id hvd-store.com
ilmen.net hydropump.su
imagefinder.site icaseclub.ru
images-free.net id-forex.com
impisr.ru ilikevitaly.com
in-tandem.co imagez.co
infektsii.com imgata.com
infowarcraft.ru infazavr.ru
innodgfdriverhm.aircus.com infoupdate.org
interfucks.net instasexyblog.com
ip-guide.com insurple.com
iqoption.com io9.com
isitpaleo.info iomoio.net
itag.pw iplusbit.blogspot.co.za
james13prix.info irkutsk.online-podarki.com
jav-fetish.site iskalko.ru
javspace.net ispac.org
jeremyeaton.co itsdp3.com
juliaworld.net jasonpartington.com
justkillingti.me jerseywholesaleelitestore.com
kadashihotel.com jestr.org
keywordsuggest.org jillepille.com
kinky-fetishes.com jmat.cn
jrcigars.com
justdating.online
kamagragelusa.net
kambasoft.com
katjimej.blog.fc2.com
kevblog.top
keywordhut.com
keywordsuggests.com
kino-ecran.ru
kinobest.pl
kinocccp.net
kit-opt.ru
knowsitall.info knowsitall.info
konkursov.net knowyournextmove.com
kosova.de kongruan.com
kung-fu-ru.com konica.kz
kvartir-remont.biz ktotut.net
lanasshop.ru kutikomi.net
landinez.co lafourmiliaire.com
lapolis.it lanadelreyfans.us
lasvegaslockandsafe.com lawyers.cafe
law-check-two.xyz le-clos-des-alouettes.com
ledpolice.ru ledis.top
lenpipet.ru lidoradio.com
lernur.net light.ifmo.ru
lescinq.com lineavabit.it
lider-zhaluzi.kiev.ua
limads.men
linkarena.com linkarena.com
linkbuddies.com linkbux.ru
liverpool.gsofootball.com linkdebrideur.xyz
livesex-amateure.info lion.misba.us
llastbuy.ru lion.snapmap.us
locatellicorretor.com.br lionking-1994.blogspot.com
lombia.co listiseltemournlan.gq
louboutinreplica.xyz ljusihus.se
m-google.xyz lol-smurfs.com
m.b00kmarks.com longgreen.info
mackeeper-land-672695126.us-east-1.elb.amazonaws.com lordzfilmz.me
madot.onlinewebshop.net louboutinbooties.xyz
mall.uk m3gadownload.pl
manimpotence.com magicplayer-s.acestream.net
masserect.com mastershef.club
masthopehomes.com
matb3aa.com matb3aa.com
max-eclat.men matchpal-a.akamaihd.net
mcnamaratech.com mature.free-websites.com
mebel-vstroika.ru medicinacom.ru
medicines-choice.com meendo-free-traffic.ga
medmajor.ru mejoresfotos.eu
member-quiz.com meltwater.com
methodsmarketing.com memberty.com
micasainvest.com mexicotravelnet.com
michaelkorsoutletstore.net minneapoliscopiers.com
michaelkorssaleoutletonline.net mobplayer.ru
midst.eu moinozhki.com
minet.club mompussy.net
miniature.io mountainstream.ms
mmoguider.ru mrbojikobi4.biz
mobile-appster.ru muycerdas.xxx
mobile.ok.ru myonigroup.com
modnie-futbolki.net myphotopipe.com
moesen-ficken.com myprintscreen.com
moesexy.com nakozhe.com
mojpregled.com nalogovyykodeks.ru
monarhs.info new-apps.ru
monclerboots.xyz new-post.tk
moncleroutletonline.win newstraveller.ru
monetizationking.net newstudio.tv
moneyteam24.com newyorkhotelsmotels.info
moomi-daeri.com
movpod.in
moyakuhnia.ru
mrwhite.biz
mtmtv.info
musezone.ru
mustwineblog.com
muzaporn.com
my-big-family.com
mydoctorok.ru
myhydros.org
myiptest.com
mysexpics.ru
nakrutka.cc
naperehresti.info
networkcheck.xyz
newfilmsonline.ru
nfljerseyscheapbiz.us
ngps1.ru
nisuturnetdgu.tk
normalegal.ru normalegal.ru
norththeface.store notaria-desalas.com
nudo.ca notebook-pro.ru
o00.in nuker.com
obiavo.in oakleyglassesonline.us
onlinemegax.com obiavo.com
ooo-gotovie.ru offer.camp
open-odyssey.org oneclickfiles.com
openstat.com online-hit.info
outclicks.net onlinebay.ru
oyster-green.com onlinefilmz.net
pandarastore.top onlinewritingjobs17.blogspot.ru
paperwritingservice17.blogspot.ru onload.pw
parajumpersstore.online onthemarch.co
partners-ship.pro ooomeru.ru
pcads.ru outpersonals.com
pdn-4.com outrageousdeal-a.akamaihd.net
pensplan4u.com pc-services.ru
pepperstyle.ru pcimforum.com
percin.biz.ly pechikamini.ru
perm-profnastil.ru pinup-casino1.ru
petrovka-online.com
pic2fly.com
piccshare.com
pictures-and-images.net
pinstake.com
pio.polytopesexempt.com pio.polytopesexempt.com
pistonclasico.com pixgood.com
pivka.xyz pizda.lol
pl-top.pl plastgran.com
pl-vouchers.com play.leadzupc.com
platezhka.net polybuild.ru
playlott.com popserve.adscpm.net
polska-poezja.com pornhubforum.tk
pony-business.com pornofeuer.com
popunder.ru pornoinn.com
porno-chaman.info pornonik.com
postclass.com powc.r.ca.d.sendibm2.com
priceg.com pozdravleniya-c.ru
prizestohandle.club pozdrawleniya.ru
pozvonim.com
prefersurvey.net
preparevideosafesystem4unow.site
prize44.com
pro-tec.kz
profitfx.online profitfx.online
projefrio.com.br profitwithalex.info
promodj.com
promoforum.ru promoforum.ru
prostitutki-almata.org prostitutki-kiev.org
prostitutki-belgoroda.org pssucai.info
ptr.ruvds.com purchasepillsnorx.com
qexyfu.bugs3.com purplestats.com
rapevideosmovies.com pussysaga.com
raschtextil.com.ua pussyspace.net
quick-seeker.com
rangjued.com
rankia.com
ranksignals.com
ranksonic.com
rasteniya-vs-zombi.ru
razorweb-a.akamaihd.net razorweb-a.akamaihd.net
rebrand.ly redirectingat.com
reckonstat.info reimageplus.com
redhotfreebies.co.uk
reklama-i-rabota.ru reklama-i-rabota.ru
remedyotc.com relatodelpresente.com.ar
remmling.de remont-komputerov-notebook.ru
remontvsamare.su remont-ustanovka-tehniki.ru
rennlist.com remybutler.fr
retailwith.com rniaeba.ga
rewardit.com robotixix.com
rfd-split.hr royalads.net
rome2rio.com
royal-investments.net
rozalli.com rozalli.com
ruclicks.com ru-mediaget.ru
rybalka-opt.ru rubbed.us
sady-urala.ru rucrypt.com
salmonfishingsacramentoriver.com russintv.fr
saveriopiazza.it rvtv.ru
sawin.beth.webd.pl sa-rewards.co.za
sax-sex.com saletool.ru
sbwealthsolutions.ca samara.rosfirm.ru
scanner-ivan.top sanjosestartups.com
scanner-julianna.top santasgift.ml
scanner-margo.top sardinie.us
scanspyware.net satoristudio.net
scottbywater.com scanner-don.top
sdelai-prosto.ru scanner-jack.top
search-error.com scanner-nelson.top
searchinterneat-a.akamaihd.net scopich.com
securityallianceservices.com searchwebknow-a.akamaihd.net
seo-traffic-ranking.info semaltmedia.com
seoanalyses.com seokicks.de
seofirmreviewsus.info seorankinglinks.us
seoholding.com serdcenebolit.com
serpstat.com serpstat.com
serptehnika.ru service.adtech.us
servisural.ru sex-znakomstva.online
sexkontakteao.info sexspornotub.com
sexobzor.info sharebutton.to
sexpartygirls.net shop-electron.ru
sexs-foto.com
sexuria.net
sharebutton.org
sherlock.se
shijian.ac.cn
sibtest.ru
sibvitr.ru sibvitr.ru
sildenafil-tadalafil.info sildenafilcitratemed.com
simply.net sisiynas.ru
sirpornogratis.xxx site-analyzer.com
siteonomy.com slutloadlive.com
sklad-24.ru snjatie-geroinovoy-lomki.ru
skypasss.com soaksoak.ru
sladkoevideo.com socialmonkee.com
sloopyjoes.com solnplast.ru
social-s-iii.xyz soserfis.com
soft-terminal.ru sotechco.co
soheavyblog.com spamnuker.com
solmarket.by spb-plitka.ru
space4update.pw spidtest.org
spbchampionat.ru spylog.com
spywarenuker.com spymac.net
ssconstruction.co sterva.cc
stal-rulon.ru stillmiracle.com
stauga.altervista.org stop-gepatit.te.ua
sticken.co stromerrealty.com
streetfire.net stroymonolit.su
summerlinhomes411.info
superiends.org superiends.org
surflinksmedical.com svnuppsalaorebro.se
surintech.ac.th sweepstakes.rewardit.com
svarkagid.com swimpool.ca
taaaak.com tagil.zrus.org
tds-advert005.info tattooreligion.ru
telefonsex-ohne0900.net teastory.co
telsis.com tecnoteakviareggio.it
templates.franklinfire.co tedxrj.com
tengohydar.tk teenporn18.net
thaisamkok.com telefonsexkostenlos.tk
the-torrent-tracker.blogspot.com terrafootwear.us
thecoral.com.br testingads.pro
timecrimea.ru thebestweightlosspills.ovh
tizanidinehydrochloride4mgstreetvalue.blogspot.com thedownloadfromwarez.blogspot.com
tizanidinesleepaid.blogspot.com thepokertimer.com
tkanorganizma.ru timhost.ru
tizanidineonline.blogspot.com
tmearegion26.com tmearegion26.com
toondinsey.com tmtrck.com
topsy.com token-lab.org
tourismvictoria.com top10registrycleaners.com
tradedeals.biz topanasex.com
trafficfactory.biz topsiteminecraft.com
trailer.cinemaflix.website track112.site
twinderbella.com triplepanda.xyz
twittruth.com truck-rece.life
tzritel.tk trustl.life
ufa.xrus.org tubo360.com
uggsale.online tupper-posuda.ru
umnovocaminho.com twincitiescarservice.com
unblocksit.es ua-company.ru
uptime-as.net ucoz.ru
uptime-gamma.net udayavani.com
uptime.com ukrtextbook.com
uzporno.mobi unacittaconte.org
vabasa.inwtrade.com usacasino.com
valsalud.com v720hd.ru
veerotech.com vacances-voyages.info
velobikestock.com valkiria-tk.ru
vapsy.com
vduplo.ru
ventelnos.com ventelnos.com
vesnatehno.com vilingstore.net
vesnatehno.ru
vezuviy.su
videositename.com
vinsit.ru
vintontech.info vintontech.info
vitalads.net vinylvault.co.uk
vkontarkte.com vizag.kharkov.ua
voluumtracker1.com vkgaleria.com
warezkeeper.com vmnmvzsmn.over-blog.com
wcb.su vod.com.ua
webadvance.club vodkoved.ru
website-analyzer.info vseigru.one
website-speed-checker.site vtcdns.com
wellcome2slovenia.ru vucms.com
wineitudes.wordpress.com vzlomtw.com
winx-play.ru walkme.com
world-mmo.com web-betting.ru
web.cvut.cz
webalan.ru
weprik.ru
westum.se
whereverdesperate.gq
wieseversa.no
wingsofrefuge.net
woman-h.ru
worldmusicfests.com worldmusicfests.com
worst-sites.online
writingservices17.blogspot.ru writingservices17.blogspot.ru
wurr.voila.net wstroika.ru
www.pinnacle-bets.com www.get-free-traffic-now.com
x-diesel.com xchat26.myfreecams.com
xn----itbkqkfiq.xn--p1ai xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn--80aeahghtf8ac5i.xn--p1ai xn--80aafb2a.xn--p1ai
xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai xn--b1agm2d.net
xn--80aodinpgi.xn--p1ai xolodremont.ru
xvideosporn.biz xtrafficplus.com
xxx-cam.webcam xxxnatelefon.ru
yaoguangdj.com yellowads.men
yogamatsexpert.com yougetsignal.com
yougame.biz youjizz.com
youporn-forum.ga yurgorod.ru
youporn-ru.com zajm-pod-zalog-nedvizhimosti.ru
yourads.website zaloadi.ru
yourtemplatefinder.com zarabotat-na-sajte.ru
youtube-downloader.savetubevideo.com zebradudka.com
yuarra.pluto.ro zeroredirect6.com
zajm-zalog-krasnodar.ru zeroredirect8.com
zeleznobeton.ru zhuravlev.info
zeroredirect7.com zmoda.hostreo.com
zeroredirect9.com zoominfo.com
zolotoy-lis.ru zvezdagedon.ru
zryydi.com zx6.ru
zytpirwai.net

474
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1008.su 00it.com
51.la 03e.info
aarbur.com 0daymusic.org
acortarurl.es 101raccoon.ru
affrh2015.com 1millionusd.xyz
air-edem.ru 21h2o.com
albamargroup.com 273-fz.ru
alert.scansafe.net 4webmasters.org
all4invest.info addray.pro
alphacarolinas.com adtiger.tk
alvtank.se advokateg.ru
alfapro.ru
alice.it
allsilver925.co.il
allstatesugarbowl.org
allwidewallpapers.com
am-se.com
amyfoxfitness.com amyfoxfitness.com
analytics-ads.xyz androids-store.com
animalia-life.club animaltoplist.com
animalrank.com
api.stathat.com api.stathat.com
apibatbrowsecom-a.akamaihd.net app-ready.xyz
approved.su appfastplay.com
artel116.ru applyneedy.xyz
asdfz.pro ayerbo.xhost.ro
atlasvkusov.ru backiee.com
atmovs.com balans.shahterworld.org
atofilms.com bankmib.ru
audiofree.ru basedecor.ru
aviav.org becuo.com
azlex.uz belinvestdom.by
babieca.com bestwaystogetpaid.us
bankiem.pl
bannerpower.com
barnfurnituremart.com
berlininsl.com
best-way.men
bestempresas.es
bestmarriages.com
betwinservice.com betwinservice.com
bio-japan.net bnt-team.com
bitporno.sx bolitgorlo.net
bizzliving.com bongacams.com
bjorkbacken.nu bonky.biz
blockworld.ru book-bets.com
bochemit.com.ua bosman.pluto.ro
bonkers.name businesn.men
brainzpod.win buy-cheap-pills-order-online.com
braip.com.br buyantiviralwp.com
buyk.host.sk buyclomidonlaine.com
canrioloadm.gq buzzonclick.com
cartierwatch.xyz callawaygolfoutlet.online
cdncash.org carsplate.com
cenokos.ru cete.ru
cheapsergiorossi.online chee-by.biz
ciproandtizanidine.blogspot.com chinaelitecheapjerseys.com
cjs.com.ru citysecurity.nu
clayvasedesigns.tk cleansearch.net
clickbank.net
clickso.com clickso.com
cocolyze.com colehaanoutlet.store
compliance-ivan.xyz com-supportcenter.website
compliance-mary.top
compucelunlock.net compucelunlock.net
conciergegroup.org coolwallpapers-hd.com
connectingsingles.com coolyarddecorations.com
credit-online.ws copro.pw
dating-time-now.com counterbot.com
decenttools.com crackguru.tk
dfiles.me crackzplanet.com
diminishedvalueoforegon.com cranly.net
diy-handmade-ideas.com crirussian.ru
dodlive.mil cubook.supernew.org
downtuptv.gq cvety24.by
drivotracker.com darmebel.com.ua
drupa.com daymusam.com
dtm-spain.com dcj-nn.ru
dustyorate.com dev78.dioniqlabb.se
eets.net digital-sale.su
efkt.jp divatraffic.com
ekspertmed.com dojki365.online
eloxal.ru dumuelave.xyz
enthuse.computernetworksonline.com elkacentr.ru
erotag.com essaypro.com
everypony.ru estelight.ru
evidencecleanergold.com euroskat.ru
evrotekhservis.ru exchangeit.gq
exonline.info expresstoplivo.ru
ezigarettekaufen2.mpbloggar.se extremal-blog.com
f00kclan.de ezigarettenshop2.postbit.com
fandlr.com f-loaded.de
farm26.ru fatfasts-4tmz.com
finejewelryshop.ru fermersovet.ru
firstsiteguide.com ferretsoft.com
fitness-video.net flashlarevista.com
flauntyoursite.com footbalive.org
frameimage.org fotosfotos.eu
freewareseek.com fotoxxxru.com
freshmac.space foxydeal.com
fugarif.ga freejabs.com
fxtips.ru freshsuperbloop.com
fxund.us frustrated-favorable.gq
gateway.zscalertwo.net fullfileaccess.com
gepezz.info g7m.pl
getyourimage.club gamebackyard.com
gheus.altervista.org garciniaxt.us
goforexvps.com gaytube.com
grizzlysgrill.com ge0ip.com
gsasearchenginerankerdiscount.com geoads.com
help.tpu.ru girlsatgames.ru
hostnow.men greatgrace.ru
hoterika.com gsasearchengineranker.space
howlongdoestizanidinestayinyoursystem.blogspot.com gta-club.ru
hyipmanager.in hausfrauensex18.com
iccornacircri.cf hd-film.pl
ifirestarter.ru hdxnxxtube.mobi
ifmo.ru helvetia.com.ua
iframe-toloka.com herokuapp.com
infektsii.com histats.com
homeart.space
hotcore.info
hotelcrocenzi.sm
hothot.ru
hoverboard360.de
hulfingtonpost.com
ideawheel.com
imguramx.pw
insider.pro
investmac.com investmac.com
investyb.com inzn.ru
iplusbit.blogspot.co.za istanbulit.com
iqs.biz.ua izhevsk.zrus.org
ispaniya-costa-blanca.ru janavibekken.no
jagg.info jose.mulinohouse.co
jerseychinabizwholesale.com kchaxton.com
jerseyschinabizwholesale.us
jerseywholesalechinabiz.com
kaac.ru
kamen-e.ru
kenaba.su kenaba.su
keyhantercume.com keywordsuggests.com
keywordblocks.com kino2018.cc
keywords-monitoring-success.com kinosed.net
kino-rating.ru krasivye-devushki.net
kollesa.ru ktotut.net
komp-pomosch.ru ku6.com
konica.kz
koopilka.com
kumuk.info
kvartira-sutochno.com kvartira-sutochno.com
labplus.ru lacloop.info
leadwayau.com
lerporn.info lerporn.info
light.ifmo.ru lignofix.ua
lovi-moment.com.ua lomza.info
lumb.co lumb.co
magda-gadalka.ru m3gadownload.pl
menhealed.net mac-shield.com
metarip.ru marcoislandvacations.net
mezaruk.info martlinker.com
minecraft-rus.org maximilitary.ru
mir-business-24.ru mb140.ru
mistr-x.org meddesk.ru
mlvc4zzw.space megahdporno.net
moonci.ru meget.co.za
morocco-nomad-excursions.com michaelkorssaleoutletonline.net
morpicert.pw mir-betting.ru
mouselink.co mister-shop.com
mnogolok.info
modenamebel.ru
mojaocena.com
moncleroutletonline.win
mosdverka.ru
mozello.ru mozello.ru
mrlmedia.net
musezone.ru
musirc.com musirc.com
my-aladin.com national-today-winning-winner.club
my-bc.ru net-radar.com
namecrumilchlet.tk notfastfood.ru
niroo.info novodigs.com
nottyu.xyz novosti-hi-tech.ru
odoratus.net nsatc.net
offside2.5v.pl olgacvetmet.com
online-x.ru online247.ml
openmediasoft.com ororodnik.goodbb.ru
optitrade24.com outpersonals.com
pateaswing.com painting-planet.com
pc-virus-d0l92j2.pw paparazzistudios.com.au
pharmacyincity.com pattersonsweb.com
phobia.us perosan.com
pixgood.com peugeot-club.org
ploenjitmedia.azurewebsites.net phpdevops.com
plusnetwork.com pijoto.net
porndairy.in pinstake.com
pornoted.com porn-w.org
pornzone.tv porndroids.com
pozdrawleniya.com pornolook.net
predmety.in.ua pornowarp.info
pukaporn.com porto.abuilder.net
puteshestvennik.com princevc.com
qpypcx.com promalp-universal.ru
rankingchart.de pyramidlitho.webs.com
rebrand.ly quebec-bin.com
reckonstat.info ranksays.com
redirect.trafficreceiver.club rapidsites.pro
rent2spb.ru redirlock.com
roleforum.ru robotixix.com
royalcar-ufa.ru sakhboard.ru
rubanners.com scanner-elena.top
saletool.ru scanner-jane.top
scanner-ivan.top schoolfiles.net
scanner-josh.top searchtooknow-a.akamaihd.net
scripted.com seorankinglinks.us
search.xtconnect.com sex-foto.pw
securesmrt-dt.com sexs-foto.com
see-your-website-here.com shemalegalls.blogporn.in
seksvideoonlain.com sherlock.se
sexkrasivo.net skladvaz.ru
sexy-pings.com softxaker.ru
sexytrend.ru sribno.net
sfj-ror.no stockspmb.info
shariki-zuma-lines.ru stop-zavisimost.com
shmetall.com.ua stpicks.com
siteonomy.com suchenindeutschland.com
sky-mine.ru supermodni.com.ua
skytraf.xyz sureone.pro
spy-app.info svetodiodoff.ru
stopnarco.ru tattooha.com
studentguide.ru templates.radiodigital.co
sumo.com tvand.ru
taximytishi.ru twinderbella.com
techart24.com twitlinks.com
thebitcoincode.com ukkala.xyz
themeforest.net ukrtvir.com.ua
tizanidine4mgstreetvalue.blogspot.com unblocksit.es
tizanidinepain.blogspot.com velen.io
tizanidinevscyclobenzaprine.blogspot.com versaut.xxx-cam.webcam
tradedeals.biz viagraneggrx.com
trafficfactory.biz viberdownload10.com
trk-4.net vipromoffers.com
tvteleport.ru vkontarkte.com
u555u.info voltrknc1.com
ugogo.info vozbujdenie.com
ultimateclassicrock.com webads.co.nz
unionmarkt.de webcamtalk.net
unrealcommander.com welck.octopis.com
urlopener.blogspot.com.au wholinkstome.com
video-production.com wikes.20fr.com
vip2ch.com womens-journal.net
vipsexfinders.com xev.ru
vizag.kharkov.ua xkaz.org
w3data.co xn--80aafb2a.xn--p1ai
website-speed-up.top xsfetish.org
wejdz-tu.pl xxxdatinglocal.us
westvilletowingservices.co.za youradulthosting.com
wetgames.ru yourtemplatefinder.com
womensplay.net youtubedownload.org
woodyguthrie.se zahvat.ru
xmlinde.com zebradudka.com
yellowads.men zero1.it
yubikk.info zeroredirect5.com
yur-p.ru zolotoy-lis.ru
zajm-pod-zalog-nedvizhimosti.ru
zarajbuilders.com
zdesoboi.com
zeleznobeton.ru
zeroredirect10.com
zhongwenlink.com
zionstar.net
zlatnajesen.com
zoodrawings.com
zoogdinsney.com
zophim.me zophim.me

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.1.170 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3406 ### Version: V4.2022.10.3407
### Updated: Wed Oct 12 22:01:27 UTC 2022 ### Updated: Thu Oct 13 10:04:21 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3406 ### Version: V4.2022.10.3407
### Updated: Wed Oct 12 22:01:27 UTC 2022 ### Updated: Thu Oct 13 10:04:21 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##