vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 02:29:58 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.02.2943 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-02-25 10:09:01 +00:00
parent 9ae3581ae5
commit bf2f4da203
47 changed files with 88144 additions and 88154 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2942 #### Version: V4.2022.02.2943
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2942 #### Version: V4.2022.02.2943
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.02.2942 #### Version: V4.2022.02.2943
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10882
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.02.2942 #### Version: V4.2022.02.2943
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10886
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10886
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10886
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10886
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 1; 10.1.1.227 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 0; 10.1.1.227 0;
127.0.0.1 0; 127.0.0.1 0;

10886
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 1; 10.1.1.227 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 0; 10.1.1.227 0;
127.0.0.1 0; 127.0.0.1 0;

10886
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 1; 10.1.1.227 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 0; 10.1.1.227 0;
127.0.0.1 0; 127.0.0.1 0;

10886
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2942 2943

10886
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10886
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.14.160 0; 20.185.191.113 0;

10886
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 1; 10.1.1.227 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 0; 10.1.1.227 0;
127.0.0.1 0; 127.0.0.1 0;

10886
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 1; 10.1.1.227 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 0; 10.1.1.227 0;
127.0.0.1 0; 127.0.0.1 0;

10886
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 1; 10.1.1.227 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 0; 10.1.1.227 0;
127.0.0.1 0; 127.0.0.1 0;

10886
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.14.160 0; 20.185.191.113 0;

10886
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.14.160 0; 20.185.191.113 0;

10886
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 1; 10.1.1.227 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Extreme Picture Finder Asterias
GalaxyBot HaosouSpider
Net Vampire Harvest
Netcraft Intelliseek
PECL::HTTP MFC_Tear_Sample
Re-re Pixray
SuperHTTP SocialRankIOBot
V-BOT Xenu
WebWhacker facebookscraper
seostar seoscanners

302
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
ALittle Client 404checker
ASPSeek ADmantX
Aboundex
AfD-Verbotsverfahren
AiHitBot
Alexibot Alexibot
Ankit AlphaBot
Anarchie
Anarchy99
Anthill
Apexoo
Attach
AwarioRssBot
BBBike BBBike
BDFetch BDFetch
BLEXBot
BackDoorBot BackDoorBot
BackStreet BacklinkCrawler
Badass
Bandit
Barkrowler Barkrowler
Battleztar Bazinga BatchFTP
Brandprotect Bigfoot
Buddy Bitacle
Bullseye Blackboard
BunnySlippers BlowFish
CCBot Boardreader
BuiltWith
CODE87 CODE87
CSHttp
CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect CensysInspect
ChinaClaw CheeseBot
Chlooe Chlooe
Cloud mapping Cloud mapping
Cocolyzebot
Cogentbot Cogentbot
Copier Collector
Craftbot CrazyWebCrawler
Crescent Crescent
Custo Custo
DBLBot
DIIbot
DTS Agent
DatabaseDriverMysqli DatabaseDriverMysqli
DigitalPebble Deusu
Discobot Digincore
DnBCrawler-Analytics Dispatch
DomCopBot DittoSpyder
DomainSigmaCrawler DomainAppender
Domains Project ECCP/1.0
Download Wonder
Dragonfly
Drip
EMail Siphon EMail Siphon
EMail Wolf
Ebingbong Ebingbong
Ecxi
EirGrabber EirGrabber
Exabot
Express WebPictures
ExtLinksBot ExtLinksBot
Extreme Picture Finder FemtosearchBot
FDM Fimap
FlashGet FlashGet
Foobot Flunky
Fuzz Fuzz
FyberSpider
Fyrebot Fyrebot
GalaxyBot GetRight
Genieo
GermCrawler
GetWeb GetWeb
Getintent Go!Zilla
Gotit Grafula
GrabNet GrapeshotCrawler
GrapeFX
GridBot
HEADMasterSEO
HMView HMView
HTMLparser
HTTP::Lite
HTTrack
Haansoft
HaosouSpider HaosouSpider
Havij Harvest
Hloader Hloader
IDBTE4M HonoluluBot
Humanlinks
HybridBot
IDBot
IRLbot IRLbot
Image Fetch
Image Sucker Image Sucker
InterGET IndeedBot
Intelliseek
Internet Ninja Internet Ninja
InternetSeer
Iria Iria
Iskanie Iskanie
JamesBOT JOC Web Spider
JennyBot Jbrofuzz
Joomla
Jorgee
JustView JustView
Jyxobot
Kenjin Spider Kenjin Spider
Keyword Density Kozmosbot
Kinza LNSpiderguy
LWP::Simple Lanshanbot
Larbin
Leap Leap
LexiBot LeechFTP
Lftp
Libwhisker Libwhisker
LieBaoFast Lightspeedsystems
Likse
LinkWalker
Linkbot Linkbot
Linkdexbot Linkdexbot
LinksManager LinksManager
LinqiaMetadataDownloaderBot Lipperhey
LinqiaRSSBot Lipperhey Spider
Litemage_walker Litemage_walker
Ltx71 Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc MJ12bot
Majestic SEO MQQBrowser
Majestic-SEO MSIECrawler
MarkMonitor MarkMonitor
MarkWatch
Masscan Masscan
Mata Hari Mata Hari
MauiBot
MeanPath Bot MeanPath Bot
Meanpathbot Meanpathbot
MicroMessenger MegaIndex.ru
Microsoft URL Control Microsoft URL Control
Minefield Minefield
Mojeek Mojeek
Mojolicious Mojolicious
MolokaiBot MolokaiBot
Mozlila Mr.4x3
Name Intelligence Musobot
NICErsPRO
NPbot
Nameprotect
Navroad Navroad
Needle NearSite
Nessus Net Vampire
NetAnts NetMechanic
NetLyzer NetSpider
Nettrack NetZIP
Netvibes
Nibbler
Niki-bot Niki-bot
NimbleCrawler Nmap
Nimbostratus Nuclei
Ninja
Nutch Nutch
OpenLinkProfiler Offline Explorer
Openvas OrangeSpider
OutclicksBot
PECL::HTTP PECL::HTTP
PHPCrawl
Page Analyzer Page Analyzer
PageAnalyzer PageAnalyzer
PageGrabber
PageScorer
PageThing.com
Papa Foto Papa Foto
Pavuk Petalbot
Picscout PictureFinder
Piepmatz
Pixray Pixray
Pockey
Probethenet Probethenet
Pu_iN Pu_iN
Pump Pump
PxBroker QueryN Metasearch
PyCurl
RSSingBot
RankActive
RankActiveLinkBot RankActiveLinkBot
Reaper RankurBot
RocketCrawler Re-re
SBIder RedesScrapy
SEOkicks SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SalesIntelligent SEOprofiler
SISTRIX
SMTBot
ScanAlert
Scanbot Scanbot
Scrapy ScoutJet
SentiBot Screaming
Shodan SearchmetricsBot
Siphon Seekport
Site Sucker SemanticJuice
SiteCheckerBotCrawler SemrushBot
SiteExplorer
SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker
Sitebeam
Siteimprove Siteimprove
Sitevigil
SmartDownload SmartDownload
Snapbot SocialRankIOBot
Sogou web spider Sogou web spider
Sottopop Sottopop
Spammen
SpankBot SpankBot
Spbot
SputnikBot SputnikBot
Sqlmap Sqlmap
Sqlworm Sqlworm
Sucuri Sqworm
SuperHTTP Sucker
Suzuran Surfbot
Szukacz Szukacz
T0PHackTeam T0PHackTeam
Telesphoreo TeleportPro
TightTwatBot Telesoft
Toata Telesphorep
The Intraformant
Titan
Trendiction Trendiction
Turnitin True_Robot
URLy Warning
UnisterBot UnisterBot
Upflow
VB Project VB Project
VCI
Vacuum
VidibleScraper VidibleScraper
VoidEYE Voil
WASALive-Bot WBSearchBot
WEBDAV
WISENutbot WISENutbot
WWW-Mechanize Wallpapers
Wallpapers/3.0
WeSEE WeSEE
Web Enhancer
WebAuto
WebCollage WebCollage
WebCopier
WebPix WebPix
WebSauger WebSauger
WebStripper
WebSucker WebSucker
WebBandit WebmasterWorldForumBot
WebEnhancer
WebGo IS
WebReaper
WebSucker
Webalta
WebsiteQuester WebsiteQuester
Webster Webster
Whacker Whacker
Wonderbot Whatweb
Woobot Widow
Wprecon WinHTTrack
Xaldon WebSpider Zade
YoudaoBot Zermelo
ZoominfoBot Zeus
ZumBot ZoomBot
archive.org_bot
arquivo.pt
coccocbot coccocbot
crawler4j cognitiveseo
dataforseo.com
demandbase-bot
domainsproject.org domainsproject.org
evc-batch
gopher gopher
instabid heritrix
ips-agent ips-agent
isitwp.com iubenda-radar
lwp-trivial magpie-crawler
meanpathbot mediawords
page scorer oBot
pcBrowser
plumanalytics plumanalytics
polaris version
probe-image-size
ripz ripz
satoristudio.net
scan.lol
seobility seobility
seocompany.store seoscanners
sexsearcher serpstatbot
siteripz sogouspider
sp_auditbot sp_auditbot
spyfu
sysscan
tAkeOut tAkeOut
trendiction.de webpros.com
ubermetrics-technologies.com
webgains-bot
x09Mozilla x09Mozilla
x22Mozilla zauba.io

308
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker
404enemy 404enemy
80legs 80legs
ADmantX
Abonti Abonti
Aboundex Aboundex
Aboundexbot
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
Anarchie AllSubmitter
Anarchy99 Atomseobot
Ankit AwarioSmartBot
BBBike BBBike
BackDoorBot BDCbot
BackWeb BLEXBot
BackStreet
Backlink-Ceck Backlink-Ceck
Badass
Bandit
Barkrowler Barkrowler
Bigfoot BetaBot
Bitacle Black Hole
BlackWidow BlackWidow
Blackboard BlowFish
Blow
Boardreader Boardreader
Buck Bolt
BuiltBotTough BotALot
BuiltWith Buddy
BunnySlippers CATExplorador
BuzzSumo
CODE87 CODE87
CazoodleBot Calculon
Cegbfeieh
CensysInspect CensysInspect
CheTeam
CheeseBot CheeseBot
CherryPicker Chlooe
CopyRightCheck Claritybot
Copyscape Cliqzbot
Cosmos Cocolyzebot
Craftbot Crescent
Crawling at Home Project
CrunchBot
Curious Curious
Deusu DatabaseDriverMysqli
Demon
Devil
Digincore
DigitalPebble DigitalPebble
Dirbuster Dirbuster
DittoSpyder Discobot
Discoverybot
Dispatch
DnBCrawler-Analytics
DomCopBot
DomainCrawler
DomainSigmaCrawler
Domains Project Domains Project
Dotbot Dotbot
Download Wonder
Dragonfly
Drip Drip
ECCP/1.0 EMail Wolf
EasyDL
EroCrawler EroCrawler
Evil Ezooms
Exabot FHscan
Express WebPictures
Extreme Picture Finder
EyeNetIE
FemtosearchBot FemtosearchBot
Firefox/7.0 FlashGet
Foobot Flunky
Freeuploader
FrontPage FrontPage
Fuzz Fuzz
G-i-g-a-b-o-t FyberSpider
GalaxyBot GT::WWW
GermCrawler GermCrawler
GetWeb GetWeb
Getintent Gotit
Gigabot
Go!Zilla
Go-Ahead-Got-It
GrabNet GrabNet
GrapeFX GrapeshotCrawler
HaosouSpider HEADMasterSEO
HTTP::Lite
HTTrack
Havij Havij
HonoluluBot Humanlinks
Id-search IRLbot
IlseBot IlseBot
InfoNaviRobot Image Sucker
IndeedBot
Indy Library
InfoTekies InfoTekies
Intelliseek
InterGET InterGET
Internet Ninja Jbrofuzz
InternetSeer JennyBot
Iria
Iskanie
IstellaBot
JOC Web Spider
JamesBOT
JetCar JetCar
Kenjin Spider Jetty
Jorgee
JustView
Jyxobot
Keyword Density Keyword Density
Kozmosbot Kozmosbot
LNSpiderguy
Larbin
Leap Leap
LeechGet
LexiBot LexiBot
Lftp LibWeb
Lightspeedsystems
Likse
Linkbot Linkbot
LinkextractorPro LinkextractorPro
LinkpadBot LinkpadBot
LinqiaRSSBot
Lipperhey
Lmspider Lmspider
Ltx71
MIDown tool MIDown tool
MJ12bot Magnet
MQQBrowser
MTRobot
Mag-Net
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic SEO
MarkMonitor Majestic-SEO
Majestic12
MarkWatch MarkWatch
Mata Hari Masscan
MauiBot Mb2345Browser
MeanPath Bot
Meanpathbot Meanpathbot
MegaIndex.ru Mediatoolkitbot
Moblie Safari Metauri
Morfeus Fucking Scanner Microsoft URL Control
Name Intelligence Mister PiX
NearSite Mojeek
Nessus MolokaiBot
Mr.4x3
NICErsPRO
NPbot
Nameprotect
NetAnts NetAnts
NetSpider Netvibes
NetZIP Nibbler
Nettrack NimbleCrawler
Niki-bot
Ninja Ninja
Offline Navigator Nmap
OrangeBot Nuclei
OutclicksBot PHPCrawl
PECL::HTTP
PageAnalyzer PageAnalyzer
PageGrabber Papa Foto
Picsearch PeoplePal
Petalbot
Pi-Monster
Picscout
PictureFinder
Pixray Pixray
Pu_iN PleaseCrawl
PxBroker Pockey
ProPowerBot
ProWebWalker
Psbot
Pump
QueryN Metasearch QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot
RankActive
RankActiveLinkBot RankActiveLinkBot
RankFlex RankFlex
RankingBot2 RankingBot2
RankurBot
Re-re Re-re
RealDownload
Reaper Reaper
Recorder RebelMouse
RedesScrapy RedesScrapy
Ripper RepoMonkey
RocketCrawler Rogerbot
SEOkicks
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler
SMTBot
SalesIntelligent
Scanbot Scanbot
ScreenerBot ScoutJet
ScrepyBot Scrapy
Screaming
SearchmetricsBot
Seekport
SemanticJuice SemanticJuice
SemrushBot
SentiBot
SeobilityBot SeobilityBot
Siphon Seomoz
Site Sucker
SiteLockSpider SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker
Sitebeam Sitebeam
Sitevigil Sitevigil
Sogou web spider Snake
Spammen Snoopy
Spbot SpankBot
Sqworm Sqlmap
SuperBot Sqlworm
SuperHTTP Sucker
Surfbot
SurveyBot SurveyBot
Suzuran T8Abot
Swiftbot
Szukacz
Teleport Teleport
TeleportPro Telesoft
Telesphoreo TightTwatBot
Telesphorep
Titan
Toata Toata
True_Robot Tracemyfile
Turnitin Trendiction
UnisterBot Trendictionbot
TurnitinBot
TwengaBot
Twice
URLy.Warning
VB Project VB Project
VCI Vacuum
Vagabondo
VelenPublicWebCrawler
VidibleScraper VidibleScraper
Voil Voil
Voltron
WASALive-Bot WASALive-Bot
WEBDAV WEBDAV
WPScan WPScan
WWW-Collector-E WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers
WallpapersHD
WeSEE WeSEE
Web Auto
Web Collage Web Collage
Web Enhancer Web Enhancer
Web Fetch Web Sauger
Web Sucker WebCopier
WebAuto
WebBandit
WebEnhancer WebEnhancer
WebFetch
WebFuck WebFuck
WebGo IS WebGo IS
WebPix
WebSucker
WebZIP WebZIP
Webshag
Website Quester
WebsiteExtractor WebsiteExtractor
Webster Webster
Whack Whacker
Whatweb Whatweb
Who.is Bot Woobot
Wonderbot
Wotbox
Wprecon Wprecon
Xaldon WebSpider Xaldon WebSpider
Xenu Xaldon_WebSpider
Zade Zade
Zermelo
Zeus
Zitebot
ZmEu
ZoomBot ZoomBot
adscanner archive.org_bot
arquivo-web-crawler arquivo.pt
cah.io.community check1.exe
com.plumanalytics com.plumanalytics
crawler.feedback
dataforseo.com dataforseo.com
eCatch heritrix
evc-batch
instabid instabid
ips-agent
iubenda-radar
lwp-request lwp-request
lwp-trivial satoristudio.net
magpie-crawler scalaj-http
oBot
probe-image-size
scan.lol scan.lol
seobility seobility
seocompany.store
seoscanners
seostar seostar
serpstatbot sexsearcher
siteripz
sp_auditbot
tAkeOut
trendiction.de trendiction.de
webmeup-crawler webgains-bot
xpymep1.exe
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
compliance-ivan.xyz cfjump.com
filesclub.net fuckingawesome.com
jurajskie.info iloveitaly.ru
liveinternet.ru par-fallen.ga
metallosajding.ru partnerads.men
porto.abuilder.net py100.ru
soda.media rabotaetvse.ru
traderzplanet.in tizanidinedosageforsleep.blogspot.com
your-bearings.com xtraffic.plus
zoomovies.org yougotanewdomain.com

922
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com 0lovespells0.blogspot.com
10bet.com 1001desert.com
1bet.com 123cha.com
1hwy.com 14b.info
202ch.com 1j7740kd.website
3weekdiet.com 2020iscoming.info
4kplayer.pl 3-letter-domains.net
51unlim.ru 4istoshop.com
9icmzvn6.website 4ureyesonly.com
9med.net 4webmasters.com
aac500b7a15b2646968f6bd8c6305869d7.com 4webmasters.org
aac52006ec82a24e08b665f4db2b5013f7.com 5elementov.ru
abcguru.xyz 5u.com
activepr.ru 7zap.com
adclickthru.net 8kisses.eu
adloads.net 991mostfm.co.id
adsloads.com aa14ab57a3339c4064bd9ae6fad7495b5f.com
adultfriendfinder.com aaronabel.com
advokateg.ru acc.eu.org
agahinameh.com ad-words.ru
alarmobninsk.ru adman.gr
ald2014.org admanaerofoil.com
alive-ua.com admitad.com
alldubai.biz adpremium.org
allmarketsnewdayli.gdn adsland.men
allpdfmags.net adtech.fr
alphavisions.net adtology.com
alta-realestate.com advocatemsk.ru
ageofclones.com
air-edem.ru
alevinefesleri.com
alpinism.ru
am-se.com
amehdaily.com amehdaily.com
ananumous.ru amung.us
anaseracresar.tk amyfoxfitness.com
angkortours.vn android-vsem.org
angry-fermi-7633.arukascloud.io animaltoplist.com
anonymousfox.co apibetweenlinesn-a.akamaihd.net
apartmentratings.com apple.com-webbrowsing-security.review
api.stathat.com arabgirls.adultgalls.com
apiholdingmypage-a.akamaihd.net architecturebest.com
apper.de arcteryxstore.online
artel116.ru
asdfg.pro asdfg.pro
asophoto.com autoplate.info
astana.xxxkaz.org aviapanda.ru
atelielembrancaqueencanta.com.br avirasecureserver.com
atleticpharm.org aviva-limoux.com
autobudpostach.club avtolombard-krasnodar.com
autorn.ru azazaporn.com
avon-ukraine.com babyguns.ru
avramstroy.ru bamo.xsl.pt
azadnegar.com bangla-chat-uk-paralud.ga
baladur.ru bankofthewext.com
balkanfarma.org basedecor.ru
banki76.ru battle.net
bankiem.pl bear.gotcher.us
battlecarnival.su belsetirehafi.tk
bazaronline24.ru
bbsoldes.fr
beachdriveblog.com
beachpics.com
bedcapdealers.com
beetpics.pw
belinka.com.ua
best-deals-products.com best-deals-products.com
bestbrainprod.win bestmobilityscooterstoday.com
bestprofits-there.com bestofpicture.com
bestserials.com bet365.com
betterscooter.com beyan.host.sk
bharatdefencekavach.com bezsporno.ru
biketank.ga bigpictures.club
biocn.dx.am bingoporno.com
biteg.xyz
blackhatworth.com
blacktwhite.com
blogrankers.com blogrankers.com
bloke.com blogs.rediff.com
bonkers.name bnt-team.com
bookhome.info bolitgorlo.net
bpro1.top bradleylive.xyz
braintobe.win braip.com.br
bryansk.zrus.org brakehawk.com
buketeg.xyz brateg.xyz
burger-imperia.com brus-vsem.ru
business-suggested.tk builtwith.com
businesxxl.com bumskontakte.org
buy-cheap-pills-order-online.com callmd5map.com
buzz-porno.info candypeople.se
c-english.ru cannibalcheerleader.com
cacheimages.com canvas.gloverid.site
callawaygolfoutlet.online canyougethighofftizanidine.blogspot.com
cand.jp capsderfudd.tk
carina-sy.de cardsharp1.ru
cartechnic.ru cartierlove2u.com
cartierbracelet.xyz cartierreplica.win
cartierlovebraceletreplica.xyz cbb1smartlist12.click
cfacarrosserie74.com cenokos.ru
checktext.org cezartabac.ro
chinacheapelitejerseys.com chcu.net
cheapbelstaff.online
cheapestjerseysonwholesale.com
cheapmarmot.online
checkm8.com
christianlouboutinreplica.pw christianlouboutinreplica.pw
ci.ua christianlouboutinreplica.win
citetick.com cicaki.net
cineacademy.ru
citizenclsdriveri7.pen.io citizenclsdriveri7.pen.io
clayvasedesigns.tk cleanallspyware.com
codq.info clickcash.com
coindirect.io
colobit.biz
cometorussia.net
commoncrawl.org commoncrawl.org
companies-catalog.com compliance-brian.xyz
compliance-elena.xyz compliance-john.top
compliance-fred.top compliance-julianna.top
compliance-josh.top compliance-walt.top
compliance-mark.top computernetworksonline.com
comsysnet.com
concordexoticrentals.com concordexoticrentals.com
continent-e.tv coolwallpapers-hd.com
cookielawblog.wordpress.com corta.co
countbertwistdisp26.soup.io costablue.xyz
covid-schutzmasken.de counter.yadro.ru
cowlmash.com country-chic.ru
crafthubs.com crazyprotocol.com
crest3d.ru creativehutindia.com
crynet.cc custom-electric-guitar.com
cubs-jerseys.us customcatchcan.com
culturevie.info cv.wallhade.co
dailyrank.net dandiyabeats.in
danashop.ru danschawbel.com
daptravel.com daydream-studio.ru
daretodonate.co decorationspcs.com
darmebel.com.ua detki-opt.ru
datract.com deutschehobbyhuren.net
ddospanel.com
ddsoldes.fr
dealwifi.com
dear-diary.ru
delfin-aqua.com.ua
derevesendeco.com
devochki.top devochki.top
dfwu1013.info diy-handmade-ideas.com
dichvuvesinhhanoi.com djihispano.com
domain-submit.info dnepr-avtospar.com.ua
domain.webkeyit.com doctissimo.fr
domainsatcost.com documentsite.net
domoysshop.ru dogclothing.store
domznaniy.ru domaineaneblanc.com
domcran.net
domik-derevne.ru
dominterior.org
donna7753191.ru donna7753191.ru
dorratex.tn dostavka-v-krym.com
doxysexy.com dprtb.com
duawitchrarato.tk drupa.com
dustyorate.com druzhbany.ru
dverimegapolis.ru dscaas.website
dvrlists.com e90post.com
elektrischezi.canalblog.com ekspresihati.info
elitepcgames.com elektrischezigarette1.blog.pl
elvenar.com elektrischezigarettekaufen2.cowblog.fr
embedle.com elidelcream.weebly.com
en.home-task.com elitesportsadvisor.com
enpolis.ru en.altezza.travel
enews.tech
escort-russian.com escort-russian.com
escortslet.net este-line.com.ua
eshop.md excaliburfilms.com
exci.ru executehosting.com
expdom.com
ezigarettekaufen.myblog.de
ezigarettekaufen1.hpage.com ezigarettekaufen1.hpage.com
f-online.de ezigarettekaufen2.mpbloggar.se
f5mtrack.com ezigarettenkaufen1.deviantart.com
fable.in.ua ezigarettenkaufen2.dreamwidth.org
falllow.gq ezigarettenshop1.yolasite.com
fantasticpixcool.com f-loaded.de
falopicm.pw
fashion.stellaconstance.co
fashionavenuegame.com fashionavenuegame.com
fealq.com fastfixing.tech
finejewelryshop.ru feargames.ru
fishingwholesale.us femmesdenudees.com
flauntyoursite.com filesdatabase.net
filesvine.com
filmi-v.online
finstroy.net
fishtauto.ru
flash4fun.com
flex4launch.ru flex4launch.ru
flprog.com floridamhca.org
focalink.com flytourisme.org
forex.osobye.ru forex.osobye.ru
formulaf1results.blogspot.com forms-mtm.ru
forpostlock.ru foxinsocks.ru
fotosfotos.eu frameimage.org
fr-bearings.ru
fr.netlog.com
frbizlist.com frbizlist.com
freakycheats.com free-social-buttons.xyz
free-laptop-reward.com freshsuperbloop.com
fuckbuddybestgilf.info front.to
funponsel.com frustrated-favorable.gq
funtoonez.com gael-s.ru
gamedayhouse.com gambarkatabaru.com
gameplexcity.com game-mmorpg.net
gameprimary.com gamesprite.me
gateway.zscalerone.net geileweiber.tk
gcup.ru genetworx.com
gdebestkupit.ru girlsatgames.ru
ge0ip.com gk170.ru
ge0ip.org glavtral.ru
geckoandfly.com globalscam.ga
getresponse.com glogow.pl
gigapeta.com goldenggames.com
girlporn.ru golfresa.lucania.se
glall.ru goodwinmetals.co
go2album.com graphid.com
goldadpremium.com gsasearchengineranker.pw
golden-catalog.pro gsasearchengineranker.top
goo.ne.jp guiadeserraazul.com
goodhousekeeping.com gymi.name
gosreg.amchs.ru habermetre.com
gotcher.us hasshe.com
grtyi.com hdwallpapers-free.com
hamptonoaks.ca helpmymacfaster.trade
hatdc.org hkladys.com
hebr.myddns-flir.com homeart.space
himazin.info hostingclub.lk
hobild.net hoverboardforsaledirect.com
hoholikik.club hundejo.com
home.myplaycity.com hustoon.over-blog.com
hotcore.info ibmdatamanagement.co
housediz.com id-forex.com
houseofgaga.ru ideawheel.com
hoverboard360.se ideibiznesa2015.ru
howopen.ru impotentik.com
hunthillfarmtrust.org inboundlinks.win
i-hobot.ru inboxdollars.com
iamsport.org inclk.com
icaseclub.ru indetiske.ya.ru
ie.57883.net infektsii.com
igadgetsworld.com insta-add.pro
ilo134uloh.com investyb.com
imagerydatabase.com invitefashion.com
imgarit.pw inzn.ru
iqbazar.ru istanbulit.com
iqoption.com itbc.kiev.ua
it-max.com.ua ivoiretechnocom.ci
janerikholst.se jerseywholesaleelitestore.com
jobgirl24.ru jewelryandfiligree.com
jose.mulinohouse.co jjbabskoe.ru
justdating.online jrcigars.com
kakablog.net justkillingti.me
kambasoft.com jyvopys.com
kandidos.com kasino-money.pw
karpun-iris.ru keywesthideaways.co
keenoutlet.online keywordbasket.com
ketrzyn.pl kichenaid.ru
keyword-suggestions.com kino-filmi.com
keywords-monitoring-success.com kinosed.net
keywordsdoctor.com klikbonus.com
keywordspay.com kosmetyki.tm.pl
kino-ecran.ru kretpicf.pw
kino-rating.ru
kmd-pto.ru
kmgamex.cf
komputernaya-pomosh-moscow.ru
kountrylife.com
kovesszucs.atw.hu
krasnodar.xrus.org
kurwa.win
lacasamorett.com
ladov.ru
ladsblue.com
ladsup.com ladsup.com
lanadelreyfans.us lampokrat.ws
landscapebackgrounds.blogspot.com lancheck.net
laptoper.net legionalpha.com
law-check-seven.xyz lenpipet.ru
lennatin.info letto.by
lescinq.com light.ifmo.ru
likesdesign.com liran-locks.com
limtu.ifmo.ru livejasmin.com
linkrr.com ljusihus.se
lion.bolegapakistan.com lombardfinder.ru
liverpool.gsofootball.com lonely-mature.com
loadopia.com longlifelomilomi.info
locatellicorretor.com.br lotto6888.com
lonfon.xyz louboutinreplica.xyz
lookingglassemb.com lovelycraftyhome.com
lottospring.com lowephotos.info
lunchrock.co ltvperf.com
machicon-ueno.info m3gadownload.pl
mainhp.net
make-money-online.com
mall.uk mall.uk
manyresultshub-a.akamaihd.net manyresultshub-a.akamaihd.net
masserect.com matrixalchemy.com
maximpartnerspr.com mebel-alait.ru
medi-fitt.hu medicinacom.ru
medispainstitute.com.au medicovi.com
meltwater.com mejoresfotos.eu
members.ghanaweb.com mere.host.sk
metalonly.info micasainvest.com
methodsmarketing.com michaelkorsoutletstore.net
minecraft-neo.ru
minet.club minet.club
minneapoliscopiers.com miniads.ca
moesonce.com misswell.net
mojowhois.com mixtapetorrent.com
mobplayer.net
mojpregled.com
monclercoats.xyz monclercoats.xyz
moncleroutletonline.xyz mosdverka.ru
money-for-placing-articles.com most-kerch.org
morf.snn.gr moyakuhnia.ru
msk-diplomat.com mug-na-chas-moscow.ru
mttwtrack.com
musicas.baixar-musicas-gratis.com musicas.baixar-musicas-gratis.com
my-bc.ru mwtpludn.review
my-cash-bot.co my-aladin.com
mydownloadengine.com myblogregistercm.tk
myfreecams.com mydirtyhobby.com
myliveblog.ru mylesosibirsk.ru
myonlinepayday.co nash-krym.info
myperiod.club nastroyke.net
mypornfree.ru navalwiki.info
myra.top nerudlogistik.ru
namecrumilchlet.tk news-readers.ru
national-today-winning-winner.club newyorkhotelsmotels.info
news-speaker.com next-dentists.tk
newstudio.tv nfljerseys.online
nfvsz.com nhl09.ru
no-rx.info nhl17coinsforps3.gratisblog.biz
nomuos.it nikitsyringedrivelg.pen.io
npoet.ru nottyu.xyz
nzfilecloud.weebly.com noumeda.com
obiavo.com novgorod.xrus.org
obiavo.ru o00.in
offside2.5v.pl offer.wpsecurity.website
okonich.com.ua office-windows.ru
online-hit.info onlineserialy.ru
online247.ml onlineslotmaschine.com
onlinedomains.ru openstat.com
onlyforemont.ru optitrade24.com
onlyporno.ru p-business.ru
openfrost.com paccohichetoti.ml
opinionreelle.com pacificair.com
outrageousdeal-a.akamaihd.net
paintingplanet.ru paintingplanet.ru
paleohub.info paparazzistudios.com.au
partners-ship.pro parajumpersoutlet.online
perso.wanadoo.es partnerworkroom.men
peugeot-club.org pastaleads.com
phobia.us pc-test.net
pinwallpaper.top pdns.cz
pixgood.com pdns.download
pizdopletka.club pearlisland.ru
pkr1hand.com perm.xrus.org
play-mp3.com phelissota.xyz
photostudiolightings.com
phrcialiled.com
picphotos.net
picsforkeywordsuggestion.com
picture-group.com
picturesboss.com
pigrafix.at
piluli.info
pk-pomosch.ru
playtap.us playtap.us
polimga.pw pochtovyi-index.ru
popup-hgd.xyz polytopesexempt.com
porno-play.net pons-presse.com
porno-raskazy.ru popup.matchmaker.com
prchecker.info porndl.org
pricheski-video.com pornhubforum.tk
prizefestival.mobi pornoblood.com
pornofeuer.com
pornoplen.com
pornosee.info
portal-eu.ru
pourvous.info
powitania.pl
pr-ten.de
pravoholding.ru
privacyassistant.net
prizesbook.online prizesbook.online
prodess.ru
productarium.com productarium.com
profitkode.com prointer.net.ua
progonrumarket.ru promoheads.com
prombudpostach.com.ua
pronorm.fr
prostitutki-belgoroda.org
prostitutki-yaroslavlya.org prostitutki-yaroslavlya.org
rada.ru quizzitch.net
rapidokbrain.com r-control.ru
rapidsites.pro rabotaetvse.ru
rating-bestcasino.com rarbg.to
recordpage-a.akamaihd.net realmonte.net
relax.ru realtytimes.com
remote-dba.de rebuildermedical.com
rentalcarnavi.info reining.lovasszovetseg.hu
rethinkwasteni.info remont-comp-pomosh.ru
rialp.getenjoyment.net rentaremotecomputer.com
rimedia.org respublica-otel.ru
robot-forex.biz reversing.cc
roznica.com.ua reyel1985.webnode.fr
rrutw.com rossanasaavedra.net
rukino.org ru-mediaget.ru
runstocks.com runstocks.com
rus-teh.narod.ru sady-urala.ru
russkoe-zdorovie.ru samo-soznanie.ru
rvtv.ru sanatorrii.ru
sack.net sarahmilne.top
salmonfishingsacramentoriver.com savefrom.com
samolet.fr scanner-jack.top
sbf441.com scanner-jessica.top
scanner-mark.top scanner-walt.top
scmor.ilxc.cc
scottbywater.com
search.1and1.com
search.alot.com
searchencrypt.com searchencrypt.com
seofied.com seeresultshub-a.akamaihd.net
serdcenebolit.com serdcenebolit.com
sergiorossistore.online servisural.ru
sexblog.pw sexgalleries.top
sexobzor.info sexy-pings.com
sexsaoy.com sexy-screen-savers.com
sexstream.pl sexystrippe.info
shodanhq.com sexyteens.hol.es
shop.acim.org sexytrend.ru
shopsellcardsdumps.com shop.xz618.com
shopvilleroyboch.com.ua sigmund-freud.co.uk
sientalyric.co silktide.com
silvercash.com sim-service.net
site-analyzer.com sinel.info
slonechka.ru sitevalued.com
smart-scripts.com skuteczna-dieta.co.pl
slim.sellany.ru
smartpet.ru
smichovbike.cz
smsactivator.ru
snjatie-geroinovoy-lomki.ru
snsdeainavi.info snsdeainavi.info
social-s-hhh.xyz socas.pluto.ro
socialbutton.xyz social-s-iii.xyz
sofit-dmd.ru social-vestnik.ru
solmarket.by socialtrade.biz
softonicads.com
solitaire-game.ru
solution4u.com solution4u.com
songplanet.ru spidtest.org
sosdepotdebilan.com sstroy44.ru
spinazdrav.ru stairliftstrue.com
sprttrack.com stard.shop
staff.prairiesouth.ca
stauga.altervista.org
streha-metalko.si streha-metalko.si
suhanpacktech.com strigkaomsk.ru
sumo.com studiokamyk.com.pl
superfish.com stuff-about-money.com
superiends.org suggest-keywords.com
supervesti.ru supermesta.ru
surffoundation.nl tacbibirfa.tk
swinon.site tamada69.com
tanieaukcje.com.pl
tasteidea.com tasteidea.com
templates.radiodigital.co teencastingporn.com
terrafootwear.us telegramdownload10.com
the-trader.net tengohydar.tk
thebluffs.com teslathemes.com
thexart.club thaismartloan.com
timer4web.com thenetinfo.com
tizanidineformigraines.blogspot.com thepantonpractice.co.uk
tizanidinehcl2mg.blogspot.com tizanidineandgabapentin.blogspot.com
tizanidineingredients.blogspot.com tizanidinehcl4mgtab.blogspot.com
tmtrck.com tizanidinehcl4mgtabinfo.blogspot.com
toloka.hurtom.com tizanidineinteractions.blogspot.com
tomck.com tomck.com
top-deal.com.pl top-l2.com
topbestgames.com top250movies.ru
topkarkas.com topcasinoratings.ru
topmira.com toyota.7zap.com
torrents.life tracxn.com
toy-shop.top traffic2money.com
tozup.com trahic.ru
track2.shop trion.od.ua
traffictrade.life tt-ipd.info
trahvid.com tubeline.biz
truck-land.life tuberkulezanet.ru
tube8.com turvgori.ru
tuckermktg.com tversvet.ru
tvory.predmety.in.ua twiclub.in
tx41tclega.ru ua-company.ru
txxx.com
typimga.pw
u17795.netangels.ru
uac.net.au
uasb.ru uasb.ru
uggsale.online ucoz.ru
ultimateclassicrock.com upproar.com
unlimitdocs.net urdoot.win
url-img.link usadacha.net
v-doc.co vashsvet.com
vekzdorov.ru videochat.world
veopornogratis.xxx viktoria-center.ru
viagengrarx.com vilingstore.net
vip-parfumeria.ru virus-respirators.com
vkak.ru volume-pills.biz
voronezh.xrus.org voprosotvet24.ru
watchdogs-2.ru waterpurifier.club
webads.co.nz webmasters.stackexchange.com
webs.com weburok.com
website-speed-up.top wesharepics.site
wetgames.ru wetgames.ru
whatsappdownload10.com while.cheapwebsitehoster.com
whengirlsgowild.com whiteelephantwellington.com
whereverdesperate.gq win-spy.com
works.if.ua windowssearch-exp.com
worldtraveler.world wingsofrefuge.net
www.ehscloud.cn woman-tampon.ru
wormix-cheats.ru
wurr.voila.net
www.souvenirua.com www.souvenirua.com
wzgyyq.com wyniki-lista.pl
x-diesel.biz xaylapdiendanang.com
xcombear.ru xjlottery.com
xfluro.com xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xolodremont.ru
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai xpresscare.ru
xn--80ab4aa2g.xn--p1ai xuki.us
xzlive.com xvideospornoru.com
youandcredit.ru xxart.ru
youghbould.wordpress.com ynymnwbm.bloger.index.hr
youraticles.pl youporn-forum.ga
yourporn.com zarabotok--doma.ru
youtuhe.com
ytmnd.com
yugo-star.ru
zaidia.xhost.ro
zero1.it zero1.it
zeroredirect5.com zeroredirect6.com
zlatnajesen.com zhorapankratov7.blogspot.com
zoogdiesney.com znakom.sibtest.ru
zoominfo.com
zootoplist.com
zscaler.net
zscalerone.net
zscalertwo.net
zvuker.net

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
12bet.com 4ureyesonly.com
1hwy.com 5i2.net
40cg.com 70casino.online
7fon.ru a96527gi.beget.tech
991mostfm.co.id activecampaign.dreamhosters.com
aa625d84f1587749c1ab011d6f269f7d64.com ad-words.ru
adconscious.com addbin.men
adloads.net advancedsoftwaresupport.com
afslankpillen2017nl.eu agadelha.com.br
agreda.pluto.ro aihelen.net
alert-fjg.xyz alborzan.com
amazon-adsystem.com allmarketsnewdayli.gdn
amung.us amazingninja.com
android-systems.ru api.stathat.com
anlimebel.kiev.ua apioasisspacenet-a.akamaihd.net
apiadanaknet-a.akamaihd.net aquarium-pro.ru
apiluckyleapnet-a.akamaihd.net arcteryxsale.online
appleid-verification.com autobudpostach.club
asdfg.pro awaybird.ru
babyfactory.fr basisches-wasser.net
bensbargains.net beatifullandscape.co
bestmarriages.com beauty-bracelet.com
big-boards.info bimatoprost-careprost.com
big-cash.net blackbabesporn.com
bimatoprost-careprost.com.ua bolegapakistan.com
bioinnovate.co bonky.biz
bjetjt.com brains2.biz
blackle.com brandehk.dk
blogos.kz bum.com.ru
bolitgorlo.net buy-meds24.com
brokergid.ru buyessay3.blogspot.ru
bryansk.zrus.org buyhoverboard.com
buttons-for-website.com buzzzg.men
buttons-for-your-website.com cah.io.community
callstevens.com callstevens.com
canacopegdl.com
canvas.gloverid.site
carapics.com
cartierring.xyz cartierring.xyz
cbb1smartlist12.click ccbill.com
cctva.tv cellfun.mobi
centre-indigo.org.ua chocolateslim-u-srbiji.com
chatroulette.online cialischmrx.com
chcu.net cigarpass.com
cheapcigarettesc.info cityadspix.com
cheatcode-lita12.rhcloud.com clicksor.com
chiptuninger.com com-onlinesupport.host
cialisndbrx.com compliance-walter.top
cinemaflix.website connectionstrenth.com
ciproandtizanidine.blogspot.com contextualyield.com
classicakuhni.ru crackguru.tk
comissionka.net dangphoto.trade
compliance-alex.xyz davebestdeals.com
compliance-barak.xyz daymusam.com
concordexoticrentals.com diy-handmade-ideas.com
coolyarddecorations.com dnsrsearch.com
craftinsta.ru domik-derevne.ru
cryptoswap.biz dumuelave.xyz
dentfidemountpreach.tk
detective01.ru
diamond-necklace.info
divatraffic.com
divku.ru
domashneeruporno.com
draniki.org
dtnlyss.com
duplicashapp.com duplicashapp.com
electrik-avenue.com elektrischezi.canalblog.com
energydiet24.ru elektrischezigaretten2.beeplog.com
estibot.com elektronischezigarette2.mex.tl
exmasters.com elektrozigaretten1.postbit.com
ezaz.info enge-fotzen.info
fapgon.com enpolis.ru
ferieboligkbh.dk eralph.tk
fickenprivat.info erectile.bid
finejewelryshop.ru erotikstories.ru
firesub.pl eu-cookie-law.blogspot.com
fix-website-errors.com ewebarticle.info
focalink.com falopicm.pw
forex21.ru fearcrow.com
forum.doctissimo.fr filmania-x.ru
frameimage.org findanysex.com
free-share-buttons.blogspot.com fireads.men
free-today.com flashahead.info
freelifetimefuckbook.com flex4launch.ru
freeuploader.com franklinfire.co
freewebs.com
freshdz.com freshdz.com
freza-sverlo.ru fungamelands.com
fxtips.ru gagrasector.ru
fyl.com.ru gameonasia.com
gambarxkata.co geckoandfly.com
geilehausfrauen.net genericlowlatencyasiodriverhq.aircus.com
generalporn.org getaclueamerica.com
geoads.com graphid.com
glass-msk.ru hd-filmy.net
goodhumor24.com hdpixion.com
gox.com.ua health-medical-portal.info
growshop.es himazin.info
guitar-master.org histock.info
gwagka.com hit-kino.ru
hamptonoaks.ca homemature.net
headpharmacy.com i-hobot.ru
hermesbracelets.xyz i-midias.net.br
hexpilot.com iblogpress.xyz
hidefiles.org img.wallpaperstock.net
homeinns.com inboundlinks.win
host-protection.com intim-uslugi.info
hostcritique.com iqupdatetmz.win
hoverboard360.nl irunfar.com
hyipmanager.in iscblog.info
ictizanidinehcl4mg.blogspot.com islamtoday.co.za
ifirestarter.ru izhevsk.zrus.org
ilmexico.com izismile.com
imgarit.pw janavibekken.no
imicrovision.com javidol.site
infobabki.ru kaac.ru
iomoio.net
isitwp.com
istizanidineacontrolledsubstance.blogspot.com
jasonpartington.com
javlibrary.cc
jwss.cc
kamagragelusa.net
kareliatobacco.ru kareliatobacco.ru
kazrent.com kenaba.su
keywordsdoctor.com ketrzyn.pl
kinobaks.com kiinomaniak.pl
klumba55.ru kino-filmi.com
koleso24.com.ua kinosed.net
kurbappeal.info kinozapas.com
landreferat.ru kreativperlen.ch
legalrc.biz kriokomora.info
lexixxx.com labvis.host.sk
linkbux.ru laexotic.com
littlesunraiser.com landscaping.center
liverpool.gsofootball.com layola.biz.tc
long-beach-air-conditioning.com lazy-z.com
losangeles-ads.com leadn.pl
luchshie-topcasino.ru letmacworkfaster.site
lumb.co lineavabit.it
m0r0zk0-krava.ru linkpulse.com
mebel-ekb.com lion.conboy.us
meget.co.za m-google.xyz
micasainvest.com magda-gadalka.ru
mindeyegames.com magento-crew.net
mir-betting.ru mature.free-websites.com
mirobuvi.com.ua members.ghanaweb.com
mobioffertrck.com mensandals.xyz
moje-recenze.cz michellblog.online
monthlywinners.com microsearch.ru
montredemarque.nl mir-business-24.ru
mustwineblog.com mobplayer.net
mydownloadengine.com mojpreskumanie.com
monclerboots.xyz
monclerjacketsoutlet.win
more-letom.ru
morpicert.pw
moscow.online-podarki.com
mototsikl.org
muz-shoes.ru
muznachas-service.ru
mydearest.co
myfreetutorials.com
mylida.org
myxdate.info
nailsimg.com nailsimg.com
nomuos.it naj-filmy24.pl
noumeda.com
o-o-6-o-o.ru o-o-6-o-o.ru
obiavo.net obiavo.kz
oivcvx.website obsessionphrases.com
online247.ml onblastblog.online
oren-cats.ru onclkads.com
papasdelivery.ru optom-deshevo.ru
pc-services.ru ourville.info
percin.biz.ly paydayonlinecom.com
pila.pl pekori.to
pix24x7.com pinup-casino1.ru
pizda.lol pixell.club
pl.id-forex.com platezhka.net
pliks.pl ploenjitmedia.azurewebsites.net
porto.abuilder.net plohaya-kreditnaya-istoriya.ru
posible.net pontiacsolstice.info
progonrumarket.ru pornowarp.info
projectforte.ru powitania.pl
pronorm.fr preparevideosafesystem4unow.space
prostitutki-yaroslavlya.org priceg.com
qor360.com produkto.net
razyboard.com pron.pro
remedyotc.com ptr.ruvds.com
replicalouboutin.xyz qitt.ru
resant.ru quality-traffic.com
rewardpoll.com rankia.com
rfserial.net rarbg.to
rucrypt.com razleton.com
realmonte.net
regdefense.com
rfid-locker.co
ringtonepartner.com
ros-ctm.ru
royalcar-ufa.ru
runstocks.com runstocks.com
russian-postindex.ru salutmontreal.com
russintv.fr santechnik.jimdo.com
sanatorrii.ru sarm.tk
sarahmilne.top scanner-irvin.top
scanner-don.top scmor.ilxc.cc
screentoolkit.com searchkut.com
searchmywindow-a.akamaihd.net searchtooknow-a.akamaihd.net
seeingmeerkat.com semp.net
sextracker.be semrush.com
sfd-chess.ru seo18.su
shareyards.com sex-tracker.com
snow.nvr163.com sexreliz.com
soft-terminal.ru sharebutton.to
solinf.co siteaero.com
space4updating.win sivs.ru
starpages.net skladvaz.ru
stats-public.grammarly.io skypasss.com
stockquotes.wooeb.com small-games.biz
store-rx.com smallseotools.com
svnuppsalaorebro.se socialbutton.xyz
swagbucks.com songplanet.ru
tabakur77.com souvenirua.com
tds-advert002.info spylog.com
tedy.su stackthatbucks.com
telegraf.by stillmiracle.com
thefotosgratis.eu sucsesofinspiration.com
tizanidinerestlesslegsyndrome.blogspot.com tagil.zrus.org
tizanidinestreetvalue.blogspot.com talant-factory.ru
tobeyouday.win tetracsaudi.com
tootoo.to thexart.club
top-study.work tizanidineformigraines.blogspot.com
trackzapper.com tizanidineforopiatewithdrawal.blogspot.com
tubo360.com tizanidineinteractions.blogspot.com
tv-spoty.info tizanidinesleepaid.blogspot.com
tvorozhnaja-zapekanka-recept.ru traffic2money.com
twitlinks.com treasuretrack-a.akamaihd.net
ua.tc triplepanda.xyz
udayavani.com turizm.bz
uk-zheu20.ru ufa.xrus.org
ultimatesetnewfreeallsoftupgradesystems.pw umityangin.net
varikoz24.com urll.eu
velobikestock.com urlopener.com
verymes.xyz vapeface.club
video-chat.love videochat.tv.br
vkontarkte.com videogamesecrets.com
voronezh.xrus.org videospornogratisx.net
wallpapers-best.com vigrx-original.ru
wedding0venues.tk virtualbb.com
whatistizanidine2mg.blogspot.com watchdogs-2.ru
while.cheapwebsitehoster.com weburlopener.com
white-truck.life whereverdesperate.gq
wholesalejerseychinashop.com wholesalejerseysgaa.com
wholesalenfljerseys.us.com worldhistory.biz
woman-orgasm.ru worldoffiles.ru
xn----itbkqkfiq.xn--p1ai wyniki-lista.pl
xn--80aafb2a.xn--p1ai x-mix.info
xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai x5market.ru
yebocasino.com xbaboon.com
youandcredit.ru xjrul.com
yourothersite.com xn--d1acah0c.xn--p1ai
youtubedownload.org xn--oogle-wmc.com
you-shall-not-pass.is74.ru
zagadki.in.ua zagadki.in.ua
zaloro.com zerocash.msk.ru
zeikopay.com zeroredirect11.com
zhongwenlink.com zeroredirect7.com
zone-kev717.info

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.152 0; 10.1.1.227 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2942 ### Version: V4.2022.02.2943
### Updated: Thu Feb 24 22:01:31 UTC 2022 ### Updated: Fri Feb 25 10:03:20 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2942 ### Version: V4.2022.02.2943
### Updated: Thu Feb 24 22:01:31 UTC 2022 ### Updated: Fri Feb 25 10:03:20 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##