vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 02:29:58 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.01.2873 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-01-23 22:07:13 +00:00
parent d52ca56fd1
commit ccce39a455
47 changed files with 92403 additions and 92393 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2872 #### Version: V4.2022.01.2873
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2872 #### Version: V4.2022.01.2873
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.01.2872 #### Version: V4.2022.01.2873
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11412
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.01.2872 #### Version: V4.2022.01.2873
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11416
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11416
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11416
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11416
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

11416
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

11416
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

11416
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2872 2873

11416
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11416
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.109.186.241 0; 168.62.177.159 0;

11416
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

11416
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

11416
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

11416
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.109.186.241 0; 168.62.177.159 0;

11416
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.109.186.241 0; 168.62.177.159 0;

11416
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 1; 10.1.0.122 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Brandprotect FHscan
Cloud mapping HTMLparser
EirGrabber LWP::Simple
GetRight Musobot
LeechGet Name Intelligence
Lmspider Openvas
SurveyBot The Intraformant
Voltron Voil
Wonderbot scalaj-http
netEstate NE Crawler x22Mozilla

316
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker
404enemy
80legs
ADmantX ADmantX
AIBOT AIBOT
ALittle Client
ASPSeek ASPSeek
Abonti Aboundex
Acunetix Aboundexbot
AfD-Verbotsverfahren AfD-Verbotsverfahren
Aipbot AiHitBot
Anarchie Alligator
Ankit Ankit
Anthill Aspiegel
Asterias
Atomseobot
Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot BDFetch
BDCbot
BLEXBot BLEXBot
BackDoorBot BackStreet
Backlink-Ceck Backlink-Ceck
BacklinkCrawler Barkrowler
Badass Battleztar Bazinga
Bandit Bitacle
BatchFTP BlackWidow
Black Hole Blackboard
BlowFish
Bolt
BotALot BotALot
Brandprotect Brandprotect
Brandwatch Brandwatch
Buck Buck
BuiltBotTough Buddy
BuiltWith BuiltWith
BunnySlippers BunnySlippers
CATExplorador BuzzSumo
CODE87 CCBot
CSHttp Cegbfeieh
CazoodleBot Cliqzbot
CheTeam Cocolyzebot
CheeseBot Copier
ChinaClaw
Cogentbot
CopyRightCheck
Cosmos Cosmos
Craftbot
Crawling at Home Project Crawling at Home Project
Crescent Crescent
Curious Curious
Custo
DBLBot DBLBot
DTS Agent DTS Agent
DataCha0s DataCha0s
Devil
Digincore
Dirbuster
Discobot Discobot
Discoverybot
DnyzBot
DomCopBot DomCopBot
DomainAppender DomainAppender
DomainCrawler Domains Project
Download Wonder
Dragonfly Dragonfly
EMail Wolf EMail Siphon
EasyDL Ecxi
EroCrawler
Evil
Exabot Exabot
Express WebPictures ExtLinksBot
ExtractorPro Extractor
Extreme Picture Finder
EyeNetIE
FDM FDM
FHscan FHscan
Firefox/7.0 Fimap
FlashGet Freeuploader
Flunky Fuzz
Fyrebot
GalaxyBot
GetRight GetRight
GetWeb Getintent
Go!Zilla Go!Zilla
GrabNet
Grafula Grafula
GrapeFX GrapeFX
GridBot GrapeshotCrawler
HMView HMView
HTMLparser
HTTrack HTTrack
Haansoft Haansoft
HaosouSpider HaosouSpider
Harvest Havij
Humanlinks
HybridBot HybridBot
IDBTE4M IRLbot
Id-search
IlseBot
Image Sucker Image Sucker
IndeedBot
Indy Library Indy Library
InfoNaviRobot
InfoTekies
Intelliseek Intelliseek
InterGET
JOC Web Spider JOC Web Spider
JamesBOT
Jbrofuzz Jbrofuzz
Jorgee Jetty
Kenjin Spider JustView
Keyword Density LNSpiderguy
Kinza LexiBot
LWP::Simple
Leap
Libwhisker Libwhisker
Lightspeedsystems Lightspeedsystems
Likse
LinkScan LinkScan
LinkWalker LinkWalker
LinksManager Linkbot
LinqiaScrapeBot Linkdexbot
LinkpadBot
Lipperhey
Litemage_walker Litemage_walker
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MIDown tool
MSFrontPage MSFrontPage
MSIECrawler MTRobot
Mag-Net Magnet
Majestic-SEO Majestic SEO
Majestic12 MarkWatch
MarkMonitor Mb2345Browser
MauiBot
MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot MicroMessenger
Metauri
Microsoft URL Control Microsoft URL Control
Moblie Safari MolokaiBot
Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mr.4x3
Msrabot
Musobot
NICErsPRO
NPbot
Nameprotect Nameprotect
Navroad NearSite
Needle Needle
Nessus NetLyzer
NetZIP NetSpider
Netcraft Nettrack
Netvibes Niki-bot
Nibbler Ninja
Nikto Offline Explorer
Nmap
Nutch
Offline Navigator Offline Navigator
OpenLinkProfiler
OpenVAS
Openfind Openfind
OutclicksBot OrangeBot
PECL::HTTP Page Analyzer
PageAnalyzer PageGrabber
PageThing.com Picsearch
PeoplePal
Pi-Monster
Pixray
PleaseCrawl PleaseCrawl
ProPowerBot ProPowerBot
Probethenet Pu_iN
Psbot QueryN Metasearch
Pump Quick-Crawler
PxBroker
PyCurl
RSSingBot
RankActiveLinkBot
RankFlex
RankingBot
RankingBot2
Rankivabot Rankivabot
Reaper ReGet
Recorder Recorder
Ripper RedesScrapy
RocketCrawler SEOkicks
Rogerbot
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SalesIntelligent
ScoutJet
Scrapy
Screaming Screaming
ScrepyBot Searchestate
SemrushBot SearchmetricsBot
Semrush
SentiBot
SeoSiteCheckup SeoSiteCheckup
Site Sucker Siphon
SiteCheckerBotCrawler
SiteSucker
Sitebeam Sitebeam
Siteimprove
Sitevigil
SlySearch
Snake Snake
SocialRankIOBot Snoopy
Sociscraper
Sottopop Sottopop
Spbot SpaceBison
Spammen
Spanner
Spinn3r Spinn3r
SputnikBot SputnikBot
Sqworm Sqlmap
Steeler Sucker
Stripper SurveyBot
Szukacz Suzuran
T8Abot
Teleport Teleport
Telesphoreo The Intraformant
Telesphorep Toata
Thumbor
Toweyabot Toweyabot
Tracemyfile Tracemyfile
Turnitin Trendiction
Trendictionbot
True_Robot
TwengaBot
URLy Warning
URLy.Warning URLy.Warning
VCI V-BOT
Vacuum
Vagabondo Vagabondo
VeriCiteCrawler VelenPublicWebCrawler
VidibleScraper Virusdie
Voltron VoidEYE
WEBDAV
WISENutbot WISENutbot
WPScan WPScan
WWW-Collector-E Web Auto
WWW-Mechanize Web Collage
WWWOFFLE Web Fetch
Wallpapers Web Fuck
Wallpapers/3.0
Web Enhancer
Web Pix
Web Sauger Web Sauger
Web Sucker Web Sucker
WebAuto WebAuto
WebReaper WebCollage
WebSauger WebCopier
WebStripper WebFuck
WebZIP WebGo IS
Webalta WebLeacher
Webshag WebSucker
Webster Website Quester
Whacker WebsiteQuester
Whatweb Widow
Xaldon WebSpider Woobot
Xenu Xaldon_WebSpider
YoudaoBot
Zermelo
Zeus
Zitebot Zitebot
ZmEu
ZoominfoBot
ZumBot
ZyBorg
adscanner adscanner
arquivo-web-crawler archive.org_bot
autoemailspider clark-crawler
cah.io.community crawler.feedback
crawl.sogou.com crawler4j
dataforseo.com dataforseo.com
domainsproject.org evc-batch
gopher facebookscraper
heritrix
ips-agent ips-agent
isitwp.com
iubenda-radar
mediawords mediawords
muhstik-scan
netEstate NE Crawler
oBot oBot
page scorer
pcBrowser pcBrowser
plumanalytics
polaris version polaris version
ripz probe-image-size
scalaj-http scalaj-http
scan.lol
seobility seobility
seocompany.store
seoscanners
seostar seostar
serpstatbot serpstatbot
sitechecker.pro sitechecker.pro
spyfu sogouspider
tAkeOut webmeup-crawler
ubermetrics-technologies.com x09Mozilla
webprosbot xpymep1.exe
zgrab

328
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404enemy 404checker
ASPSeek ADmantX
Aboundex AIBOT
Aboundexbot Acunetix
AhrefsBot AhrefsBot
AiHitBot
Alexibot
Anarchie Anarchie
Anarchy
Ankit Ankit
Anthill Apexoo
Aspiegel Aspiegel
AwarioRssBot Asterias
BackWeb BDFetch
Bandit BackDoorBot
Barkrowler Backlink-Ceck
Bitacle Badass
Black Hole Battleztar Bazinga
Blow Bigfoot
Bolt Blackboard
BlowFish
Boardreader
Brandprotect Brandprotect
BunnySlippers Buck
CCBot Buddy
Calculon BuiltBotTough
Cegbfeieh BuiltWith
CATExplorador
CheTeam CheTeam
ChinaClaw Chlooe
Cliqzbot Citoid
Claritybot
Cocolyzebot
Cogentbot
Collector Collector
Copier
CopyRightCheck CopyRightCheck
Copyscape
Cosmos Cosmos
CrazyWebCrawler Crawling at Home Project
CrunchBot Curious
Custo
CyotekWebCopy CyotekWebCopy
DBLBot DBLBot
DataCha0s DIIbot
Demon DTS Agent
Deusu DatabaseDriverMysqli
DigitalPebble
Dirbuster Dirbuster
Discobot Discoverybot
Dispatch DnyzBot
DomCopBot
DomainCrawler DomainCrawler
Domains Project DomainSigmaCrawler
Dotbot DomainStatsBot
Download Wonder
Dragonfly Dragonfly
EMail Siphon Drip
ECCP/1.0
Ebingbong
EirGrabber EirGrabber
Evil EroCrawler
ExtractorPro Extreme Picture Finder
Ezooms Ezooms
FDM
FHscan
FemtosearchBot FemtosearchBot
Fimap FlashGet
Foobot FrontPage
Freeuploader Genieo
GT::WWW GermCrawler
GalaxyBot Go!Zilla
GetRight Go-Ahead-Got-It
Gotit Grabber
GrabNet
Grafula
GrapeFX
GrapeshotCrawler GrapeshotCrawler
GridBot
HEADMasterSEO HEADMasterSEO
HTTP::Lite
Haansoft Haansoft
HonoluluBot Havij
Image Fetch Hloader
Image Sucker IDBot
IndeedBot IRLbot
InfoNaviRobot Iblog
Id-search
IlseBot
Indy Library
InfoTekies
Intelliseek Intelliseek
InternetSeer InterGET
Iskanie Iria
JOC Web Spider
JamesBOT
Jbrofuzz
JennyBot JennyBot
JetCar Jorgee
Jetty JustView
Jyxobot Jyxobot
Kenjin Spider Keyword Density
Kozmosbot Kinza
LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot
LeechFTP LeechFTP
LeechGet
LexiBot
LibWeb LibWeb
Libwhisker Libwhisker
Lightspeedsystems Linkdexbot
Likse LinkpadBot
LinkScan LinqiaScrapeBot
LinkWalker Lipperhey
Lmspider Litemage_walker
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc MJ12bot
MQQBrowser
MSIECrawler MSIECrawler
Mag-Net
Magnet Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic SEO
Majestic12 Majestic-SEO
Masscan MarkMonitor
MarkWatch
Mass Downloader
Mata Hari Mata Hari
Mb2345Browser
Metauri
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Minefield
Mojolicious Mojolicious
Morfeus Fucking Scanner MolokaiBot
Mozlila Mr.4x3
Msrabot NPbot
Musobot Nameprotect
Net Vampire Navroad
Netcraft NearSite
Netvibes Needle
NetMechanic
NetSpider
NetZIP
Nibbler Nibbler
Nimbostratus Niki-bot
Nikto
Ninja Ninja
Nmap Not
Nuclei Nuclei
Nutch Offline Explorer
Offline Navigator OnCrawl
OpenLinkProfiler
Openfind
OrangeSpider OrangeSpider
PECL::HTTP PECL::HTTP
PageGrabber PHPCrawl
PageScorer POE-Component-Client-HTTP
Pandalytics Page Analyzer
Papa Foto PageAnalyzer
Pavuk
Petalbot Petalbot
Picsearch PictureFinder
Probethenet PleaseCrawl
ProPowerBot
Psbot
Pump Pump
PxBroker PyCurl
QueryN Metasearch
RankActive RankActive
RankActiveLinkBot RankingBot2
Rankivabot Rankivabot
Re-re
ReGet ReGet
Reaper RebelMouse
RedesScrapy RedesScrapy
RepoMonkey
Ripper Ripper
RocketCrawler
Rogerbot
SBIder SBIder
SISTRIX SEOkicks
ScoutJet SEOlyticsCrawler
SalesIntelligent
Scrapy
Screaming
ScreenerBot
ScrepyBot ScrepyBot
SearchmetricsBot SentiBot
SemanticJuice SeoSiteCheckup
SemrushBot SeobilityBot
Seomoz Seomoz
Shodan SiteLockSpider
Site Sucker
SiteCheckerBotCrawler
SiteSnagger
Sitebeam
Siteimprove
SmartDownload SmartDownload
Snake Snapbot
Sogou web spider Snoopy
Sosospider Sociscraper
Sottopop
SpaceBison SpaceBison
Spammen SpankBot
Spbot Spbot
Spinn3r
SputnikBot SputnikBot
Sqlmap Sqworm
Sqlworm Sucuri
Sucker
SuperHTTP
Surfbot
Suzuran Suzuran
Swiftbot Swiftbot
Teleport Teleport
TeleportPro TightTwatBot
Telesphoreo Toata
Telesphorep Tracemyfile
The Intraformant
TheNomad
Titan
Trendiction Trendiction
Trendictionbot True_Robot
Turnitin
TurnitinBot
URLy Warning
URLy.Warning URLy.Warning
UnisterBot UnisterBot
Vacuum VCI
VelenPublicWebCrawler VelenPublicWebCrawler
VidibleScraper VeriCiteCrawler
Virusdie
VoidEYE VoidEYE
Voil Voil
Voltron
WASALive-Bot WASALive-Bot
WBSearchBot WEBDAV
WISENutbot WPScan
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WeSEE WeSEE
Web Collage Web Collage
Web Fetch Web Fuck
Web Pix Web Sucker
WebAuto WebImageCollector
WebBandit WebLeacher
WebCollage
WebCopier
WebGo IS
WebSauger WebSauger
WebSucker WebSucker
WebmasterWorldForumBot Webalta
WebsiteQuester WebsiteQuester
Webster
Whack Whack
Whatweb Whatweb
Who.is Bot Who.is Bot
WinHTTrack WinHTTrack
WiseGuys Robot WiseGuys Robot
Wonderbot Wotbox
Wprecon Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Xenu Xenu
YoudaoBot Zade
Zeus ZoomBot
ZmEu
ZumBot ZumBot
ZyBorg adscanner
arquivo-web-crawler arquivo-web-crawler
autoemailspider autoemailspider
cognitiveseo cah.io.community
crawl.sogou.com com.plumanalytics
dataforseo.com domainsproject.org
eCatch
evc-batch
gopher gopher
heritrix
instabid
internetVista monitor internetVista monitor
iubenda-radar lwp-trivial
meanpathbot magpie-crawler
muhstik-scan netEstate NE Crawler
oBot pcBrowser
polaris version polaris version
s1z.ru scalaj-http
satoristudio.net seostar
scan.lol sitechecker.pro
trendiction.com spyfu
voyagerx.com trendiction.de
x22Mozilla
xpymep1.exe xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
abusora.com 4k-player.pl
advancedmassagebysara.com azinoofficial777.ru
avto-oligarh.ru hunthillfarmtrust.org
buynorxx.com medictube.ru
chocolateslim-en-france.com remontgruzovik.ru
letto.by russkoe-zdorovie.ru
seasaltwithfood.com stairliftsarea.com
tomatis.gospartner.com tourcroatia.co.uk
top1-seo-service.com viel.su
websiteexplorer.info your-bearings.com

920
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
007angels.com 000free.us
1000n1.ru 1-99seo.com
101billion.com
101lesbian.xyz
1millionusd.xyz 1millionusd.xyz
24-ak.ru 2rich4bitches.com
4kplayer.pl 3-letter-domains.net
3xforum.ro
4rent.ru
4replicawatch.net 4replicawatch.net
5-steps-to-start-business.com 5000-cotydzien.com
5ws.dating-app.ru 5elementov.ru
8xv8.com 8jn.dating-app.ru
9med.net adcash.com
aa8780bb28a1de4eb5bff33c28a218a930.com adimmix.com
acads.net
actionnooz.com
adconscious.com
adloads.com adloads.com
admeasures.com adminshop.com
adsloads.com admitad.com
adtech.de adzerg.com
adtech.fr agysya.ru
adult3dgames.com aksonural.ru
aerogo.com aktivator-windows10.blogspot.com
agapovdg.ru alborzan.com
agreda.pluto.ro alkoravto.ru
akama.com all-number.com
albuteroli.com allblogroll.com
algarveglobal.com altamayoztourism.com
algerianembassy.co.in amateurlivechat.org
allboard.xobor.de amateurmatch.com
alot.com amospalla.es
alphaforum.ru android4fun.org
alt-servis.ru animenime.ru
am15.net
amanda-porn.ga
amazon-seo-service.com
analytics-ads.xyz
andpolice.com
androids-store.com
animal-drawings.com
anlimebel.kiev.ua anlimebel.kiev.ua
anonymizeme.pro anonymous-redirect.com
apiadanaknet-a.akamaihd.net aosheng-tech.com
apiappenableinfo-a.akamaihd.net ap.senai.br
apiluckyleapnet-a.akamaihd.net apartment.ru
app-ready.xyz apparel-offer.com
arendakvartir.kz arabgirls.adultgalls.com
asrvvv-a.akamaihd.net arendovalka.xyz
astrochicks.com arraty.altervista.org
audiofree.ru artdeko.info
autobudpostach.club as5000.com
avek.ru asiengirls.net
b-styles.xyz asrv-a.akamaihd.net
b00kmarks.com asrvrep-a.akamaihd.net
babyguns.ru atdedinotuho.tk
banan.tv atelielembrancaqueencanta.com.br
bannerspace.com atmagroup.ru
barnfurnituremart.com audiobangout.com
basedecor.ru azbuka-mo.ru
bayareaaeroclub.org azbukadiets.ru
belsetirehafi.tk azinoofficial777.ru
bestdraws.com baersaratov.ru
bestpriceninja.com bagcionderlab.com
bang-hotties.com
bankofthewext.com
bannerbank.ru
barbourjackets.online
bensbargains.net
bestchoice.cf
bestfortraders.com
bestmarriages.com
bestmobilityscooterstoday.com
bestssaker.com
bestwebsiteawards.com bestwebsiteawards.com
beta.hotkeys.com bestwebsitesawards.com
betteroffers.review
bget.ru bget.ru
big-boards.info bif-ru.info
big-cash.net binomo.com
binomo.kz bjorkbacken.nu
birzha-truda.eu blog.remote-computer.de
blog.f00kclan.de blpmovies.com
bloke.com book-bets.com
bmwhighperformers.com
bolegapakistan.com
bookmark4you.com.biz
boole.onlinewebshop.net
bosefux.esy.es bosefux.esy.es
bpro1.top brains2.biz
brandbucket.com brakehawk.com
brandehk.dk brus-vsem.ru
brateg.xyz brus.city
brewdom.ru
brokergid.ru
bugof.gq bugof.gq
buketeg.xyz buyessaynow.biz
businesn.men buzzzg.men
buy-loft.ru bytimedance.ru
buzzsumo.com calcularpagerank.com
bwlx.prepedu.cn callstevens.com
canvas.gloverid.site camdolls.com
canyougethighofftizanidine.blogspot.com candlewooddentalcentre.com.au
carapics.com
cartierbracelet.xyz cartierbracelet.xyz
cartierlove2u.xyz
cartierlovestore.com cartierlovestore.com
cdncash.org cartierreplica.win
centre-indigo.org.ua cartierreplica.xyz
cherrypointplace.ca censys.io
chidporn.com check-host.net
christianlouboutinoutlet.win chimiver.info
chuckguilford.com cityadspix.com
circlesl.com clash-clans.ru
cl.s7.exct.net clean-start.net
classiquebijoux.ru cleanersoft.com
coldfilm.ru clickprotects.com
com-supportcenter.website clknsee.com
compliance-irvin.top codq.info
crazyboost.pro compliance-brian.top
creditwell.ru compliance-donald.xyz
custom-product-labels.com compliance-jane.top
customboxes4less.com compliance-margo.top
customcollegeessays.net cougfan.info
cpabegins.ru
cpajunkies.com
cranly.net
credit-online.ws
cryptoswap.biz
cupday.com
custom-electric-guitar.com
cutt.us
cwetochki.ru cwetochki.ru
czat.wp.pl danceuniverse.ru
d0t.ru dandiyabeats.in
daisye.top
davidovic.info davidovic.info
ddospanel.com dbmkfhqk.bloger.index.hr
delitime.info ddpills.com
de.zapmeta.com
decorationspcs.com
delayreferat.ru
demenageur.com demenageur.com
dev.citetick.com detki-opt.ru
dickssportinggoods.com detskie-zabavi.ru
digital-sale.su dfiles.me
dildofotzen.net dieswaene.com
directrev.com digitalassetmanagement.site
dispo.de dimfour.com
dokumentalkino.net dimkino.ru
dollartree.info discountbarbour.online
domik-derevne.ru documentsite.net
domini.cat doeco.ru
dorratex.tn dominterior.org
dotnetdotcom.org doxyporno.com
doublepimp.com drugs-no-rx.info
downloadkakaotalk.com dtm-spain.com
dpihatinh.gov.vn dumuelave.xyz
droidlook.net eachdayisagift.review
drugspowerstore.com efkt.jp
dynainbox.com egovaleo.it
e-biznes.info ekosmetyki.net.pl
easyshoppermac.com elektrischezi.canalblog.com
easytuningshop.ru elektrischezigarette2.devhub.com
ekobata.ru elektrozigarette1.dreamwidth.org
elektronischezigarettekaufen1.beeplog.com elektrozigaretten1.postbit.com
elektrozigaretten2.yn.lt
elidelcream.weebly.com
elitesportsadvisor.com
elmacho.xyz
elstal.com.pl
empis.magix.net empis.magix.net
envaseslotusama.com energydiet-info.ru
eonpal.com eropho.com
escort-russian.com essay-edu.biz
estelight.ru este-line.com.ua
euromasterclass.ru europeanwatches.ru
everflora.ru euroskat.ru
exdocsfiles.com excaliburfilms.com
executehosting.com eyes-on-you.ga
extremepornos.net f5mtrack.com
ezigarettekaufen2.blox.pl
ezigarettenkaufen2.dreamwidth.org
ezigaretteshop.webs.com
facialporntube.com
family1st.ca
faptitans.com
faracontrol.ir
favornews.com favornews.com
filesclub.net fetroshok.ru
financehint.eu fickenprivat.info
finteks.ru filmgo.ru
flashbannernow.com finuse.com
floridamobilebillboards.com firesub.pl
fm-upgrade.ru forum20.smailik.org
forum69.info forums.d2jsp.org
fotosfotos.eu foxinsocks.ru
foxydeal.com freakycheats.com
free-gluten.ru free-laptop-reward.com
free-share-buttons.top free-today.com
freemags.cc front.ru
freenom.link frvo.alptandem.ru
futbolkisales.ru funnel.co.za
fx-brokers-review.com g33.org
galaxyflowers.ru game300.ru
gamblingnerd.com gamesprite.me
gamezblox.com gaypornmovie.net
gardene.ru genericviagrasildenafiled.net
gazporno.com getmyads24.com
get-free-social-traffic.com getresponse.com
get-free-traffic-now.com gg.zzyjxs.com
ghazel.ru
gilbertbanda.net gilbertbanda.net
glicol.kz gktt.ru
globalscam.ga glopages.ru
gloverid.site gloverid.site
goodwinmetals.co goldandcard.ru
gotorussia.com gorodservis.ru
gosreg.amchs.ru
gourcy.altervista.org
gratis-sexkontakte.com gratis-sexkontakte.com
greatgrace.ru greatgrace.ru
greatidea.marketing gribokstop.com
grupografico-pilar.com.ar grizzlysgrill.com
gsmlab.pl
gsmtlf.ru
gta-top.ru gta-top.ru
hamilton.ca guge.io
hcate.com guildebzh.info
hem.passagen.se gwebtools.com
hifidesign.ru hahashka.ru
historichometeam.com hamptonoaks.ca
hitcpm.com hannasolution.ru
hao123.com
hasshe.com
hauleddes.com
hitmuzik.ru
hiwibyh.bugs3.com hiwibyh.bugs3.com
host-protection.com hotsocialz.com
hostcritique.com housemilan.ru
hoste.octopis.com humanorightswatch.org
hotchatdate.com
hotxnights.info
hoverboard360.at
huhn.altervista.org
iblogpress.xyz
id-forex.com id-forex.com
iframe-toloka.com ideibiznesa2015.ru
ido3.com
ihc2015.info
ilmen.net
imadedinner.net
img.wallpaperstock.net
imperia31.ru
importchinacoach-teach.com
inboundlinks.win
inclk.com inclk.com
increasewwwtraffic.info
infazavr.ru
infowarcraft.ru infowarcraft.ru
internetproviderstucson.com instabid.tech
iqs.biz.ua internetartfair.com
irkutsk.zrus.org investmac.com
islamtoday.co.za invivo.hu
ivanovo.zrus.org iplogger.org
jabimgo.pw iscblog.info
james13prix.info istripper.com
jetsli.de ixora.pro
johannesburgsingles.co.za javcoast.com
joingames.org javlibrary.cc
kamalsinha.com javstock.com
kaz.kz jestr.org
kerch.site jimmychoosale.online
kerwinandcariza.com jolic2.com
ketrzyn.pl kalandranis.gr
kino-key.info kasino-money.pw
klikbonus.com kazinogames.lv
knowsitall.info keywordblocks.com
kumuk.info kino-ecran.ru
ladsblue.com kino-rating.ru
lalalove.ru kinobest.pl
landoftracking.com kinoplen.ru
laptop-4-less.com kolotiloff.ru
lerporn.info kongoultry.net
letmacworkfaster.site konkursov.net
letolove.ru kostenlos-sexvideos.com
lexiangwan.com krasula.pp.ua
lider-zhaluzi.kiev.ua kuzinsp.ru
liffa.ru labelwater.se
linkredirect.biz ladov.ru
lob.com.ru larose.jb2c.me
locimge.pw lawyers.cafe
lonerangergames.com lawyersinfo.org
louboutinshoes.xyz lazy-z.com
love-baby.cz lazymanyoga.com
letto.by
lifebyleese.com
light.ifmo.ru
likrot.com
linkdebrideur.xyz
lion.misba.us
lirunet.ru
livejasmin.com
liver-chintai.org
loadingpages.me
louboutinbooties.xyz
lubetube.com lubetube.com
luckyday.world luchshie-topcasino.ru
m.b00kmarks.com luxmagazine.cf
mainhp.net luxup.ru
makenahartlin.com lyrics.home-task.com
malwareremovalcenter.com mafcards.ru
marinetraffic.com magiadinamibia.blogspot.com
markjaybeefractal.com magnetic-bracelets.ru
masturbate.co.uk mainlinehobby.net
maximilitary.ru manifestation.betteroffers.review
mbiologi.ru map028.com
mcadamssupplyco.com maridan.com.ua
mebel-vstroika.ru marmitaco.cat
medanestesia.ru meccadumps.net
medicineseasybuy.com megahdporno.net
meduza-consult.ru methodsmarketing.com
meetlocalchicks.com mikrobiologies.ru
mejoresfotos.eu mindbox.co.za
meteostate.com missvietnam.org
midst.eu mister-shop.com
milblueprint.com mncrftpcs.com
mir-limuzinov.ru monarhs.info
mirtorrent.net moneymaster.ru
mmgq.ru moroccosurfadventures.com
mmofreegames.online mp3downloadhq.com
mobicover.com.ua mrlmedia.net
mobot.site msfsaar.de
modenamebel.ru musezone.su
mokrayakiska.com musicstock.me
money-for-placing-articles.com muztops.ru
musezone.ru mydoctorok.ru
musicspire.online myftpupload.com
musirc.com myghillie.info
mydownloadengine.com myhitmp3.club
mygameplus.com myindospace.com
mysex21.com myonlinepayday.co
myshopmatemac.com mypornfree.ru
myxdate.info naturalpharm.com.ua
naturalshair.site naturtreenspicerx.pw
newtechspb.ru new-post.tk
nomuos.it newsperuse.com
nottyu.xyz newstaffadsshop.club
noumeda.com nextrent-crimea.ru
novosibirsk.xrus.org nfljerseyscheapchinabiz.com
obiavo.by nippon-bearings.ru
oda.as nodup.ru
office-windows.ru november-lax.com
ohmyrings.com o-o-8-o-o.ru
omsk.xrus.org omgtnc.com
onemactrckr.com oops-cinema.ru
openlibrary.org
optitrade24.com
orsonet.ru
outclicks.net
ovirus.ru ovirus.ru
paintingplanet.ru paccohichetoti.ml
paparazzistudios.com.au panicatack.com
parajumpersoutlet.online parvezmia.xyz
partner-high.men
partner-print.men
pateaswing.com
pb-dv.ru pb-dv.ru
pc-virus-d0l92j2.pw pearlisland.ru
pdns.cz penisvergrotendepillennl.ovh
pensplan.com perederni.net
pensplan4u.com php-market.ru
picswe.com pisanieprac.info
picturesfrom.com piter.xrus.org
pihl.se pivka.xyz
pipki.r.acdnpro.com pizza-tycoon.com
pladform.ru pl-top.pl
plastjulgranar.se play-mp3.com
plastweb.ru podshipniki-nsk.ru
plusnetwork.com pogodnyyeavarii.gq
pobeiranie.pl polybuild.ru
poffet.net pooleroadmedicalcentre.co.uk
polcin.de porno-home365.com
pomoc-drogowa.cba.pl pornofiljmi.com
pornoinn.com
pornoklad.net pornoklad.net
pornokorol.com pornoklad.ru
pornotubs.com pornonik.com
pornowarp.info pornosmola.info
postclass.com portnoff.od.ua
potoideas.us potoideas.us
pozvonim.com privatefx.all4invest.info
prefersurvey.net proekt-gaz.ru
preventheadacheguide.info profitkode.com
printingpeach.com promoforum.ru
profitsport.club psoriasis-file.trade
prointer.net.ua
promgirldresses.xyz
prostitutki-novgoroda.org
ptr.ruvds.com
pts163.ru
pukaporn.com pukaporn.com
pyrodesigns.com.au purplestats.com
quebec-bin.com quickbuck.com
quick-offer.com registratciya-v-moskve.ru
rabot.host.sk relayblog.com
rainbowice.ru resultshub-a.akamaihd.net
rankexperience.com retreatia.com
ranking2017.ga riralmolamsaca.tk
rating-casino2021.ru rock-cafe.info
redirectingat.com rockprogblog.com
referencemoi.com
remontgruzovik.ru
renhacklids.tk
rightenergysolutions.com.au
ring4rhino.com
robertefuller.com
roofers.org.uk
rossmark.ru
rostov.xrus.org
royalads.net royalads.net
sammyweaver.com ruex.org.ua
sanyuprojects.com rus-pornuha.com
sarahmilne.top rutor.group
scanner-donald.top rvtv.ru
samoiedo.it
satellite.maps.ilovevitaly.com
sbf441.com
scanmarine.info
scanmyphones.com
scanner-john.top
scanner-julia.top scanner-julia.top
scanner-walter.top scanner-margo.top
scmor.ilxc.cc scanner-mary.top
searchengineranker.email schlampen-treffen.com
searchimage.co sdelai-prosto.ru
searchimpression.com searchencrypt.com
searchlock.com seccioncontrabajo.com
searchtooknow-a.akamaihd.net securityallianceservices.com
semalt.com semalt.com
seobility.net seo18.su
seorankinglinks.xyz seoboxes.com
service-core.ru seokicks.de
sexobzor.info serdcenebolit.com
sfj-ror.no servisural.ru
sex-tracker.com
sexpornotales.com
sextracker.com
shama-rc.net
sharebutton.net sharebutton.net
shijian.ac.cn shinikiev.com.ua
shillyourcoins.com shmetall.com.ua
silverage.ru shtaketniki.kz
site-speed-checker.site sibtest.ru
skachat-besplatno-obrazcy.ru sientalyric.co
simple-share-buttons.com
simplepooltips.com
site-analyzer.com
sitevaluation.org
skypasss.com
slavia.info
slowmac.tech slowmac.tech
social-s-iii.xyz soda.media
sovetogorod.ru solnplast.ru
speechfoodie.com spacash.com
spravka-medosmotr.ru space-worry.ml
sptslmtrafms.com spidtest.org
spy-app.info spitfiremusic.com
spywarenuker.com spylog.com
stard.shop stal-rulon.ru
steame.ru stats-public.grammarly.io
stillmiracle.com stroicol.net
stockquotes.wooeb.com
stopnarco.ru
strag-invest.ru
strana-krasoty.ru
stroiminsk.org stroiminsk.org
stroyplus.ru subj.ukr-lit.com
suchenindeutschland.com swinginwithme.ru
sudexpert66.ru syvertsen-da.no
suggest-keywords.com taihouse.ru
sundrugstore.com tanieaukcje.com.pl
supernew.org teenporn18.net
surcentro.com tehngr.ru
svbur.ru telefonsexsofort.tk
syndicate.fun teletype.in
tarad.com template-kid.com
teenfuck.tv the-trader.net
thefotosgratis.eu thefarmergame.com
thetoiletpaper.com thegioixekhach.com
thin.me.pn
titslove.yopoint.in titslove.yopoint.in
tizanidine4mgstreetprice.blogspot.com tizanidine4mgstreetprice.blogspot.com
tizanidinedruginteractions.blogspot.com tizanidineandgabapentin.blogspot.com
tizanidinehcl4mgtablet.blogspot.com tizanidinedrugtest.blogspot.com
tizanidinesideeffects.blogspot.com tizanidinehcl4mgtab.blogspot.com
tksn.ru tizanidinehclsideeffects.blogspot.com
tobeyouday.win tizanidinehydrochloride2mg.blogspot.com
tongkatmadura.info tizanidinenarcotic.blogspot.com
tooplay.com tizanidineorflexeril.blogspot.com
top-karkas.ru tomatis.gospartner.com
topsiteminecraft.com topmira.com
toy-shop.top trichizobswiv.agddns.net
tozup.com
track.deriv.com
tracksz.co
trafficshaper.com
traffique.net
traffpartners.com
trahic.ru
truck-addzilla.life truck-addzilla.life
truck-rece.life trustedhealthtips.com
turbo-suslik.org tsyndicate.com
tvand.ru tupper-shop.ru
tvteleport.ru twinderbella.com
udayavani.com
uggbootsoutletsale.us uggbootsoutletsale.us
ugogo.info uginekologa.com
ukkala.xyz ukr-lit.com
ukrobstep.com uptimechecker.com
underthesite.com urlopener.blogspot.com.au
uni.me vacances-voyages.info
univerfiles.com varikozdok.ru
usadacha.net vashsvet.com
vabasa.inwtrade.com versaut.xxx-cam.webcam
valkiria-tk.ru videochat.mx
varikoz24.com videositename.com
veles.shop vids18.site
vellings.info virtualbb.com
veselokloun.ru vysigy.su
via-energy-acquistare.com vzlomfb.com
videochat.ph waysbetter.cn
virtuagirl.com webalan.ru
vladimir.xrus.org webmasterhome.cn
vroze.com webradiology.ru
vtc.pw while.cheapwebsitehoster.com
vvon.co.uk whiteproduct.com
vvpg.ru wholesalecheapjerseysfree.com
w3data.co whosonmyserver.com
webtherapy.ru worldoffiles.ru
wedding0venues.tk xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
wemarketing.se xnxxandxvideos.com
white-truck.life xpresscare.ru
winner7777.net xrus.org
wnoz.de xsion.net
wonderfulflowers.biz youporn-forum.ga
workle.website your-bearings.com
worldinternetauthority.com yourothersite.com
worldtraveler.world yousense.info
wrz0iuebwhp5fg.freeddns.com youtube-downloader.savetubevideo.com
www.bookmaker-bets.com ytmnd.com
x-stars.ru yuarra.pluto.ro
xchat26.myfreecams.com yugo-star.ru
xmronta.com zahvat.ru
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai zaimhelp.ru
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai zastroyka.org
xn----ctbigni3aj4h.xn--p1ai zeroredirect10.com
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai zeroredirect6.com
xn--80aanaardaperhcem4a6i.com zigarettenonlinekaufen1.blox.pl
xn--80abgj3a5acid6ghs.top zigzog.ru
xn--h1ahbi.com.ua zirondelli.it
xwatt.ru znakomstvaonlain.ru
xxxdatinglocal.us zog.link
xxxhdvideo.site zojirushi-products.ru
yadro.ru
youjizz.vc
youraticles.pl
za-music.mymobiplanet.com
zaloadi.ru
zapiszto.pl
zawyna.ua
zeroredirect.com
zeroredirect5.com
zlatnajesen.com
zmoda.hostreo.com
znakomstva-moskva77.ru
zophim.me

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01apple.com 123kuma.com
13tabs.com 12masterov.com
3-letter-domains.net 2daytrendingnews.com
4replicawatch.net 70casino.online
5kstore.com absugars.com
aa08daf7e13b6345e09e92f771507fa5f4.com
aa8780bb28a1de4eb5bff33c28a218a930.com
aaa38852e886ac4af1a3cff9b47cab6272.com
abcdeg.xyz
academiacsmendoza.org academiacsmendoza.org
actulite.com addray.pro
adexprts.com adzpower.com
admatic.com.tr aliexpresscashback.club
advokateg.ru apidiamondatanet-a.akamaihd.net
affordablewebsitesandmobileapps.com apiwebwebgetcom-a.akamaihd.net
agardomains.com arendakvartir.kz
alert-fjg.xyz arkartex.ru
alldownload.pw ayakino.net
ample-awards-today.us ayodhya.co
analnoeporno.tv
animenime.ru
anniemation.com
apparel-offer.com
asacopaco.tk
autonew.biz
avtoarenda.by
azazu.ru azazu.ru
azbukadiets.ru
babieca.com babieca.com
baditri.com babs.com.ua
barnfurnituremart.com best-seo-offer.com
bestcoin.cc bestimagecoollibrary.com
bestmobilityscooterstoday.com bet-prognoz.com
bestserials.com beta.hotkeys.com
bestwrinklecreamnow.com birzha-truda.eu
bioca.org bitcoin-ua.top
bird1.ru brokergid.ru
bitcoinpile.com bscodecs.com
bloggers.nl call-of-duty.info
blpmovies.com camdolls.com
bobinoz.com candelluxsklep.pl
bonanza-fortune.men canyougethighofftizanidine.blogspot.com
braindaily.xyz cardsharp1.ru
brains2.biz carfax.com.ua
burkesales.com carsplate.com
butstrap.space cartierlove.xyz
canrioloadm.gq cartujano-pre.de
cascadelink.org chiblackhawks-jerseys.com
centrdebut.ru chuckguilford.com
chatroulette.si ciproandtizanidine.blogspot.com
cienum.fr cityadspix.com
circlesl.com club-musics.ru
clickpapa.com codq.info
com-supportcenter.website compliance-irvin.top
concealthyself.com connectionstrenth.com
concordexoticrentals.com continent-e.tv
coolwallpapers-hd.com converse.ddsoldes.fr
cowblog.fr cookingmeat.ru
coolgramgoods.com
corta.co
counciltally.com
countercrazy.com
cowlmash.com cowlmash.com
culad.com cracksplanet.com
cyse.tk customsua.com.ua
dailystrength.org dame-ns.kz
dangphoto.trade daooda.com
diminishedvalueoforegon.com day-news.info
dirtpics.pw dealitright.click
domik-derevne.ru degerlund.net
dustyorate.com delfin-aqua.com.ua
e-collantes.com derevesendeco.com
ege-essay.ru design-lands.ru
elektronischezigarettekaufen1.beeplog.com dev.citetick.com
erot.co dichvuvesinhhanoi.com
dinosaurus.site
dipstar.org
doctormakarova.ru
domaincrawler.com
domcran.net
dyshagi.ru
e-commerce-seo1.com
editmedios.com
eldiariodeguadalajara.com
elstal.com.pl
emediate.eu
engines-usa.com
erotikchat-24.com
etm-consult.de
event-tracking.com
evrotekhservis.ru evrotekhservis.ru
extstat.com exonline.info
ezigarettekaufen1.hpage.com familienzahnaerzte.com
fable.in.ua familyholiday.ml
failingmarriege.blogspot.com fashionindeed.ml
fanrto.com feorina.ru
fba-mexico.com fitnesspiks.com
fettefrauen.net folowsite.com
filesvine.com forsex.info
flashlarevista.com forum-engineering.ru
free-social-buttons.xyz foto-telok.net
freecamdollars.com foxydeal.com
frighteningremain.cf freesitetest.com
frivgame250.com freshberry.com.ua
gaytube.com gallerily.com
ge0ip.com gamedayassist.com
gomusix.com gamingspark.com
goroda-vsego-mira.ru gateway.zscalerone.net
gq-catalog.gq genericviagrasildenafiled.net
handicapbathtubarea.com get-your-social-buttons.info
help.tpu.ru girlspicsa.com
homeandhealth.ru go2album.com
homegardenlova.com goodnightjournal.com
hspline.com gosarhivrt.ru
hyhj.info gwebtools.com.br
idegenvezeto.eu h2monline.com
impresagaia.it hacktougroup.ru
inet-traffic.com hao123.com
internet-apteka.ru healing-dysplasia.ru
iptool.xyz hundejo.com
iqbazar.ru ibmdatamanagement.co
isoveti.ru igadgetsworld.com
itsdp3.com imadedinner.net
jbl-charge.info imgata.com
journeydownthescale.info imperiafilm.ru
junglenet-a.akamaihd.net intelhdgraphicsgtdrive6w.metroblog.com
kazan.xrus.org iphantom.com
keki.info iron-age.info
kerwinandcariza.com jus0wil.pp.ua
kol-energo.ru kamagragelusa.net
kosmetyki.tm.pl keywordspay.com
kupit-adenu.ru khafre.us
lapitec.eu kinobest.pl
ldrtrack.com kotaku.com
le-clos-des-alouettes.com krafte.ru
lion.playtap.us kraljeva-sutjeska.com
lionking-1994.blogspot.com lacave.ntic.fr
littlesunraiser.com loadopia.com
llastbuy.ru longadventure.com
lombia.co losangeles-ads.com
lookingglassemb.com magento-crew.net
louboutinshoes.xyz markjaybeefractal.com
lsp-awak-perikanan.com marsgatan.com
lumb.co massage-info.nl
machicon-ueno.info masterseek.com
magicplayer-s.acestream.net mastershef.club
maglid.ru mathgym.com.au
manipulyator-peterburg.ru matrixalchemy.com
marktforschung-stuttgart.com mcadamssupplyco.com
mbiologi.ru medanestesia.ru
meddesk.ru mezaruk.info
medkritika.ru midst.eu
mega-bony-2017.pl millionare.com
mega-polis.biz.ua mindbox.co.za
megagrabber.ru misswell.net
mexicotravelnet.com mockupui.com
minyetki.ru monclerjacketsoutlet.pw
mnimmigrantrights.net moscow-clining.ru
mojowhois.com muschisexbilder.com
moroccosurfadventures.com myhealthcare.com
moyaterapiya.ru najaden.no
mp3ritm.top nakozhe.com
msk.afora.ru netcheckcdn.xyz
mxgetcode.com nodup.ru
myhitmp3.club oktube.ru
narutonaruto.ru one.net.in
naturtreenspicerx.pw online-templatestore.com
nfvsz.com oxford-book.com.ua
nordstar.pro paleohub.info
notebook-pro.ru pammik.ru
noumeda.com penisvergrotendepillennl.ovh
novosti-hi-tech.ru perfectpracticeweb.com
ontargetseo.us perm-profnastil.ru
orel-reshka.net pio.polytopesexempt.com
papasdelivery.ru playboyfiles.xblog.in
park.above.com
partnersafe.men
pictures-and-images.com
pinkduck.ga
piski.top
playfortuna-play.ru playfortuna-play.ru
playtap.us pornhub-forum.ga
pochtovyi-index.ru pornoblood.com
pomoc-drogowa.cba.pl pornoslive.net
pony-business.com portal-eu.ru
porn555.com powc.r.ca.d.sendibm2.com
pourvous.info pozdravleniya-c.ru
pr0fit-b0x.com pravoholding.ru
praisong.net prostitutki-odessa.org
prchecker.info pskcijdc.bloger.index.hr
pro-poly.ru
proposal-engine.com
prostitutki-rostova.org
prostitutki-tyumeni.org
psvita.ru psvita.ru
quangcaons.com quick-seeker.com
regdefense.com quickbuck.com
remorcicomerciale.ro quickchange.cc
renewablewealth.com radiodigital.co
rank-checker.online
ratemodels.net
rating-bestcasino.com
realting-moscow.ru
registrysweeper.com
ricorsogiustizia.org ricorsogiustizia.org
rixpix.ru riders.ro
ruscoininvest.company rockma.se
sanatorrii.ru ros-ctm.ru
sarm.tk ru-mediaget.ru
saveriopiazza.it rumamba.com
scanner-elena.top running-line.ru
scanner-jane.top rus-pornuha.com
scanspyware.net rus-teh.narod.ru
sdi-pme.com rvzr-a.akamaihd.net
seadragonherbery.com sack.net
searchmywindow-a.akamaihd.net sape.top
seo-traffic-ranking.info scanner-george.top
serptehnika.ru scanner-julia.top
sheerseo.com scanner-marwin.top
shtaketniki.ru scanner-walt.top
sirpornogratis.xxx scrapinghub.com
sisiynas.ru search-goo.com
site-speed-check.site seorussian.ru
spb-plitka.ru sevenstars7.com
spb.ru sexad.net
starpages.net sextracker.de
stroimajor.ru shinikiev.com.ua
sukarame.net shopvilleroyboch.com.ua
supermama.top sitevalued.com
surcentro.com skanninge.se
technopellet.gr skutecznetabletkinaporostwlosow.pl
the-torrent-tracker.blogspot.com snsdeainavi.info
thepantonpractice.co.uk social-buttons.xyz
thepokertimer.com strv.se
ticketsys.inetwd.com suchenindeutschland.com
sugvant.ru
survival.betteroffers.review
tattooha.com
terraclicks.com
time-japan.ru time-japan.ru
titelhelden.eu tizanidinecost.blogspot.com
tizanidinepills.blogspot.com tizanidinenarcotic.blogspot.com
torrents.life tomatis.gospartner.com
track112.site topdownloads.ru
trackzapper.com tramadolandtizanidine.blogspot.com
trustedmaccleaner.com trk-4.net
tula.howotorg.ru uni.me
turizm.bz uprour.com
turizmus.us
ugguk.online
ukkelberg.no
ultimateclassicrock.com
uptimebot.net uptimebot.net
valmetrundan.se urlcut.ru
varbergsvind.se urlopener.com
vavilone.com
viagengrarx.com
videochat.life
videochat.world videochat.world
vitoriacabos.com vkgaleria.com
vladimir.xrus.org vsexkontakte.net
voltrknc1.com websites-reviews.com
voprosotvet24.ru weekes.biz.tc
vpnmouse.com westen-z.life
vse-pesni.com worldis.me
vvpg.ru woweb.com.ua
vysigy.su xaylapdiendanang.com
waterpurifier.club xmlinde.com
webenlace.com.ar xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
weblo.com xn--80ahdheogk5l.xn--p1ai
webnode.me xn--90acenikpebbdd4f6d.xn--p1ai
webtherapy.ru xxxguitars.com
wma-x.com xxxnatelefon.ru
worldmusicfests.com yatrk.xyz
xuki.us yellowads.men
yaoguangdj.com zlatnajesen.com
yellowstonevisitortours.com zootoplist.com
yoshkarola.zrus.org
zeroredirect1.com

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.97 0; 10.1.0.122 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2872 ### Version: V4.2022.01.2873
### Updated: Sun Jan 23 10:03:25 UTC 2022 ### Updated: Sun Jan 23 22:01:23 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2872 ### Version: V4.2022.01.2873
### Updated: Sun Jan 23 10:03:25 UTC 2022 ### Updated: Sun Jan 23 22:01:23 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##