vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 11:10:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.01.3541 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-01-08 22:07:24 +00:00
parent 84daa6fa60
commit c6da7b6bcd
47 changed files with 97367 additions and 97360 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3540 #### Version: V4.2023.01.3541
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3540 #### Version: V4.2023.01.3541
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2023.01.3540 #### Version: V4.2023.01.3541
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12026
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2023.01.3540 #### Version: V4.2023.01.3541
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12030
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12030
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12030
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12030
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 1; 10.1.0.27 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 0; 10.1.0.27 0;
127.0.0.1 0; 127.0.0.1 0;

12030
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 1; 10.1.0.27 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 0; 10.1.0.27 0;
127.0.0.1 0; 127.0.0.1 0;

12030
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 1; 10.1.0.27 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 0; 10.1.0.27 0;
127.0.0.1 0; 127.0.0.1 0;

12030
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3540 3541

12030
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12030
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.45.204.50 0; 138.91.87.48 0;

12030
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 1; 10.1.0.27 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 0; 10.1.0.27 0;
127.0.0.1 0; 127.0.0.1 0;

12030
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 1; 10.1.0.27 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 0; 10.1.0.27 0;
127.0.0.1 0; 127.0.0.1 0;

12030
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 1; 10.1.0.27 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 0; 10.1.0.27 0;
127.0.0.1 0; 127.0.0.1 0;

12030
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.45.204.50 0; 138.91.87.48 0;

12030
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.45.204.50 0; 138.91.87.48 0;

12030
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 1; 10.1.0.27 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Citoid G-i-g-a-b-o-t
DomainCrawler Lightspeedsystems
HTMLparser LinksManager
NetZIP Pavuk
PxBroker Pockey
RankActive Probethenet
Spbot Rankivabot
lwp-trivial VCI
serpstatbot Voil
zgrab sogouspider

282
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 404enemy
80legs
AIBOT AIBOT
ASPSeek ASPSeek
Abonti Aboundex
Acunetix AfD-Verbotsverfahren
AiHitBot
Aipbot Aipbot
Alexibot AllSubmitter
AlphaBot Anarchy99
Ankit
Anthill
Apexoo Apexoo
Asterias Asterias
BDCbot Atomseobot
AwarioRssBot
BDFetch BDFetch
BLEXBot
BackDoorBot BackDoorBot
BatchFTP BackStreet
Battleztar Bazinga BacklinkCrawler
Bigfoot Bandit
BlackWidow BetaBot
Blackboard Bitacle
Blow
Boardreader
Bolt Bolt
BotALot BotALot
Brandprotect
Buck Buck
BuiltBotTough Buddy
BuiltWith
Bullseye
BunnySlippers BunnySlippers
CATExplorador BuzzSumo
CCBot CCBot
CODE87 CODE87
Calculon
Cegbfeieh Cegbfeieh
CensysInspect ChinaClaw
Chlooe Chlooe
Citoid
Claritybot
Cloud mapping Cloud mapping
Cogentbot Copyscape
Copier Cosmos
Craftbot Craftbot
CrazyWebCrawler Crescent
Curious Curious
CyotekWebCopy CyotekWebCopy
DBLBot
DSearch DSearch
DataCha0s DataCha0s
Demon Devil
Deusu Digincore
Dirbuster DigitalPebble
Disco
DnBCrawler-Analytics DnBCrawler-Analytics
DomainAppender DnyzBot
DomainSigmaCrawler DomainSigmaCrawler
Domains Project Domains Project
Dotbot Dotbot
Download Wonder EMail Wolf
Dragonfly Ebingbong
EMail Siphon
EirGrabber EirGrabber
EroCrawler EroCrawler
Evil Evil
Exabot Extreme Picture Finder
Ezooms
FDM
FHscan FHscan
Fimap
Flunky Flunky
Foobot
FyberSpider FyberSpider
GT::WWW Genieo
GermCrawler
GetRight GetRight
GetWeb
GoZilla GoZilla
Gotit Gotit
GrabNet GrabNet
GrapeFX
GridBot GridBot
HTTP::Lite HTMLparser
Havij
Heritrix Heritrix
HybridBot Hloader
IDBot Humanlinks
IRLbot Image Fetch
Iblog Image Sucker
IndeedBot IndeedBot
InfoNaviRobot Internet Ninja
InfoTekies Iskanie
IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT Jbrofuzz
JetCar
Jetty Jetty
Joomla JikeSpider
JustView
Jyxobot Jyxobot
Keybot Translation-Search-Machine
Keyword Density
Kozmosbot Kozmosbot
LWP::Simple Lftp
Lanshanbot
Larbin
Libwhisker
Lightspeedsystems
LinkScan
Linkbot Linkbot
LinkpadBot LinkextractorPro
LinqiaMetadataDownloaderBot LinksManager
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot MFC_Tear_Sample
Lipperhey Spider MIDown tool
Lmspider
MIIxpc MIIxpc
MJ12bot MJ12bot
Mag-Net MSFrontPage
Majestic SEO Majestic-SEO
MarkWatch MarkMonitor
Mass Downloader Mass Downloader
Masscan Masscan
Mata Hari Mediatoolkitbot
Meanpathbot
MegaIndex.ru MegaIndex.ru
Metauri
Microsoft Data Access Microsoft Data Access
Minefield
Moblie Safari Moblie Safari
Morfeus Fucking Scanner Mojeek
Mozlila Mojolicious
NICErsPRO NICErsPRO
Nameprotect Name Intelligence
Navroad NetMechanic
Net Vampire NetZIP
Nettrack
Netvibes Netvibes
Niki-bot Nimbostratus
NimbleCrawler Ninja
Nmap Nmap
Octopus Not
Offline Explorer Nuclei
Nutch
Offline Navigator
OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS OutclicksBot
OrangeSpider
PECL::HTTP PECL::HTTP
PageScorer POE-Component-Client-HTTP
PageThing.com PageThing.com
Panscient Panscient
Pavuk Petalbot
Picscout Picscout
Pixray Picsearch
PleaseCrawl PleaseCrawl
Pockey
ProWebWalker ProWebWalker
Probethenet
Psbot
Pump Pump
PyCurl PyCurl
QueryN Metasearch Quick-Crawler
RankingBot RankurBot
RankingBot2 RealDownload
Rankivabot Ripper
RebelMouse RocketCrawler
Recorder Rogerbot
SEOkicks SEOlyticsCrawler
SEOprofiler SEOprofiler
SEOstats SEOstats
SMTBot
ScanAlert ScanAlert
Scanbot
ScoutJet
Scrapy Scrapy
Screaming Screaming
ScrepyBot Seekport
Searchestate Semrush
SemrushBot SentiBot
SeoSiteCheckup
SeobilityBot
Seomoz Seomoz
SiteSucker SiteSnagger
SocialRankIOBot SlySearch
Snapbot
Snoopy
Sociscraper Sociscraper
Sogou web spider Sogou web spider
Sosospider Sosospider
Sottopop
SpaceBison
Spammen Spammen
SpankBot SpankBot
Spanner
Spbot
Spinn3r Spinn3r
SputnikBot SputnikBot
Sqlmap Sqlmap
Sqlworm Sqworm
Steeler Sucker
Sucuri SuperBot
Surfbot
SurveyBot SurveyBot
Suzuran
Swiftbot Swiftbot
T0PHackTeam T8Abot
Teleport Teleport
Telesoft TeleportPro
Telesphorep Telesphorep
Thumbor TheNomad
TightTwatBot TightTwatBot
Titan
Toweyabot Toweyabot
Trendiction Turingos
True_Robot
TurnitinBot TurnitinBot
TwengaBot TwengaBot
Typhoeus
URLy.Warning URLy.Warning
UnisterBot UnisterBot
Upflow V-BOT
Vacuum
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VidibleScraper VidibleScraper
VoidEYE Voil
Voltron Voltron
WBSearchBot WBSearchBot
WISENutbot
WWW-Collector-E
WWW::Mechanize WWW::Mechanize
Wallpapers/3.0 WWWOFFLE
WallpapersHD Wallpapers
Web Auto
Web Collage Web Collage
Web Fetch Web Fuck
Web Sucker Web Sucker
WebBandit WebBandit
WebEnhancer
WebFuck
WebPix
WebReaper
WebSauger WebSauger
WebStripper WebStripper
WebWhacker WebWhacker
Website Quester WebZIP
Webster Webshag
WebsiteExtractor
Whack Whack
WinHTTrack Wonderbot
Wprecon
Xaldon WebSpider
Xaldon_WebSpider
YoudaoBot YoudaoBot
Zauba Zeus
Zermelo ZmEu
ZumBot ZumBot
autoemailspider arquivo-web-crawler
coccocbot arquivo.pt
cognitiveseo clark-crawler
com.plumanalytics com.plumanalytics
crawl.sogou.com eCatch
crawler.feedback facebookscraper
domainsproject.org gopher
heritrix heritrix
iubenda-radar instabid
linkdexbot
lwp-trivial lwp-trivial
muhstik-scan meanpathbot
page scorer
pcBrowser pcBrowser
plumanalytics polaris version
satoristudio.net ripz
scalaj-http s1z.ru
scan.lol scan.lol
seoscanners seobility
sexsearcher sexsearcher
sitechecker.pro
siteripz siteripz
sogouspider sogouspider
sysscan tAkeOut
trendiction.com x09Mozilla
webmeup-crawler x22Mozilla
webpros.com zgrab
webprosbot

308
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker
80legs 80legs
ADmantX ADmantX
AIBOT AIBOT
Abonti ALittle Client
Aboundexbot Aboundexbot
AfD-Verbotsverfahren
AiHitBot
Aipbot
Alligator
AlphaBot
Anarchie Anarchie
Anarchy
Asterias Asterias
AwarioRssBot AwarioRssBot
BDFetch AwarioSmartBot
Bandit BLEXBot
Barkrowler Backlink-Ceck
BetaBot BacklinkCrawler
Bigfoot Badass
BatchFTP
Bitacle Bitacle
Black Hole Black Hole
BlackWidow Blackboard
Blow Blow
BlowFish Buck
Brandprotect BuiltWith
Brandwatch CSHttp
Buddy
BuiltBotTough
Bullseye
BuzzSumo
CCBot
Cegbfeieh Cegbfeieh
CensysInspect
CheTeam CheTeam
CherryPicker CherryPicker
ChinaClaw ChinaClaw
Citoid Chlooe
Cogentbot Cliqzbot
Collector Cloud mapping
Cocolyzebot
Cosmos
Craftbot Craftbot
Crescent Crawling at Home Project
CrunchBot CrunchBot
DBLBot
DTS Agent
DatabaseDriverMysqli
Deusu Deusu
Digincore DigitalPebble
DnBCrawler-Analytics Disco
DomainAppender Discobot
Dispatch
DnyzBot
DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
Download Wonder
Dragonfly
Drip Drip
ECCP/1.0 ECCP/1.0
EirGrabber
Evil
Exabot Exabot
EyeNetIE Express WebPictures
ExtLinksBot
Extractor
ExtractorPro
Extreme Picture Finder
FDM
FHscan FHscan
Fimap FemtosearchBot
FrontPage Firefox/7.0
GT::WWW Flunky
G-i-g-a-b-o-t
Genieo
GetRight GetRight
Go!Zilla Getintent
Grabber Gigabot
Go-Ahead-Got-It
GoZilla
GrabNet
GrapeshotCrawler
GridBot GridBot
HEADMasterSEO HEADMasterSEO
HTMLparser HTMLparser
HTTP::Lite HTTrack
Haansoft HaosouSpider
Havij
Heritrix Heritrix
Hloader HonoluluBot
IDBot HybridBot
Iblog IRLbot
IlseBot
Image Fetch Image Fetch
Indy Library
InfoNaviRobot
InfoTekies InfoTekies
Intelliseek InterGET
Internet Ninja
InternetSeer InternetSeer
IstellaBot JOC Web Spider
JamesBOT Jbrofuzz
JennyBot Jyxobot
Jorgee
Kenjin Spider Kenjin Spider
Keybot Translation-Search-Machine
Keyword Density
Kinza Kinza
Kozmosbot
LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot Leap
Larbin
LeechGet
LexiBot LexiBot
Lftp
Libwhisker
LieBaoFast
Lightspeedsystems Lightspeedsystems
Likse LinkextractorPro
LinkScan MIDown tool
Litemage_walker MJ12bot
Ltx71
MIIxpc
MQQBrowser MQQBrowser
MSIECrawler
Magnet
Majestic SEO
Majestic12 Majestic12
MarkMonitor
MarkWatch MarkWatch
MauiBot Mass Downloader
Mb2345Browser Mata Hari
MeanPath Bot MeanPath Bot
Mediatoolkitbot Microsoft URL Control
Minefield
Mister PiX Mister PiX
Mojeek
Mojolicious Mojolicious
Mozlila Morfeus Fucking Scanner
Mr.4x3 Mr.4x3
Nameprotect Msrabot
NPbot
Name Intelligence
Navroad Navroad
Needle Net Vampire
Nessus NetAnts
NetLyzer Netcraft
NetMechanic Nettrack
Netvibes Netvibes
Niki-bot Nibbler
Nikto
Nimbostratus
Not Not
Nutch
Octopus Octopus
Offline Navigator Offline Explorer
OnCrawl OpenVAS
OpenLinkProfiler Openfind
OrangeBot OrangeBot
OrangeSpider OrangeSpider
OutclicksBot OutclicksBot
PageAnalyzer PECL::HTTP
Panscient PageThing.com
Pavuk Pandalytics
PeoplePal PeoplePal
Picscout Petalbot
Picsearch
PictureFinder PictureFinder
Piepmatz
Pimonster Pimonster
PleaseCrawl Pixray
Psbot Pockey
Pu_iN Pu_iN
PxBroker Pump
RSSingBot Quick-Crawler
RankFlex RankFlex
RankingBot2 RankingBot2
Rankivabot
Re-re
ReGet ReGet
RebelMouse RealDownload
SISTRIX Reaper
Recorder
SEOkicks
SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler
SEOstats
SMTBot SMTBot
ScanAlert Scanbot
Scrapy
Screaming Screaming
ScreenerBot
SearchmetricsBot SearchmetricsBot
Semrush Semrush
SemrushBot SentiBot
SeoSiteCheckup SeobilityBot
Seomoz Seomoz
Shodan
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
Sitebeam SiteSnagger
Siteimprove
Sitevigil
SmartDownload SmartDownload
Snake
Snapbot Snapbot
SocialRankIOBot Snoopy
Sociscraper Sociscraper
SpaceBison Sogou web spider
Sosospider
Spammen
Spbot Spbot
Sqlmap Sqworm
Sucuri Steeler
SuperHTTP Stripper
SurveyBot Sucker
Suzuran Swiftbot
Szukacz T0PHackTeam
Teleport T8Abot
Telesphorep Telesphorep
TheNomad The Intraformant
Thumbor
TightTwatBot
Titan
Toata Toata
Tracemyfile Toweyabot
Turingos True_Robot
Turnitin
TwengaBot TwengaBot
Typhoeus Typhoeus
UnisterBot
Upflow Upflow
V-BOT V-BOT
VCI Vagabondo
Vacuum
VelenPublicWebCrawler
Virusdie Virusdie
Voil Voltron
WASALive-Bot WWW::Mechanize
WEBDAV
WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD WeSEE
Web Auto Web Fuck
Web Fetch
Web Pix
WebBandit
WebCollage WebCollage
WebEnhancer WebCopier
WebFetch WebGo IS
WebImageCollector
WebLeacher
WebPix WebPix
WebReaper WebStripper
WebSucker WebSucker
WebWhacker WebWhacker
Whacker Whack
Whatweb
Widow Widow
Wonderbot WiseGuys Robot
Woobot Woobot
Wotbox Wotbox
Wprecon
Xaldon WebSpider Xaldon WebSpider
Xenu
Zauba
Zermelo Zermelo
ZmEu ZoomBot
ZoominfoBot
ZyBorg
archive.org_bot
arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider autoemailspider
backlink-check backlink-check
cah.io.community
cognitiveseo
crawl.sogou.com crawl.sogou.com
crawler4j
demandbase-bot demandbase-bot
evc-batch eCatch
facebookscraper
gopher gopher
heritrix
ips-agent
iubenda-radar
linkdexbot linkdexbot
lwp-request lwp-request
meanpathbot meanpathbot
muhstik-scan
netEstate NE Crawler
oBot oBot
page scorer
plumanalytics plumanalytics
polaris version
probe-image-size
satoristudio.net
seobility
seocompany.store
seostar seostar
serpstatbot
sitechecker.pro
sogouspider
sp_auditbot
sysscan sysscan
voyagerx.com tAkeOut
zgrab trendiction.de
webgains-bot
webmeup-crawler
xpymep1.exe
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
2nt.ru adsloads.com
amospalla.es adtech.us
autogrep.ru casinorewards.info
backlinkwatch.com co.lumb.co
doxysexy.com elektrozigaretten1.postbit.com
filunika.com.ru goforexvps.com
ihtec2019.org happy.new.yeartwit.com
storehouse.ua journeydownthescale.info
tradedeals.biz mebel-alait.ru
trahic.ru xn--c1acygb.xn--p1ai

938
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
000free.us 000free.us
08800.top 1008.su
11235813.webzdarma.cz 101raccoon.ru
1j7740kd.website 12bet.com
2drittel.de 12u.info
4webmasters.org 13tabs.com
5kstore.com 1bet.com
6128786.com 1qingdao.com
7ooo.ru 1xbet4you.com
991mostfm.co.id 4webmasters.com
aa8b68101d388c446389283820863176e7.com 51unlim.ru
aa9d046aab36af4ff182f097f840430d51.com 7fon.ru
aac52006ec82a24e08b665f4db2b5013f7.com 7wind.ru
abbanreddy.com 7yue.org
academiacsmendoza.org abclauncher.com
activepr.ru acads.net
adhome.biz acgs.tk
adpremium.org adclickservice.com
adtech.de adclickthru.net
adult-shop.com.ua adconscious.com
addbin.men
adimmix.com
admanaerofoil.com
admitad.com
adtech.fr
adventureparkcostarica.com adventureparkcostarica.com
ahrefs.com agecheckadult.com
aktobe.xkaz.org akuhni.by
alborzan.com alborzan.com
alekseevec.ru
alert-fdm.xyz
alert-fjg.xyz alert-fjg.xyz
alibestsale.com alessandraleone.com
alkoravto.ru all4wap.ru
allkrim.com allproblog.com
alot.com allwomen.info
angigreene.com alpharma.net
animali.deagostinipassion.it altamayoztourism.com
animalrank.com android-vsem.org
animaltoplist.com anmysite.com
anonymizeme.pro apartmentbay.ru
apartment.ru apiqualitinknet-a.akamaihd.net
apartmentratings.com approved.su
apelsinnik.website artdeko.info
apiallgeniusinfo-a.akamaihd.net atelielembrancaqueencanta.com.br
applicationg29.com atleticpharm.org
arcteryxsale.online atout-energie-69.com
asia-forum.ru automobile-spec.com
asseenontv.ru aviav.ru
atlasvkusov.ru aviav.ru.com
audiobangout.com avtovolop.ru
autogrep.ru awency.com
autolombard-krasnodar.ru ayakino.net
autotop.com.ua ayerbo.xhost.ro
autowebmarket.com.ua baditri.com
avirasecureserver.com balitouroffice.com
avramstroy.ru bamo.xsl.pt
avtolombard-krasnodar.ru bank.uz
azartmsl.com bausparen.bz.it
babespcs.com bdsmgalls.net
bannerbank.ru beetpics.pw
bannerpower.com
batanga.net
beachpics.com
bear.gotcher.us
beauby.ru
beauty-b0x.pl
beauty-things.com
begalka.xyz
berdasovivan.ru
bestbrainprod.win bestbrainprod.win
bestfortraders.com bet-prognoz.com
bestmouthwash.club bezcmexa.ru
bestofpicture.com bitcoinremote.com
beta.hotkeys.com bjanshee.ru
bharatdefencekavach.com bjorkbacken.nu
blavia.00author.com blog-hits.com
blog.koorg.ru bloggerads.net
blog.remote-computer.de bluerobot.info
bongacams.com bonky.biz
bonzbuddy.com brainsandeggs.blogspot.com
brandov.ru bugof.gq
bristolhotel.com.ua burger-tycoon.com
bumskontakte.org buron.pw
buyclomidonlaine.com businesn.men
bvlgariwallet.xyz buy-cheap-pills-order-online.com
cactussoft.biz buyessayonline19.blogspot.ru
canacopegdl.com buyhoverboard.com
candiceloves.us bvlgariring.xyz
candypeople.se bycontext.com
capsderfudd.tk camel-beach.com
carrauterie.be canoncdriverq3.pen.io
cartierbracelet.xyz casinorewards.info
cartierlovestore.com cercacamion.it
cartierlovestore.xyz cfsrating.sonicwall.com
cartierring.xyz cheapjerseysap.com
casinopinup-wins.com cherubinimobili.it
casinos4dummies.co.uk chidporn.com
cdn.walkme.com chinacheapelitejerseys.com
cdnanalytics.xyz cialis-samples.com
certifywebsite.win cleanersoft.com
cfjump.com clickzzs.nl
chatroulette.life clodo.ru
cheatcode-lita12.rhcloud.com clubfashionista.com
checktext.org coindirect.io
chee-by.biz
chiblackhawks-jerseys.com
christianlouboutinreplica.pw
cienum.fr
cjs.com.ru
clarithromycin500mg.com
cleanallvirus.com
cmhomestayagency.com
coinsspb.com coinsspb.com
comissionka.net com-onlinesupport.site
compliance-alex.top compliance-andrew.top
compliance-alexa.xyz compliance-irvin.xyz
compliance-fred.xyz compliance-jack.top
compliance-walt.top compliance-john.top
concordexoticrentals.com concordexoticrentals.com
converse.ddsoldes.fr conocer-sanabria.com
cookie-law-enforcement-aa.xyz cookie-law-enforcement-ee.xyz
counter.yadro.ru cosmeticswomens-womensports.rhcloud.com
covetnica.com cougfan.info
crazyprotocol.com covid-schutzmasken.de
cxpromote.com cranly.net
dentfidemountpreach.tk cyse.tk
deplim.com d2jsp.org
dev33.dioniqlabb.se daymusam.com
dev78.dioniqlabb.se ddospanel.com
dgfitness.co dekorkeramik.ru
directivepub.com detki-opt.ru
dev.citetick.com
div.as div.as
dogbreedspicture.net dktr.ru
dollartree.info dmmspy.com
domain-submit.info dnsrsearch.com
domain2008.com dokumentalkino.net
domainsphoto.com dolohen.com
download-wallpaper.net domaincheck.io
downloader12.ru domainsatcost.com
downloadmefiranaratb1972.xpg.com.br domination.ml
droidlook.net donna7753191.ru
drunkenstepfather.com dorratex.tn
drupa.com downloads-whatsapp.com
ecookna.com.ua downvids.net
educontest.net drugspowerstore.com
electro-prom.com dwomlink.info
elektrischezigarettekaufen2.cowblog.fr e2click.com
elektroniksigaraankara.info ed-shop01.ru
elektrozigaretten2.yn.lt edge.sharethis.com
elvenmachine.com education-cz.ru
emergencyneeds.org egovaleo.it
empirepoker.com ekaterinburg.xrus.org
englishgamer.com electrik-avenue.com
eropho.com elektrischezigaretten2.beeplog.com
evehemming.blogspot.com.au elidelcream.weebly.com
evrotekhservis.ru elmacho.xyz
extstat.com eng-lyrics.com
es5.com
eshop4u.jp
essayservicewriting.org
este-line.com.ua
etenininrade.ga
everypony.ru
evidencecleanergold.com
exct.net
eyessurgery.ru
ezigarettekaufen1.hpage.com ezigarettekaufen1.hpage.com
ezigarettenshop1.yolasite.com ezigarettenshop1.yolasite.com
falcoware.com ezigaretteshop2.vefblog.net
fable.in.ua
facebook-mobile.xyz
fanoboi.com fanoboi.com
fdzone.org fapgon.com
fettefrauen.net fashion.stellaconstance.co
fikasound.tk fast-torrent.ru
filmbokep69.com ferieboligkbh.dk
fix-website-errors.com ff30236ddef1465f88547e760973d70a.com
floridamhca.org fickkontaktehobbyhuren.com
flowersbazar.com fifa-coins.online
footballfarrago.com fil.ru
forsex.info filmetricsasia.com
frameimage.org filmidivx.com
freakycheats.com financehint.eu
freemaintenancesysforpcandmac.top fireads.men
fsalas.com firstsiteguide.com
fuel-gas.com flash4fun.com
gateway.zscalertwo.net focalink.com
ge0ip.com fodelsedagspresenter.nu
germetiki.com.ua forum69.info
getmyads24.com foxjuegos.com
getprismatic.com freeuploader.ml
ghernnqr.skyrock.com freza-sverlo.ru
ghostvisitor.com fuckbuddybestgilf.info
glall.ru fyxabomiw.ru
glass-msk.ru g.starmoe.xyz
gloverid.site g7m.pl
goldpanningtools.com gabeshop.ru
grand-chlen.ru galeria-zdjec.com
gsou.cf gambarkataku.co
guigyverpo.cf gamebackyard.com
guildebzh.info gamevalue7.weebly.com
gamewrath.com
gdebestkupit.ru
gearcraft.us
gemara.com
get-seo-domain.com
getresponse.com
go2album.com
golden-praga.ru
goroda-vsego-mira.ru
gosarhivrt.ru
gox.com.ua
guiadeserraazul.com
gwhwpxbw.bloger.index.hr gwhwpxbw.bloger.index.hr
handicapvansarea.com
handicapvantoday.com handicapvantoday.com
handsandlegs.ru handsandlegs.ru
hd-film.pl hdapp1008-a.akamaihd.net
hdwallpapers-free.com
heartofbeijing.blogspot.com
herehloadibs.cf herehloadibs.cf
hermesbelts.xyz hidemyass.com
hermesreplica.pw hit-kino.ru
hetmanship.xyz hoverboard360.at
hobild.net hoverboard360.nl
homemature.net hunthillfarmtrust.org
houston-vikings.com hyipmanager.in
howopen.ru idc.com.ua
hoztorg-opt.ru ideawheel.com
idealtits.net ifirestarter.ru
igru-xbox.net iideaidekonkatu.info
ihc2015.info ilovevitaly.xyz
ilmexico.com imperia31.ru
impisr.ru indiakino.net
iplusbit.blogspot.co.za infobanks.ru
isitwp.com insurple.com
istizanidinelikexanax.blogspot.com interfucks.net
itis4you.com intimshop-fantasy.ru
izoll.ru investyb.com
jerseysbizwholesalecheap.com iqbazar.ru
jobgirl24.ru iqoption.pro
joingames.org iron-age.info
jrcigars.com it-max.com.ua
kadashihotel.com itag.pw
kevblog.top ivanovo.zrus.org
keywordsdoctor.com ivoiretechnocom.ci
kfon.eu javrip.net
kinozapas.org javxxx18.com
kit-opt.ru jobmarket.com.ua
kiwi237au.tk joessmogtestonly.com
kleine-titten.biz jyvopys.com
kashubadesign.ru
kasino-money.pw
kazrent.com
kchaxton.com
keywesthideaways.co
kidd.reunionwatch.com
kinohall.ru
kinosed.net
klejonka.info klejonka.info
kndxbkdx.bloger.index.hr
knowyournextmove.com
kol-energo.ru kol-energo.ru
kongruan.com kostenlos-sexvideos.com
koversite.info kountrylife.com
krasnodar.ru kvartir-remont.biz
krasnodar.zrus.org kwzf.net
landreferat.ru landreferat.ru
lazy-z.com laptop-4-less.com
letslowbefast.site larutti.ru
lidoradio.com linkbolic.com
life-instyle.com linkpulse.com
liffa.ru littlecity.ch
limads.men live-sexcam.tk
liver-chintai.org livingroomdecor.info
liverpool.gsofootball.com locustdesign.co
lombardfinder.ru loftdigital.eu
lordzfilmz.me lomza.info
lotto6888.com lookingglassemb.com
lyngdalhudterapi.no lyngdalhudterapi.no
m0r0zk0-krava.ru macfix.life
magda-gadalka.ru mafcards.ru
makis.nu maslenka.kz
malwareremovalcenter.com matrixalchemy.com
masterseek.com maximpartnerspr.com
matpre.top mazda-roadsters.com
maxxtor.eu
mcnamaratech.com mcnamaratech.com
me-ke.com mebel-ekb.com
mediafresh.online medispainstitute.com.au
medmajor.ru mega-bony2017.pl
megavolt.net.ua meltwater.com
mesbuta.info mesbuta.info
minecraft-neo.ru mexicosleevegastrectomy.com
minet.club michaelkorsoutlet.store
michaelkorssaleoutletonline.net
midst.eu
mikozstop.com
mirtorrent.net mirtorrent.net
mlf.hordo.win mitrasound.ru
mmoguider.ru mobot.site
mmstat.com mobstarr.com
moesen-ficken.com moivestiy.biz
monarhs.info moncleroutletonline.win
moncleroutletonline.xyz motherboard.vice.com
mosdverka.ru mountainstream.ms
musicas.baixar-musicas-gratis.com mouselink.co
musirc.com movies-in-theaters.net
mustwineblog.com mrbitsandbytes.com
my-bc.ru mrvideospornogratis.xxx
myanyone.net mtmtv.info
mybinaryoptionsrobot.com musicdaddy.net
mydirtyhobby.com muz-baza.net
myperiod.club my-cash-bot.co
na-telefon.biz my-floor.in.ua
nakozhe.com myblogregistercm.tk
narkologiya-penza.ru mydearest.co
naval.jislaaik.com mydeathspace.com
navalwiki.info mydownloadengine.com
networkcheck.xyz nagdak.ru
newstaffadsshop.club naphukete.ru
njkmznnb.ru netoil.no
noclegonline.info next-dentists.tk
norththeface.store nfljerseys.online
novosibirsk.xrus.org nfljerseyscheapbiz.us
nrjmobile.fr nippon-bearings.ru
nsatc.net no-fuel.org
oecnhs.info notaria-desalas.com
oil-td.ru o00.in
obiavo.by
obiavo.in
onemantrip.com onemantrip.com
online.ktc45.ru onlainbesplatno.ru
online247.ml onthemarch.co
onlinedomains.ru openfrost.com
ops.picscout.com optibuymac.com
optom-deshevo.ru orion-v.com
oralsexfilme.net parajumpersstore.online
origin-my.ru partner-host.men
osagonline.ru pavlodar.xkaz.org
outclicks.net pawli.eu
oynat.info pc-services.ru
p-business.ru pearlisland.ru
pagesense.com pensplan4u.com
panicatack.com phobia.us
papasdelivery.ru picswe.com
parajumpersoutlet.online picturesboss.com
paramountmarble.co.uk pierrehardysale.online
pastaleads.com pinwallpaper.xyz
patol01.pw plohaya-kreditnaya-istoriya.ru
photo.houseofgaga.ru pochtovyi-index.ru
photokitchendesign.com poffet.net
picturesfrom.com pogruztehnik.ru
pillscheap24h.com poiskzakona.ru
pinkduck.ga porngalleries.top
pizza-tycoon.com pornogad.com
pk-pomosch.ru
play-mp3.com
pobeiranie.pl
pochemychka.net
popunder.ru
pornoblood.com
pornokajf.com
pornosok.ru
pornowarp.info pornowarp.info
pornpost.in positive2b.ru
posible.net pourvous.info
praisong.net pozvonim.com
privat-girl.net preg.marketingvici.com
privatbank46.ru princeadvantagesales.com
prizesbook.online prodess.ru
profitfx.online proekt-mos.ru
prospekt-st.ru promalp-universal.ru
prostitutki-tyumeni.org ptr.ruvds.com
pskcijdc.bloger.index.hr
purchasepillsnorx.com
pussysaga.com pussysaga.com
puteshestvennik.com qitt.ru
qor360.com rabot.host.sk
qualitymarketzone.com radiodigital.co
quickchange.cc ragecash.com
ranksonic.org rapidokbrain.com
redbottomheels.xyz ratemodels.net
remmling.de razyboard.com
remont-komputerov-notebook.ru realresultslist.com
rebrand.ly
relayblog.com
rentehno.ru
resellerclub.com
responsinator.com
restorator-msk.ru restorator-msk.ru
reward-survey.net ricorsogiustizia.org
ring4rhino.com rn-to-bsn.com
rostov.xrus.org rniaeba.ga
royal-investments.net robotixix.com
roznica.com.ua rocis.site
runofilms.ru russian-postindex.ru
ruscopybook.com saddiechoua.com
saitevpatorii.com sarafangel.ru
sakhboard.ru sarahmilne.top
santechnik.jimdo.com sarm.tk
sardinie.us saugatuck.com
sashagreyblog.ga sax-sex.com
satoristudio.net scalerite.co.za
savefrom.com scanner-andrew.top
scanner-alexa.top scanner-john.top
scanner-julianna.top screpy.com
school-diplomat.ru se-welding.ru
scrapy.org searchtooknow-a.akamaihd.net
seansonline24.pl securityallianceservices.com
search.alot.com sex-znakomstva.online
searchwebknow-a.akamaihd.net sexad.net
segol.tv sexspornotub.com
semalt.com sexyteens.hol.es
seoanalyses.com sharebutton.to
seokicks.de shemale-sex.net
seorussian.ru shemalegalls.blogporn.in
sexblog.pw shiksabd.com
sexfreepornoxxx.com shoplvlv.us
sexiporno.net shopvilleroyboch.com.ua
sexvporno.ru sildenafil-tadalafil.info
sexyali.com site-analyzer.com
share-buttons-for-free.com site-auditor.online
shoppytoolmac.com
shopwme.ru
si-unique.com
similardeals.net
sispe.com.br
sitevaluation.org
sklad-24.ru sklad-24.ru
skutecznetabletkinaporostwlosow.pl
slkrm.ru
slutloadlive.com
smailik.org smailik.org
smsactivator.ru smartbalanceworld.com
soaksoak.ru snworks.com
social-search.me soc-econom-problems.ru
socialsignals24.com social-s-ggg.xyz
sodexo.com soda.media
softxaker.ru softonicads.com
spymac.net solitaire-game.ru
stal-rulon.ru solmarket.by
start.myplaycity.com solution4u.com
stop-gepatit.te.ua sotechco.co
stop-zavisimost.com soviet-portal.do.am
stopnarco.ru spinazdrav.ru
store-rx.com sports-supplements.us
streha-metalko.si statustroll.com
steebook.com
stockquotes.wooeb.com
strfls.com
strongholdsb.ru
students-cheapskate.ml students-cheapskate.ml
sukarame.net subj.ukr-lit.com
sukirgenk.dvrlists.com svarbit.com
supermama.top svetodiodoff.ru
suralink.com swingerseiten.com
surgut.zrus.org
sweepstakes.rewardit.com
symphonyintegratedhealthcare.com
tako3.com tako3.com
tampabaywatch.org talant-factory.ru
tattooreligion.ru tattomedia.com
teguh.info td-33.ru
telemetryverification.net telemetryverification.net
telesvoboda.ru teletype.in
texbaza.by templates.radiodigital.co
textads.men tfxiq.com
thegioixekhach.com thejournal.ru
thegolfclub.info titan-cloud.life
thruport.com tizanidinehydrochloride4mgstreetvalue.blogspot.com
tizanidine4mgstreetvalue.blogspot.com tizanidineingredients.blogspot.com
tizanidineandcipro.blogspot.com tizanidineinteractions.blogspot.com
tizanidinehcl2mgtablet.blogspot.com tizanidineonline.blogspot.com
tizanidinestreetprice.blogspot.com tizanidineshowupondrugtest.blogspot.com
tmm-kurs.ru tizanidinewithdrawal.blogspot.com
tongkatmadura.info tonivedu.it
toon-families.com topanasex.com
top10-online-games.com topdownloads.ru
topflownews.com torontoplumbinggroup.com
track2.shop tovaroboom.vast.ru
traffic100.com toy-shop.top
trafficnetzwerk.de tracklead.net
trailer.cinemaflix.website traffic-club.info
trainoffend.ml trafficreceiver.club
trudogolik.net traffixer.com
tula.mdverey.ru tramadolandtizanidine.blogspot.com
tupper-posuda.ru trkdf.com
turn-up-life.life trumpetedextremes.com
tversvet.ru uchetunet.su
tvorozhnaja-zapekanka-recept.ru uchil.net
twittrading.com ugguk.online
twu.com.ua umg-stroy.ru
ukrtextbook.com unimodemhalfduplefw.pen.io
ultimateclassicrock.com usbggettwku.ga
undergroundcityphoto.com usswrite.com
url-img.link v-doc.co
usacasino.com vbabule.net
ustion.ru vchulkax.com
utrolive.ru vegan-foods.us
vadimkravtcov.ru videokrik.net
vapsy.com vinylvault.co.uk
velpanex.ru vipsexfinders.com
vertaform.com visitcambridge.org
video-chat.cn voluumtrk.com
videositename.com vote-up.ru
videtubs.pl walkme.com
vkgaleria.com watchmyfb.pl
voltrknc1.com webcamdevochka.com
vtc.pw webnode.me
vykupavto-krasnodar.ru weddingdresses.xyz
vysigy.su wesharepics.com
wahicbefa31.soup.io wjgony.com
watchinf.com world-mmo.com
wcb.su wrz0iuebwhp5fg.freeddns.com
webinstantservice.com wsgames.ru
weprik.ru wurr.voila.net
wesharepics.info ww2awards.info
westum.se xedserver.com
winx-play.ru xitjw.info
wladimirpayen.com xjrul.com
woman-orgasm.ru xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
word-vorlagen.xyz xn--80aaagvmjabrs1aoc9luc.xn--p1ai
wordpresscore.com xn--b1ag5cfn.xn--p1ai
worldlovers.ru xn--d1acah0c.xn--p1ai
worldtraveler.world xxart.ru
wowcasinoonline.ooo xxxdatinglocal.us
www.get-free-traffic-now.com xxxmania.top
www.jbetting.com xxxrus.org
www.pinnacle-bets.com yeartwit.com
x-musics.com yellowads.men
x-porno.video yellowfootprints.com
xcombear.ru yellowproxy.net
xjlottery.com youthreaders.com
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai zahvat.ru
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai zaim-pod-zalog-krasnodar.ru
xn--80ahvj9e.xn--p1ai zaobao.com.sg
xn--80atua3d.xn--p1ai znakomstva-moskva77.ru
xnxxandxvideos.com znakomstvaonlain.ru
xvideosporn.biz zojirushi-products.ru
xxxtube69.com zscaler.net
xz618.com
yellowstonesafaritours.com
yellowstonevisitortours.com
yougotanewdomain.com
yourtemplatefinder.com
ytmnd.com
yun56.co
zaapplesales.blogspot.com
zarabiaj-dzis.pl
zazagames.org
zerocash.msk.ru
zzbroya.com.ua

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
03p.info
0lovespells0.blogspot.com
10bet.com 10bet.com
5forex.ru 2020iscoming.info
8558l.jobs.net 21h2o.com
aanapa.ru 4ur.click
academiacsmendoza.org 5-steps-to-start-business.com
acgs.tk 8kisses.eu
adman.gr ad-words.ru
adprotect.net addtoadd.men
adultfullhd.com advokateg.xyz
adultnet.in agreda.pluto.ro
advertex.info akusherok.ru
advocatemsk.ru alfa9.com
afmuseum.com
algarveglobal.com
all-streaming-media.com all-streaming-media.com
alphacarolinas.com
amateurgalls.com
ample-awards-today.us
anaksma.info anaksma.info
android4fun.org android-systems.ru
anniemation.com angel1777.kz
anonymousfox.co angry-fermi-7633.arukascloud.io
arabseyes.com animaltoplist.com
arate.ru anonymous-redirect.com
arendakvartir.kz apiadanaknet-a.akamaihd.net
atley.eu.pn apple.com-cleaner.systems
azadnegar.com arcteryxsale.online
b00kmarks.com ardimobileinfo.ml
backiee.com autovideobroadcast.com
baditri.com avon-severozapad.ru
bankofthewext.com azartclub.org
azlex.uz
bannertesting.com bannertesting.com
beachpics.com basedecor.ru
bedandbreakfast.com bbsoldes.fr
best-mam.ru bcmp.org
best-way.men bestcurs.org
beztuberkuleza.ru bio.trade-jp.net
blogrankers.com biteg.xyz
bloke.com bizfly.info
boom-celebs.com blagovest-med.ru
bracketsmackdown.com blogs.rediff.com
brainboostingsupplements.org butstrap.space
brainsandeggs.blogspot.com buy-cheap-online.info
brateg.xyz buyessay3.blogspot.ru
break-the-chains.com buyviagraa.blogspot.com
briomotor.co camdolls.com
bumble.cheapwebsitehoster.com candy-glam-hp.com
calcularpagerank.com cartierlovebracelet.xyz
capsderfudd.tk cdn.walkme.com
clickhype.com cezartabac.ro
cmd.kz chee-by.biz
cocyq.inwtrade.com children-learningreading.info
codq.info clicksor.com
creditmoney.com.ua craftinsta.ru
crynet.cc custom-product-labels.com
custodieva.ru daooda.com
customcollegeessays.net
davebestdeals.com
dawlenie.com
dealitright.click
dieta-personalna.pl
dikx.gdn dikx.gdn
djihispano.com discovertreasurenow.com
documentbase.net doctissimo.fr
domination.ml dojki365.online
downloads-whatsapp.com domaincdn.xyz
doxysexy.com downvids.net
drunkmoms.net dptaughtme.com
dtnlyss.com dynainbox.com
earl-brown.info egvar.net
easyshoppermac.com
ecommerce-seo.org
ed-shop01.ru
ekobata.ru
elektrischezigarette1.blog.pl
elektroniksigaraankara.info elektroniksigaraankara.info
empathica.com
empis.magix.net
enge-fotzen.info
erotikstories.ru erotikstories.ru
esfchat.tk escort-russian.com
euronis-free.com essay-edu.biz
ezigaretteshop2.mywapblog.com etur.ru
f012.de europeanwatches.ru
fashionavenuegame.com extads.net
favorcosmetics.com extremepornos.net
financehint.eu ezigarettekaufen2.blox.pl
find1friend.com facialporntube.com
flooringinstallation-edmonton.com fitness-video.net
formulaf1results.blogspot.com formseo.com
forum.tvmir.org free-social-buttons.xyz
forums.toucharcade.com freemaintenancesysforpcandmac.top
freakycheats.com fuckingawesome.com
frvo.alptandem.ru funtoonez.com
fungamelands.com fxtips.ru
fungirlsgames.net gamingspark.com
galeria-zdjec.com
gay-file.com
gearsadspromo.club
geile-lelly.eu
gembird.com
genetworx.com
getaclueamerica.com getaclueamerica.com
girlsfuckdick.com glavprofit.ru
greenzaim.ru glavtral.ru
grizzlysgrill.com globalscam.ga
growboxbl.ru globatur.ru
h2monline.com golmau.host.sk
hamacapty.com gosreg.amchs.ru
havepussy.com gratuitbaise.com
hd-film.pl gribokstop.com
homemature.net guardlink.com
hothot.ru hccoder.info
hoverboard360.se hdseriale.pl
hvd-store.com hesteel.pl
idegenvezeto.eu hobbyhuren-datenbank.com
iinstalll-fii1leis.jus0wil.pp.ua hornymatches.com
hothor.se
i-service.kz
imagecoolpub.com imagecoolpub.com
imagine-ex.co imediadesk.com
infobanks.ru inboxdollars.com
interferencer.ru internet-apteka.ru
iscblog.info investingclub.ru
iusstf.org iplusbit.blogspot.co.za
kartiniresto.com istripper.com
kasino-money.pw ix20.ru
koopilka.com jerseyswholesalechinalimited.com
labplus.ru jimmychoosale.online
lashstudia.ru jpcycles.com
lechtaczka.net jyvopys.com
lexaprogeneric.link karting196.ru
likesuccess.com kazinogames.lv
linkbux.ru kellyonline.xyz
lirunet.ru ketrzyn.pl
loadingpages.me keywordblocks.com
lowephotos.info kinomaniatv.pl
mainhp.net kleine-titten.biz
marketingtechniques.info konkursov.net
mediaoffers.click landoftracking.com
medmajor.ru languagecode.com
mielec.pl laxob.com
mobile-appster.ru lazy-z.com
moivestiy.biz letmacwork.world
moneymaster.ru letslowbefast.today
moneyzzz.ru likrot.com
morocco-nomad-excursions.com lineavabit.it
mosrif.ru lineshops.biz
moyakuhnia.ru littleberry.ru
liumimgx.pw
liverpool.gsofootball.com
lombia.com
lulea-auktionsverk.se
mainlinehobby.net
marcoislandvacations.net
mbiologi.ru
medkletki.ru
meltwater.com
monarchfind-a.akamaihd.net
moncleroutletonline.pw
mp3downloadhq.com
mrsdalloways.com
mttwtrack.com
muzaporn.com
my-big-family.com my-big-family.com
nagdak.ru myghillie.info
netvouz.com mystats.xyz
nootrino.com myyour.eu
nuker.com nakrutka.cc
obiavo.by navalwiki.info
obiavo.net nfljerseysforsalewholesaler.com
offside2.5v.pl notfastfood.ru
okuos.com oren-cats.ru
onko-24.com painting-planet.com
online-x.ru partner-cdn.men
onlinemeetingnow.com partner-trustworthy.men
onlinewritingjobs17.blogspot.ru picphotos.net
osnova3.ru pillscheap24h.com
ossmalta.com piluli.info
pacificair.com piratecams.com
paighambot.com pirateday.ru
paleohub.info pisanieprac.info
pandarastore.top popunder.net
partner-pop.men popunder.ru
pavlodar.xkaz.org pornobrazzers.biz
perfection-pleasure.ru portal-eu.ru
photosaga.info porto.abuilder.net
pic-re.blogspot.com proekt-mos.ru
picsearch.com prohoster.info
plugingeorgia.com pskcijdc.bloger.index.hr
poffet.net
pojdelo.weebly.com
pokerniydom.ru
printdirectforless.com
printie.com
prodvigator.ua
prostitutki-almata.org
prpops.com
pulseonclick.com
pyramidlitho.webs.com
qexyfu.bugs3.com qexyfu.bugs3.com
raisedseo.com qwesa.ru
randki-sex.com rainbowice.ru
redirect.trafficreceiver.club rankings-analytics.com
registrysweeper.com regionshop.biz
renhacklids.tk remont-komputerov-notebook.ru
rollercoin.com rentaremotecomputer.com
rootandroid.org replica-watch.ru
sad-torg.com.ua resellerclub.com
saddiechoua.com respublica-otel.ru
saneitconsulting.com reversing.cc
sethrollins.net reward-survey.net
sexfreepornoxxx.com risparmiocasa.bz.it
sibecoprom.ru rosbalt.com.ua
sirpornogratis.xxx sakhboard.ru
slowmacfaster.trade saveriopiazza.it
slutloadlive.com scanner-olga.top
smart-scripts.com schoolfiles.net
soft1.ru search-goo.com
sogimlecal.tk search.alot.com
splendorsearch-a.akamaihd.net searchlock.com
sport7777.net segol.tv
spywarebegone.com seoprofiler.com
sr-rekneskap.no ship-marvel.co.ua
srecorder.com shtora66.ru
star61.de simpoed.ufop.br
statistici.ro site-speed-checker.site
stauga.altervista.org slowmac.tech
steame.ru smallseotools.com
stretchmate.net songoo.wz.cz
strigkaomsk.ru space4updating.win
studentguide.ru starwars.wikia.com
takeprofitsystem.com stop-gepatit.te.ua
taqywu51.soup.io superstats.com
technika-remont.ru survival.betteroffers.review
thiegs.reco.ws szucs.ru
tksn.ru tabakur77.com
tooplay.com tako3.com
topquality.cf thedownloadfreeonlinegames.blogspot.com
torrents.cd time-japan.ru
torrnada.ru tizanidinecost.blogspot.com
tour-line.net tizanidinehcl4mgtablet.blogspot.com
tracksz.co tizanidinehclsideeffects.blogspot.com
tula.howotorg.ru top-deal.com.pl
turkeyreport.tk topmira.com
tvand.ru traffic-club.info
tvnewsclips.info trucri.me
viel.su trudogolik.net
vip-parfumeria.ru u17795.netangels.ru
vk-mus.ru urlopener.com
vksaver-all.ru usdx.us
voloslove.ru ussearche.cf
vsexkontakte.net veerotech.com
waysbetter.cn vepad.com
wcb.su versaut.xxx-cam.webcam
websiteaccountant.de video--production.com
webstatsdomain.org vodkoved.ru
westsextube.com vsdshnik.com
westvilletowingservices.co.za vseigry.fun
whatistizanidinehclusedfor.blogspot.com vvpg.ru
whatzmyip.net w7s.ru
whos.amung.us web-analytics.date
xkaz.org weblo.com
xmlinde.com weddingdresses.xyz
xn--24-glceagatoq7c2a6ioc.xn--p1ai wholinkstome.com
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai worldoffiles.ru
xxxguitars.com wpthemedetector.co.uk
yaoguangdj.com wzgyyq.com
zeroredirect7.com x-diesel.org
zkjovpdgxivg.ga xcc24.pl
xdoza.com
xn----dtbndd4ae7eub.top
yaderenergy.ru
yaminecraft.ru
zatjmuzu.info
zigarettenonlinekaufen.tumblr.com
zophim.me
zvetki.ru

153
dev-tools/test_units/ratelimittest-master.txt
View file

@ -8,79 +8,6 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -494,6 +421,76 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
@ -584,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.117 0; 10.1.0.27 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3540 ### Version: V4.2023.01.3541
### Updated: Fri Jan 6 22:01:15 UTC 2023 ### Updated: Sun Jan 8 22:01:24 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3540 ### Version: V4.2023.01.3541
### Updated: Fri Jan 6 22:01:15 UTC 2023 ### Updated: Sun Jan 8 22:01:24 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##