vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-07 13:10:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.01.3552 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-01-20 22:07:23 +00:00
parent 7d92f60be0
commit 7a52909efc
47 changed files with 102113 additions and 102113 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3551 #### Version: V4.2023.01.3552
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3551 #### Version: V4.2023.01.3552
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2023.01.3551 #### Version: V4.2023.01.3552
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12630
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2023.01.3551 #### Version: V4.2023.01.3552
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12634
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12634
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12634
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12634
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

12634
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

12634
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

12634
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3551 3552

12634
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12634
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.175.194 0; 13.88.61.192 0;

12634
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

12634
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

12634
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

12634
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.175.194 0; 13.88.61.192 0;

12634
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.175.194 0; 13.88.61.192 0;

12634
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Ankit Attach
BlackWidow BacklinkCrawler
Cliqzbot Harvest
Litemage_walker Linkbot
OutfoxBot RankActive
PeoplePal Voil
Pu_iN Web Collage
WebLeacher Zauba
isitwp.com check1.exe
seostar muhstik-scan

286
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 01h4x.com
404enemy AIBOT
80legs
ALittle Client ALittle Client
Abonti Aboundexbot
Acunetix AfD-Verbotsverfahren
AiHitBot Alexibot
Alligator Alligator
Anarchy Anarchie
Anarchy99 Anarchy99
Ankit Ankit
Anthill Anthill
Aspiegel Aspiegel
Atomseobot BDCbot
AwarioSmartBot
BBBike
BDFetch
BackDoorBot BackDoorBot
BackWeb BackWeb
BacklinkCrawler Bandit
Blackboard Barkrowler
Blow BetaBot
BlowFish BlowFish
Buck Bolt
Buddy BotALot
BuiltBotTough BuiltBotTough
BuzzSumo BuzzSumo
CATExplorador
CODE87
CSHttp CSHttp
Cegbfeieh CheeseBot
Chlooe
Citoid
Claritybot
Cloud mapping
Copier
Cosmos Cosmos
Crawling at Home Project CyotekWebCopy
CrazyWebCrawler
Curious
DBLBot DBLBot
DSearch DSearch
DTS Agent
DataCha0s DataCha0s
Deusu DatabaseDriverMysqli
DigitalPebble Demon
Discobot Devil
Discoverybot Disco
DittoSpyder DittoSpyder
DnyzBot DomainAppender
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Dotbot Dotbot
Ebingbong Download Wonder
Drip
EMail Siphon
EMail Wolf
EasyDL
Ecxi Ecxi
EirGrabber Extractor
EroCrawler
Express WebPictures
ExtractorPro
Extreme Picture Finder
EyeNetIE EyeNetIE
Firefox/7.0 Ezooms
Fimap
Flunky Flunky
Freeuploader Foobot
FrontPage
Fuzz Fuzz
Genieo Go-Ahead-Got-It
GrabNet GoZilla
GrapeFX
GrapeshotCrawler
GridBot GridBot
HMView HTMLparser
HTTrack HTTP::Lite
Haansoft HaosouSpider
Harvest Harvest
Havij
Hloader
HonoluluBot HonoluluBot
Humanlinks
IRLbot
Id-search Id-search
IndeedBot Image Sucker
Indy Library Intelliseek
InfoNaviRobot Internet Ninja
JOC Web Spider JOC Web Spider
JamesBOT Jbrofuzz
JetCar Jetty
Joomla Joomla
Jyxobot JustView
Kenjin Spider Kenjin Spider
Kinza
Kozmosbot
LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot Lanshanbot
Larbin
Leap Leap
LeechFTP LexiBot
Lftp LieBaoFast
Lightspeedsystems
Likse Likse
LinkWalker LinkWalker
LinkextractorPro LinkpadBot
LinqiaScrapeBot
Lipperhey
Lipperhey Spider Lipperhey Spider
Litemage_walker MIDown tool
Lmspider MIIxpc
MQQBrowser
MSFrontPage
MTRobot MTRobot
Mag-Net
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic-SEO
Majestic12 Majestic12
MarkMonitor Mata Hari
Mb2345Browser Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot
MegaIndex.ru
Metauri
MicroMessenger MicroMessenger
Microsoft URL Control
Mister PiX Mister PiX
Mojeek
MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Msrabot
Mr.4x3
Musobot Musobot
NPbot NPbot
Navroad Nessus
Needle NetSpider
NetZIP NetZIP
Netvibes Netvibes
Nimbostratus NextGenSearchBot
Ninja Niki-bot
Nmap Nuclei
Not
Nutch Nutch
Offline Explorer Offline Explorer
OnCrawl
OpenLinkProfiler
OpenVAS OpenVAS
Openfind
Openvas Openvas
OrangeBot OrangeSpider
OutclicksBot OutclicksBot
PECL::HTTP
PHPCrawl PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageThing.com Page Analyzer
PageAnalyzer
Pandalytics Pandalytics
Panscient
Pavuk
Picscout
PictureFinder PictureFinder
Pimonster Pimonster
Pixray Pixray
Pockey ProPowerBot
Probethenet Probethenet
Psbot
Pu_iN Pu_iN
PxBroker QueryN Metasearch
RankActive RankFlex
RankingBot RankingBot2
RankurBot Rankivabot
Reaper RealDownload
RebelMouse
Recorder Recorder
RedesScrapy RedesScrapy
Ripper RocketCrawler
Rogerbot Rogerbot
SBIder SEOprofiler
SEOkicks SEOstats
SEOlyticsCrawler
ScanAlert ScanAlert
ScreenerBot Scanbot
ScrepyBot
Searchestate
SearchmetricsBot SearchmetricsBot
SemanticJuice Semrush
SemrushBot SemrushBot
SentiBot
SeobilityBot SeobilityBot
SiteCheckerBotCrawler Sitebeam
SiteSucker Siteimprove
SlySearch Sitevigil
SmartDownload
Snapbot Snapbot
SocialRankIOBot Snoopy
Sogou web spider Sociscraper
SpaceBison SpaceBison
SpankBot Spbot
Spanner
Spinn3r Spinn3r
Sqlmap Sqlmap
Sqlworm Sqworm
Sucuri Stripper
SuperBot
SurveyBot SurveyBot
Suzuran Suzuran
Szukacz Swiftbot
T8Abot TeleportPro
Telesoft Telesoft
Telesphorep Titan
TheNomad
Toweyabot
Tracemyfile Tracemyfile
Trendiction Turnitin
Trendictionbot Twice
True_Robot Typhoeus
TurnitinBot URLy Warning
VB Project URLy.Warning
Upflow
VCI VCI
VeriCiteCrawler VelenPublicWebCrawler
Voil VidibleScraper
Voltron VoidEYE
WBSearchBot WBSearchBot
WEBDAV
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE
Wallpapers
Wallpapers/3.0
WeSEE WeSEE
Web Enhancer
Web Sucker
WebAuto WebAuto
WebCollage WebCollage
WebCollage
WebCopier WebCopier
WebFetch WebFetch
WebReaper WebImageCollector
WebSauger WebSauger
WebSucker
WebWhacker
Webshag Webshag
WebsiteExtractor
Website Quester Website Quester
WebsiteExtractor
Webster
Whack
Who.is Bot Who.is Bot
Widow Widow
WiseGuys Robot Wonderbot
Wotbox Woobot
Xaldon_WebSpider Wprecon
Xenu Zitebot
Zauba ZmEu
Zermelo
ZumBot ZumBot
ZyBorg ZyBorg
adscanner
archive.org_bot archive.org_bot
arquivo-web-crawler arquivo-web-crawler
autoemailspider arquivo.pt
backlink-check
cognitiveseo
crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j crawler4j
demandbase-bot demandbase-bot
evc-batch facebookscraper
instabid
internetVista monitor internetVista monitor
linkdexbot isitwp.com
lwp-request lwp-trivial
magpie-crawler
mediawords mediawords
muhstik-scan muhstik-scan
oBot pcBrowser
plumanalytics plumanalytics
s1z.ru s1z.ru
satoristudio.net scalaj-http
scan.lol
seobility seobility
seocompany.store
seoscanners seoscanners
seostar serpstatbot
sexsearcher
sitechecker.pro sitechecker.pro
siteripz trendiction.com
sp_auditbot ubermetrics-technologies.com
spyfu
trendiction.de
voyagerx.com voyagerx.com
x22Mozilla x22Mozilla
xpymep1.exe xpymep1.exe
zgrab zauba.io

284
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider
404checker 404checker
404enemy 80legs
AIBOT AIBOT
ALittle Client Abonti
Aboundexbot Aboundex
Acunetix AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot
Aipbot
Alexibot Alexibot
AlphaBot Alligator
Ankit Ankit
Apexoo Apexoo
Aspiegel
Attach Attach
AwarioRssBot AwarioRssBot
BDFetch BDFetch
BackStreet BackWeb
Barkrowler Backlink-Ceck
Battleztar Bazinga Badass
Bandit
BetaBot
Bitacle Bitacle
Black Hole Black Hole
BlackWidow Blackboard
Boardreader BlowFish
Brandprotect Bolt
BuiltWith BuiltBotTough
Bullseye
BunnySlippers BunnySlippers
Calculon
CazoodleBot
Cegbfeieh Cegbfeieh
CheeseBot CensysInspect
CheTeam
CherryPicker
Chlooe Chlooe
Cliqzbot Claritybot
Cosmos Cocolyzebot
Crawling at Home Project Collector
Copier
CopyRightCheck
Craftbot
CrazyWebCrawler CrazyWebCrawler
Curious Crescent
Custo Custo
CyotekWebCopy CyotekWebCopy
DIIbot
Demon Demon
Deusu
Devil Devil
Digincore
DigitalPebble DigitalPebble
Discobot Dirbuster
DomCopBot Disco
DomainAppender DittoSpyder
DnyzBot
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Dragonfly Domains Project
ECCP/1.0 Download Wonder
Drip
EMail Siphon
EMail Wolf
Ebingbong Ebingbong
EirGrabber EroCrawler
Exabot Express WebPictures
ExtLinksBot Ezooms
Extreme Picture Finder
EyeNetIE
FDM FDM
FHscan FHscan
FemtosearchBot
Fimap
FlashGet
Flunky Flunky
Foobot Freeuploader
Fuzz FrontPage
FyberSpider
Fyrebot
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW GT::WWW
GalaxyBot Genieo
GetWeb GetWeb
Getintent Getintent
GoZilla
GrabNet
Grafula
GrapeFX
GrapeshotCrawler
HMView HMView
HTTP::Lite HTTP::Lite
HaosouSpider Hloader
Harvest
Havij
HonoluluBot
Humanlinks Humanlinks
IDBTE4M
IDBot IDBot
IRLbot IRLbot
Iblog
Id-search
IlseBot IlseBot
Image Fetch Image Fetch
InfoTekies InfoTekies
Intelliseek
Internet Ninja
InternetSeer
Iskanie
IstellaBot
JOC Web Spider
Jbrofuzz Jbrofuzz
JennyBot
Jetty Jetty
JikeSpider JikeSpider
Jorgee
JustView JustView
Kenjin Spider Kinza
Keyword Density
LWP::Simple LWP::Simple
Larbin LeechFTP
LeechGet
LexiBot
Lftp Lftp
LibWeb
Libwhisker Libwhisker
Lightspeedsystems Lightspeedsystems
LinkScan LinkWalker
LinkpadBot
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
MFC_Tear_Sample Lmspider
MJ12bot Ltx71
Mag-Net MIIxpc
MTRobot
Majestic SEO
Majestic12 Majestic12
MarkMonitor
Masscan Masscan
Mediatoolkitbot MauiBot
MeanPath Bot
Metauri
Microsoft URL Control Microsoft URL Control
Minefield Minefield
Mister PiX Moblie Safari
Mojolicious Mojolicious
MolokaiBot
Mr.4x3 Mr.4x3
Needle Msrabot
Name Intelligence
Net Vampire
NetAnts NetAnts
NetLyzer
NetSpider NetSpider
NetZIP NetZIP
Nikto Nettrack
Nmap Nibbler
Offline Explorer Nimbostratus
OnCrawl Octopus
OpenLinkProfiler OpenLinkProfiler
OpenVAS
Openvas
PECL::HTTP PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP
PageGrabber PageGrabber
PageScorer PageScorer
Papa Foto
Pavuk Pavuk
Petalbot
Picscout
PictureFinder
Piepmatz
Pimonster
Pixray Pixray
ProPowerBot
Probethenet
PxBroker
PyCurl
QueryN Metasearch QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot RSSingBot
RankActive RankActive
RankActiveLinkBot RankActiveLinkBot
RankingBot RankFlex
RankingBot2
Rankivabot
RebelMouse RebelMouse
Recorder Recorder
RedesScrapy
Ripper
RocketCrawler RocketCrawler
SEOkicks SEOkicks
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SEOprofiler
SISTRIX
SMTBot
SalesIntelligent SalesIntelligent
Scanbot ScanAlert
SearchmetricsBot ScoutJet
Seekport Screaming
ScrepyBot
Semrush Semrush
SemrushBot SemrushBot
SeoSiteCheckup
Shodan Shodan
Siphon
Site Sucker
SiteSnagger
SiteSucker
Sitebeam
SlySearch SlySearch
Snake
Snapbot Snapbot
Snoopy SocialRankIOBot
Sociscraper Sosospider
Spammen Spammen
SpankBot SpankBot
Spbot SuperHTTP
Spinn3r Swiftbot
Sqlmap TeleportPro
Sqworm Telesphoreo
Stripper
Sucuri
SuperBot
SurveyBot
Suzuran
Szukacz
T0PHackTeam
Telesphorep Telesphorep
Thumbor Trendictionbot
Toata Turnitin
Toweyabot TurnitinBot
Tracemyfile
URLy Warning
URLy.Warning URLy.Warning
UnisterBot
Upflow
VB Project
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
Virusdie
VoidEYE
Voil Voil
WASALive-Bot
WEBDAV WEBDAV
WPScan WPScan
WWW::Mechanize WWW-Mechanize
WallpapersHD WallpapersHD
Web Auto Web Collage
Web Sucker Web Fetch
WebCopier
Web Fuck Web Fuck
WebGo IS WebCollage
WebFuck
WebImageCollector
WebLeacher
WebPix
WebSauger WebSauger
WebSucker WebStripper
WebWhacker
WebZIP
Webalta
WebmasterWorldForumBot
Webshag Webshag
WebsiteExtractor
WebsiteQuester WebsiteQuester
Whack Webster
Whacker
Whatweb
Widow Widow
WinHTTrack WiseGuys Robot
Wonderbot Woobot
Xenu Xaldon_WebSpider
YoudaoBot
Zauba Zauba
Zeus
Zitebot
ZmEu ZmEu
ZoomBot ZumBot
ZyBorg
adscanner adscanner
arquivo.pt arquivo-web-crawler
check1.exe backlink-check
com.plumanalytics com.plumanalytics
dataforseo.com
demandbase-bot demandbase-bot
gopher eCatch
evc-batch
heritrix
instabid instabid
internetVista monitor
linkdexbot linkdexbot
lwp-request lwp-request
lwp-trivial lwp-trivial
magpie-crawler
meanpathbot meanpathbot
pcBrowser mediawords
plumanalytics
polaris version
ripz
s1z.ru s1z.ru
scan.lol
seoscanners seoscanners
seostar seostar
sitechecker.pro serpstatbot
sp_auditbot sogouspider
sysscan trendiction.de
tAkeOut ubermetrics-technologies.com
webprosbot voyagerx.com
x09Mozilla webmeup-crawler
webpros.com
x22Mozilla x22Mozilla
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
backuperwebcam.weebly.com bgviagrachrx.com
crafthubs.com bird1.ru
damasarenai.info getyourimage.club
duplicashapp.com gsasearchengineranker.xyz
fuckingawesome.com gwebtools.com.br
i-hobot.ru klikbonus.com
pornogig.com mebelcomplekt.ru
rabot.host.sk panamaforbeginners.com
seoprofiler.com thenews-today.info
taraz.xkaz.org topkarkas.com

912
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
007angels.com
00it.com 00it.com
1001watch.com.ua 1-99seo.com
100dollars-seo.com 101billion.com
101raccoon.ru 101raccoon.ru
123cha.com 11235813.webzdarma.cz
123movies.love 1xbet4you.com
1688.com 256bit.by
18ps.ru 3weekdiet.com
1kinobig.ru 4kplayer.pl
273-fz.ru 4replicawatch.net
2kata.ru 5forex.ru
2pxg8bcf.top 8lufu.com
2x2fan.ru a.frcls.fr
404.mn a96527gi.beget.tech
5kstore.com aa9d046aab36af4ff182f097f840430d51.com
7yue.org aac500b7a15b2646968f6bd8c6305869d7.com
8si.ru aarbur.com
98oi.ru abcdeg.xyz
aa8780bb28a1de4eb5bff33c28a218a930.com abovetherivernc.com
activecampaign.dreamhosters.com absolutelyanalog.com
adclickthru.net acc.eu.org
advancedmassagebysara.com activepr.ru
afora.ru addbin.men
ahrntal.verymes.xyz addray.pro
albuteroli.com admanaerofoil.com
adtology.com
adultactioncam.com
agecheckadult.com
ageofclones.com
aktobe.xkaz.org
alert-fjg.xyz
algarveglobal.com
alkoravto.ru alkoravto.ru
all4invest.ru allbizne.co.ua
aloofly.com altamayoztourism.com
alot.com
aludecor.info aludecor.info
ameblo.top amazon-seo-service.com
anal-acrobats.hol.es amyfoxfitness.com
analnoeporno.tv ananumous.ru
android-style.com animal-drawings.com
apiitsthirteende-a.akamaihd.net api.stathat.com
apioasisspacenet-a.akamaihd.net apiappenableinfo-a.akamaihd.net
apisecretsaucebi-a.akamaihd.net apibrowsesmartne-a.akamaihd.net
apispringsmartne-a.akamaihd.net apimountainbikei-a.akamaihd.net
apptester.tk appfixing.space
arenda-yeisk.ru approvedlocal.co.za
arkkivoltti.net arpelsreplica.xyz
arraty.altervista.org
asacopaco.tk
ascat.porn ascat.porn
autogrep.ru asrvrep-a.akamaihd.net
autoseo-traffic.com auspolice.com
avkzarabotok.info auto-moto-elektronika.cz
avtolombard-krasnodar.com autoblogger24.info
ayeartoforget.com autotop.com.ua
azarttoto.com
azbukadiets.ru azbukadiets.ru
babespcs.com azbukafree.com
back.dog balook.com
baditri.com bankmib.ru
beauty-b0x.pl bedandbreakfast.com
begalka.xyz
belstaffstore.online belstaffstore.online
berlininsl.com bestbrainprod.win
best-coupon-offer.com besthomemadepornsites.com
bestcalovebracelet.cn bestmarriages.com
bestcoin.cc betshuckclean.com
besthoro.ru
bestimagecoollibrary.com
bet365.com
betwinservice.com
bezlimitko.xyz
big-boards.info big-boards.info
bildsuche.ru bigames.online
bio-japan.net bioscorp.ru
bizfly.info bizcheapjerseyswholesalechina.com
blackle.com blavia.00author.com
blockworld.ru blog.f00kclan.de
boanonihaca.tk bloggerads.net
boo-arts.com blue-square.biz
brains2.biz bmsco.co
brimstonehillfortress.org board.f00d.de
bonvillan.com
brg8.com
brus.city
btc4u.ru
bumble.cheapwebsitehoster.com bumble.cheapwebsitehoster.com
bumskontakte.org buytizanidineonlinenoprescription.blogspot.com
bupropion-sr-150-mg.us buyviagraa.blogspot.com
burkesales.com buzz-porno.info
businescoop.men callawaygolfstore.online
buy-forum.ru callejondelpozo.es
buyclomidonlaine.com candlehandmade.com
byme.se cartierjusteunclou.xyz
c1.onedmp.com cartierreplica.win
callmd5map.com casablancamanor.co.za
callstevens.com casite-513049.cloudaccess.net
cartechnic.ru cb.iphantom.com
cartierlovestore.com cdnanalytics.xyz
ccbill.com cete.ru
centrumcoachingu.com cfacarrosserie74.com
cheapjerseysbizwholesale.us cfsrating.sonicwall.com
cheapsergiorossi.online chatroulette.world
checkpagerank.net cheapmoncler.pw
chinajerseyswholesalecoupons.com chimiver.info
chocoslim.pro chocolateslim-en-france.com
christianlouboutinreplica.win christianlouboutinreplica.win
citysecurity.nu cindymatches.com
codysbbq.com citetick.com
collegeessay19.blogspot.ru claytransformations.info
compliance-barak.top clickprotects.com
compliance-checker-7.info clippingphotoindia.com
compliance-fred.xyz codq.info
compliance-jessica.top coindirect.io
compliance-mary.top collegerentals.com
crazyprotocol.com compliance-elena.top
cscwtalkto.site compliance-walter.top
csgo4.win conciergegroup.org
customsua.com.ua coolingoods.com
cyclobenzaprinevstizanidine.blogspot.com copyrightclaims.org
cymbaltaandtizanidine.blogspot.com customerguru.in
dandiyabeats.in dailystrength.org
defenderxtactical.com dairyindia.in
delfin-aqua.com.ua daisye.top
destinationrealestate.com damasarenai.info
dfiles.me ddlmega.net
diffbot.com ddsoldes.fr
dildofotzen.net demian.kz
discountbarbour.online digital-sale.su
dnsrsearch.com directivepub.com
docsportal.net disruptingdinnerparties.com
dollartree.info dnepr-avtospar.com.ua
domain-submit.info dnepropetrovsk.mistr-x.org
doreenblog.online dscaas.website
dostavka-v-krym.com
downloads-whatsapp.com
drunkenstepfather.com
drunkmoms.net
dustyorate.com dustyorate.com
eandsgallery.com e90post.com
eldiariodeguadalajara.com education-cz.ru
elektrischezigaretten1.blogse.nl empirepoker.com
elstal.com.pl esopini.com
eupornstar.info eugenevaultstorage.com
europeanwatches.ru europeanwatches.ru
exportshop.us exct.net
extener.org ezigarettekaufen.myblog.de
extstat.com
ezigarettenshop2.myblog.de
ezigarettenshop2.postbit.com ezigarettenshop2.postbit.com
facialporntube.com
familienzahnaerzte.com
familyholiday.ml
fantasticpixcool.com
femmesdenudees.com
fenoyl.batcave.net
fetlifeblog.com fetlifeblog.com
filesmonster.porn fifa-coins.online
filmetricsasia.com filefilter.weebly.com
filmania-x.ru
fishtauto.ru fishtauto.ru
fitness-video.net flash4fun.com
fitnesspiks.com flashahead.info
fix-website-errors.com foxweber.com
flowersbazar.com free-floating-buttons.com
forexunion.net freemaintenancesysforpcandmac.top
forminecrafters.ru frequiry.com
foto-sisek.porngalleries.top frustrated-favorable.gq
foxjuegos.com fuckmill.com
free-social-buttons.xyz fun-mobi.pl
freelifetimefuckbook.com fxgallery.com
freewebs.com g7m.pl
freshwallpapers.info gallery.rennlist.com
fun2cell.net
gabeshop.ru
gagrasector.ru
galleryawesome.com
gamerextra.com gamerextra.com
gay-site.store garciniaxt.us
gepezz.info gaz-voshod.ru
get-seo-domain.com ge0ip.org
gettpromos.com gearcraft.us
generalporn.org
geoads.com
get-free-traffic-now.com
getoutofdebtfree.org
getridofstretchmarks.org
gkvector.ru gkvector.ru
glavtral.ru glastecfilms.com.my
global-ics.co.za globetrotting-culture.ru
globatur.ru golaya.pw
gne8.com golden-praga.ru
goldadpremium.com goodly.pro
goodhousekeeping.com
goodwinmetals.co
googlefeud.com googlefeud.com
gpirate.com gosarhivrt.ru
greenidesign.co gov.yanao.ru
grand-chlen.ru
greatfind-a.akamaihd.net
greatzip.com
growmyfunds.ca growmyfunds.ca
gsasearchenginerankerdiscount.com gstatey.net
gwagka.com guardlink.com
gwebtools.com.br handsandlegs.ru
habermetre.com hatedriveapart.com
hackers-crackers.tk hchha.com
hamptonoaks.ca health-medical-portal.info
hanink.biz.ly heartofpayne.xyz
headpharmacy.com
healbio.ru
healthcarestore.info
hermesbracelets.xyz hermesbracelets.xyz
heroz.fr hitmuzik.ru
highland-homes.com hotelcrocenzi.sm
hit-men.men hvd-store.com
hiwibyh.bugs3.com iamsport.org
homeandhealth.ru iboss.com
homedecoguide.info imgarit.pw
homedo.fabpage.com imicrovision.com
hotsocialz.com impotentik.com
housediz.com insurple.com
hplaserjetpdriver8y.pen.io interesnie-faktu.ru
hyiphunter.org intimshop-fantasy.ru
i-service.kz invivo.hu
iideaidekonkatu.info ipornox.xxx
imagine-ex.co istizanidineacontrolledsubstance.blogspot.com
inboundlinks.win iusstf.org
incest-ru.com janerikholst.se
incomekey.net janettabridal.com
indiakino.net javspace.net
infowarcraft.ru jerseywholesalechinabiz.com
inmate-locator.us karadene.com
innodgfdriverhm.aircus.com keywordsdoctor.com
insomniagamingfestival.com kidskunst.info
interfucks.net kino-rf.ru
internetartfair.com kino2018.club
invitefashion.com klejonka.info
inzn.ru knigonosha.net
iqs.biz.ua
iron-age.info
iwanttodeliver.com
ix20.ru
j-times.ru
javatex.co.id
javstock.com
jeremyeaton.co
jerseyswholesalechinalimited.com
jerseywholesaleelitestore.com
jurajskie.info
kaidalibor.de
kareliatobacco.ru
karusel-market.ru
knogg.net
kometars.xyz kometars.xyz
kongruan.com
konturkrasoty.ru konturkrasoty.ru
kosynka-games.ru krasivye-devushki.net
labvis.host.sk krasnodar-avtolombards.ru
ladsblue.com krasula.pp.ua
lampokrat.ws laapp.com
larger.io labelwater.se
law-check-eight.xyz lanasshop.ru
leadn.pl languagecode.com
lexixxx.com lapitec.eu
linkdebrideur.xyz lazy-z.com
linkrr.com lechenie-gemorroya.com
live-cam6.info linkbolic.com
live-sexcam.tk littleberry.ru
liveinternet.ro littlesunraiser.com
livingroomdecor.info livejasmin.com
longlifelomilomi.info locksmith.jp
lordzfilmz.me locustdesign.co
louboutinreplica.pw lodki-pvh.dp.ua
lovi-moment.com.ua lolnex.us
lyrster.com lombia.com
mactechinfo.info lonerangergames.com
manifestyourmillion.com louboutinreplica.xyz
manve.info love-baby.cz
marmitaco.cat luxup.ru
martlinker.com magazin-pics.ru
maju.bluesalt.co
make-money-online.com
manipulyator-peterburg.ru
mansiondelrio.co
mapquestz.us
massage-info.nl
masserect.com
master-muznachas.ru master-muznachas.ru
matb3aa.com
max-p.men max-p.men
maximilitary.ru
me-ke.com
mebel-vstroika.ru mebel-vstroika.ru
mebeldekor.com.ua mediaoffers.click
mega-polis.biz.ua members.ghanaweb.com
microsoftportal.net memberty.com
mil-stak.com michaelkorsoutlet.store
milblueprint.com millionare.com
mitrasound.ru mmostrike.ru
mlf.hordo.win mobile-appster.ru
mmstat.com
mobsfun.net mobsfun.net
modenamebel.ru moncleroutletonline.pw
moesexy.com monthlywinners.com
mojaocena.com morlat.altervista.org
moje-recenze.cz mosrif.ru
moomi-daeri.com moviezbonkerssk.cf
mp3ritm.top moyakuhnia.ru
mrsdalloways.com musicstock.me
mturkcontent.com my-bc.ru
muschisexbilder.com nac-bearings.ru
musezone.ru nastydollars.com
mvpicton.co.uk naturalpharm.com.ua
mydoctorok.ru naturalshair.site
myfreetutorials.com netpics.org
mykings.pw nextlnk12.com
myseoconsultant.com nfljerseyscheapbiz.us
mysexpics.ru
nardulan.com
net-profits.xyz
newsperuse.com
nextconseil.com
nikhilbahl.com
nina.az nina.az
nrv.co.za nonews.co
nuup.info novgorod.xrus.org
o-dachnik.ru obnal.org
o-o-8-o-o.com oconto.ru
oda.as oda.as
odywpjtw.bloger.index.hr office-windows.ru
ohmyrings.com omoikiri-japan.ru
ok.ru
onescreen.cc
onko-24.com
online-x.ru online-x.ru
onlineku.com onlinebay.ru
onlywoman.org onload.pw
onthemarch.co ontargetseo.us
optom-deshevo.ru openfrost.net
openmultipleurl.com
oranga.host.sk oranga.host.sk
osoznanie-narkotikam.net owathemes.com
ourville.info
outclicks.net
ownshop.cf ownshop.cf
oyster-green.com owohho.com
oz-offers.com palocco.it
paleohub.info pandroid.co
panamaforbeginners.com parajumpersstore.online
paphoselectricianandplumber.com partner-cdn.men
pechikamini.ru pavlodar.xkaz.org
perfectpracticeweb.com pawli.eu
perosan.com pc-test.net
pertlocogasilk.tk php-market.ru
photo.houseofgaga.ru pihl.se
pics-group.com pivka.xyz
pipki.r.acdnpro.com pix-hd.com
piski.top pk-services.ru
piter.xrus.org pl-top.pl
piuminiita.com poem-paying.gq
pixell.club
pl-vouchers.com
plastgranar.nu
playtap.us
polcin.de polcin.de
pontiacsolstice.info popup-fdm.xyz
popunder.net poquoson.org
porn-w.org
porno.simple-image.com.ua porno.simple-image.com.ua
pornohub.me pornoreino.com
pornotubexxx.name pornosemki.info
pospr.waw.pl pornoslive.net
pricheski-video.com pornosmola.info
privacyassistant.net pornstartits.xblog.in
privacylocationforloc.com poshiv-chehol.ru
prizefestival.mobi predominant-invent.tk
qor360.com pro-poly.ru
qwarckoine.com prod2016.com
razvratnoe.org profitfx.online
repeatlogo.co.uk prom23.ru
replicalouboutin.xyz promoforum.ru
rfserial.net promover.org
riralmolamsaca.tk pst2017.onlinewebshop.net
rn-to-bsn.com qualitymarketzone.com
rocketchange.ru raavidesigns.com
radiodigital.co
rangjued.com
ranking2017.ga
raschtextil.com.ua
razamicroelectronics.com
redirect.trafficreceiver.club
reeyanaturopathy.com
referencemoi.com
reimageplus.com
reklama-i-rabota.ru
remont-fridge-tv.ru
rentaremotecomputer.com
replicaclub.ru
retailwith.com
ringtonepartner.com
romhacking.ru
rossanasaavedra.net rossanasaavedra.net
royalads.net royal-betting.net
royalvegascasino.com ruscoininvest.company
sack.net s.lollypopgaming.com
sadaholding.com s8-nowy-wygraj.comli.com
samolet.fr santasgift.ml
sanidumps.com santechnik.jimdo.com
sape.top sarahmilne.top
satoristudio.net sashagreyblog.ga
saveindex.xyz saveindex.xyz
scanner-brian.top sbf441.com
scanner-don.top scalerite.co.za
scanner-jack.top scanmarine.info
scanner-willy.top scanner-margo.top
screentoolkit.com sdsjweb.com
scrnet.biz.ua seansonline24.pl
seo-platform.com searchengineranker.email
seocdvig.ru secretscook.ru
seoheap.com security60-e.com
sex-tracker.de semxiu.com
sex-watch.com seofirmreviewsus.info
serialsway.ucoz.ru
sex-foto.pw
sex-tracker.com
sexflirtbook.com sexflirtbook.com
sexs-foto.com sexsaoy.com
sexstream.pl shikiso.info
sexuria.net shiksabd.com
shijian.ac.cn shop.xz618.com
shoppytoolmac.com shoppingmiracles.co.uk
sientalyric.co silvermature.net
simply.net sim-service.net
siteheart.net simple-share-buttons.com
sivs.ru simpoed.ufop.br
sklad-24.ru siteexpress.co.il
slim.sellany.ru skladvaz.ru
soc-econom-problems.ru sky-mine.ru
social-button.xyz sladkoevideo.com
social-buttons.com smadihome.com
smart-scripts.com
sneakyboy.com
social-fun.ru social-fun.ru
social-s-ggg.xyz sohoindia.net
softxaker.ru sophang8.com
soietvousmaime.fr sortthemesitesby.com
solnplast.ru sovinsteel.ru
sonnikforme.ru
sotechco.co
spammen.de
speechfoodie.com
speedup-my.site speedup-my.site
spidtest.org spinnerco.ca
spymac.net stair.registrydomainservices.com
standardchartered-forex.com starwars.wikia.com
stickers-market.ru stromerrealty.com
stockquotes.wooeb.com suhanpacktech.com
stockspmb.info supermama.top
strana-krasoty.ru svetodiodoff.ru
stroydetali.ru svnuppsalaorebro.se
studiokamyk.com.pl swsociety.se
sudexpert66.ru tagil.zrus.org
superfish.com
swagbucks.com
taqywu51.soup.io taqywu51.soup.io
tedxrj.com tds-advert005.info
teesdaleflyballclub.co.uk tedy.su
telefonsexi.com telegraf.by
telemetryverification.net telsis.com
thaismartloan.com tengohydar.tk
thepantonpractice.co.uk the-usa-games.blogspot.com
theplacetoupdating.pw theprofitsmaker.net
theporndude.com thruport.com
timhost.ru titan-ads.life
tizanidinedrugtest.blogspot.com tivolibasket.it
tizanidineandgabapentin.blogspot.com
tizanidineduringpregnancy.blogspot.com
tizanidinehcl2mgsideeffects.blogspot.com
tizanidinehcl4mgtab.blogspot.com tizanidinehcl4mgtab.blogspot.com
tooplay.com tizanidinehcl4mgtablet.blogspot.com
top-karkas.ru tizanidinehydrochloride2mg.blogspot.com
top10-way.com tizanidineusedfor.blogspot.com
topads.men tkanorganizma.ru
topkarkas.com tmtrck.com
tradgardspartner.se toloka.hurtom.com
trafficfactory.biz toondinsey.com
trahic.ru top10registrycleaners.com
tube8.com top250movies.ru
tubeline.biz topcar-krasnodar.ru
turn-up-life.life topmira.com
tyumen.xrus.org toursmaps.com
uasb.ru tracksz.co
ultimatesetnewfreeallsoftupgradesystems.pw tradedeals.biz
umg-stroy.ru traderzplanet.in
unece.org trafficnetzwerk.de
unpredictable.ga trahvid.com
urzedowski.eu trainoffend.ml
trkur.com
truck-rece.life
trustl.life
tryrating.com
ttsq.fr
tupper-shop.ru
turvgori.ru
twinderbella.com
twittruth.com
twodollarshows.com
u-cheats.ru
u555u.info
uhod-za-sobakoj.ru
unacittaconte.org
unimodemhalfduplefw.pen.io
unrealcommander.biz
upstore.me
usdx.us usdx.us
utiblog.fr viagengrarx.com
utrolive.ru viagra-soft.ru
uzungil.com video-woman.com
vasileostrovsky-rayon.ru videos-for-your-business.com
vbtracker.net videosbox.ru
veopornogratis.xxx videospornogratisx.net
video-chat.cn vipps.com.my
vip.51.la viteonlusarezzo.it
virus-respirators.com vladimir.xrus.org
vizag.kharkov.ua voditeltrezviy.ru
vkak.ru volume-pills.biz
voloo.ru w7s.ru
vremya.eu wallpaperaccess.com
vrnelectro.ru
vsesubwaysurfers.com
vykupavto-krasnodar.ru
vzlomtw.com
wanker.us wanker.us
warezkeeper.com wdrake.com
watchdogs-2.ru webradiology.ru
webads.co.nz websites-reviews.com
webix.biz whatsupinfoley.com
webjam.com whos.amung.us
webstatsdomain.org windowssearch-exp.com
weburok.com
wetgames.ru
whatsappbot.flyland.ru
wildcattube.com
wildnatureimages.com
wineitudes.wordpress.com wineitudes.wordpress.com
wnoz.de woodyguthrie.se
womensplay.net word-vorlagen.net
wordkeyhelper.com woweb.com.ua
xedserver.com wrona.it
xep.info www.ehscloud.cn
www.labves.ru
x-true.info
xblognetwork.com
xerox-douglas.cf
xfluro.com xfluro.com
xingzi-vision.com xgftnlrt.bloger.index.hr
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai xn--80aeahghtf8ac5i.xn--p1ai
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai xn--80ak6aa92e.com
xn--80ahvj9e.xn--p1ai xvideospornoru.com
xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai xwatt.ru
xn--b1agm2d.net yebocasino.com
xrus.org yourads.website
xxlargepop.com yoursite.com
yatrk.xyz youtube-downloader.savetubevideo.com
yeartwit.com yuarra.pluto.ro
yellowstonesafaritours.com zarenica.net
yginekologa.com zebradudka.com
youandcredit.ru zigarettenonlinekaufen1.bloog.pl
youjizz.com zirondelli.it
yousense.info zootoplist.com
zajm-pod-zalog-nedvizhimosti.ru zx6.ru
zakonobosago.ru
zed21.net
zoogdisany.com
zrizvtrnpale.tk
zvetki.ru
zvooq.eu
zvuker.net
zzbroya.com.ua

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1-99seo.com 1001desert.com
100searchengines.com 214jaluwobapef.cf
1pamm.ru 5tey7463.weebly.com
1webmaster.ml 6128786.com
2kata.ru albumsuper.info
404.mn angel1777.kz
4kepics.com animebox.com.ua
7a2rnuey1tw9ar.ru apartmentratings.com
aarbur.com apimossnetinfo-a.akamaihd.net
abtasty.com app-ready.xyz
admeasures.com aproposde.com
adult-shop.com.ua aquarium-pro.ru
aimayangzhi.com atlasvkusov.ru
all-number.com autoloans.com
alldubai.biz b2b-lounge.com
alloysteel.ru balois.worldbreak.com
alvtank.se banan.tv
ameblo.top bannerbank.ru
anaseracresar.tk berdasovivan.ru
animaltoplist.com best-businessman.ru
anonymousfox.co bestwaystogetpaid.us
apifasterlightin-a.akamaihd.net bif-ru.info
apiqualitinknet-a.akamaihd.net bigpenisguide.com
apper.de bitnote.co
asscenihotosidea.blogspot.com blackhatworth.com
atdedinotuho.tk blogs.rediff.com
autotop.com.ua bmw-ark.ru
aversis.net bnt-team.com
back.dog board.f00d.de
badavit.com.ua
bausparen.bz.it
best-seo-offer.com
bestssaker.com
blockworld.ru
blpmovies.com
boeuklad.com
boltalko.xyz boltalko.xyz
bristolhotel.com.ua bombla.org
businesxxl.com book-bets.com
bvlgariring.xyz bookhome.info
cars-modification.net bradleylive.xyz
cdncash.com brandov.ru
cfsrating.sonicwall.com bravegirlsclub.com
chinacheapelitejerseys.com breget74.ru
cialischstgerts.com burger-tycoon.com
citetick.com buttons-for-website.com
clmforexeu.com candlehandmade.com
co.lumb.co cete.ru
collegeessay19.blogspot.ru cfcl.co.uk
compliance-ivan.top cfjump.com
compliance-julia.top classiquebijoux.ru
coolbar.pro clickintext.net
cornerstone-countertops.com clicksor.com
cpabegins.ru club-lukojl.ru
damasarenai.info coccoc.com
delitime.info comeondog.info
detmebel.su compliance-barak.top
deutschehobbyhuren.net compliance-ivan.xyz
dfwu1013.info compliance-jack.top
dildofotzen.net compliance-walter.top
dinkolove.ya.ru credit-online.ws
divku.ru creditmoney.com.ua
dollartree.info creditnation.ru
doublepimp.com cv.wallhade.co
elektronischezigarettekaufen1.myblog.de czeshop.info
eloconcream.blogspot.com dating-time-now.com
erot.co dipstar.org
espaceinventoristes.com dividendo.ru
eyelike.com.ua divisioncore.com
eywords-monitoring-your-success.com djstools.com
fdzone.org driving.kiev.ua
fearcrow.com drupa.com
felizporno.com easync.io
fickkontakte.org el-nation.com
filmi-v.online elektrischezigarette1.onsugar.com
flowersbazar.com elektroniksigaraankara.info
forensicpsychiatry.ru elektrozigarette2.wordpressy.pl
epngo.bz
ereko.ru
ezigarettenkaufen1.pagina.gr
f1nder.org
failingmarriege.blogspot.com
fickenprivat.info
film-one.ru
finansov.info
fireads.men
flex4launch.ru
formulaantiuban.com formulaantiuban.com
fortwosmartcar.pw foto-telok.net
free-share-buttons.com frbizlist.com
get-free-social-traffic.com freenode.info
get-free-traffic-now.com fsakhalin.ru
glall.ru fuckmill.com
gogps.me games.kolossale.ru
goodwriterssales.com gay-file.com
gotcher.us geckoandfly.com
graphics8.info gepezz.info
hackers-crackers.tk girlsfuckdick.com
hawaiielectriclight.com gpirate.com
hol.es gstatey.net
homemade.gq gtopstats.com
homik.pw guidefs.ru
hostsshop.ru h2monline.com
hulfingtonpost.com hausfrauensex18.com
hustoon.over-blog.com hchha.com
iflycapetown.co.za hdimagegallery.net
ihtec2019.org hdseriale.pl
ilovevitaly.xyz headpharmacy.com
import-sales.com homedecorpicture.us
incest-ru.com hotloans.ru
iopeninghours.co.uk hoverboard360.nl
ipostroika.ru ilmen.net
iqoption.com immobiliaremassaro.com
irkutsk.online-podarki.com infospot.pt
isotoner.com insta-add.pro
istanbulit.com int.search.tb.ask.com
iusstf.org iplusbit.blogspot.co.za
kakablog.net ispac.org
kamalsinha.com itservicesthatworkforyou.com
kerwinandcariza.com ivoiretechnocom.ci
keywesthideaways.co jerseychinabizwholesale.com
kol-energo.ru johannesburgsingles.co.za
karaganda.xkaz.org
kihi.gdn
kinohall.ru
kmd-pto.ru
konkursowo-24.pl konkursowo-24.pl
kovrenok.ru kretpicf.pw
krasnodar-avtolombards.ru kung-fu-ru.com
kredit-blog.ru lalalove.ru
ktotut.net
law-check-seven.xyz
ledpolice.ru ledpolice.ru
lemon-ade.site
lflash.ru lflash.ru
liverpool.gsofootball.com limtu.ifmo.ru
lombia.co lineshops.biz
lsp-awak-perikanan.com liumimgx.pw
lutherstable.org lottospring.com
m0r0zk0-krava.ru lunamedia.co
maglid.ru m292.info
maju.bluesalt.co maju.bluesalt.co
max-p.men makenahartlin.com
mensandals.xyz mamasuper.prom.ua
metallosajding.ru marmitaco.cat
mirtorrent.net mbiologi.ru
misterjtbarbers.com mcnamaratech.com
mobicover.com.ua militarysale.pro
mobilemedia.md mindbox.co.za
mobstarr.com mirobuvi.com.ua
mojowhois.com mobot.site
moneyviking-a.akamaihd.net most.gov.iq
more-letom.ru mrwhite.biz
moscow-clining.ru muabancantho.info
moscow-region.ru muz-baza.net
mrbojikobi4.biz mxgetcode.com
myindospace.com my-aladin.com
myxdate.info my-bc.ru
nextlnk12.com mydeathspace.com
nikitsyringedrivelg.pen.io nailsimg.com
nisuturnetdgu.tk negociosdasha.com
normalegal.ru nfljerseyscheapbiz.us
o-o-6-o-o.com nicefloor.co.uk
odoratus.net noclegonline.info
offergroup.info nodding-passion.tk
oksrv.com nw-servis.ru
oneminutesite.it obiavo.ru
oops-cinema.ru ofanda.com
openmultipleurl.com offer.wpsecurity.website
ownshop.win offside2.5v.pl
oz-offers.com ops.picscout.com
outrageousdeal-a.akamaihd.net
owathemes.com
panicatack.com
partybunny.ru partybunny.ru
pavlodar.xkaz.org perederni.net
pay2me.pl perso.wanadoo.es
pekori.to physfunc.ru
petrovka-online.com piccshare.com
picturesboss.com pl.id-forex.com
pigrafix.at polcin.de
poem-paying.gq pornhub-ru.com
poppen-nw.net pornotubexxx.name
portal-eu.ru pr0fit-b0x.com
printingpeach.com preparevideosafesystem4unow.space
privatbank46.ru privatefx.all4invest.info
promgirldresses.xyz rank-checker.online
rankings-analytics.com rating-bestcasino.com
rapidokbrain.com razyboard.com
rapidsites.pro reelheroes.net
rednise.com regdefense.com
reeyanaturopathy.com registrysweeper.com
registrationdomainsite.com restorator-msk.ru
registrydomainservices.com rockprogblog.com
rentalcarnavi.info roll123.com
rentehno.ru running-line.ru
retailwith.com runstocks.com
rusvideos.su samara.rosfirm.ru
saveindex.xyz sarf3omlat.com
sbf441.com saugatuck.com
sbornik-zakonov.ru screpy.com
scanner-andrew.top sdelai-prosto.ru
search-goo.com serdcenebolit.com
searchlock.com servisural.ru
sell-fb-group-here.com sexgalleries.top
seo18.su sexystrippe.info
seomarketings.online shijian.ac.cn
sexkontakteao.info shmetall.com.ua
sfj-ror.no shoesonlinebuy.cn
shoppingjequiti.com.br siha.de
shymkent.xkaz.org sispe.com.br
sierraapps.com siteimprove.com
sitevaluation.com socialbookmarksubmission.org
slimcdn.com softonicads.com
sogimlecal.tk solnplast.ru
spidtest.org strangeduckfilms.com
spywarebegone.com stuff-about-money.com
swimpool.ca sumo.com
symbaloo.com supermodni.com.ua
td-l-market.ru tacbibirfa.tk
timdreby.com takethatad.com
tizanidineduringpregnancy.blogspot.com tds-advert002.info
token-lab.org thecounter.com
top10-online-games.com tizanidineandgabapentin.blogspot.com
tracklead.net tizanidineandhydrocodone.blogspot.com
top-l2.com
topflownews.com
topquality.cf
trafficshaper.com
treasuretrack-a.akamaihd.net treasuretrack-a.akamaihd.net
tula.howotorg.ru twitlinks.com
tupper-posuda.ru tyumen.xrus.org
turbabitload.weebly.com ua-company.ru
tvnewsclips.info unimodemhalfduplefw.pen.io
u-cheats.ru us-america.ru
umaseh.com ustion.ru
unmaroll.ya.ru vepad.com
utiblog.fr vipps.com.my
vacuumscleaner.com vkgaleria.com
validccseller.com voloomoney.com
viagra-soft.ru westen-v.life
viberdownload10.com worst-sites.online
videojam.tv xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
vietimgy.pw xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
voltrknc1.com xn--80aeahghtf8ac5i.xn--p1ai
vrotike.ru xn--d1acah0c.xn--p1ai
vvon.co.uk xxxtube69.com
weburok.com yellowproxy.net
wejdz-tu.pl yoluxuryevents.com
whatsupinfoley.com yoopsie.com
whengirlsgowild.com yoshkarola.zrus.org
wingsofrefuge.net youandcredit.ru
wma-x.com yougetsignal.com
works.if.ua youtubologia.it
worldinternetauthority.com ypmuseum.ru
ww1943.ru zagadki.in.ua
x-stars.ru zawyna.ua
xbaboon.com zvooq.eu
xn--80ab4aa2g.xn--p1ai zygophyceous.womanstars.site
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xnxx699.com
xnxxandxvideos.com
yourporn.com
yourporngay.com
zaimhelp.ru
zaloro.com
zionstar.net
zirondelli.it
zlatnajesen.com
znakom.sibtest.ru
znaturaloriginal.com
zog.link

22
dev-tools/test_units/ratelimittest-master.txt
View file

@ -1,11 +1,21 @@
<!DOCTYPE html>
<html> <html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
ml><html> <html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>
@ -571,14 +581,4 @@ ml><html>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.156 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3551 ### Version: V4.2023.01.3552
### Updated: Fri Jan 20 10:03:21 UTC 2023 ### Updated: Fri Jan 20 22:01:22 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3551 ### Version: V4.2023.01.3552
### Updated: Fri Jan 20 10:03:21 UTC 2023 ### Updated: Fri Jan 20 22:01:22 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##