vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.01.3535 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-01-02 22:07:27 +00:00
parent e0489e32a7
commit 4c2eb5e5d2
48 changed files with 94329 additions and 94309 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3534 #### Version: V4.2023.01.3535
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3534 #### Version: V4.2023.01.3535
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2023.01.3534 #### Version: V4.2023.01.3535
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11654
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2023.01.3534 #### Version: V4.2023.01.3535
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11658
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11658
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11658
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11658
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 1; 10.1.0.99 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 0; 10.1.0.99 0;
127.0.0.1 0; 127.0.0.1 0;

11658
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 1; 10.1.0.99 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 0; 10.1.0.99 0;
127.0.0.1 0; 127.0.0.1 0;

11658
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 1; 10.1.0.99 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 0; 10.1.0.99 0;
127.0.0.1 0; 127.0.0.1 0;

11658
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3534 3535

11658
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11658
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.71.18 0; 13.87.244.145 0;

11658
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 1; 10.1.0.99 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 0; 10.1.0.99 0;
127.0.0.1 0; 127.0.0.1 0;

11658
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 1; 10.1.0.99 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 0; 10.1.0.99 0;
127.0.0.1 0; 127.0.0.1 0;

11658
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 1; 10.1.0.99 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 0; 10.1.0.99 0;
127.0.0.1 0; 127.0.0.1 0;

11658
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.71.18 0; 13.87.244.145 0;

11658
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.71.18 0; 13.87.244.145 0;

11658
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 1; 10.1.0.99 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Badass AhrefsBot
BuiltWith AiHitBot
Demon Aipbot
Dirbuster Cogentbot
HTMLparser Flunky
Ltx71 LinkWalker
NextGenSearchBot Majestic12
PECL::HTTP Ripper
PageThing.com Spammen
Pockey Zeus

316
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 01h4x.com
404enemy
ADmantX
ALittle Client ALittle Client
Aboundex
Acunetix Acunetix
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot
Aipbot Aipbot
AllSubmitter Alexibot
AlphaBot Anarchie
Anarchy Anarchy
Aspiegel Anarchy99
Ankit
Apexoo
Attach
BLEXBot BLEXBot
BackDoorBot BackDoorBot
BackStreet
Backlink-Ceck
BacklinkCrawler
Badass
Bandit Bandit
Bigfoot
Bitacle Bitacle
Blow Blackboard
BlowFish BlowFish
Boardreader
Bolt Bolt
BotALot Brandwatch
Buddy
BuiltWith BuiltWith
BunnySlippers BunnySlippers
BuzzSumo BuzzSumo
CheTeam CODE87
CazoodleBot
CheeseBot
CherryPicker CherryPicker
Cloud mapping ChinaClaw
Cogentbot Chlooe
CopyRightCheck Cliqzbot
Craftbot Cocolyzebot
Crawling at Home Project CrazyWebCrawler
DSearch CrunchBot
Curious
DTS Agent
DataCha0s
DatabaseDriverMysqli
Demon Demon
Devil Devil
Dirbuster Digincore
Disco Disco
Discoverybot Discobot
DittoSpyder DittoSpyder
DomainCrawler DnyzBot
DomainSigmaCrawler DomCopBot
DomainStatsBot EMail Siphon
Download Wonder EasyDL
Dragonfly
ECCP/1.0
Ecxi Ecxi
EirGrabber Express WebPictures
EroCrawler ExtLinksBot
Evil
Extractor Extractor
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
EyeNetIE
FHscan FHscan
Fimap Firefox/7.0
FlashGet FrontPage
Foobot
Fyrebot Fyrebot
GermCrawler GT::WWW
Getintent GetWeb
Go!Zilla Grafula
Go-Ahead-Got-It GrapeFX
GoZilla
Gotit
Grabber
GrapeshotCrawler
HEADMasterSEO
HMView HMView
Harvest HTMLparser
Hloader HTTP::Lite
Humanlinks Haansoft
HybridBot HaosouSpider
IDBot Havij
Id-search HonoluluBot
IDBTE4M
IlseBot IlseBot
Image Fetch IndeedBot
InfoNaviRobot
InfoTekies InfoTekies
InterGET InternetSeer
IstellaBot IstellaBot
JetCar JOC Web Spider
Jetty
JikeSpider
Jorgee
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Kinza Kozmosbot
LNSpiderguy LWP::Simple
LeechFTP Larbin
LexiBot LexiBot
Lftp LieBaoFast
Lightspeedsystems Lightspeedsystems
LinkWalker LinksManager
Linkbot
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot Lmspider
Lipperhey
Lipperhey Spider
MFC_Tear_Sample MFC_Tear_Sample
MTRobot MIDown tool
Magnet MJ12bot
MQQBrowser
Mag-Net
Majestic SEO Majestic SEO
Majestic-SEO Majestic-SEO
Majestic12
MarkMonitor MarkMonitor
Masscan MarkWatch
Mass Downloader
Mata Hari
Mb2345Browser Mb2345Browser
MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
MicroMessenger MicroMessenger
Microsoft Data Access Microsoft Data Access
Minefield Minefield
Mister PiX Mozlila
Mojeek Msrabot
Mojolicious Musobot
Mr.4x3
Name Intelligence Name Intelligence
Navroad Nameprotect
NearSite NearSite
NetAnts Needle
NetMechanic NetLyzer
NetSpider NetSpider
Nettrack Niki-bot
Netvibes Nikto
NimbleCrawler NimbleCrawler
Nimbostratus Nuclei
Ninja Octopus
Nutch
Offline Navigator Offline Navigator
OnCrawl
OpenVAS OpenVAS
Openvas OrangeBot
PHPCrawl PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer Panscient
Pandalytics Pavuk
Petalbot PeoplePal
Picscout
Picsearch Picsearch
PictureFinder ProPowerBot
Piepmatz RSSingBot
PleaseCrawl RankActive
Pockey RankActiveLinkBot
Psbot
Pump
PxBroker
QueryN Metasearch
Quick-Crawler
RankFlex RankFlex
RankingBot RankingBot2
Rankivabot ReGet
RankurBot
Re-re
RealDownload
Recorder Recorder
RedesScrapy RedesScrapy
RocketCrawler RocketCrawler
SBIder
SEOkicks SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOstats SEOlyticsCrawler
SalesIntelligent
Scrapy Scrapy
ScreenerBot
ScrepyBot
Searchestate Searchestate
SemanticJuice SearchmetricsBot
SemrushBot Shodan
Siphon
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteLockSpider SiteLockSpider
SiteSnagger
SiteSucker
Siteimprove
SlySearch
SmartDownload SmartDownload
Snake Snake
Snoopy Snapbot
Sogou web spider Sociscraper
Sosospider
Sottopop
SpaceBison SpaceBison
SpankBot Spammen
Spbot
Spinn3r Spinn3r
Steeler Surfbot
Stripper
Sucker
Sucuri
SuperBot
SurveyBot SurveyBot
T8Abot Suzuran
Teleport Teleport
Telesphoreo Telesoft
Telesphorep Telesphorep
The Intraformant
Thumbor Thumbor
TightTwatBot TightTwatBot
Titan
Toata Toata
Toweyabot
Trendictionbot Trendictionbot
Turnitin
TurnitinBot TurnitinBot
Twice
Typhoeus Typhoeus
URLy Warning
VCI
Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
Voil
Voltron Voltron
WWWOFFLE WBSearchBot
Wallpapers WEBDAV
WISENutbot
WPScan
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
WeSEE WeSEE
Web Collage
Web Enhancer Web Enhancer
WebCollage Web Fetch
WebCopier Web Fuck
WebEnhancer Web Pix
WebLeacher WebAuto
WebStripper WebBandit
WebFetch
WebPix
WebSucker WebSucker
WebZIP
Webalta Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
Website Quester Webshag
WebsiteExtractor WebsiteExtractor
Webster WebsiteQuester
Whatweb Whack
WinHTTrack Whacker
Wonderbot
Woobot Woobot
Zitebot Wprecon
ZmEu Xaldon WebSpider
ZoomBot Xaldon_WebSpider
ZumBot Zauba
arquivo-web-crawler Zermelo
arquivo.pt Zeus
check1.exe ZoominfoBot
ZyBorg
autoemailspider
cah.io.community
clark-crawler
com.plumanalytics
crawl.sogou.com crawl.sogou.com
evc-batch crawler4j
demandbase-bot
facebookscraper
gopher
heritrix heritrix
internetVista monitor
ips-agent
iubenda-radar
lwp-request
lwp-trivial lwp-trivial
mediawords mediawords
plumanalytics muhstik-scan
polaris version oBot
s1z.ru scalaj-http
satoristudio.net
seocompany.store
seoscanners
seostar seostar
sogouspider sexsearcher
sp_auditbot sp_auditbot
sysscan spyfu
tAkeOut
trendiction.com trendiction.com
ubermetrics-technologies.com
webmeup-crawler webmeup-crawler
webpros.com x09Mozilla
webprosbot x22Mozilla
xpymep1.exe
zauba.io zauba.io
zgrab

280
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker
404enemy 404enemy
Aboundex ADmantX
AIBOT
ASPSeek
Aboundexbot
Acunetix Acunetix
AiHitBot AiHitBot
Alexibot Alexibot
Alligator Alligator
AlphaBot
Anarchie Anarchie
Ankit Anarchy
Anthill
Asterias Asterias
AwarioRssBot Atomseobot
BBBike BBBike
BDCbot BDCbot
BDFetch BLEXBot
BackStreet BackWeb
Badass Backlink-Ceck
BatchFTP Barkrowler
Battleztar Bazinga Battleztar Bazinga
BetaBot Bitacle
Black Hole
Blow Blow
BotALot BlowFish
Buck Bolt
BuiltBotTough
BuiltWith
Bullseye
BuzzSumo BuzzSumo
CSHttp
Cegbfeieh Cegbfeieh
CensysInspect
ChinaClaw
Chlooe Chlooe
Citoid
Claritybot Claritybot
Cliqzbot Cliqzbot
Cocolyzebot
Cogentbot
Collector
Copyscape Copyscape
DBLBot Crawling at Home Project
Custo
DIIbot DIIbot
Demon Demon
Deusu Devil
Digincore DigitalPebble
Dispatch Dirbuster
DittoSpyder Disco
DomainAppender DnyzBot
DomCopBot
DomainCrawler DomainCrawler
DomainSigmaCrawler Dotbot
Domains Project
Drip
ECCP/1.0
EMail Wolf EMail Wolf
EasyDL
Ebingbong Ebingbong
EirGrabber
EroCrawler EroCrawler
Exabot Exabot
Extractor ExtLinksBot
Extreme Picture Finder Extreme Picture Finder
EyeNetIE
Ezooms Ezooms
FHscan Flunky
FemtosearchBot
FlashGet
Foobot
Freeuploader Freeuploader
GalaxyBot FrontPage
Genieo Genieo
GermCrawler Go!Zilla
GetRight
GetWeb
Go-Ahead-Got-It
GoZilla
Gotit
GrabNet GrabNet
Grafula Grabber
GrapeFX
GridBot GridBot
HEADMasterSEO
HMView HMView
Haansoft HTMLparser
HonoluluBot HTTP::Lite
Havij
HybridBot HybridBot
IDBot
Iblog Iblog
IlseBot IlseBot
Image Fetch
Image Sucker Image Sucker
IndeedBot InfoNaviRobot
InfoTekies InterGET
Internet Ninja
InternetSeer InternetSeer
Iria Iria
Iskanie
JOC Web Spider
JamesBOT JamesBOT
Jetty Jbrofuzz
JikeSpider JennyBot
Joomla Kenjin Spider
JustView Keybot Translation-Search-Machine
Kinza
Kozmosbot Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple
Larbin
Leap
LeechFTP LeechFTP
Lftp LibWeb
Libwhisker Lightspeedsystems
Likse
LinkWalker LinkWalker
LinksManager LinkpadBot
Lipperhey Spider LinqiaRSSBot
MIIxpc Lipperhey
Litemage_walker
MIDown tool
MQQBrowser
MSIECrawler
MTRobot
Mag-Net Mag-Net
Majestic SEO Majestic SEO
MarkMonitor MarkWatch
Mata Hari Mata Hari
MauiBot MauiBot
Metauri Mediatoolkitbot
MegaIndex.ru
MicroMessenger
Microsoft Data Access Microsoft Data Access
Microsoft URL Control
Minefield
Mister PiX
Moblie Safari
Mojeek Mojeek
Mozlila MolokaiBot
Mr.4x3 Musobot
NICErsPRO NICErsPRO
NPbot
Name Intelligence
Nameprotect
Navroad
Needle Needle
Net Vampire
NetAnts NetAnts
NetLyzer Netcraft
NetZIP Netvibes
Niki-bot Nibbler
NimbleCrawler
Nimbostratus Nimbostratus
Ninja Octopus
Nutch Openfind
OrangeBot OutclicksBot
PECL::HTTP PHPCrawl
PageGrabber PageScorer
PageThing.com PageThing.com
Papa Foto Pandalytics
Pavuk Petalbot
PeoplePal
Pi-Monster
Picscout
Picsearch
Pimonster Pimonster
Pump Pockey
PxBroker ProPowerBot
ProWebWalker
Probethenet
Psbot
QueryN Metasearch QueryN Metasearch
Rankivabot RankActive
RankActiveLinkBot
RankFlex
RankingBot
Re-re
RebelMouse RebelMouse
RedesScrapy RedesScrapy
RepoMonkey RepoMonkey
SBIder SEOkicks
SEOkicks-Robot SEOlyticsCrawler
SEOprofiler SEOprofiler
SEOstats Scanbot
SalesIntelligent ScoutJet
ScanAlert ScreenerBot
Screaming ScrepyBot
Searchestate Searchestate
SemanticJuice
Semrush Semrush
SeoSiteCheckup Seomoz
Shodan
Siphon Siphon
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteSnagger SiteExplorer
Siteimprove SiteLockSpider
SmartDownload
Snoopy Snoopy
SocialRankIOBot Sociscraper
Sosospider Sogou web spider
Spammen SpaceBison
SpankBot SpankBot
Spanner Spanner
Spbot
SputnikBot SputnikBot
Sqlmap
Sqworm Sqworm
Stripper Stripper
SuperHTTP Sucuri
SuperBot
Surfbot Surfbot
Swiftbot
T0PHackTeam T0PHackTeam
Teleport TeleportPro
Telesoft
Telesphorep Telesphorep
TheNomad TheNomad
Titan Titan
Tracemyfile Tracemyfile
Trendiction Trendiction
Trendictionbot Trendictionbot
Twice TurnitinBot
Typhoeus
URLy Warning URLy Warning
Upflow
V-BOT V-BOT
VB Project VB Project
VCI Vacuum
Vagabondo Virusdie
VelenPublicWebCrawler Voltron
VoidEYE WBSearchBot
WEBDAV WEBDAV
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WeSEE Web Enhancer
Web Auto
Web Fetch
Web Fuck
Web Pix Web Pix
Web Sauger WebEnhancer
Web Sucker
WebBandit
WebFetch
WebFuck
WebReaper
WebSauger
WebSucker WebSucker
WebWhacker
Webshag Webshag
Whacker Website Quester
Who.is Bot
WinHTTrack WinHTTrack
WiseGuys Robot
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider
Xenu
Zermelo
Zeus
Zitebot Zitebot
ZmEu ZmEu
ZoomBot ZyBorg
archive.org_bot arquivo-web-crawler
autoemailspider arquivo.pt
cah.io.community check1.exe
clark-crawler coccocbot
crawl.sogou.com crawler.feedback
demandbase-bot eCatch
domainsproject.org
facebookscraper facebookscraper
gopher gopher
ips-agent iubenda-radar
isitwp.com
linkdexbot linkdexbot
lwp-request
lwp-trivial
meanpathbot meanpathbot
mediawords mediawords
muhstik-scan
netEstate NE Crawler netEstate NE Crawler
pcBrowser
plumanalytics plumanalytics
s1z.ru
scalaj-http
scan.lol scan.lol
seocompany.store seocompany.store
serpstatbot serpstatbot
sexsearcher sexsearcher
siteripz siteripz
sogouspider
sp_auditbot
tAkeOut tAkeOut
trendiction.com
trendiction.de trendiction.de
webmeup-crawler webmeup-crawler
webpros.com
webprosbot webprosbot
x09Mozilla x09Mozilla
xpymep1.exe
zauba.io zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
adrenalin-stk.ru bristolhotel.com
alldezservice.kz freza-sverlo.ru
calc-for-credit.ru goosefishpost.bid
kchaxton.com kiinomaniak.pl
lolnex.us life-instyle.com
mmog-play.ru rankingchart.de
picsforkeywordsuggestion.com royalvegascasino.com
shillyourcoins.com soft-program.com
topappspro.com ucoz.ru
worldmusicfests.com zhorapankratov7.blogspot.com

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1008.su 1008.su
11235813.webzdarma.cz 11235813.webzdarma.cz
12masterov.com 13tabs.com
178evakuator178.ru 2kata.ru
24x7-server-support.site 4pp13.com
2728fb936f0.com 4rent.ru
2nt.ru 51.la
2x2fan.ru 8si.ru
300richmond.co.nz DomainStatsBot
3weekdiet.com aadroid.net
5kstore.com
5ws.dating-app.ru
883zy.com
a1security.com.ua
aaronabel.com
absolutelyanalog.com
academiacsmendoza.org academiacsmendoza.org
acads.net
activepr.ru activepr.ru
adconscious.com ads-seo.men
adloads.net adserver-e7.com
adman.se adservme.com
adnotbad.com affordablewebsitesandmobileapps.com
ads-cool.pro aksonural.ru
adtiger.tk all-streaming-media.com
ahrefs.com allbizne.co.ua
ahrntal.verymes.xyz
aibolita.com
alert-fjg.xyz
alienwheel.es
allblogroll.com
allboard.xobor.de allboard.xobor.de
alpharma.net allornamenti.com
altamayoztourism.com amung.us
ameblo.top android-style.com
anaksma.info angkortours.vn
andpolice.com apelsinnik.website
animal36.com apioasisspacenet-a.akamaihd.net
anmysite.com apioutoboxnet-a.akamaihd.net
apartamentwroclaw.eu apispringsmartne-a.akamaihd.net
apartment.ru apparel-offer.com
api.stathat.com
app5.letmacworkfaster.world
apper.de apper.de
appfastplay.com
appiq.mobi
apple.com-webbrowsing-security.science
approved.su
arclk.net
arenanews.com.ua
arenda-avtoprokat-krasnodar.ru
arvut.org
as5000.com as5000.com
asdfz.pro asscenihotosidea.blogspot.com
asseenontv.ru atovh.local-finders.com
asseenontvonline.ru autochoixspinelli.com
atlasvkusov.ru ayakino.net
audiobangout.com backiee.com
audiofree.ru
ausergrubhof.info
avramstroy.ru
azartmix.com
azazu.ru
azlex.uz
babs.com.ua
backlink4u.net backlink4u.net
baersaratov.ru balitouroffice.com
balla-boo.se bankiem.pl
banksrf.ru
bannerspace.com
basedecor.ru
basisches-wasser.net basisches-wasser.net
battleforupdating.site bedcapdealers.com
beetpics.pw begalka.xyz
best-seo-software.xyz
bestbuy.ca
bestcalovebracelet.cn bestcalovebracelet.cn
betteroffers.review bestchoice.cf
bidbuy.co.kr bestkfiledxd.cf
biglistofwebsites.com beyan.host.sk
bimatoprost-careprost.com.ua bibys.com
bio-optomarket.ru billiard-classic.com.ua
biocn.dx.am bioscorp.ru
blackplanet.com bjorkbacken.nu
blagovest-med.ru blockety.co
blog.f00kclan.de blog.remote-computer.de
bloggerads.net bloke.com
blue-square.biz blue-square.biz
bmw.afora.ru bolegapakistan.com
bombla.org bongacams.com
bonzybuddy.com brainsandeggs.blogspot.com
bookmark4you.com.biz
braintobe.win
breget74.ru
brewdom.ru brewdom.ru
buyclomidonlaine.com bristolhostel.com
byme.se brk-rti.ru
bzero1jewelry.net bryansk.zrus.org
callcafe.info btc4u.ru
canacopegdl.com btvn.ru
candiceloves.us buscarfoto.com
carstrends2015.com callstevens.com
cartierlovebracelet.xyz camsex.xxx-cam.webcam
casinos4dummies.co.uk candy-glam-hp.com
cementaresearch.se carfax.com.ua
charmstroy.info cars-modification.net
chatroulette.online cartierreplica.pw
cheapestjerseysonwholesale.com cdncash.com
checkm8.com cdncash.org
chocolateslim-en-france.com cejewelry.xyz
cezartabac.ro
changde.58.com
chatroulette.life
cheap-trusted-backlinks.com
checktext.org
chiblackhawks-jerseys.com
christianlouboutinshoes.xyz christianlouboutinshoes.xyz
cialis-samples.com cialgenisrx.com
cialischsrx.com clash-clans.ru
cleanersoft.com cleansearch.net
clmforexeu.com clickzzs.nl
clodo.ru clknsee.com
cntravelre.com club-samodelkin.ru
compliance-don.top clubfashionista.com
compliance-walter.top coaching-netz.info
comsysnet.com cocolyze.com
coprofam.org colehaanoutlet.store
copypaste.traderzplanet.in com-onlinesupport.host
covadhosting.biz compliance-andrew.top
cpajunkies.com compliance-barak.top
crazyboost.pro converse.ddsoldes.fr
cookielawblog.wordpress.com
craftburg.ru
cranly.net
credit-respect.ru
creditmoney.com.ua
crest-poloski.ru
crirussian.ru
cubook.supernew.org cubook.supernew.org
cubs-jerseys.us cyclobenzaprinevstizanidine.blogspot.com
customcatchcan.com damasarenai.info
customergrowthsystems.com daooda.com
davidovic.info datract.com
dear-diary.ru delfin-aqua.com.ua
dekorkeramik.ru demian.kz
deshevo-nedorogo.ru design-lands.ru
deutschland123.de detoxmed24.ru
dev78.dioniqlabb.se dev33.dioniqlabb.se
diminishedvalueoforegon.com directrev.com
dnsrsearch.com disruptingdinnerparties.com
domashneeruporno.com dollartree.info
domain.webkeyit.com
domain2008.com
doska-vsem.ru doska-vsem.ru
dostavimvdom.ru dosugrostov.site
dwomlink.info dotmass.top
eachdayisagift.review dprtb.com
earncash.com.ua drupa.com
ed-shop01.ru dscaas.website
ege-essay.ru dyt.net
eksprognoz.ru e-biznes.info
electro-prom.com e-commerce-seo1.com
elektrischezigaretten1.blogse.nl edudocs.net
elektronischezigarettekaufen2.tumblr.com eleimgo.pw
enpolis.ru elektrozigarettekaufen1.devhub.com
erectile.bid elitepcgames.com
ereko.ru emailaccountlogin.co
es5.com eng-lyrics.com
eshop.md erotik0049.com
eu-cookie-law.blogspot.com erotikchat-24.com
eu-cookie-law.info escort-russian.com
eupornstar.info esopini.com
everflora.ru etm-consult.de
everytalk.tv eurosamodelki.ru
expertblog.info evogarage.com
eyelike.com.ua evrotekhservis.ru
eyessurgery.ru exchanges-bet.com
ezigaretteshop2.vefblog.net exci.ru
f1nder.org expdom.com
expresstoplivo.ru
ez8motelseaworldsandiego.com
ezaz.info
ezigarettekaufen1.hpage.com
falopicm.pw falopicm.pw
farmprofi.net family1st.ca
fashion.stellaconstance.co familyholiday.ml
fetishinside.com fashion-mk.net
fettefrauen.net fashionavenuegame.com
fickluder69.com fealq.com
filesvine.com fickblock18.com
fillmewithhappiness.com fickenbumsen.net
financetip.eu fifa-coins.online
findclan.org filmi-v.online
fitfloponline.store financehint.eu
fodelsedagspresenter.nu find1friend.com
forex-procto.ru findpik.com
forms-mtm.ru folowsite.com
formseo.com forexgb.ru
forpackningsutveckling.se forodvd.com
fortevidyoze.net forum.doctissimo.fr
forum69.info forums.toucharcade.com
froggytube.com freesitetest.com
freeuploader.ml
freewebs.com
front.to
fungamelands.com
funnypica.com
galleryawesome.com
gamblingnerd.com gamblingnerd.com
gazobeton-p.com.ua game-top.su
gcup.ru gaytube.com
ge0ip.org gdebestkupit.ru
gelezki.com generalporn.org
getridofstretchmarks.org genetworx.com
getyourimage.club getpopunder.com
girlsfuckdick.com getprismatic.com
gk-atlant.info gheus.altervista.org
glall.ru ghostvisitor.com
glavtral.ru gigapeta.com
go2album.com gilbertbanda.net
goldadpremium.com
gonextmedia.com
googlefeud.com
googlemare.com
googleseo.com.tr
gosarhivrt.ru gosarhivrt.ru
gosreg.amchs.ru gosreg.amchs.ru
gq-catalog.gq gourcy.altervista.org
greatzip.com gpirate.com
gsou.cf greatidea.marketing
gwebtools.com.br greenzaim.ru
hasshe.com guardlink.org
hazardky.net guod.me
hdhc.ru hannasolution.ru
hdfreeporno.net
headpress.ru headpress.ru
hentai-manga.porn highland-homes.com
hermesbirkinhandbagoutlets.com hledejvshopech.cz
hidemyass.com hotenergy.ru
highstairs-a.akamaihd.net hotkeys.com
hmywwogw.bloger.index.hr htmlcorner.com
honyaku.yahoofs.jp hyipmanager.in
hopeonthestreet.co.uk i-midias.net.br
horoshieokna.com imicrovision.com
hostingclub.lk imperiafilm.ru
hotblog.top incomekey.net
hotdl.in intelhdgraphicsgtdrive6w.metroblog.com
hotloans.ru
housediz.com
howtoclean.club
howtowhitenteethfast.xyz
hscsscotland.com
https-legalrc.biz
huimin764128.com
hystersister.com
ilmen.net
ilovevitaly.com
images.gyffu.com
imitex-plus.ru
impotentik.com
incest-ru.com
incolors.club
infokonkurs.ru
infoupdate.org
innodgfdriverhm.aircus.com
intellektmedia.at
internetartfair.com
invitefashion.com invitefashion.com
istanbulit.com iqs.biz.ua
itis4you.com iradiology.ru
japan-bearings.ru isitpaleo.info
japfm.com iskalko.ru
joyceblog.top istmira.ru
jrpmakati.com ix20.ru
jurajskie.info izismile.com
jyvopys.com janerikholst.se
keywordsuggests.com jaxcube.info
khadastoafarde.tk junglenet-a.akamaihd.net
kino-filmi.com kadashihotel.com
kopihijau.info kaz.kz
kovesszucs.atw.hu keki.info
krynica.info kenaba.su
kutikomi.net kicknights.gq
ladov.ru kinoduh.ru
lankarns.com kiwi237au.tk
laptop-4-less.com klumba55.ru
laserpen.club kndxbkdx.bloger.index.hr
lechenie-gemorroya.com kongoultry.net
lemon-ade.site konica.kz
lernur.net konpax.com
koopilka.com
kosova.de
krasnodar.ru
kuzinsp.ru
laxob.com
leadwayau.com
lechtaczka.net
letmacworkfaster.site letmacworkfaster.site
letsrepair.in lexixxx.com
letto.by likesdesign.com
lexaprogeneric.link liky.co.ua
lignofix.ua lineshops.biz
linerdrilling.com linkrr.com
liver-chintai.org liveinternet.ru
ljusihus.se lomb.co
lkbennettstore.online longlifelomilomi.info
lonely-mature.com louboutinreplica.pw
lookingglassemb.com lrac.info
m0r0zk0-krava.ru m5home.ru
marketingvici.com machicon-ueno.info
marktforschung-stuttgart.com mackeeper-center.club
massage-info.nl magda-gadalka.ru
matrixalchemy.com makis.nu
mb140.ru malwareremovalcenter.com
mebel-alait.ru managerpak204.weebly.com
meddesk.ru mantramusic.ru
medicovi.com manualterap.roleforum.ru
mega-polis.biz.ua mararoom.ru
michellblog.online mbiologi.ru
miniature.io mcar.in.ua
misandesign.se meble-bogart.info
mmog-play.ru med-zdorovie.com.ua
monclercoats.xyz mega-bony2017.pl
money-every-day.com metalonly.info
mostantikor.ru metaxalonevstizanidine.blogspot.com
msk.afora.ru michaelkorsoutletstore.net
muizre.ru midst.eu
musflashtv.com mielec.pl
musicstock.me minegam.com
musicvidz.ru minet.club
mobilierland.com
modnie-futbolki.net
mojpreskumanie.com
morepoweronmac.trade
moscow.xrus.org
mp3ritm.top
mrcsa.com.au
msfsaar.de
musicspire.online
mustat.com
muycerdas.xxx muycerdas.xxx
mysexpics.ru muz-shoes.ru
najaden.no mvpicton.co.uk
narco24.me mwtpludn.review
netoil.no myhydros.org
netpics.org myiptest.com
ngps1.ru myliveblog.ru
nhl17coinsforps3.gratisblog.biz nakrutka.cc
nikitsyringedrivelg.pen.io naphukete.ru
no-rx.info nastydollars.com
nosecret.com.ua national-today-winning-winner.club
naughtyconnect.com
negral.pluto.ro
nerudlogistik.ru
newyorkhotelsmotels.info
nextlnk12.com
nightvision746.weebly.com
notfastfood.ru notfastfood.ru
novgorod.xrus.org o-o-11-o-o.com
novosti-hi-tech.ru obiavo.kz
o-dachnik.ru offers.bycontext.com
obiavo.ru offside2.5v.pl
oconto.ru offtopic.biz
oecnhs.info
offf.info
omgtnc.com omgtnc.com
one.net.in one-gear.com
onlinedomains.ru
onlineslotmaschine.com onlineslotmaschine.com
openstat.com openstat.com
opinionreelle.com
optikremont.ru optikremont.ru
otbelivanie-zubov.com ordernorxx.com
ourville.info ownshop.cf
partner-trustworthy.men paidonlinesites.com
partybunny.ru panchro.co.uk
panouri-solare-acoperis.com
paramountmarble.co.uk
partner-print.men
pechikamini.ru pechikamini.ru
pelfind.me photo-clip.ru
peugeot-club.org physfunc.ru
piski.top picsfair.com
plastgran.com pinwallpaper.top
plastgranar.nu pix-hd.com
porno-transsexuals.ru plugingeorgia.com
pornoforadult.com popugaychiki.com
pornogad.com porn555.com
pornogig.com
pornohub.me
pornokajf.com pornokajf.com
pornowarp.info pornoted.com
poshiv-chehol.ru pornotubexxx.name
pospr.waw.pl pornpost.in
powc.r.ca.d.sendibm2.com prefersurvey.net
pozdravleniya-c.ru preparevideosafesystem4unow.site
predominant-invent.tk
pricheskaonline.ru pricheskaonline.ru
privatov-zapisi.ru profitwithalex.info
privetsochi.ru proftests.net
prizefestival.mobi promodj.com
prizesbook.online prostitutki-almata.org
profitfx.online prostitutki-kharkova.org
prointer.net.ua prostitutki-rostova.org
pron.pro proxyelite.biz
prosperent.com prpops.com
pussyfleet.com pulseonclick.com
pussyspace.net qcstrtvt.bloger.index.hr
pwwysydh.com qor360.com
rank-checker.online quick-offer.com
ranksignals.com ranksays.com
rapidsites.pro reimageplus.com
realresultslist.com reklama1.ru
research.ifmo.ru remont-ustanovka-tehniki.ru
rixpix.ru renhacklids.tk
romhacking.ru resant.ru
rukino.org resellerclub.com
russkoe-zdorovie.ru responsive-test.net
rvtv.ru resultshub-a.akamaihd.net
sa-live.com rock-cafe.info
saltspray.ru rospromtest.ru
sanyuprojects.com roznica.com.ua
sawin.beth.webd.pl ru-mediaget.ru
rubanners.com
rusoft-zone.ru
rustic-quiver.win
rusvideos.su
saecsa.co
sanatorrii.ru
sanidumps.com
sarf3omlat.com
sbwealthsolutions.ca
scanner-andrew.top scanner-andrew.top
scanner-fred.top score-ads.men
scanner-ivan.top
scrnet.biz.ua scrnet.biz.ua
searchwebknow-a.akamaihd.net seadragonherbery.com
selectads.men search.pch.com
seoexperimenty.ru securesmrt-dt.com
seokicks.de semaltmedia.com
service-core.ru seo-prof1.xyz
service.adtech.fr seo-traffic-ranking.info
service.adtech.us seofied.com
sevendays.com.ua seorankinglinks.com
sex-tracker.de servingnotice.com
sex-watch.com sex-foto.pw
sexblog.pw sharebutton.net
sexpornotales.com shell-pmr.ru
sexvideo-sex.com shop.xz618.com
sexvporno.ru shtaketniki.ru
sexxdate.net simplepooltips.com
sexyebonyteen.com siteonomy.com
shivafurnishings.com slim.sellany.ru
si-unique.com slowmac.tech
sibtest.ru slowmacfaster.trade
similardeals.net slujbauborki.ru
siteimprove.com snomer1.ru
sledstvie-veli.net snowplanes.com
smartadserver.com social-s-hhh.xyz
smichovbike.cz
snow.nvr163.com
snsdeainavi.info
snworks.com
socas.pluto.ro
socialbutton.xyz socialbutton.xyz
sockshares.tv socialsignals24.com
soft-terminal.ru softomix.com
softtor.com soheavyblog.com
softwaretrend.net spacash.com
songplanet.ru sportbetfair.com
soochi.co spy-sts.com
soundfrost.org stauga.altervista.org
spaceshipad.com steame.ru
spb.ru stroilka.info
sports-supplements.us suggest-keywords.com
ssconstruction.co superstarfloraluk.com
stal-rulon.ru surflinksmedical.com
star61.de surgut.zrus.org
stockspmb.info survival.betteroffers.review
strangeduckfilms.com svtrd.com
success-seo.com t-bygg.com
superinterstitial.com tacbelarus.ru
supermesta.ru taihouse.ru
superoboi.com.ua tako3.com
svolze.com tandvardshuset.net
tabletkinaodchudzanie.com.pl taqplayer.info
teenforporn.com taqywu51.soup.io
teesdaleflyballclub.co.uk
tehngr.ru tehngr.ru
templates.radiodigital.co telefonsexi.com
terrafootwear.us tetracsaudi.com
thedownloadfreeonlinegames.blogspot.com thebluenoodle.com
thegameriders.com titan-ads.life
thfox.com tizanidine4mgprice.blogspot.com
timdreby.com tizanidineandgabapentin.blogspot.com
timetorelax.biz tizanidineinteractions.blogspot.com
tizanidineduringpregnancy.blogspot.com torrents-tracker.com
tizanidinesideeffects.blogspot.com toyota.7zap.com
tomck.com traffic100.com
tongkatmadura.info
topclickguru.com
torontoplumbinggroup.com
torrentgamer.net
toursmaps.com
tpu.ru
tracksurf.daooda.com
trackzapper.com
trafficfactory.biz trafficfactory.biz
tri-slona.org tramadolandtizanidine.blogspot.com
trion.od.ua
trkdf.com
truck-addzilla.life
trustaffs.com trustaffs.com
try-rx.com turvgori.ru
ttrraacckkrr.com tv-spoty.info
tubeline.biz tvand.ru
tuckpointingmasonrysystems.com twu.com.ua
twojebook.pl txxx.com
unrealcommander.org uchetunet.su
uroffer.link umaseh.com
ussearche.cf uptimebot.net
uytmaster.ru urlopener.com
uvozdeckych.info
valsalud.com
vavilone.com vavilone.com
vduplo.ru vchulkax.com
viagra.pp.ua vclicks.net
video-chat.cn vedomstvo.net
videotuber.ru venerologiya.com
vip-dom.in ventelnos.com
vipsexfinders.com vepad.com
viteonlusarezzo.it vesnatehno.ru
vodkoved.ru voluumtracker1.com
volgograd.xrus.org vtcdns.com
vostoktrade.info vykup-avto-krasnodar.ru
vpnhowto.info vzglyadriv.kg
vsexkontakte.net w3data.co
warezaccess.com wait3sec.org
watchinf.com waycash.net
wcb.su wcb.su
web-analytics.date webadvance.club
web-betting.ru website-audit.com.ua
webcamdevochka.com welcomeauto.ru
website-speed-checker.site westum.se
welck.octopis.com wholesalejerseychinaoutlet.com
wesharepics.site whosonmyserver.com
wfb.hatedriveapart.com ws.ampower.me
wheelchairliftsarea.com wufak.com
where-toget.com
wholesalejerseychinashop.com
wholinkstome.com
wildcattube.com
wiosenny-bon-1500.pl
worldinternetauthority.com
worldmusicfests.com
worldtraveler.world
www.kabbalah-red-bracelets.com
x-diesel.biz
x-rates.ru x-rates.ru
x-stars.ru xfire.com
xcc24.pl xkaz.org
xchangetrak.com xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xev.ru xn--80aanaardaperhcem4a6i.com
xgftnlrt.bloger.index.hr xn--80ak6aa92e.com
xlovecam.com
xmronta.com
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
xn--b1agm2d.net xn--b1agm2d.net
xn--oogle-wmc.com xn--e1aggki3c.xn--80adxhks
xnxxandxvideos.com xn--lifehacer-1rb.com
xrus.org xvideosporn.biz
yadro.ru xxxasianporn.net
youghbould.wordpress.com yaminecraft.ru
yourtemplatefinder.com yellocloud.be
za-fun-offer.com yourdesires.ru
zaobao.com.sg zambini.ru
zeg-distribution.com zarepta.com
zog.link zionstar.net
zone-kev717.info zoompegs.com
zrus.org zvetki.ru
zscalertwo.net
zvuker.net
zx6.ru

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
123kuma.com 2x2fan.ru
12u.info 4senses.co
2015god.org 4ureyesonly.com
aanapa.ru 5-steps-to-start-business.com
aaronabel.com 66cpwgln.space
alfabot.xyz 7xc4n.com
alldezservice.kz abclauncher.com
alvtank.se acunetix-referrer.com
amyfoxfitness.com ads-cool.pro
adsland.men
adultfriendfinder.com
adultwebhosting.info
aero2.ru
agardomains.com
akita.kz
albamargroup.com
alborzan.com
alloysteel.ru
animalrank.com
anlimebel.kiev.ua anlimebel.kiev.ua
apiallgeniusinfo-a.akamaihd.net apimegabrowsebiz-a.akamaihd.net
apiportalmorecom-a.akamaihd.net aramaicmedia.org
apisecretsaucebi-a.akamaihd.net arendovalka.xyz
ardimobileinfo.ml arkartex.ru
aucoinhomes.com autowebmarket.com.ua
aviva-limoux.com avon-novosib.ru
b2b-lounge.com bank.uz
bang-hotties.com beachtoday.ru
bear.gotcher.us beetpics.pw
best-games.today best-seo-solution.com
bestofupload.info bio-japan.net
bibys.com bioscorp.ru
blogsfunky672.weebly.com blog.koorg.ru
bradleylive.xyz boazpower.com
brateg.xyz businesxxl.com
break-the-chains.com buyk.host.sk
buyessayonline19.blogspot.ru canrole.com
cartierbraceletsreplica.pw cartierloveringreplica.xyz
cartierlovebraceletreplica.xyz cartiertrinity.xyz
castingbank.ru casinopinup-wins.com
cayado.snn.gr cfacarrosserie74.com
cbox.ws children-learningreading.info
centrumcoachingu.com chiptuninger.com
cgi2.nintendo.co.jp clksite.com
chcu.net clodo.ru
cheapjerseysbizwholesale.us cognitiveseo.com
cheapmoncler.pw compliance-fred.xyz
chinacheapelitejerseys.com compliance-jane.top
chocolateslim-en-france.com cookingmeat.ru
classicakuhni.ru crest3d.ru
cleansearch.net customsua.com.ua
coin-hive.com cxpromote.com
compliance-barak.xyz cyclobenzaprinevstizanidine.blogspot.com
compliance-don.xyz darkages.info
compliance-donald.xyz darmebel.com.ua
cookie-law-enforcement-ff.xyz datract.com
coolgamechannel.com ddpills.com
country-chic.ru den-noch24.ru
crystalslot.com dgfitness.co
dayibiao.com divatraffic.com
dear-diary.ru dmmspy.com
decoratinghomes.ga domain2008.com
deplim.com domaincdn.xyz
dimkino.ru dostavimvdom.ru
dktr.ru dumuelave.xyz
domainsatcost.com eandsgallery.com
domoysshop.ru edge.sharethis.com
drpornogratisx.xxx edudocs.net
e-collantes.com ekn-art.se
e-poker-2005.com elektronischezigarettekaufen2.tumblr.com
easycommerce.cf elvenar.com
elektir.ru englate.com
elektrischezigarette2.devhub.com erotikchat-24.com
elektronischezigarette2.mex.tl escortplius.com
enhand.se event-tracking.com
eropho.com ezigarettenshop2.myblog.de
escort-russian.com ezigarettenshop2.postbit.com
essayassist.com falllow.gq
everflora.ru family1st.ca
ezb.elvenmachine.com familyholiday.ml
f07.de fastfixing.tech
fanpagerobot.com fenoyl.batcave.net
farmprofi.net fickkontakte.org
ff30236ddef1465f88547e760973d70a.com filmania-x.ru
filmidivx.com firesub.pl
fitnesspiks.com formulaf1results.blogspot.com
fordsonmajbor.cf fortwosmartcar.pw
forex-indextop20.ru foxjuegos.com
forpostlock.ru fr-bearings.ru
forumprofi.de frameimage.org
fotzen-ficken.com free-traffic.xyz
freemags.cc freejabs.com
fuck-paid-share-buttons.xyz ftns.ru
funcrushgames.com funcrushgames.com
fungamelands.com fungamelands.com
gay-file.com gaytube.com
geile-lelly.eu goodnightjournal.com
geileweiber.tk goodprotein.ru
get-seo-domain.com googlemare.com
getpopunder.com goroda-vsego-mira.ru
go2album.com greenzaim.ru
goldandcard.ru grizzlysgrill.com
goodhumor24.com growshop.es
goosefishpost.bid grupografico-pilar.com.ar
graphid.com haikuware.com
gsasearchengineranker.pw herehloadibs.cf
gwhwpxbw.bloger.index.hr hermesbelts.xyz
gz2.bbsoldes.fr home-task.com
hamacapty.com homelygarden.com
headpress.ru houseofrose.com
homemature.net howtoclean.club
hostcritique.com ihc2015.info
images-free.net iideaidekonkatu.info
incest-ru.com ikritikimou.gr
indianmedicaltourismshop.com impisr.ru
iplusbit.blogspot.co.za incomekey.net
iqs.biz.ua infodocsportal.com
itch.io insurple.com
izhevsk.xrus.org intim-uslugi.info
japan-bearings.ru investpamm.ru
javlibrary.cc ipchicken.com
jobgirl24.ru jyrxd.com
journalhome.com kaz.kz
kaidalibor.de keywordblocks.com
kazrent.com keywordglobal.co.za
keywordbasket.com kinofak.net
kinocccp.net kinostorm.net
krasnodar-avtolombards.ru knigonosha.net
kunstaktien.info life-instyle.com
lafriore.ru lion.conboy.us
laxdrills.com lizace.com
lineavabit.it luciddiagnostics.in
linkbolic.com m5home.ru
lol-smurfs.com magiadinamibia.blogspot.com
longlifelomilomi.info mailemedicinals.com
magicplayer-s.acestream.net manuscript.su
malwareremovalcenter.com medkritika.ru
marcoislandvacations.net metallosajding.ru
masturbate.co.uk metascephe.com
med-zdorovie.com.ua mex-annushka.ru
metalonly.info mmstat.com
mmgq.ru mncrftpcs.com
monitorwebsitespeed.com mojowhois.com
monsterdivx.com
monsterdivx.tv monsterdivx.tv
mrsdalloways.com morlat.altervista.org
mylida.org moscow.online-podarki.com
narkologiya-orel.ru mundoaberrante.com
naturtreenspicerx.pw newhairstylesformen2014.com
neodownload.webcam no-fuel.org
nevansk.ru nomuos.it
new7ob.com nonews.co
newyorkhotelsmotels.info obiavo.net
nrv.co.za onclkads.com
nudejapan.net onlinedomains.ru
okayimage.com onlyporno.ru
onlinadverts.com panchro.xyz
origin-my.ru parlament.biz
osnova3.ru partner-trustworthy.men
oyster-green.com pensplan4u.com
pagesense.com phormchina.com
par-fallen.ga pierrehardysale.online
pijoto.net pinwallpaper.top
pipki.r.acdnpro.com popugaychiki.com
pliks.pl pornonik.com
popup-jdh.xyz portside.cc
porno-dojki.net postclass.com
pornoslive.net pregnant.guru
poshiv-chehol.ru priceg.com
pospr.waw.pl priscilarodrigues.com.br
powenlite24.ru prizefestival.mobi
pretty-mart.com pro-tec.kz
prize44.com producm.ru
prodess.ru profit-opportunity.com
profitsport.club purplesphere.in
pussyfleet.com qwertty.net
putitin.me rangjued.com
pyrodesigns.com.au remorcicomerciale.ro
quickloanbank.com sale-japan.com
realtytimes.com
responsinator.com
rightenergysolutions.com.au
rixpix.ru
rvi.biz
sdrescher.net sdrescher.net
seobility.net selectads.men
seoholding.com seofirmreviewsus.info
seorussian.ru seorankinglinks.xyz
servisural.ru serialsway.ucoz.ru
sexytrend.ru serviporno.com
shemalegalls.blogporn.in sevendays.com.ua
sibecoprom.ru sexblog.pw
siha.de sexpartygirls.net
sildenafil-tadalafil.info sexsearch.com
sispe.com.br sharpchallenge.com
smart-balancewheel.com shoppytoolmac.com
snjatie-geroinovoy-lomki.ru shopvilleroyboch.com.ua
social-s-iii.xyz sideeffectsoftizanidine.blogspot.com
socialbuttons.xyz sientalyric.co
soundfrost.org similardeals.net
space4updating.win site3.free-share-buttons.com
specialfinanceoffers.com slowmac.tech
sportbetfair.com smartpet.ru
sribno.net soblaznu.net
stard.shop socialmonkee.com
store-rx.com softomix.com
solmarket.by
sport7777.net
start.myplaycity.com
steelmaster.lv
strigkaomsk.ru strigkaomsk.ru
superlist.biz stroiminsk.org
takeflyte.com strongholdsb.ru
td-l-market.ru stuff-about-money.com
teguh.info sukarame.net
template-kid.com surintech.ac.th
terraclicks.com taboola.com
tgtclick.com timdreby.com
theautoprofit.ml tooplay.com
theendivechronicles.com totu.info
thetattoohut.com tourismvictoria.com
thin.me.pn tovaroboom.vast.ru
thruport.com toys.erolove.in
tiandeural.ru turbo-suslik.org
tizanidinehcl4mgtab.blogspot.com
trahic.ru
triplepanda.xyz
tula.howotorg.ru
turist-strani.ru turist-strani.ru
twodollarshows.com tzritel.tk
tytoona.com urengoy.pro
uggbootsoutletsale.us valoresito.com
urlopener.com varikozdok.ru
vclicks.net
vduplo.ru
vekzdorov.ru
vgoloveboli.net vgoloveboli.net
vietimgy.pw videonsk.com
visa-china.ru vigrx-original.ru
warningwar.ru vipsiterip.org
webcamtalk.net vita.com.hr
webmasters.stackexchange.com webix.me
websites-reviews.com weburlopener.com
white-truck.life wellcome2slovenia.ru
wingsoffury2.com wiosenny-bon-1500.pl
wladimirpayen.com wma-x.com
www.event-tracking.com woweb.com.ua
x-lime.com xmnb.net
x-lime.net xn----7sbaphztdjeboffeiof6c.xn--p1ai
xblog.in xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xgames-04.com yhit.press
xn----9sbubg3ambdfl1j.xn--p1ai yougame.biz
xxx-treker.ru yourporngay.com
xxxguitars.com zaloro.com
xz618.com zero1.it
yottos.com zeroredirect9.com
your-bearings.com zigarettenonlinekaufen1.blox.pl
youradexchange.com zog.link
youtuhe.com zooggames.com
yur-p.ru
zakon-ob-obrazovanii.ru
zaobao.com.sg
zapatosenventa.info
zarabotok--doma.ru
zetmaster.ru
zoogdinsney.com
zryydi.com
zvuker.net

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

12
dev-tools/test_units/ratelimittest-master.txt
View file

@ -1,11 +1,21 @@
<!DOCTYPE html>
<html> <html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
ml><html> <html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.166 0; 10.1.0.99 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3534 ### Version: V4.2023.01.3535
### Updated: Mon Jan 2 10:03:35 UTC 2023 ### Updated: Mon Jan 2 22:01:23 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3534 ### Version: V4.2023.01.3535
### Updated: Mon Jan 2 10:03:35 UTC 2023 ### Updated: Mon Jan 2 22:01:23 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##