vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-06 20:50:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3484 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-21 22:07:01 +00:00
parent 61215f7815
commit 30a51861d7
47 changed files with 86926 additions and 86936 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3483 #### Version: V4.2022.11.3484
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3483 #### Version: V4.2022.11.3484
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3483 #### Version: V4.2022.11.3484
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10728
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3483 #### Version: V4.2022.11.3484
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10732
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10732
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10732
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10732
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.1.29 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.1.29 0;
127.0.0.1 0; 127.0.0.1 0;

10732
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.1.29 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.1.29 0;
127.0.0.1 0; 127.0.0.1 0;

10732
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.1.29 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.1.29 0;
127.0.0.1 0; 127.0.0.1 0;

10732
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3483 3484

10732
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10732
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.120.199 0; 20.228.81.243 0;

10732
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.1.29 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.1.29 0;
127.0.0.1 0; 127.0.0.1 0;

10732
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.1.29 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.1.29 0;
127.0.0.1 0; 127.0.0.1 0;

10732
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.1.29 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.1.29 0;
127.0.0.1 0; 127.0.0.1 0;

10732
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.120.199 0; 20.228.81.243 0;

10732
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.120.199 0; 20.228.81.243 0;

10732
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 1; 10.1.1.29 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Anthill AllSubmitter
BuiltWith BetaBot
GetRight Claritybot
Indy Library DBLBot
ScreenerBot Image Fetch
SiteLockSpider Minefield
Sqlmap ScanAlert
TightTwatBot ZyBorg
WebsiteQuester trendiction.de
polaris version x09Mozilla

298
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404enemy ADmantX
ASPSeek Aboundexbot
Abonti Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AiHitBot
Aipbot Aipbot
Alexibot Alligator
AllSubmitter AlphaBot
Anthill Anarchy99
Apexoo
Asterias
Atomseobot Atomseobot
Attach Attach
BDFetch BDFetch
BackWeb BackWeb
BatchFTP BacklinkCrawler
Battleztar Bazinga Bandit
BlackWidow BetaBot
Bolt Bitacle
BotALot Blow
Brandwatch Boardreader
Buddy Buddy
BuiltBotTough
BuiltWith BuiltWith
Bullseye Bullseye
BunnySlippers
Calculon
CensysInspect CensysInspect
CheTeam
CherryPicker CherryPicker
ChinaClaw
Citoid Citoid
Claritybot Claritybot
Cloud mapping Cliqzbot
Cogentbot Cocolyzebot
Copier CopyRightCheck
Craftbot Copyscape
Custo Cosmos
DSearch CrazyWebCrawler
DTS Agent Curious
DataCha0s CyotekWebCopy
DIIbot
DatabaseDriverMysqli DatabaseDriverMysqli
Devil Devil
DigitalPebble DigitalPebble
Dirbuster Disco
Discobot Discoverybot
Dispatch
DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DomainAppender DnyzBot
DomainCrawler
Download Wonder
Dragonfly
Drip
EMail Siphon
EMail Wolf EMail Wolf
EasyDL EasyDL
Ebingbong Ebingbong
Ecxi
EirGrabber
EroCrawler
Exabot Exabot
Express WebPictures Express WebPictures
Extractor Extractor
FDM ExtractorPro
FHscan Extreme Picture Finder
FemtosearchBot Fimap
Freeuploader Firefox/7.0
FyberSpider Foobot
Fyrebot FrontPage
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW
GalaxyBot GalaxyBot
Getintent Genieo
GrabNet GermCrawler
Grabber Gigabot
Grafula Go-Ahead-Got-It
HTTP::Lite GrapeshotCrawler
Harvest HMView
IRLbot HTTrack
Haansoft
HaosouSpider
Hloader
IDBTE4M
Iblog
IlseBot IlseBot
Image Fetch
Image Sucker Image Sucker
IndeedBot IndeedBot
Indy Library Indy Library
InfoNaviRobot Intelliseek
InternetSeer InternetSeer
Iria
Iskanie Iskanie
JOC Web Spider IstellaBot
Jorgee JennyBot
Jetty
JikeSpider
Joomla
JustView
Kenjin Spider Kenjin Spider
Keybot Translation-Search-Machine Kinza
Keyword Density Kozmosbot
Lanshanbot LNSpiderguy
Larbin LeechFTP
LexiBot
Lftp Lftp
Libwhisker Libwhisker
LinkScan Likse
LinkWalker LinkWalker
Linkbot Linkbot
LinkextractorPro
LinkpadBot LinkpadBot
LinksManager LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey Spider MFC_Tear_Sample
Litemage_walker MQQBrowser
Lmspider
MIIxpc
MTRobot MTRobot
Mag-Net Mag-Net
Magnet
Majestic SEO Majestic SEO
Majestic-SEO Majestic-SEO
MarkMonitor MarkMonitor
Masscan MarkWatch
Mata Hari MegaIndex.ru
MauiBot
Mediatoolkitbot
MicroMessenger
Microsoft URL Control Microsoft URL Control
Mister PiX Moblie Safari
Mojolicious Mojolicious
Morfeus Fucking Scanner Mozlila
Mr.4x3
Msrabot
Musobot Musobot
NICErsPRO
NPbot
Name Intelligence
NearSite
Nessus
Net Vampire Net Vampire
NetLyzer NetMechanic
Netcraft Nettrack
Netvibes Netvibes
Niki-bot NextGenSearchBot
Nibbler
Nikto
Ninja Ninja
Nuclei Nutch
Octopus Octopus
Openfind OpenLinkProfiler
Openvas OpenVAS
PHPCrawl
Page Analyzer
PageAnalyzer PageAnalyzer
PageScorer PageGrabber
Pavuk
PeoplePal PeoplePal
Piepmatz Piepmatz
Pimonster Pimonster
Pixray Pockey
ProPowerBot ProPowerBot
ProWebWalker Probethenet
Pu_iN Pu_iN
Pump QueryN Metasearch
RankActive RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankingBot RankingBot
ReGet Rankivabot
RocketCrawler RankurBot
SEOkicks Re-re
SEOprofiler Ripper
ScanAlert Rogerbot
Scanbot SEOlyticsCrawler
ScoutJet
Scrapy Scrapy
Screaming Screaming
Searchestate ScreenerBot
SemanticJuice Seekport
Semrush Semrush
SentiBot SentiBot
Site Sucker SeoSiteCheckup
SiteExplorer SiteLockSpider
SiteSnagger Sitebeam
SiteSucker Siteimprove
Sitevigil
Snake
Snapbot Snapbot
Sociscraper
Sogou web spider Sogou web spider
SpaceBison SpankBot
Spammen Spanner
Spbot Spbot
Spinn3r Spinn3r
SputnikBot SputnikBot
Sqlmap
Sqlworm Sqlworm
Sqworm
Steeler Steeler
Stripper Stripper
Sucker
SuperBot
SuperHTTP SuperHTTP
Suzuran Suzuran
Szukacz Teleport
Telesoft The Intraformant
TheNomad Thumbor
TightTwatBot Tracemyfile
Toata Turnitin
Toweyabot TurnitinBot
True_Robot Typhoeus
Twice VeriCiteCrawler
URLy Warning VidibleScraper
URLy.Warning
Virusdie
VoidEYE VoidEYE
WASALive-Bot Voil
WBSearchBot WWW-Collector-E
WISENutbot
Wallpapers Wallpapers
Wallpapers/3.0 WallpapersHD
WeSEE WeSEE
Web Enhancer Web Enhancer
Web Pix Web Pix
Web Sucker WebCopier
WebCollage
WebFuck WebFuck
WebGo IS WebImageCollector
WebLeacher
WebSucker WebSucker
WebWhacker WebWhacker
WebZIP WebZIP
Webalta Webshag
Website Quester
WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster Webster
Whack Whack
Whacker Whacker
Widow Widow
WiseGuys Robot
Wonderbot
Woobot
Wprecon
Xaldon WebSpider Xaldon WebSpider
YoudaoBot Xaldon_WebSpider
Xenu
Zade Zade
Zermelo
Zitebot Zitebot
ZmEu ZoominfoBot
ZumBot
ZyBorg ZyBorg
autoemailspider arquivo.pt
check1.exe check1.exe
coccocbot cognitiveseo
com.plumanalytics com.plumanalytics
crawl.sogou.com crawl.sogou.com
domainsproject.org crawler.feedback
crawler4j
dataforseo.com
evc-batch evc-batch
instabid
internetVista monitor internetVista monitor
iubenda-radar ips-agent
linkdexbot isitwp.com
lwp-trivial netEstate NE Crawler
meanpathbot
mediawords
pcBrowser pcBrowser
plumanalytics polaris version
probe-image-size
ripz
s1z.ru s1z.ru
seobility scalaj-http
seocompany.store scan.lol
seoscanners
seostar
serpstatbot
sitechecker.pro
siteripz siteripz
sogouspider sogouspider
sp_auditbot tAkeOut
spyfu ubermetrics-technologies.com
sysscan
trendiction.de
webmeup-crawler webmeup-crawler
x22Mozilla webprosbot
zauba.io xpymep1.exe

320
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider
80legs 80legs
Abonti ADmantX
Aboundexbot AIBOT
Acunetix ALittle Client
ASPSeek
Aboundex
AhrefsBot
AiHitBot AiHitBot
Aipbot Aipbot
Anarchy AllSubmitter
Ankit Anarchie
Apexoo Anarchy99
Aspiegel Aspiegel
Asterias
Attach
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike
BDCbot
BDFetch
BLEXBot BLEXBot
BacklinkCrawler Backlink-Ceck
Barkrowler Bigfoot
Battleztar Bazinga
Bitacle
Black Hole
BlackWidow BlackWidow
BlowFish Blow
BuiltBotTough Boardreader
BuzzSumo Bolt
CCBot Brandprotect
CODE87 BuiltWith
CATExplorador
CSHttp CSHttp
Calculon Calculon
CensysInspect CazoodleBot
CherryPicker Cegbfeieh
CheTeam
ChinaClaw
Chlooe Chlooe
Citoid Cogentbot
Claritybot Cosmos
Cloud mapping
Collector
Copier
Copyscape
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
CrunchBot Curious
DBLBot
DIIbot
DSearch DSearch
DTS Agent Demon
Deusu
Digincore
Dirbuster
Disco Disco
Discobot
Dispatch Dispatch
DittoSpyder DittoSpyder
DnyzBot
DomCopBot DomCopBot
DomainCrawler
DomainStatsBot DomainStatsBot
Domains Project Domains Project
Dotbot Drip
Dragonfly Ebingbong
EMail Wolf EroCrawler
EasyDL
Ecxi
Evil
Exabot Exabot
Express WebPictures Extractor
ExtLinksBot
ExtractorPro
EyeNetIE EyeNetIE
FDM
FHscan
Fimap Fimap
Firefox/7.0 FlashGet
Foobot Foobot
Freeuploader Freeuploader
FrontPage Fuzz
G-i-g-a-b-o-t Fyrebot
GalaxyBot GT::WWW
GetWeb Genieo
GermCrawler
GetRight
Getintent
Gigabot Gigabot
GoZilla Go!Zilla
GrapeshotCrawler GrabNet
Grabber
GrapeFX
GridBot
HEADMasterSEO HEADMasterSEO
HMView HMView
Haansoft HTMLparser
Havij HonoluluBot
Heritrix HybridBot
IRLbot
Iblog Iblog
Id-search Id-search
IlseBot IlseBot
Image Fetch Image Sucker
InfoNaviRobot IndeedBot
Intelliseek InfoTekies
Iria InterGET
Internet Ninja
InternetSeer
Iskanie Iskanie
IstellaBot IstellaBot
JOC Web Spider Jetty
JikeSpider
Jorgee
Keybot Translation-Search-Machine
Kozmosbot Kozmosbot
LWP::Simple LNSpiderguy
Lanshanbot LexiBot
Leap Lightspeedsystems
LeechFTP LinqiaRSSBot
Lftp
Libwhisker
LinksManager
Lipperhey Spider Lipperhey Spider
Lmspider Lmspider
Ltx71 MJ12bot
MQQBrowser
MSFrontPage MSFrontPage
Mag-Net
Magnet
Majestic-SEO
MarkMonitor MarkMonitor
MarkWatch MarkWatch
MauiBot
Mb2345Browser Mb2345Browser
Meanpathbot
Mediatoolkitbot Mediatoolkitbot
MegaIndex.ru Metauri
Microsoft URL Control
Minefield
MolokaiBot
Mozlila Mozlila
Msrabot Mr.4x3
NICErsPRO
NPbot
Name Intelligence Name Intelligence
Nameprotect Nameprotect
Navroad NetLyzer
Needle
Net Vampire
NetSpider
Nibbler
Nikto
NimbleCrawler NimbleCrawler
Nimbostratus Nimbostratus
Nmap Nuclei
Not Nutch
Openvas Octopus
OutfoxBot Offline Explorer
Offline Navigator
PECL::HTTP PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageThing.com Page Analyzer
Pavuk Petalbot
Picscout Pimonster
PictureFinder
Pixray Pixray
ProWebWalker PleaseCrawl
ProPowerBot
Psbot Psbot
Pu_iN PxBroker
Pump
QueryN Metasearch
Quick-Crawler Quick-Crawler
RankActive RSSingBot
RankFlex RankActiveLinkBot
RankurBot Rankivabot
RealDownload RealDownload
RebelMouse Reaper
Recorder Recorder
Ripper RedesScrapy
RocketCrawler RepoMonkey
Rogerbot
SBIder
SEOkicks SEOkicks
SEOlyticsCrawler
SEOprofiler
SISTRIX SISTRIX
SalesIntelligent
ScanAlert ScanAlert
Scanbot Scanbot
ScoutJet Screaming
Searchestate ScreenerBot
ScrepyBot
SearchmetricsBot
Seekport Seekport
SemanticJuice SeoSiteCheckup
Seomoz Shodan
Siphon Site Sucker
SiteLockSpider SiteExplorer
SiteSucker
Sitebeam
Siteimprove
Sitevigil Sitevigil
SlySearch SmartDownload
Snapbot Snoopy
Sosospider Spammen
SpaceBison Spinn3r
Spanner Sqlworm
SputnikBot
Sqlmap
Sqworm Sqworm
Stripper Steeler
Sucuri SuperHTTP
Surfbot
SurveyBot SurveyBot
Swiftbot
Szukacz Szukacz
T8Abot
TeleportPro TeleportPro
Tracemyfile Telesphorep
The Intraformant
TheNomad
Thumbor
TightTwatBot
Toweyabot
Trendictionbot Trendictionbot
Turnitin Turnitin
TwengaBot TurnitinBot
Upflow Typhoeus
VB Project Vagabondo
Vacuum VeriCiteCrawler
VelenPublicWebCrawler Virusdie
VidibleScraper
VoidEYE VoidEYE
Voltron Voltron
WBSearchBot WISENutbot
WPScan
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE
Wallpapers/3.0
WeSEE WeSEE
Web Collage Web Auto
Web Enhancer Web Fetch
Web Sauger Web Sauger
Web Sucker Web Sucker
WebBandit WebCopier
WebCollage
WebEnhancer WebEnhancer
WebFetch WebFetch
WebStripper WebGo IS
WebPix
WebReaper
WebSucker
WebWhacker WebWhacker
Webshag Webalta
WebsiteExtractor
Whack
Whatweb
Who.is Bot
WiseGuys Robot WiseGuys Robot
Wonderbot Wonderbot
Wotbox
Xaldon_WebSpider
Xenu Xenu
Zauba
Zitebot
ZmEu
ZoomBot ZoomBot
ZumBot archive.org_bot
arquivo-web-crawler arquivo-web-crawler
backlink-check backlink-check
cah.io.community
check1.exe
clark-crawler
coccocbot
cognitiveseo cognitiveseo
com.plumanalytics
crawler.feedback
crawler4j crawler4j
demandbase-bot demandbase-bot
eCatch facebookscraper
gopher gopher
heritrix
instabid instabid
isitwp.com internetVista monitor
iubenda-radar
linkdexbot linkdexbot
lwp-trivial
meanpathbot meanpathbot
oBot pcBrowser
plumanalytics polaris version
probe-image-size seobility
ripz
seocompany.store
seoscanners
seostar seostar
sitechecker.pro siteripz
sogouspider sogouspider
sp_auditbot
spyfu spyfu
trendiction.de tAkeOut
trendiction.com
ubermetrics-technologies.com
voyagerx.com voyagerx.com
webgains-bot webmeup-crawler
webprosbot
x09Mozilla x09Mozilla
x22Mozilla
xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
actulite.com allblogroll.com
bytimedance.ru automatic-seo.com
candytiner.com biteg.xyz
censys.io brainboostingsupplements.org
chatseo.com freewebs.com
forsex.info mhi-systems.ru
hatedriveapart.com prosmibank.ru
inet-traffic.com rocis.site
mysex21.com sosdepotdebilan.com
russintv.fr superlist.biz

954
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
2015god.org 000free.us
00it.com
1000n1.ru
1001desert.com
11pikachu.ru
12u.info
1688.com
256bit.by
2kata.ru 2kata.ru
4k-player.pl 5000-cotydzien.com
4rent.ru 5ws.dating-app.ru
4replicawatch.net a.frcls.fr
4ureyesonly.com aa81bf391151884adfa3dd677e41f94be1.com
aadroid.net aab94f698f36684c5a852a2ef272e031bb.com
abusora.com acheterviagrafr24.com
acortarurl.es
acunetix-referrer.com
adimmix.com
admatic.com.tr admatic.com.tr
adserver-e7.com adobereader-free.ru
adultactioncam.com adultnet.in
adultgalls.com advokateg.xyz
advokat-grodno.by affrh2015.com
advokateg.ru agysya.ru
aimayangzhi.com
albamargroup.com albamargroup.com
albumsuper.info ald2014.org
alfa-img.com
algarveglobal.com
alibestsale.com alibestsale.com
alkoravto.ru alt-servis.ru
all4wap.ru aludecor.info
allornamenti.com ample-awards-today.us
allpdfmags.net android-style.com
am-se.com angkortours.vn
amanda-porn.ga
amateurlivechat.org
amazon-seo-service.com
anaksma.info
android-vsem.org
anime.dougasouko.com
anlimebel.kiev.ua anlimebel.kiev.ua
apartamentwroclaw.eu ap.senai.br
apiholdingmypage-a.akamaihd.net apple.com-cleaner.systems
apilinkswiftco-a.akamaihd.net approvedlocal.co.za
apioasisspacenet-a.akamaihd.net arcadeplayhouse.com
applicationg29.com arraty.altervista.org
arpelsreplica.xyz asdfg.pro
autobrennero.it atmovs.com
aviav.org audiobangout.com
ayanaspa.com autonew.biz
azartmix.com aviav.eu
backiee.com avtovolop.ru
badavit.com.ua azarttoto.com
bambi.ck.ua azinoofficial777.ru
bankofthewext.com baltgem.com
battlecarnival.su bank.uz
bausparen.bz.it bannerads.de
baoxaydung.com.vn
bazaronline24.ru bazaronline24.ru
bcmp.org bear.gotcher.us
beachpics.com beget.tech
benchmarkcommunications.co.uk besthomemadepornsites.com
berdasovivan.ru beyan.host.sk
berlininsl.com bfz.biz
bestimagecoollibrary.com biglistofwebsites.com
betterhdporn.com bimatoprost-careprost.com
bezlimitko.xyz bingo8888.com
big-cash.net bitnote.co
blackwitchcraft.ru bjgugu.net.ua
blog.koorg.ru blackbabesporn.com
bloggen.be blackcurranthumidifiers.site
bmw-ark.ru blockworld.ru
bochemit.com.ua bmw.afora.ru
bodybuilding-shop.biz boleznikogi.com
bolegapakistan.com bracketsmackdown.com
bonky.biz brainboosting.club
boo-arts.com brains2.biz
braintobe.win brainxs.us
brandedleadgeneration.com browsepulse-a.akamaihd.net
browserprotecter.com bumble.cheapwebsitehoster.com
buigas.00it.com burger-imperia.com
builtwith.com buyantiviralwp.com
buqayy0.livejournal.com buyclomidonlaine.com
buqyxa.rincian.info bvlgaribracelet.xyz
burger-tycoon.com californianews.cf
buron.pw canacopegdl.com
buyk.host.sk candy-glam-hp.com
cah.io.community capacitacionyaprendizaje.com
calvet.altervista.org capsderfudd.tk
camsex.xxx-cam.webcam carson.getenjoyment.net
canoncdriverq3.pen.io cbb1smartlist12.click
capture-room.com cementaresearch.se
cartierbracelet.xyz censys.io
cartiertrinity.xyz cerev.info
ccbill.com chatroulette.life
centrdebut.ru cheap-trusted-backlinks.com
cete.ru cheapmarmot.online
chatroulette.world check-host.net
cheapcigarettesc.info children-learningreading.info
checkm8.com chinaelitecheapjerseys.com
chocoslim.pro chocolateslim-en-espana.com
ci.ua cindymatches.com
cialischsrx.com citetick.com
clean-virus-mac.com citysecurity.nu
clevernt.com clickpapa.com
click2pawn.com clickprotects.com
clickbanksites.info clicksotrk.com
clipartnew.com cocolyze.com
clknsee.com coderstate.com
cmd.kz codysbbq.com
co.lumb.co collegeessay19.blogspot.ru
com-onlinesupport.site comissionka.net
cometorussia.net companies-catalog.com
compliance-barak.xyz compliance-alexa.xyz
concordexoticrentals.com compliance-fred.top
coolgamechannel.com compliance-mark.top
cowlmash.com coop-gamers.ru
cottageofgrace.com
counter.yadro.ru
cranly.net
credit-online.ws
creditnation.ru creditnation.ru
creditwell.ru cubook.supernew.org
culmimg.pw culad.com
customboxes4less.com customboxes4less.com
cutalltheshit.com cutalltheshit.com
d0t.ru cymbaltaandtizanidine.blogspot.com
ddlmega.net dalavia.ru
de.zapmeta.com dangphoto.trade
decoratinghomes.ga daooda.com
deda-moroza-zakaz.ru darodar.com
dfiles.me data1.scopich.com
diffbot.com datsun-do.com
ddospanel.com
degerlund.net
delitime.info
design-ideas.info
digital-sale.su
digitalfaq.com
dignitasdata.se dignitasdata.se
diminishedvalueoforegon.com dms-sw.ru
discountliv.com dnsrsearch.com
discovertreasurenow.com downloadeer.net
divisioncore.com dprtb.com
djonwatch.ru druzhininevgeniy63.blogspot.com
djstools.com dynainbox.com
dnepr-avtospar.com.ua e2click.com
dogbreedspicture.net electrik-avenue.com
domainanalyzing.xyz eleimgo.pw
domainsatcost.com elektrischezigaretten1.blogse.nl
dosugrostov.site emergencyneeds.org
downloader12.ru eng-lyrics.com
draniki.org erotag.com
easyshoppermac.com esnm.ru
easyukraine.com etm-consult.de
editors.choice6912650.hulfingtonpost.com exchangeit.gq
eksprognoz.ru extremez.net
elidelcream.weebly.com ezigarettenshop1.yolasite.com
en.altezza.travel f-loaded.de
ero-advertising.com farmingworm.com
ertelecom.ru fatmaelgarny.com
evrotekhservis.ru feel-planet.com
exci.ru fenoyl.batcave.net
excitacao.com feorina.ru
extlinks.com fermersovet.ru
ezigarettekaufen1.hpage.com filunika.com.ru
facebook-mobile.xyz fitness-video.net
familyphysician.ru fix-website-errors.com
fearcrow.com flashahead.info
filefilter.weebly.com folowsite.com
filesvine.com fototravel.eu
filmbokep69.com free-share-buttons.blogspot.com
findpics.pw free411games.com
finemanteam.com freenode.info
finstroy.net freesitetest.com
fitfloponline.store fugarif.ga
forexunion.net fym.com.ru
forodvd.com galeria-zdjec.com
fotzen-ficken.com gambarxkata.co
foxjuegos.com gavuer.ru
frameimage.org
freelinkbuilding.website.tk
freeseedsonline.com
freevpn.space
freewareseek.com
fuckmill.com
fullfileaccess.com
futbolkisales.ru
fyl.com.ru
gamezblox.com
gate5.co.za
gazoblok.net.ua gazoblok.net.ua
ge0ip.com generic-pills-online.com
getmyads24.com gerhardhealer.com
ghostvisitor.com get-free-traffic-now.com
girlporn.ru get-your-social-buttons.info
girls-ufa.ru getyourimage.club
glavtral.ru ggiaro.com
gloverid.site gobongo.info
gogalleryawesome.com golfresa.lucania.se
good-mummy.ru goodprotein.ru
goodhumor24.com goosefishpost.bid
goodwinmetals.co gov.yanao.ru
googlepositions.com gratuitxblcodes.com
gotcher.us guardlink.com
gsasearchengineranker.space gymi.name
gsasearchenginerankerdiscount.com habermetre.com
hacktougroup.ru hausfrauensex18.com
hem.passagen.se hdapp1008-a.akamaihd.net
hermesreplica.pw heliko.no
hobbyhuren24.net help.tpu.ru
homelygarden.com hikesearch.net
hotblog.top hitcpm.com
ibmdatamanagement.co hoholikik.club
igadgetsworld.com hol.es
ikearugs.xyz homedecorpicture.us
ilbe.club houdom.net
ilmen.net hoverboardforsaledirect.com
ilovevitaly.com hvd-store.com
imagine-ex.co ilo134uloh.com
img.wallpaperstock.net imagefinder.site
immigrational.info images-free.net
inclk.com imperia31.ru
indianmedicaltourismshop.com importchinacoach-teach.com
indo-export.ru impotentik.com
infostatsvc.com impresagaia.it
inspiring-desperate.tk infazavr.ru
instabid.tech infektsii.com
iplogger.org int.search.mywebsearch.com
ipornox.xxx intelhdgraphicsgtdrive6w.metroblog.com
iradiology.ru investingclub.ru
iscblog.info iqs.biz.ua
istanbulit.com isitpaleo.info
ixora.pro isitwp.com
izhevsk.zrus.org istizanidineanopiate.blogspot.com
javstock.com it-max.com.ua
jennyfire.ru ivearchenceinflu.cf
jerseyswholesalechinalimited.com jackwolfskinoutlet.online
jerseywholesalechinabiz.com javxxx18.com
jjbabskoe.ru joessmogtestonly.com
journeydownthescale.info jovencitas.gratis
joy-penguin.com justucalling32211123456789.tk
junglenet-a.akamaihd.net kareliatobacco.ru
kabbalah-red-bracelets.com keki.info
kamorel.com kevblog.top
keenoutlet.online keywords-monitoring-success.com
kicknights.gq keywordspay.com
keywordsuggest.org
kino-ecran.ru
kinoflux.net kinoflux.net
klin3952.ru kiskinhouse.com
klumba55.ru kiwe-analytics.com
komputers-best.ru kmd-pto.ru
ks1234.com kndxbkdx.bloger.index.hr
kursy-ege.ru koleso24.com.ua
kwzf.net kongoultry.net
kosova.de
kovesszucs.atw.hu
krasnodar.ru
kupiproday.com.ua
landinez.co
larger.io
laudit.ru laudit.ru
legalrc.biz lawyersinfo.org
lego4x4.ru letslowbefast.today
letto.by lider-zhaluzi.kiev.ua
linkarena.com lion.misba.us
ljusihus.se lsex.xyz
lkbennettstore.online luckyshop.net.ua
lob.com.ru m5home.ru
lonerangergames.com mac-shield.com
louboutinreplica.pw mactechinfo.info
louboutinshoes.xyz maju.bluesalt.co
m1media.net manimpotence.com
mackeeper-land-672695126.us-east-1.elb.amazonaws.com manualterap.roleforum.ru
macnewtech.com map028.com
magento-crew.net markjaybeefractal.com
makenahartlin.com mebel-vstroika.ru
manve.info medanestesia.ru
marmotstore.online medkletki.ru
matematikus.info
max-eclat.men
max-p.men
maximilitary.ru
mcnamaratech.com
medicineseasybuy.com
medkritika.ru medkritika.ru
meendo-free-traffic.ga meet-flirt-dating.com
michaelkorsoutlet.store mega-bony-2017.pl
michaelkorsoutletstore.net megaindex.ru
michellblog.online metafilter.com
microstatic.pl mexicosleevegastrectomy.com
mmog-play.ru misswell.net
mobifunapp.weebly.com mnimmigrantrights.net
mobplayer.ru mnogabukaff.net
modnie-futbolki.net modabutik.ru
mompussy.net moivestiy.biz
monarhs.info moje-recenze.cz
moncleroutletonline.pw morepoweronmac.trade
monetizer.com-01.site morf.snn.gr
moneyzzz.ru moscow.online-podarki.com
montazhnic.ru most.gov.iq
moroccosurfadventures.com motherboard.vice.com
moviemail-online.co.uk mrcsa.com.au
mycouponizemac.com muizre.ru
myfreetutorials.com musezone.ru
naturalshair.site music7s.me
nikitsyringedrivelg.pen.io myblogregistercm.tk
nw-servis.ru myindospace.com
o-o-6-o-o.com mylida.org
oakridgemo.com mypets.by
obiavo.by mysexpics.ru
odywpjtw.bloger.index.hr myshopmatemac.com
okroshki.ru naturtreenspicerx.pw
newfilmsonline.ru
nicovideo.jp
nippon-bearings.ru
nisuturnetdgu.tk
noclegonline.info
nrv.co.za
nudepatch.net
nuker.com
o-o-11-o-o.com
o-o-8-o-o.ru
obiavo.kz
ochistka-stokov.ru
ofanda.com
okayimage.com
olympescort.com
omoikiri-japan.ru omoikiri-japan.ru
online-hd.pl one-gear.com
onlinewritingjobs17.blogspot.ru oneshotdate.com
ontargetseo.us online-x.ru
openlibrary.org onlineporno.site
openstat.com onlineserialy.ru
onlyforemont.ru
opinionreelle.com opinionreelle.com
osb.se11.ru origin-my.ru
oxford-book.com.ua otbelivanie-zubov.com
pateaswing.com p-business.ru
pathwhelp.org pavlodar.xkaz.org
patol01.pw pc-services.ru
pb-dv.ru pcimforum.com
pc-virus-d0l92j2.pw pdns.download
pcgroup.com.uy pechikamini.ru
pearlisland.ru pelfind.me
phelissota.xyz pendelprognos.se
phormchina.com pepperstyle.ru
photokitchendesign.com pigrafix.at
photorepair.ru pila.pl
picanalyzer.data-ox.com pinkduck.ga
pijoto.net pinsdaddy.com
pinstake.com pl.aasoldes.fr
pinwallpaper.top pl.id-forex.com
pirateday.ru placid-rounded-coast.glitch.me
pkr1hand.com pladform.ru
plaff-go.ru pobeiranie.pl
platezhka.net popcash.net
ploenjitmedia.azurewebsites.net pornhive.org
plugingeorgia.com
pod-muzyku.club
pornophoto.xyz pornophoto.xyz
poshiv-chehol.ru portside.cc
pozdrawleniya.com presleycollectibles.com
pravoholding.ru princevc.com
printingpeach.com prizesbook.online
prlog.ru promoheads.com
proekt-gaz.ru prostitutki-astana.org
profitsport.club prpops.com
progonrumarket.ru psa48.ru
projectforte.ru pushdata.sendpulse.com
prom23.ru qld10000.net
prombudpostach.com.ua ranksays.com
pronorm.fr razyboard.com
propranolol40mg.blogspot.com recipedays.ru
pulseonclick.com redhotfreebies.co.uk
purchasepillsnorx.com remont-komputerov-notebook.ru
putitin.me remorcicomerciale.ro
quality-traffic.com rentaremotecomputer.com
queerspace.com replicaclub.ru
quickcashlimited.com resultshub-a.akamaihd.net
raavidesigns.com rff-cfal.info
radiodigital.co rixpix.ru
rebelmouse.com rocis.site
redbottomheels.xyz rockprogblog.com
rednise.com rubbed.us
referencemoi.com
registry-clean-up.net
reklama-i-rabota.ru
reklama1.ru
relatodelpresente.com.ar
remybutler.fr
restorator-msk.ru
rightenergysolutions.com.au
rogervivierforsale.com
rosbalt.com.ua
rp9.ru
runofilms.ru
rusoft-zone.ru
russkie-gorki.ru russkie-gorki.ru
saitevpatorii.com saddiechoua.com
savememoney.co.za samo-soznanie.ru
sbetodiodnye-lampy.ru santechnik.jimdo.com
sbricur.com sawin.beth.webd.pl
scanner-jess.top sbprabooks.com
scanner-brian.top
scanner-john.top scanner-john.top
scanner-viktor.top searchencrypt.com
scanner-walt.top searchkut.com
scanspyware.net semalt.com
seo18.su semprofile.com
seoboxes.com semrush.com
seorank.info seokicks.de
seozoom.it sexpornotales.net
serdcenebolit.com sextracker.be
sexad.net shiksabd.com
sexkontakteao.info shohanb.com
sexkvartal.com shop.acim.org
sexyteens.hol.es shopcheermakeup.info
shikiso.info shopwme.ru
shinikiev.com.ua sibecoprom.ru
shivafurnishings.com sibtest.ru
sientalyric.co simon3.ru
silktide.com skytraf.xyz
silvermature.net slujbauborki.ru
simplesite.com smadihome.com
sims-sims.ru smartadserver.com
sklad-24.ru snworks.com
skyway24.ru social-button.xyz
slavic-magic.ru socialsignals24.com
slayerlife.com soheavyblog.com
snyatie-lomki-v-stacionare.ru
socialtrade.biz
sockshare.net
sodexo.com
soft-terminal.ru
sonata-arctica.wz.cz sonata-arctica.wz.cz
songplanet.ru souvenirua.com
spabali.org soviet-portal.do.am
spaceshipad.com space-worry.ml
specialfinanceoffers.com spinnerco.ca
speeddream.xyz spravka130.ru
sportbetfair.com spy-app.info
spywarenuker.com spymac.net
stair.registrydomainservices.com stoki.ru
stairliftsarea.com stopnarco.ru
star61.de stretchmate.net
statoutlook.info studworks.org
steebook.com sugvant.ru
stillmiracle.com superoboi.com.ua
streetfooduncovered.com supers.com.ua
students-cheapskate.ml surgut.zrus.org
sugarkun.com svbur.ru
summerlinhomes411.info sweepstakes.rewardit.com
suture.co swinginwithme.ru
swingerseiten.com t3chtonic.com
syndicate.fun tabletkinaodchudzanie.com.pl
tacbelarus.ru tagil.zrus.org
taqplayer.info tarad.com
thecounter.com tattooreligion.ru
theendivechronicles.com taxi-v-eisk.ru
themeforest.net teenbbw.yopoint.in
thesmartsearch.net telefonsexkostenlos.tk
thetattoohut.com thecoral.com.br
thfox.com tizanidineforopiatewithdrawal.blogspot.com
titan-cloud.life top-karkas.ru
tizanidineandgabapentin.blogspot.com torrnada.ru
tizanidinehcl2mgtablet.blogspot.com totu.us
tizanidinestreetprice.blogspot.com tourcroatia.co.uk
tkanorganizma.ru traffic2cash.org
top10-way.com trafficreceiver.club
tovaroboom.vast.ru tri-slona.org
tpu.ru trk-4.net
trackmedia101.com truck-addzilla.life
trafficshaper.com truck-land.life
traffpartners.com truebeauty.cc
trailer.cinemaflix.website tubeline.biz
try-rx.com tuckermktg.com
tube8.com
turbo-suslik.org turbo-suslik.org
twitlinks.com twittrading.com
txxx.com tx41tclega.ru
tzritel.tk tyumen.xrus.org
uchetunet.su uggbootsoutletsale.us
ucoz.ru uginekologa.com
uggsale.online undergroundcityphoto.com
unionmarkt.de unmaroll.ya.ru
uptime.com unpredictable.ga
urccvfmc.bloger.index.hr valoresito.com
url-extractor.xyz venerologiya.com
urll.eu verymes.xyz
vacuumcleanerguru.com vesnatehno.ru
vacuumscleaner.com videospornogratisx.net
velobikestock.com vigrx-original.ru
videositename.com vinsit.ru
vinylvault.co.uk vinylvault.co.uk
voluumtrk.com vip-parfumeria.ru
vonradio.com viteonlusarezzo.it
vroze.com viveresaniesnelli.it
vse-pesni.com vkak.ru
vtc.pw vladimir.xrus.org
vykup-avto-krasnodar.ru vsesubwaysurfers.com
vzlom-na-zakaz.com vvpg.ru
wallpapersist.com vzglyadriv.kg
wanker.us vzlomfb.com
warning.or.kr wahicbefa31.soup.io
webalan.ru watchdogs-2.ru
webmasters.stackexchange.com watsonrealtycorp.com
wechatdownload10.com webkeyit.com
welck.octopis.com website-stealer.nufaq.com
welcomeauto.ru weightatraining.com
weprik.ru win-spy.com
whatzmyip.net wineitudes.wordpress.com
whereiskentoday.com witherrom55.eklablog.fr
wholesalejerseychinaoutlet.com wjgony.com
wholesalejerseyscheapjerseys.us.com wma-x.com
wieseversa.no woodyguthrie.se
wingsofrefuge.net word-vorlagen.xyz
wiosenny-bon-1500.pl www.ehscloud.cn
woman-h.ru www.souvenirua.com
works.if.ua xbaboon.com
worst-sites.online xchangetrak.com
wovis.site xitjw.info
www.get-free-traffic-now.com xn----7sbifcamovvfggw9d.xn--p1ai
www.kabbalah-red-bracelets.com
www.pinnacle-bets.com
xaylapdiendanang.com
xev.ru
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
xn----9sbebi2bvzr7h.xn--p1ai xn----ctbbcjd3dbsehgi.xn--p1ai
xn----itbkqkfiq.xn--p1ai xn--80atua3d.xn--p1ai
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai xportvusbdriver8i.snack.ws
xn--80aeahghtf8ac5i.xn--p1ai xtraffic.plus
xvideosbay.com xtrafficplus.com
youporn-ru.com xtube.com
yourads.website yellowfootprints.com
yoursearch.me yogamatsexpert.com
zarabiaj-dzis.pl yoopsie.com
zeg-distribution.com youjizz.com
zigarettenonlinekaufen2.drupalgardens.com youradexchange.com
zionstar.net youtoner.it
zojirushi-products.ru yur-p.ru
zoompegs.com zahvat.ru
zebradudka.com
zero1.it
zerocash.msk.ru
zhacker.net
zoogdinsney.com
zscaler.net
zverokruh-shop.cz

480
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1kinobig.ru 0daymusic.org
256bit.by 1001watch.com.ua
3-letter-domains.net 108shot.com
3dracergames.com 1pamm.ru
5i2.net 2015god.org
8si.ru 21h2o.com
aa08daf7e13b6345e09e92f771507fa5f4.com 2x2fan.ru
aab94f698f36684c5a852a2ef272e031bb.com 3th.co.in
abcdeg.xyz 3w24.com
adclickservice.com 5ws.dating-app.ru
adktrailmap.com 8xv8.com
adssafeprotected.com ads-cool.pro
alt-servis.ru advokateg.xyz
ananumous.ru aludecor.info
anjalika.co.in amigobulls.com
apartamentwroclaw.eu anaksma.info
apple.com-webbrowsing-security.review apessay.com
appsecurityr.com apidiamondatanet-a.akamaihd.net
aproposde.com apiitsthirteende-a.akamaihd.net
auto-moto-elektronika.cz arabsexxxtube.com
autolombard-krasnodar.ru arcadepages.com
autorn.ru arendatora.ru
azartmix.com autoblogger24.info
azazaporn.com autotop.com.ua
balla-boo.se azartclub.org
bannertesting.com bcmp.org
bedcapdealers.com beclean-nn.ru
beremenyashka.com bget.ru
best-coupon-offer.com bidr.trellian.com
best-seo-software.xyz bingo8888.com
bodybuilding-shop.biz blacktwhite.com
book-bets.com blogig.org
brandehk.dk blogqpot.com
businescoop.men bluejays-jerseys.us
business-suggested.tk bluesman.nu
buyclomidonlaine.com brains2.biz
buyessaynow.biz buyfriend.ru
buytizanidineonlinenoprescription.blogspot.com buytizanidineonline.blogspot.com
buzzonclick.com bycontext.com
camdenmemorials.com calcularpagerank.com
caulderwoodkennels.com candlehandmade.com
cb.iphantom.com carabela.com.do
certifywebsite.win carmuffler.net
cgi2.nintendo.co.jp cartechnic.ru
chiblackhawks-jerseys.com cartierbracelet.xyz
ciekawinki.pl cartierjusteunclou.xyz
cognitiveseo.com children-learningreading.info
compliance-barak.top chinajerseyswholesalecoupons.com
compliance-george.xyz clips.ua.ac.be
compliance-jane.top codq.info
coolbar.pro compliance-donald.xyz
copenergo.ru compliance-josh.top
creatives.adbetclickin.pink coolgramgoods.com
creditwell.ru covid-schutzmasken.de
culad.com credit-online.ws
cyse.tk darkbooks.org
dalavia.ru db.speedup.gdn
data1.scopich.com ddospanel.com
dear-diary.ru detskie-konstruktory.ru
demian.kz discovertreasure-a.akamaihd.net
dengi-pod-zalog-nedvizhimosti.ru documentsite.net
derevesendeco.com downloads-whatsapp.com
digitalfaq.com drunkmoms.net
dignitasdata.se dtnlyss.com
dinosaurus.site dumuelave.xyz
djekxa.ru dwomlink.info
doctormakarova.ru dynainbox.com
domaincdn.xyz dyshagi.ru
donvito.unas.cz eas-seo.com
drillsaw.ru eduserver.net
dumpsmania24.com ekosmetyki.net.pl
ecookna.com.ua elektrischezi.canalblog.com
editmedios.com en.home-task.com
elitesportsadvisor.com energydiet-info.ru
enskedesquashclub.se errorfixing.space
european-torches.ru eu-cookie-law.blogspot.com
evening-dating-club.info exci.ru
eyelike.com.ua farmingworm.com
favoritemoney.ru fashion-stickers.ru
ferrotodo.com felizporno.com
filmci.pro firma-legion.ru
focalink.com firstdrugmall.ru
forminecrafters.ru floridahuntingfishingadventures.com
forum-engineering.ru footbalive.org
frbizlist.com foto-telok.net
free-deals.faith frankofficial.ru
freemaintenancesysforpcandmac.top freemaintenancesysforpcandmac.top
gambarxkata.co frequiry.com
gamedayhouse.com fuck-paid-share-buttons.xyz
getaclueamerica.com geoads.com
goforexvps.com gg.zzyjxs.com
goldpanningtools.com gigapeta.com
gpms.org.my goldadpremium.com
grizzlysgrill.com goroda-vsego-mira.ru
gsmlab.pl gunsvicceadadebt.tk
gwebtools.com guod.me
hamilton.ca hamacapty.com
handicapvantoday.com headpress.ru
hcate.com healbio.ru
hildinghr.se hesteel.pl
homeinns.com highstairs-a.akamaihd.net
homemade.gq historichometeam.com
hotchatdate.com hit-men.men
htmlcorner.com hitsbox.info
iccornacircri.cf hkladys.com
ictizanidinehcl4mg.blogspot.com hmmm.cz
ifmo.ru howtowhitenteethfast.xyz
ilmen.net ikritikimou.gr
ilte.info imagine-ex.co
impisr.edunsk.ru
import-sales.com import-sales.com
impresagaia.it interfucks.net
incolors.club internet-apteka.ru
indianmedicaltourismshop.com investyb.com
inet-traffic.com ipornox.xxx
ipchicken.com jerseyschinabizwholesale.us
iplogger.org jobmarket.com.ua
istizanidineanarcoticdrug.blogspot.com justbcause.com
itronics.ca kanimage.com
joessmogtestonly.com kino-key.info
katushka.net kinoflux.net
kinoplen.ru koronirealestate.gr
klejonka.info kurwa.win
kmgamex.cf lescinq.com
komputernaya-pomosh-moscow.ru linkbolic.com
konkursov.net maju.bluesalt.co
koral.se mansiondelrio.co
kreativperlen.ch mararoom.ru
landmania.ru
ledis.top
lietaer.com
likesdesign.com
livingroomdecor.info
lookingglassemb.com
mall.uk
marketingtechniques.info
mathgym.com.au mathgym.com.au
mebeldekor.com.ua
med-zdorovie.com.ua
mega-polis.biz.ua
mesto-x.com
mexicosleevegastrectomy.com mexicosleevegastrectomy.com
milblueprint.com mobilemedia.md
mnogolok.info monthlywinners.com
mobile-appster.ru moomi-daeri.com
monetizer.com-01.site mototsikl.org
moscow.xrus.org mundoaberrante.com
muz-baza.net music7s.me
mybackgroundlandscape.blogspot.com myblogregistercm.tk
nalogovyy-kodeks.ru myphotopipe.com
namecrumilchlet.tk mysex21.com
narosty.com narkologiya-orel.ru
netcheckcdn.xyz net-profits.xyz
njpalletremoval.com nfljerseyscheapchinabiz.com
nonameread45.live nosecret.com.ua
offtopic.biz noumeda.com
ok-ua.info november-lax.com
okonich.com.ua nucia.biz.ly
optibuymac.com ofanda.com
oren-cats.ru okout.ru
ownshop.win old-rock.com
paidonlinesites.com onclickpredictiv.com
pcboa.se onetravelguides.com
pdfprof.com onlinemeetingnow.com
photo-clip.ru ooo-olni.ru
picphotos.net ops.picscout.com
pipki.r.acdnpro.com orsonet.ru
play-mp3.com paradontozanet.ru
plugingeorgia.com partners-ship.pro
pobeiranie.pl perfectpracticeweb.com
pogruztehnik.ru petitions.whitehouse.gov
popserve.adscpm.net picture-group.com
popup.matchmaker.com pizza-imperia.com
pl.id-forex.com
pod-muzyku.club
porndl.org
porndroids.com
pornoblood.com
pornogratisdiario.com pornogratisdiario.com
pornoslive.net pornohub.me
pornosmola.info pornstartits.xblog.in
pricheski-video.com powitania.pl
professionaldieselcare.com prchecker.info
prostitutki-rostova.org prizestohandle.club
prostitutki-tolyatti.org producm.ru
puzzleweb.ru prostitutki-kharkova.org
qitt.ru
qld10000.net qld10000.net
quangcaons.com rabot.host.sk
ranksonic.info ragecash.com
rapidsites.pro rcb101.ru
redirect.trafficreceiver.club reeyanaturopathy.com
registrydomainservices.com regdefense.com
rossanasaavedra.net rentalcarnavi.info
rossmark.ru rentehno.ru
research.ifmo.ru
rogervivierforsale.com
rospromtest.ru
rvi.biz rvi.biz
santechnik.jimdo.com scanmyphones.com
sax-sex.com scanner-fred.top
scanner-josh.top scanner-jess.top
scanner-mary.top scanspyware.net
scenicmissouri.us sdelatmebel.ru
searchwebknow-a.akamaihd.net seorussian.ru
servicecenter.co.ua sexytrend.ru
sexphoto.site shoppingmiracles.co.uk
shakhtar-doneck.ru shtaketniki.ru
sharpchallenge.com sicfor.bcu.cc
shell-pmr.ru signx.info
shmetall.com.ua simul.co
simpoed.ufop.br siteonomy.com
soblaznu.net slutloadlive.com
socialsignals24.com smichovbike.cz
spinazdrav.ru snworks.com
spymac.net sockshare.net
statistici.ro speechfoodie.com
sucsesofinspiration.com srdrvp.com
srgwebmail.nl
steebook.com
strana-solnca.ru
strongsignal-a.akamaihd.net
sukirgenk.dvrlists.com sukirgenk.dvrlists.com
thaismartloan.com supermodni.com.ua
thefotosgratis.eu teesdaleflyballclub.co.uk
timeallnews.ru telefonsexi.com
tizanidine4mgtablets.blogspot.com tfxiq.com
tizanidineoral.blogspot.com tizanidine4mgprice.blogspot.com
toon-families.com tk-assortiment.ru
traffic2cash.org tnctrx.com
trahvid.com topappspro.com
truck-addzilla.life topvidos.ru
twodollarshows.com tour-line.net
txxx.com tourismvictoria.com
typer.one trafficgenius.xyz
ukkala.xyz turkeyreport.tk
ukrtextbook.com ukrtextbook.com
umityangin.net urlopener.com
upproar.com vacances-voyages.info
uptime-gamma.net videojam.tv
veopornogratis.xxx visionwell.com.cn
video-woman.com vroze.com
vipms.ru webradiology.ru
viteonlusarezzo.it whiteelephantwellington.com
vitoriacabos.com wineitudes.wordpress.com
warningzscaler.heraeus.com www.event-tracking.com
waysbetter.cn wygraj-teraz.com
wcb.su wzgyyq.com
wdrake.com xblog.in
webjam.com xcombear.ru
webstatsdomain.org xn--24-glceagatoq7c2a6ioc.xn--p1ai
westsextube.com xn--80ahvj9e.xn--p1ai
whereiskentoday.com xuki.us
whereverdesperate.gq xxart.ru
wildnatureimages.com xxxasianporn.net
williamrobsonproperty.com yoursearch.me
www.kabbalah-red-bracelets.com youtuhe.com
xn----itbkqkfiq.xn--p1ai zeroredirect11.com
xn--80atua3d.xn--p1ai zeroredirect8.com
xn--h1ahbi.com.ua
xn--l1aengat.xn--p1ai
xwatt.ru
yaoguangdj.com
yurgorod.ru
zetgie.com.pl zetgie.com.pl
zhuravlev.info

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.28 0; 10.1.1.29 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3483 ### Version: V4.2022.11.3484
### Updated: Mon Nov 21 10:03:34 UTC 2022 ### Updated: Mon Nov 21 22:01:15 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3483 ### Version: V4.2022.11.3484
### Updated: Mon Nov 21 10:03:34 UTC 2022 ### Updated: Mon Nov 21 22:01:15 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##