vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3499 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-29 10:09:06 +00:00
parent c90039e73f
commit 1f9778832c
47 changed files with 97931 additions and 97921 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3498 #### Version: V4.2022.11.3499
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3498 #### Version: V4.2022.11.3499
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3498 #### Version: V4.2022.11.3499
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12104
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3498 #### Version: V4.2022.11.3499
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12108
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12108
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12108
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12108
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

12108
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

12108
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

12108
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3498 3499

12108
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12108
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.169.185.237 0; 172.177.34.140 0;

12108
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

12108
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

12108
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

12108
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.169.185.237 0; 172.177.34.140 0;

12108
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.169.185.237 0; 172.177.34.140 0;

12108
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 1; 10.1.0.88 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
01h4x.com Discobot
BLEXBot Firefox/7.0
Iria Kozmosbot
NextGenSearchBot Lanshanbot
Offline Navigator SeobilityBot
OnCrawl SiteSnagger
SEOlyticsCrawler Szukacz
WebPix WebGo IS
seostar Widow
x22Mozilla ips-agent

304
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 360Spider
404enemy 404enemy
AIBOT 80legs
ADmantX
ALittle Client ALittle Client
Abonti ASPSeek
Aboundex Aboundex
Acunetix AfD-Verbotsverfahren
AhrefsBot Alligator
AiHitBot Anarchy99
Aipbot Apexoo
Anarchie AwarioRssBot
Asterias
Atomseobot
BBBike BBBike
BDFetch
BLEXBot BLEXBot
BackDoorBot BackDoorBot
Badass BackStreet
BatchFTP BackWeb
BetaBot Backlink-Ceck
Bigfoot BacklinkCrawler
BlackWidow
Blackboard
Blow Blow
BlowFish
Bolt Bolt
Brandprotect BotALot
Buck Brandwatch
BuzzSumo BuiltWith
CCBot CATExplorador
CODE87
CSHttp CSHttp
CazoodleBot Calculon
CheeseBot Cegbfeieh
ChinaClaw CensysInspect
Chlooe Chlooe
Cliqzbot Claritybot
Cloud mapping
Cocolyzebot
Cogentbot
Copyscape Copyscape
Cosmos Cosmos
Craftbot CrunchBot
Crescent Curious
DBLBot DIIbot
DataCha0s DSearch
Digincore DatabaseDriverMysqli
Deusu
DigitalPebble
Disco
Discobot Discobot
Discoverybot
Dispatch
DittoSpyder DittoSpyder
DnyzBot
DomCopBot
DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
ECCP/1.0 Download Wonder
EasyDL Dragonfly
Ecxi EMail Siphon
Evil EroCrawler
ExtLinksBot ExtLinksBot
Extractor
FDM FDM
FHscan
Flunky Flunky
Foobot
Freeuploader Freeuploader
FrontPage
Fuzz Fuzz
FyberSpider GT::WWW
Fyrebot
G-i-g-a-b-o-t
GalaxyBot GalaxyBot
GermCrawler
GetRight
GetWeb GetWeb
Gigabot
Go!Zilla
GoZilla
Gotit Gotit
Grabber Grafula
GridBot GrapeshotCrawler
HMView
HTMLparser HTMLparser
HTTP::Lite Haansoft
HTTrack
Harvest Harvest
Havij
Heritrix Heritrix
Hloader
HonoluluBot HonoluluBot
HybridBot
IDBTE4M IDBTE4M
IDBot IDBot
IRLbot Id-search
IlseBot
Image Sucker
IndeedBot
Indy Library Indy Library
InfoTekies InfoTekies
Intelliseek Iskanie
InterGET JOC Web Spider
InternetSeer JamesBOT
IstellaBot
Jbrofuzz
JennyBot
JetCar JetCar
Jetty
Joomla Joomla
Jyxobot Jyxobot
Keybot Translation-Search-Machine
Keyword Density
LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot
LeechFTP
LexiBot LexiBot
Lftp LibWeb
Likse LieBaoFast
LinkpadBot Lightspeedsystems
LinqiaMetadataDownloaderBot Linkbot
LinkextractorPro
Lipperhey Lipperhey
Litemage_walker Lipperhey Spider
MIIxpc MFC_Tear_Sample
MJ12bot
MSFrontPage
Mag-Net Mag-Net
Magnet Magnet
Mail.RU_Bot
Majestic12
MarkMonitor
MarkWatch MarkWatch
Mass Downloader
MauiBot
Mb2345Browser Mb2345Browser
MeanPath Bot Mediatoolkitbot
MegaIndex.ru
Microsoft URL Control Microsoft URL Control
Minefield Mojeek
Mister PiX
MolokaiBot MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila
NICErsPRO NICErsPRO
Name Intelligence NPbot
Nameprotect Navroad
Needle Needle
Net Vampire Net Vampire
NetSpider
NetZIP
Netcraft Netcraft
Netvibes
Nikto Nikto
Nimbostratus
Ninja Ninja
Octopus Not
Openfind Nutch
OrangeBot OpenLinkProfiler
OutclicksBot
PECL::HTTP PECL::HTTP
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageScorer PageScorer
PageThing.com PageThing.com
Papa Foto Papa Foto
Pavuk Pavuk
PeoplePal
Picsearch
PictureFinder
Piepmatz Piepmatz
Pimonster PleaseCrawl
Pixray
ProPowerBot
ProWebWalker ProWebWalker
Pu_iN Psbot
PxBroker Pump
PyCurl
QueryN Metasearch QueryN Metasearch
RSSingBot Quick-Crawler
RankActiveLinkBot RankActive
RankingBot2 RankurBot
Rankivabot
Re-re Re-re
RealDownload RealDownload
Reaper
Recorder
RedesScrapy
RocketCrawler
Rogerbot
SEOkicks SEOkicks
SEOkicks-Robot SalesIntelligent
SEOlyticsCrawler
SMTBot
ScanAlert
ScoutJet ScoutJet
ScreenerBot Scrapy
ScrepyBot
SearchmetricsBot
Seekport Seekport
SemanticJuice Siphon
Semrush
SentiBot
SeobilityBot
Shodan
Site Sucker Site Sucker
SiteExplorer SiteCheckerBotCrawler
SiteLockSpider SiteLockSpider
SiteSucker
Sitevigil
SmartDownload
Snake
Sociscraper Sociscraper
SpaceBison
SpankBot
Spanner Spanner
Spinn3r Spbot
Sqlmap SputnikBot
Stripper Steeler
Sucker Sucuri
SuperHTTP SuperHTTP
Surfbot Surfbot
Szukacz
T0PHackTeam T0PHackTeam
Telesoft
Telesphoreo Telesphoreo
TightTwatBot TheNomad
Titan
Toata
Toweyabot Toweyabot
Trendictionbot Tracemyfile
True_Robot Turingos
TwengaBot
Typhoeus
URLy Warning URLy Warning
URLy.Warning
VB Project VB Project
Voil Vacuum
Voltron VelenPublicWebCrawler
VidibleScraper
WISENutbot
WPScan WPScan
WWW-Collector-E
WWW-Mechanize
WWWOFFLE WWWOFFLE
WallpapersHD Wallpapers
Web Fetch WeSEE
Web Fuck Web Auto
Web Pix Web Collage
WebBandit WebBandit
WebImageCollector WebCopier
WebPix WebPix
WebSucker WebSauger
WebWhacker
WebZIP
Webalta Webalta
WebmasterWorldForumBot
WebsiteExtractor WebsiteExtractor
Whack Wotbox
Whatweb
WiseGuys Robot
Woobot
Wprecon Wprecon
YoudaoBot YoudaoBot
Zauba
Zeus Zeus
Zitebot Zitebot
ZmEu ZoomBot
ZyBorg ZyBorg
adscanner
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
autoemailspider arquivo.pt
backlink-check backlink-check
cah.io.community crawl.sogou.com
clark-crawler
cognitiveseo
crawler.feedback
crawler4j crawler4j
dataforseo.com
demandbase-bot
domainsproject.org
eCatch eCatch
evc-batch facebookscraper
heritrix gopher
instabid internetVista monitor
isitwp.com linkdexbot
lwp-request lwp-request
lwp-trivial magpie-crawler
netEstate NE Crawler
oBot oBot
page scorer probe-image-size
polaris version scalaj-http
ripz
scan.lol
seobility seobility
seocompany.store
seoscanners seoscanners
sexsearcher
siteripz siteripz
sogouspider sp_auditbot
spyfu spyfu
tAkeOut sysscan
trendiction.com
trendiction.de trendiction.de
webmeup-crawler voyagerx.com
x09Mozilla
zauba.io zauba.io
zgrab

322
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider
404checker
80legs 80legs
ADmantX Abonti
Aboundexbot Acunetix
AfD-Verbotsverfahren
AiHitBot AiHitBot
AllSubmitter Aipbot
Ankit Alexibot
Anarchy
Anthill Anthill
Apexoo Aspiegel
Atomseobot
Attach Attach
BackDoorBot AwarioRssBot
BacklinkCrawler BackStreet
BatchFTP Bandit
BetaBot Blow
Black Hole
BlackWidow
Boardreader
Bolt Bolt
Brandprotect Brandprotect
Buck Brandwatch
Buddy Buddy
BuiltBotTough BuiltBotTough
BuiltWith Bullseye
BunnySlippers BunnySlippers
Cegbfeieh BuzzSumo
CCBot
CODE87
ChinaClaw ChinaClaw
Citoid Cocolyzebot
Copier Cogentbot
Copyscape
Cosmos
Crawling at Home Project Crawling at Home Project
Crescent CrazyWebCrawler
Curious CrunchBot
DataCha0s DBLBot
DTS Agent
DatabaseDriverMysqli
Demon
Deusu
Digincore Digincore
DigitalPebble DigitalPebble
Dirbuster Disco
DnyzBot Discobot
Discoverybot
DomainAppender DomainAppender
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project Dotbot
Download Wonder Download Wonder
EMail Wolf Drip
EasyDL
Ebingbong
Ecxi
EirGrabber EirGrabber
EroCrawler Express WebPictures
Evil
Exabot
ExtractorPro ExtractorPro
Extreme Picture Finder EyeNetIE
Ezooms Ezooms
FDM
FHscan
FemtosearchBot FemtosearchBot
Fimap
Firefox/7.0
FlashGet FlashGet
Flunky Flunky
Fuzz
FyberSpider FyberSpider
Fyrebot GT::WWW
G-i-g-a-b-o-t
GermCrawler GermCrawler
GetWeb Getintent
GrabNet Go!Zilla
Grafula Grabber
GrapeFX GrapeFX
GrapeshotCrawler GrapeshotCrawler
GridBot HEADMasterSEO
HMView HMView
HTMLparser
HTTP::Lite HTTP::Lite
Haansoft HTTrack
Harvest HaosouSpider
Havij
Heritrix
Hloader Hloader
IRLbot IRLbot
Iblog
Id-search Id-search
Image Sucker
IndeedBot IndeedBot
Indy Library
InfoNaviRobot
InfoTekies InfoTekies
Intelliseek InterGET
InternetSeer JOC Web Spider
Iria
Jbrofuzz Jbrofuzz
JennyBot JetCar
Joomla
Jorgee Jorgee
Kinza Kinza
Kozmosbot
LWP::Simple LWP::Simple
Lanshanbot Lanshanbot
Larbin LexiBot
Leap
LeechGet
LibWeb LibWeb
LieBaoFast Libwhisker
Likse
Linkbot Linkbot
LinkpadBot LinkextractorPro
LinksManager Ltx71
Lipperhey Spider MFC_Tear_Sample
MIDown tool
MIIxpc
MSFrontPage
MSIECrawler
MTRobot MTRobot
Magnet Magnet
Majestic SEO
Majestic-SEO Majestic-SEO
Majestic12 Majestic12
MarkMonitor MarkMonitor
Mass Downloader Mass Downloader
Masscan Mata Hari
Metauri Meanpathbot
Microsoft Data Access Mediatoolkitbot
Microsoft URL Control MegaIndex.ru
Morfeus Fucking Scanner Minefield
Mozlila Mister PiX
Mojeek
NICErsPRO
NPbot
Nameprotect
Navroad Navroad
Needle Needle
Nessus
Net Vampire Net Vampire
NetAnts NetZIP
NetMechanic Nettrack
NextGenSearchBot NextGenSearchBot
Nibbler
NimbleCrawler
Nimbostratus Nimbostratus
Ninja Ninja
Not Not
Offline Explorer Nuclei
Offline Navigator OpenVAS
Openvas Openvas
OrangeBot
OrangeSpider
PECL::HTTP
PHPCrawl PHPCrawl
Page Analyzer Panscient
PageGrabber Petalbot
Papa Foto
PeoplePal
Pi-Monster
PictureFinder
Pimonster
Pockey
ProPowerBot ProPowerBot
Pu_iN ProWebWalker
Pump Pump
PxBroker RankActive
PyCurl RankFlex
RankActiveLinkBot
RankingBot2 RankingBot2
ReGet RankurBot
Recorder RealDownload
RebelMouse
RedesScrapy RedesScrapy
RepoMonkey Ripper
SEOkicks RocketCrawler
SEOkicks-Robot Rogerbot
SEOstats SEOprofiler
SISTRIX
SMTBot SMTBot
ScanAlert Screaming
Searchestate ScreenerBot
SearchmetricsBot ScrepyBot
SeoSiteCheckup Seekport
SeobilityBot Semrush
SemrushBot
Seomoz Seomoz
SiteCheckerBotCrawler Siphon
SiteLockSpider Site Sucker
SiteExplorer
SiteSnagger
Sitebeam
Siteimprove
Sitevigil Sitevigil
Snapbot SmartDownload
Sogou web spider SocialRankIOBot
Sociscraper
Sosospider Sosospider
SpaceBison Sottopop
Spammen
SpankBot SpankBot
Spanner Spinn3r
SputnikBot SputnikBot
Steeler Sqlmap
Sqlworm
Stripper Stripper
Sucker Sucuri
Szukacz SuperBot
SurveyBot
Suzuran
T8Abot T8Abot
The Intraformant Teleport
Titan Telesphoreo
Trendiction Telesphorep
True_Robot TheNomad
Thumbor
Tracemyfile
Trendictionbot
Turingos
Turnitin Turnitin
TwengaBot TurnitinBot
URLy.Warning Twice
V-BOT Typhoeus
Vacuum Upflow
Voil VB Project
WISENutbot Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper
Virusdie
WPScan
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WeSEE
Web Collage Web Collage
Web Fetch Web Fetch
WebCollage
WebFuck WebFuck
Web Pix
Web Sucker
WebCollage
WebCopier
WebFetch
WebGo IS WebGo IS
WebStripper WebLeacher
WebPix
WebReaper
WebSauger
WebZIP WebZIP
Webalta Webalta
Webshag Who.is Bot
WebsiteExtractor WiseGuys Robot
WebsiteQuester
Webster
Whack
Widow
WinHTTrack
Wonderbot
Woobot
Wprecon Wprecon
Xaldon WebSpider
YoudaoBot YoudaoBot
Zauba Zitebot
Zermelo
Zeus
ZmEu ZmEu
ZumBot
ZyBorg ZyBorg
arquivo-web-crawler adscanner
cah.io.community arquivo.pt
com.plumanalytics cognitiveseo
crawler.feedback crawler.feedback
eCatch dataforseo.com
demandbase-bot
evc-batch evc-batch
facebookscraper facebookscraper
lwp-request gopher
lwp-trivial internetVista monitor
muhstik-scan isitwp.com
netEstate NE Crawler iubenda-radar
magpie-crawler
mediawords
oBot oBot
page scorer
pcBrowser pcBrowser
plumanalytics polaris version
s1z.ru
scan.lol
seostar
serpstatbot serpstatbot
sitechecker.pro siteripz
spyfu sogouspider
sysscan
tAkeOut
trendiction.com trendiction.com
trendiction.de ubermetrics-technologies.com
voyagerx.com webmeup-crawler
webgains-bot
webpros.com
webprosbot
xpymep1.exe
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
256bit.by all-streaming-media.com
dawlenie.com freeuploader.com
digilander.libero.it ge0ip.com
ezigarettenkaufen1.pagina.gr gobongo.info
moneytop.ru kovrenok.ru
paradontozanet.ru pornoslive.net
pornokajf.com shareyards.com
pulseonclick.com vizag.kharkov.ua
seorankinglinks.xyz womensterritory.ru
voluumtracker1.com yourporngay.com

908
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0lovespells0.blogspot.com 1008.su
1-free-share-buttons.com 10bet.com
1001watch.com.ua 11pikachu.ru
202ch.com 12masterov.com
404.mn 14b.info
4istoshop.com 178evakuator178.ru
5elementov.ru 1kinobig.ru
999webdesign.com 2ads.co.uk
aac52006ec82a24e08b665f4db2b5013f7.com 4senses.co
aasoldes.fr 55wmz.ru
abctoppictures.net 6128786.com
accessoires-mode.in 7makemoneyonline.com
adclickservice.com 8gold.com
add-add.men 9med.net
adloads.net aaronabel.com
adsloads.com absolutelyanalog.com
adtiger.tk ads.gold
adultfullhd.com adsref.men
advertisingtag.net advancedcleaner.com
advocatemsk.ru advancedsoftwaresupport.com
affordablewebsitesandmobileapps.com advokat-grodno.by
ageofclones.com alfa9.com
ald2014.org
alkoravto.ru
all-number.com all-number.com
all4invest.info allnews.md
alot.com
amateurmatch.com
amehdaily.com
andpolice.com andpolice.com
androids-store.com
angkortours.vn angkortours.vn
animali.deagostinipassion.it anonymousfox.co
animalrank.com apartment.ru
apimossnetinfo-a.akamaihd.net apessay.com
appearance-cool.com apioasisspacenet-a.akamaihd.net
artel116.ru appfixing.space
articlesdirectoryme.info architecturebest.com
atleticpharm.org artefakct.com
auspolice.com asdfz.pro
autobudpostach.club autobrennero.it
autowebmarket.com.ua aviav.eu
ayerbo.xhost.ro avtoarenda.by
azlex.uz ayakino.net
balla-boo.se backuperwebcam.weebly.com
baoxaydung.com.vn balkanfarma.org
bannerspace.com
bavsac.com
bazaronline24.ru
bensbargains.net bensbargains.net
best-businessman.ru
bestcoin.cc
bestdooz.com
bestfortraders.com bestfortraders.com
bet365.com besthomemadepornsites.com
betonka.pro bestofpicture.com
bigcareer.info besttorrentknifta.weebly.com
bio.trade-jp.net bestwrinklecreamnow.com
bizlist.com.de betshuckclean.com
bkgr.se betteroffers.review
blog.koreadaily.com bezcmexa.ru
blogporn.in bigpenisguide.com
bolitgorlo.net birzha-truda.eu
boostmyppc.com bkns.vn
brateg.xyz blacktwhite.com
btvn.ru blogqpot.com
bloke.com
bnt-team.com
boanonihaca.tk
bocoarchives.org
bookhome.info
breastaugmentation.co.za
brillianty.info
briomotor.co
budilneg.xyz
bugof.gq
buscarfoto.com buscarfoto.com
buydissertation.net buyessayonline19.blogspot.ru
buyfriend.ru c-english.ru
bvlgaribracelet.xyz cakesplus.com.au
cablecar.us calstaterealty.us
cacheimages.com candypeople.se
cartechnic.ru carapics.com
cartierlovebracelet.xyz cartierreplica.xyz
cartierring.xyz cartierwatch.xyz
ccjp.eu
cdn.walkme.com cdn.walkme.com
cenoval.ru changde.58.com
cezartabac.ro chatroulette.world
cheapsergiorossi.online cheapbarbour.online
chiblackhawks-jerseys.com cherubinimobili.it
clean-start.net chorus.terakeet.com
cleanersoft.com cialisovercounteratwalmartusa.com
clevernt.com ciceros.co
clickbank.net citetick.com
clickpapa.com cleanallspyware.com
clicksor.net clicksor.net
clicksotrk.com club-samodelkin.ru
clipartnew.com clubfashionista.com
cntravelre.com compliance-alex.xyz
compiko.info compliance-andrew.xyz
compliance-elena.xyz compliance-fred.xyz
compliance-jack.top compliance-jack.top
compliance-jess.top compucelunlock.net
compliance-josh.top concordexoticrentals.com
compliance-viktor.top consorzioilmosaico.org
construmac.com.mx continent-e.tv
cookie-law-enforcement-ff.xyz cookie-law-enforcement-ff.xyz
cookie-law-enforcement-hh.xyz copyrightclaims.org
cookingmeat.ru culturevie.info
coolingoods.com custodieva.ru
cordstrap.cc
credit-respect.ru
culmimg.pw
customcatchcan.com customcatchcan.com
cutt.us d2jsp.org
daneshetabiat.com darodar.com
daydream-studio.ru data-ox.com
deadlinkchecker.com day-news.info
dealighted.com deniven.1bb.ru
dealwifi.com derevesendeco.com
dekoration.us detoxmed24.ru
design-ideas.info detskie-konstruktory.ru
deutschland123.de dev78.dioniqlabb.se
devochki.top diamond-necklace.info
dignitasdata.se diesel-parts28.ru
dominateforex.ml diffbot.com
domznaniy.ru dildofotzen.net
doxyporno.com directivepub.com
divatraffic.com
diy-handmade-ideas.com
docstoc.com
dokfilms.net
dokumentalkino.net
domainsatcost.com
donna7753191.ru
dotmass.top
dotnetdotcom.org
doxysexy.com doxysexy.com
dumpsmania24.com drpornogratisx.xxx
e-commerce-seo1.com eager-nash.188-93-233-196.plesk.page
ecommerce-seo.org ekn-art.se
editmedios.com el-nation.com
educatemetv.com electrik-avenue.com
elektronischezigarettekaufen1.myblog.de elektroniksigaraankara.info
elite-sex-finders.com eng-lyrics.com
enews.tech enpolis.ru
englate.com epngo.bz
envaseslotusama.com eropho.net
ero-advertising.com esfchat.tk
etur.ru evidencecleanergold.com
eu-cookie-law.blogspot.com extener.org
evehemming.blogspot.com.au ezigarettenkaufen1.pagina.gr
f012.de facecup.top
familyholiday.ml fanoboi.com
fashion-stickers.ru femmesdenudees.com
favornews.com fickblock18.com
fealq.com fikasound.tk
fetlifeblog.com financetip.eu
filesclub.net findclan.org
filmania-x.ru
filunika.com.ru
fishingwholesale.us
flagcounter.me
flashahead.info
flex4launch.ru
flowwwers.com flowwwers.com
fodelsedagspresenter.nu
forex.osobye.ru forex.osobye.ru
fortunejack.com freegamesplay.online
forumprofi.de freelifetimefuckbook.com
frameimage.org freenode.info
free-social-buttons.xyz freesitetest.com
freetangodownload.com freevpn.space
front.to fuck-paid-share-buttons.xyz
fxund.us fugarif.ga
game300.ru g7m.pl
garciniaxt.us gaming-journal.com
gate5.co.za genericviagrasildenafiled.net
gdzkurokam.ru geoads.com
get-your-social-buttons.info ggiaro.com
getlaid-xxxhookupdirect.com gidonline.one
gnuetella.com girls-ufa.ru
girlsfuckdick.com
go2mike.ru go2mike.ru
gobongo.info goblacked.com
gotwebsite1.com gonextmedia.com
graphics8.info goodhumor24.com
gratis-sexkontakte.com goodprotein.ru
greamimgo.pw goodwinmetals.co
greenidesign.co googleseo.com.tr
grizzlysgrill.com gox.com.ua
guge.io greendream.com.ua
guidefs.ru happy.new.yeartwit.com
guigyverpo.cf hasshe.com
hackers-crackers.tk haveinc.xyz
hao123.com hayate.biz
hauleddes.com hchha.com
hdhc.ru hdimagegallery.net
hdseriale.pl hdwallpapers-free.com
headpharmacy.com headpharmacy.com
healbio.ru
healmytrauma.info
health-medical-portal.info health-medical-portal.info
heartofbeijing.blogspot.com healthcarestore.info
help.tpu.ru help.tpu.ru
helvetia.com.ua hermesreplica.pw
hildinghr.se hikesearch.net
hmmm.cz hitcpm.com
homegardenlova.com homik.pw
homemature.net hotchatdate.com
host-tracker.com
hosting-tracker.com
hostingclub.lk
houdom.net
houseofgaga.ru houseofgaga.ru
houseofrose.com howmuchdoestizanidinecost.blogspot.com
hoverboard360.es htmlcorner.com
igrovyeavtomaty777.ru hulfingtonpost.com
iideaidekonkatu.info hully.altervista.org
iccornacircri.cf
ideashome.id
ilbe.club ilbe.club
ilikevitaly.com ilikevitaly.com
imfamous.info imadedinner.net
immigrational.info imgarcade.com
in-tandem.co imgarit.pw
infoupdate.org iminent.com
instasexyblog.com insta-add.pro
invest-pamm.ru ipchicken.com
iptool.xyz iqbazar.ru
istock-mebel.ru iron-age.info
jerseywholesalechinabiz.com iscblog.info
jjbabskoe.ru ixora.pro
jumptap.com jav-idol.com
justprofit.xyz jav-way.site
kakablog.net javrip.net
kalb.ru jerseywholesalebizchina.com
karachev-city.ru joyceblog.top
karaganda.xkaz.org jrcigars.com
kazrent.com jyrxd.com
keenoutlet.online keywordsuggests.com
kerwinandcariza.com kfon.eu
ketrzyn.pl kichenaid.ru
keywordhut.com kino-filmi.com
keywordsuggest.org kletkimehan.ru
kinogonew.ru
kit-opt.ru
kiwi237au.tk
kleine-titten.biz
klejonka.info
klitimg.pw klitimg.pw
konturkrasoty.ru
koopilka.com
krafte.ru
krasivoe-hd.net krasivoe-hd.net
krasivye-devushki.net krasnodar.xrus.org
krasnodar.ru kredit-pod-zalog-krasnodar.ru
krynica.info krynica.info
labplus.ru kunstaktien.info
kurwa.win
kutikomi.net
labvis.host.sk labvis.host.sk
lanadelreyfans.us lacapilla.info
landscaping.center lacasamorett.com
lapitec.eu lacloop.info
lashstudia.ru languagecode.com
lemon-ade.site lawyers.cafe
lexaprogeneric.link le-clos-des-alouettes.com
libertybilisim.com legionalpha.com
lescinq.com
life.biz.ua
linkdebrideur.xyz
linkredirect.biz
live-sexcam.tk live-sexcam.tk
lizace.com luckyday.world
lkbennettstore.online lyngdalhudterapi.no
locksmith.jp lyrster.com
lowephotos.info macfix.life
machicon-ueno.info
macresource.co.uk
macrotek.ru macrotek.ru
magazintiande.ru malwareremovalcenter.com
makenahartlin.com master-muznachas.ru
manve.info masterseek.com
mararoom.ru
marcoislandvacations.net
markjaybeefractal.com
masthopehomes.com masthopehomes.com
matematikus.info mbiologi.ru
matrixalchemy.com mcadamssupplyco.com
maxxtor.eu mebel-alait.ru
medicinacom.ru mebel-ekb.com
medizinreisen.de mediafresh.online
medkritika.ru megagrabber.ru
megahdporno.net
members.ghanaweb.com
menstennisforums.com menstennisforums.com
merryhouse.co.uk metarip.ru
metascephe.com
michaelkorsoutletstore.net michaelkorsoutletstore.net
mielec.pl mindbox.co.za
minecraft-rus.org minecraft-neo.ru
mirtorrent.net minegam.com
mirzonru.net misslike.ru
mobile.ok.ru mobifunapp.weebly.com
monitorwebsitespeed.com modenamebel.ru
mossmesi.com moi-glazki.ru
mostantikor.ru mojpreskumanie.com
mouselink.co monclercheap.xyz
monclercoats.xyz
moncleronline.xyz
moonci.ru
morlat.altervista.org
moroccosurfadventures.com
moscow.xrus.org
moviezbonkerssk.cf moviezbonkerssk.cf
mozello.ru mrvideospornogratis.xxx
mrsdalloways.com mtmtv.info
mrwhite.biz musflashtv.com
muizre.ru
muschisexbilder.com
muz-baza.net muz-baza.net
mycouponizemac.com muzaporn.com
myindospace.com my-aladin.com
mykings.pw mydownlodablefiles.com
mywallpaper.top myhydros.org
narkologiya-peterburg.ru mypornfree.ru
natali-forex.com nalogovyykodeks.ru
nbsproject.ru narkologiya-orel.ru
net-profits.xyz naturalbreakthroughsresearch.com
news-speaker.com naughtyconnect.com
netallergy.ru
netfacet.net
new7ob.com
newstaffadsshop.club newstaffadsshop.club
nicovideo.jp nextbackgroundcheck.gq
nmrk.ru nhl09.ru
no-fuel.org ningessaybe.me
nofreezingmac.click nonews.co
notasprensa.info nsatc.net
nullrefer.com office-windows.ru
nvformula.ru offtime.ru
obesidadealgarve.com okonich.com.ua
obiavo.com oneclickfiles.com
ofanda.com online-x.ru
offtopic.biz ooomeru.ru
ohmyrings.com openstat.com
okayimage.com oranga.host.sk
okeinfo.online orgasmus-virtual.com
onblastblog.online
online-podarki.com
online-templatestore.com
online7777.com
orenburg-gsm.ru
outclicks.net outclicks.net
owohho.com pacificair.com
oynat.info palocco.it
oz-offers.com pc-services.ru
palvira.com.ua pcimforum.com
partner-pop.men
patol01.pw
paydayonlinecom.com
pdns.cz pdns.cz
perso.wanadoo.es pdns.download
peugeot-club.org peekyou.com
picanalyzer.data-ox.com petrovka-online.com
pijoto.net phpdevops.com
piski.top pic2fly.com
piuminiita.com pojdelo.weebly.com
platesauto.com pons-presse.com
ploenjitmedia.azurewebsites.net popunder.net
pokemon-go-play.online porndl.org
porno-chaman.info porngalleries.top
pornodojd.ru pornotubexxx.name
pornogad.com profit-opportunity.com
pornokorol.com profitkode.com
pornonik.com promalp-universal.ru
pornoslive.net prostitutki-tyumeni.org
pornpost.in prpops.com
portside.xyz purplestats.com
predmety.in.ua push-ad.com
predominant-invent.tk pwwysydh.com
pretty-mart.com qcstrtvt.bloger.index.hr
preventheadacheguide.info questionmarque.ch
priora-2.com randki-sex.com
privetsochi.ru ranksignals.com
prize44.com rasteniya-vs-zombi.ru
prohoster.info rcpmda.ikan1080.xyz
prospekt-st.ru realitykings.com
prostitutki-kharkova.org refads.pro
prostitutki-rostova.org remmling.de
pskcijdc.bloger.index.hr restaurantlescampi.com
purchasepillsnorx.com retailwith.com
ragecash.com rome2rio.com
regdefense.com rutor.vip
remont-ustanovka-tehniki.ru s.lollypopgaming.com
remontbiz.ru saletool.ru
renewablewealth.com samo-soznanie.ru
rep-am.com sarafangel.ru
resultshub-a.akamaihd.net sbricur.com
reversing.cc scanner-donald.top
ricorsogiustizia.org scanner-fred.top
rn-to-bsn.com scanner-marwin.top
rosbalt.com.ua se-welding.ru
rubanners.com se.bnt-team.com
russkoe-zdorovie.ru seansonline24.pl
s-iwantyou.com search-error.com
sajatvelemeny.com search-goo.com
sammlungfotos.online searchengineranker.email
sampleletters.net searchimpression.com
sanyuprojects.com seccioncontrabajo.com
sayyoethe.blogspot.co.za semp.net
scanner-alex.top seo-platform.com
scmor.ilxc.cc seo-smm.kz
scottbywater.com seoboxes.com
sdsjweb.com sexreliz.com
search.1and1.com shakhtar-doneck.ru
searchinterneat-a.akamaihd.net shisha-swag.de
service-core.ru
sex-dating.co
sexcamamateurchat.com
shohanb.com
shopperifymac.com shopperifymac.com
sibdevice.ru shoppingjequiti.com.br
sicfor.bcu.cc sideeffectsoftizanidine.blogspot.com
sinestesia.host.sk simple-share-buttons.com
sirpornogratis.xxx site-speed-check.site
sizeplus.work site5.com
slonechka.ru skachat-besplatno-obrazcy.ru
sloopyjoes.com smartshoppymac.com
smartbalanceworld.com snworks.com
snjatie-geroinovoy-lomki.ru socialmadesimple.com
soft-program.com softomix.com
solinf.co solinf.co
spidtest.space spravka-medosmotr.ru
spl63.fr
spyfu.com spyfu.com
sstroy44.ru spywarenuker.com
stathat.com
steamoff.net
steelmaster.lv steelmaster.lv
stockspmb.info stroicol.net
streetfooduncovered.com
stroyplus.ru stroyplus.ru
stuff-about-money.com sudexpert66.ru
sukarame.net sureone.pro
sukirgenk.dvrlists.com svetlotorg.ru
svarkagid.com sweepstakes.rewardit.com
swinon.site szamponrevita.pl
taaaak.com
tabakur77.com tabakur77.com
takeprofitsystem.com tacbelarus.ru
tastyfoodideas.com taqplayer.info
td-33.ru tdsing.ru
tds-advert005.info teastory.co
technopellet.gr template-kid.com
telefonsex-ohne0900.net
telsis.com
templates.franklinfire.co templates.franklinfire.co
tetracsaudi.com thefds.net
textads.men thepornsex.org
thetoiletpaper.com timhost.ru
tiens2010.ru tivolibasket.it
timer4web.com tizanidine4mgstreetprice.blogspot.com
tizanidinecapsules.blogspot.com tizanidine6mg.blogspot.com
tizanidinehcl4mgtabinfo.blogspot.com tizanidinecost.blogspot.com
tizanidineinteractions.blogspot.com tizanidineorflexeril.blogspot.com
tizanidinepills.blogspot.com tizanidinestreetvalue.blogspot.com
tizanidinevsvalium.blogspot.com tizanidinewithdrawalsymptoms.blogspot.com
tkanorganizma.ru tnctrx.com
tksn.ru tonerbox.kz
tomck.com top1-seo-service.com
toon-families.com toproadrunner5.info
top250movies.ru traderzplanet.in
torrentdownloadhub.com traffique.net
track.deriv.com trumpetedextremes.com
tracksurf.daooda.com tula.howotorg.ru
trafficjunky.com turizm.bz
trafficnetzwerk.de turvgori.ru
trustedhealthtips.com typer.one
twelvevisionspartyofcolorado.com tytoona.com
u-cheats.ru ukkala.xyz
ua.tc unimodemhalfduplefw.pen.io
udayavani.com uogonline.com
uhod-za-sobakoj.ru uprour.com
ukkelberg.no urlopener.com
uni.me usadacha.net
unitygame3d.com valuado.com
upstore.me varikoz24.com
urlopener.blogspot.com.au vbikse.com
usacasino.com vedomstvo.net
vadimkravtcov.ru vegascosmetics.ru
vapsy.com veopornogratis.xxx
vbtracker.net verymes.xyz
vchulkax.com videtubs.pl
veles.shop vilingstore.net
video-camer.com
videogamesecrets.com
viktoria-center.ru
vip-parfumeria.ru
viteonlusarezzo.it viteonlusarezzo.it
vkgaleria.com vk-mus.ru
vodkoved.ru vkak.ru
voluumtrk.com vkmusics.ru
vksaver-all.ru
vksex.ru
vod.com.ua
vozbujdenie.com
vseigry.fun
vsesubwaysurfers.com vsesubwaysurfers.com
wallpapersdesk.info vvon.co.uk
wallpaperstock.net wait3sec.org
waterefficiency.co web-betting.ru
weblo.com weblo.com
webshoppermac.com webnode.me
welck.octopis.com website-speed-up.site
wemarketing.se website-stealer.nufaq.com
whatsappdownload10.com weprik.ru
wholesalecheapjerseysfree.com westum.se
wholesalejerseychinashop.com wholesalejerseychinaoutlet.com
wholinkstome.com windowssearch-exp.com
wineitudes.wordpress.com winner7777.net
word-vorlagen.xyz witclub.info
woweb.com.ua wordpress-crew.net
www.bookmaker-bets.com workle.website
www.ehscloud.cn worst-sites.online
www.labves.ru www.kabbalah-red-bracelets.com
x-diesel.info www.souvenirua.com
xblog.in x-mix.info
xmladserver.com x-porno.video
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai xdoza.com
xn--80aeahghtf8ac5i.xn--p1ai xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
xsion.net xn----ctbigni3aj4h.xn--p1ai
xvideospornoru.com xn--b1addnj3cah.xn--p1ai
yellowads.men xn--oogle-wmc.com
yourdesires.ru yellocloud.be
zaapplesales.blogspot.com ynymnwbm.bloger.index.hr
zakon-ob-obrazovanii.ru yogamatsexpert.com
zaloadi.ru yoopsie.com
zaloro.com yopoint.in
zapiszto.pl yoshkarola.zrus.org
zastenchivosti.net youbloodyripper.com
zeroredirect12.com youjizz.com
zetmaster.ru ypmuseum.ru
yuarra.pluto.ro
zakonobosago.ru
zarabotat-na-sajte.ru
zastroyka.org
zeroredirect6.com
zeroredirect7.com
zhongwenlink.com zhongwenlink.com
zigarettenonlinekaufen2.drupalgardens.com zigarettenonl.canalblog.com
zkjovpdgxivg.ga zigarettenonlinekaufen1.bloog.pl
znaniyapolza.ru zirondelli.it
zoogdiesney.com zmoda.hostreo.com
zootravel.com znakomstva-piter78.ru
zreloeporno.tv zooggames.com
zvezdagedon.ru zrelaya.pw

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1-99seo.com 00webcams.com
2345.com 101raccoon.ru
2itech.ru 34.gs
7xc4n.com
a96527gi.beget.tech
aaa38852e886ac4af1a3cff9b47cab6272.com aaa38852e886ac4af1a3cff9b47cab6272.com
absolutelyanalog.com abtasty.com
actulite.com acc.eu.org
adserver-e7.com adanih.com
adsloads.com adimmix.com
adult-shop.com.ua adtiger.tk
adultfriendfinder.com advertex.info
adultnet.in afmuseum.com
aero2.ru allknow.info
alert-fjg.xyz allkrim.com
allpdfmags.net alt-servis.ru
alltheviews.com am-se.com
alphacarolinas.com amt-k.ru
ameblo.jp apiadanaknet-a.akamaihd.net
apifasterlightin-a.akamaihd.net asmxsatadriverin.aircus.com
apple.com-webbrowsing-security.review autogrep.ru
asseenontvonline.ru autotop.com.ua
audiobangout.com avtocredit-legko.ru
ayeartoforget.com ayerbo.xhost.ro
azinoofficial777.ru azazaporn.com
baksman.com basedecor.ru
barnfurnituremart.com bashtime.ru
bcmp.org basisches-wasser.net
beachtoday.ru bavsac.com
becuo.com bestcoin.cc
bigames.online bestdraws.com
biketank.ga bet365.com
bimatoprost-careprost.com.ua bimatoprost-careprost.com.ua
bitcoins-live.ru blogig.org
bizru.info bluerobot.info
break-the-chains.com bmwhighperformers.com
bumskontakte.org boazpower.com
buqyxa.rincian.info bradleylive.xyz
business-suggested.tk btvn.ru
buyclomidonlaine.com buehne-fuer-menschenrechte.de
buydissertation.net burger-imperia.com
camdenmemorials.com buyparajumpers.online
carson.getenjoyment.net candycrushshop.com
casinos4dummies.co.uk cartierreplica.top
certifywebsite.win chcu.net
cfsrating.sonicwall.com cialisovercounteratwalmartusa.com
chatroulette.life coinsspb.com
cheapjerseysfootballshop.com compliance-alex.xyz
chelyabinsk.xrus.org compliance-olga.top
chocoslim.pro cookielawblog.wordpress.com
ci.ua coolgamechannel.com
cindymatches.com cosmetic.donna7753191.ru
clean-virus-mac.com cottageofgrace.com
clickintext.net decoratinghomes.ga
clothesforcash.com deda-moroza-zakaz.ru
cocolyze.com design-lands.ru
copyrightclaims.org diesel-parts28.ru
cougfan.info diffbot.com
countbertwistdisp26.soup.io dikx.gdn
customergrowthsystems.com doctissimo.fr
cypernhuset.se dollartree.info
cyse.tk domik-derevne.ru
czat.wp.pl dostavka-v-ukrainu.ru
dekoration.us dotnetdotcom.org
dekorkeramik.ru drugs-no-rx.info
digital-sale.su dstroy.su
domaincheck.io ecxtracking.com
dood.live editors.choice6912650.hulfingtonpost.com
doxyporno.com eldiariodeguadalajara.com
dscaas.website elektrischezigarette2.devhub.com
earl-brown.info europeanwatches.ru
earthmagic.info event-tracking.com
easyshoppermac.com evidencecleanergold.com
easyukraine.com ez8motelseaworldsandiego.com
ecommerce-seo.org ezigarettekaufen.myblog.de
elektronischezigarette2.mex.tl fashionavenuegame.com
emediate.eu fearcrow.com
enternet.ee filesdatabase.net
esopini.com floridahuntingfishingadventures.com
extlabs.io floridamhca.org
extrabot.com fotoxxxru.com
eyemagination.com foxinsocks.ru
ezigarettekaufen2.blox.pl franch.info
farmprofi.net frustrated-favorable.gq
fbdownloader.com fugarif.ga
fetroshok.ru funtoonez.com
findclan.org gallery.rennlist.com
futbolkisales.ru geckoandfly.com
gameplexcity.com gogalleryawesome.com
gay-site.store gojiberriess.apishops.ru
gemara.com greatfind-a.akamaihd.net
gfaq.ru gunsvicceadadebt.tk
goldenggames.com harmonyglen.us
google-liar.ru healmytrauma.info
gribokstop.com hikesearch.net
gsasearchengineranker.pw hornymatches.com
habermetre.com hostingclub.lk
handicapvantoday.com house.sieraddns.com
hasfun.com humanorightswatch.org
hauleddes.com iblogpress.xyz
hawaiisurf.com ico.re
hdapp1008-a.akamaihd.net
hermesreplica.pw
highstairs-a.akamaihd.net
hmmm.cz
housekuba.org
hully.altervista.org
husky-shop.cz
idealtits.net idealtits.net
imgarcade.com igadgetsworld.com
inboxdollars.com ilovevitaly.com
infektsii.com intim-uslugi.info
innoslicon.com irkutsk.online-podarki.com
izismile.com jackpotchances.com
jagg.info jav-fetish.site
kino-rating.ru jerseywholesaleelitestore.com
kleine-titten.biz kaac.ru
klin3952.ru kaliningrad.zrus.org
komputernaya-pomosh-moscow.ru kino2018.cc
kovrenok.ru knowyournextmove.com
kurwa.win krasnodar.ru
krasnodar.zrus.org
lancheck.net lancheck.net
laptoper.net laparfumotec.com
lawyers.cafe letto.by
letmacworkfaster.world libertybilisim.com
linkbux.ru liquimondo.com
littlecity.ch loadingpages.me
livetsomudvekslingstudent.bloggersdelight.dk lordzfilmz.me
locimge.pw louisvuittonoutletstore.net
lolnex.us marcowebonyodziezowe.pl
lyngdalhudterapi.no massage-info.nl
m5home.ru meendo-free-traffic.ga
macotool.com moesen-ficken.com
mactechinfo.info moesexy.com
mainhp.net moinozhki.com
matpre.top mole.pluto.ro
mediafresh.online money-for-placing-articles.com
medicines-choice.com morepoweronmac.trade
medicineseasybuy.com mossmesi.com
mere.host.sk mrmoneymustache.com
mezaruk.info mydoctorok.ru
minneapoliscopiers.com myhealthcare.com
mir-betting.ru myplaycity.com
mir-limuzinov.ru narutonaruto.ru
mixx.com nastroyke.net
mockupui.com nevansk.ru
modabutik.ru nl.netlog.com
moncleroutletonline.pw obiavo.kz
more-letom.ru oksrv.com
musicpro.monster oxotl.com
mustwineblog.com p-business.ru
mypets.by painting-planet.com
myphotopipe.com panchro.co.uk
new-post.tk pc4download.co
office2web.com pdns.download
oranga.host.sk pelfind.me
orion-code-access.net pestomou.info
osagonline.ru picsfair.com
palocco.it picture-group.com
paparazzistudios.com.au piratecams.com
pcimforum.com pistonclasico.com
perosan.com
pix-hd.com pix-hd.com
pizza-tycoon.com pl-vouchers.com
plaff-go.ru platinumdeals.gr
playtap.us plusnetwork.com
pliks.pl pokemongooo.ml
popmarker.com polska-poezja.com
poquoson.org popcash.net
porno-dojki.net porno-home365.com
pro-tec.kz pornogad.com
pronekut.com pornohub.me
puserving.com pozdravleniya-c.ru
quickbuck.com preconnubial.usuby.site
qwarckoine.com prizesbook.online
ratemodels.net professionaldieselcare.com
redhotfreebies.co.uk profolan.pl
reelheroes.net prombudpostach.com.ua
relax.ru pyrodesigns.com.au
robotixix.com rangoman.date
rozalli.com razorweb-a.akamaihd.net
rvtv.ru rentehno.ru
saugatuck.com resellerclub.com
sbt-aqua.ru responsive-test.net
school-diplomat.ru rewardpoll.com
scmor.ilxc.cc rialp.getenjoyment.net
screpy.com rimedia.org
seo-2-0.com runtnc.net
seocheckupx.com ryetaw.com
servingnotice.com sape.top
sexs-foto.top savingsslider-a.akamaihd.net
shareyards.com scanner-barak.top
shemale-sex.net searchencrypt.com
shitting.pro semxiu.com
sideeffectsoftizanidine.blogspot.com seolab.top
sigmund-freud.co.uk sexad.net
silvermature.net sfj-ror.no
sitechecker.pro shemalegalls.blogporn.in
skincrate.net shohanb.com
smartbalanceworld.com slavkokacunko.de
social-vestnik.ru slowmac.tech
soft-program.com slutloadlive.com
songoo.wz.cz smartpet.ru
spbchampionat.ru social-button.xyz
statistici.ro socialmadesimple.com
stevemonsen.com softomix.com
supernew.org solnplast.ru
svetodiodoff.ru spb.afora.ru
svtrd.com stmassage.ru
stretchingabuckblog.com
stroy-portal22.ru
tam-gde-more.ru tam-gde-more.ru
tamada69.com
tanieaukcje.com.pl tanieaukcje.com.pl
teesdaleflyballclub.co.uk the-usa-games.blogspot.com
teguh.info theguardlan.com
tjkckpytpnje.com timer4web.com
tobeyouday.win tizanidine4mg.blogspot.com
todohr.com tizanidineandcipro.blogspot.com
toolsky.com tonivedu.it
top10-online-games.com totu.info
topmira.com trafficgenius.xyz
torontoplumbinggroup.com trafficmonetize.org
track2.shop traffpartners.com
tsan.net turizm.bz
tuberkulezanet.ru unmaroll.ya.ru
twincitiescarservice.com uptimechecker.com
ukrobstep.com urlopener.blogspot.com.au
ukrtvir.com.ua validdomain.xyz
unpredictable.ga vasileostrovsky-rayon.ru
uroki.net vchulkah.net
uvozdeckych.info vclicks.net
venerologiya.com vkontarkte.com
video-camer.com vremya.eu
video-chat.cn vroze.com
vkontaktemusic.ru w3javascript.com
vksex.ru webextract.profound.net
vse-pesni.com webjam.com
vvon.co.uk websiteexplorer.info
vykup-avto-krasnodar.ru wedding-salon.net
webs.com wfb.hatedriveapart.com
wejdz-tu.pl whatsappbot.flyland.ru
wellcome2slovenia.ru wjgony.com
wordkeyhelper.com wladimirpayen.com
wosik-dach.service-for-web.de word-vorlagen.net
wygraj-skiny.win xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xmladserver.com xn--oogle-wmc.com
xn--1-8sbcpb0bdm8k6a.xn--p1ai yellowstonesafaritours.com
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai yoursearch.me
xn--b1ag5cfn.xn--p1ai zebramart.ru
xnxx699.com zirondelli.it
xzlive.com znakom.sibtest.ru
zaimhelp.ru zoodrawings.com
zkjovpdgxivg.ga zophim.me
zygophyceous.womanstars.site zryydi.com
zynax.ua
zytpirwai.net

12
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -1,11 +1,21 @@
<!DOCTYPE html>
<html> <html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
ml><html> <html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.226 0; 10.1.0.88 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3498 ### Version: V4.2022.11.3499
### Updated: Mon Nov 28 22:01:15 UTC 2022 ### Updated: Tue Nov 29 10:03:20 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3498 ### Version: V4.2022.11.3499
### Updated: Mon Nov 28 22:01:15 UTC 2022 ### Updated: Tue Nov 29 10:03:20 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##