vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 06:15:06 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.06.3200 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-06-30 22:07:11 +00:00
parent 99b1f29a95
commit a3b2e019f5
47 changed files with 104715 additions and 104708 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.06.3199 #### Version: V4.2022.06.3200
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.06.3199 #### Version: V4.2022.06.3200
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.06.3199 #### Version: V4.2022.06.3200
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12948
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.06.3199 #### Version: V4.2022.06.3200
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12952
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12952
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12952
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12952
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

12952
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

12952
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

12952
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3199 3200

12952
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12952
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.10.1.48 0; 20.127.78.107 0;

12952
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

12952
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

12952
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

12952
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.10.1.48 0; 20.127.78.107 0;

12952
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.10.1.48 0; 20.127.78.107 0;

12952
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 1; 10.1.1.31 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Asterias Barkrowler
Blow Devil
CSHttp Mister PiX
CopyRightCheck PageAnalyzer
LWP::Simple SEOstats
Microsoft Data Access Screaming
Name Intelligence Telesoft
NimbleCrawler WebStripper
Seomoz lwp-request
Wonderbot webgains-bot

318
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 01h4x.com
AIBOT 360Spider
ALittle Client 404enemy
Acunetix 80legs
AfD-Verbotsverfahren Aboundexbot
AhrefsBot
AiHitBot
Aipbot
AllSubmitter AllSubmitter
Anarchie Anarchie
Anarchy99
Anthill Anthill
Apexoo Aspiegel
Asterias
Attach
AwarioSmartBot AwarioSmartBot
BDCbot BBBike
BDFetch
BackStreet
BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler BatchFTP
Bandit
Battleztar Bazinga Battleztar Bazinga
Bitacle
BlackWidow BlackWidow
Blackboard
Blow Blow
BlowFish BotALot
Brandwatch Brandwatch
BunnySlippers Buck
CATExplorador CATExplorador
CSHttp CSHttp
Calculon Cegbfeieh
CensysInspect
CheTeam CheTeam
CheeseBot CheeseBot
CherryPicker CherryPicker
Citoid ChinaClaw
Cloud mapping Cloud mapping
Copier Cocolyzebot
Collector
Copyscape
Craftbot Craftbot
CrazyWebCrawler CrazyWebCrawler
CrunchBot Custo
Curious DBLBot
Deusu DTS Agent
Digincore Demon
Devil
DigitalPebble
Dirbuster Dirbuster
Discoverybot Disco
DittoSpyder DittoSpyder
DnyzBot DnyzBot
DomCopBot DomainAppender
DomainSigmaCrawler DomainSigmaCrawler
Dotbot
Dragonfly Dragonfly
Drip EMail Siphon
ECCP/1.0 EasyDL
EMail Wolf
Ebingbong Ebingbong
Exabot Evil
Express WebPictures Extractor
ExtLinksBot EyeNetIE
Fimap Ezooms
FemtosearchBot
FlashGet FlashGet
Foobot Flunky
Freeuploader
FrontPage
Fuzz Fuzz
GT::WWW
GalaxyBot
Genieo Genieo
GermCrawler Go!Zilla
Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
GrabNet GoZilla
Grafula Grafula
HMView GridBot
HTMLparser HEADMasterSEO
HTTP::Lite Harvest
Heritrix Hloader
HonoluluBot HonoluluBot
Humanlinks Iblog
IDBTE4M Id-search
IDBot IlseBot
Image Fetch
IndeedBot
InfoNaviRobot InfoNaviRobot
InfoTekies InterGET
Intelliseek Internet Ninja
Iria IstellaBot
Iskanie
JOC Web Spider JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz JennyBot
JetCar JustView
Jetty Jyxobot
JikeSpider Kenjin Spider
Kinza Keybot Translation-Search-Machine
Lanshanbot Larbin
LeechFTP LeechFTP
LeechGet
LibWeb LibWeb
Likse
Linkbot Linkbot
LinqiaMetadataDownloaderBot LinqiaRSSBot
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MTRobot MTRobot
Mag-Net
Magnet Magnet
Majestic SEO
Majestic-SEO Majestic-SEO
MarkMonitor Masscan
Mata Hari Mb2345Browser
MeanPath Bot Metauri
Meanpathbot Mozlila
MegaIndex.ru Msrabot
MicroMessenger
Microsoft URL Control
Mojolicious
MolokaiBot
Morfeus Fucking Scanner
Mr.4x3
NICErsPRO NICErsPRO
NPbot Name Intelligence
Nameprotect Net Vampire
Nessus NetSpider
NetAnts
NetMechanic
Nettrack Nettrack
Nibbler NextGenSearchBot
NimbleCrawler Nikto
Nimbostratus Nimbostratus
Nutch Ninja
Offline Navigator Not
Octopus
OpenLinkProfiler OpenLinkProfiler
OpenVAS Openvas
Openfind OrangeBot
OutfoxBot OutclicksBot
PECL::HTTP
PHPCrawl
Page Analyzer Page Analyzer
PageAnalyzer PageAnalyzer
Pandalytics PageScorer
Panscient Panscient
Papa Foto Papa Foto
PeoplePal PeoplePal
Pi-Monster Piepmatz
Picsearch Pimonster
Pockey PleaseCrawl
ProWebWalker ProPowerBot
Probethenet Probethenet
Psbot Psbot
PxBroker PxBroker
PyCurl QueryN Metasearch
RankFlex Quick-Crawler
RankingBot RankActiveLinkBot
RankingBot2
Rankivabot
Re-re Re-re
ReGet
Reaper
RebelMouse
Recorder Recorder
RedesScrapy RedesScrapy
RepoMonkey
Rogerbot
SBIder SBIder
SEOkicks SEOkicks
SEOlyticsCrawler
SEOprofiler SEOprofiler
Scanbot SEOstats
SalesIntelligent
ScoutJet
Scrapy
Screaming
ScreenerBot ScreenerBot
SearchmetricsBot SearchmetricsBot
Seekport
SemanticJuice
SentiBot SentiBot
SeoSiteCheckup SeoSiteCheckup
Seomoz Shodan
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteSnagger SiteExplorer
Siteimprove SiteLockSpider
SlySearch
SmartDownload
Snake Snake
SocialRankIOBot Snapbot
Snoopy
Sosospider Sosospider
Sottopop
SpankBot
Spbot Spbot
SputnikBot Spinn3r
Sqlmap Sucuri
Sqlworm
Sqworm
SuperHTTP SuperHTTP
Surfbot SurveyBot
Suzuran Swiftbot
Toata T0PHackTeam
T8Abot
Telesoft
Toweyabot Toweyabot
Tracemyfile
Trendiction
Trendictionbot Trendictionbot
True_Robot True_Robot
Turingos Typhoeus
Turnitin URLy Warning
TurnitinBot UnisterBot
URLy.Warning
Upflow Upflow
V-BOT
VB Project
VCI VCI
Vacuum
Vagabondo
VeriCiteCrawler VeriCiteCrawler
VidibleScraper Voltron
VoidEYE
WPScan
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
Wallpapers Wallpapers
WallpapersHD
WeSEE WeSEE
Web Fetch Web Auto
WebAuto Web Collage
Web Enhancer
Web Fuck
Web Pix
WebBandit WebBandit
WebCopier WebEnhancer
WebFuck
WebGo IS
WebImageCollector WebImageCollector
WebLeacher WebPix
WebStripper WebReaper
WebSucker WebSauger
WebWhacker
Webalta Webalta
WebmasterWorldForumBot Website Quester
Webshag WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster Webster
Whack Whatweb
Who.is Bot Who.is Bot
Widow
WinHTTrack
WiseGuys Robot WiseGuys Robot
Woobot Wonderbot
Wotbox
Xaldon_WebSpider
Zade Zade
Zermelo Zermelo
Zeus Zitebot
ZmEu
ZoomBot ZoomBot
ZumBot ZyBorg
backlink-check adscanner
cah.io.community archive.org_bot
arquivo-web-crawler
autoemailspider
check1.exe check1.exe
cognitiveseo com.plumanalytics
crawler.feedback
dataforseo.com
heritrix
instabid instabid
iubenda-radar internetVista monitor
ips-agent
isitwp.com
linkdexbot linkdexbot
lwp-trivial plumanalytics
magpie-crawler probe-image-size
meanpathbot satoristudio.net
netEstate NE Crawler
page scorer
pcBrowser
polaris version
s1z.ru
scalaj-http
seobility seobility
seoscanners seocompany.store
sexsearcher
sogouspider
sp_auditbot
tAkeOut tAkeOut
trendiction.com
trendiction.de trendiction.de
webmeup-crawler ubermetrics-technologies.com
webprosbot webgains-bot
x22Mozilla webpros.com
zgrab

316
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider
404enemy 404enemy
80legs
AIBOT AIBOT
ALittle Client ALittle Client
Acunetix Abonti
Aboundex
AhrefsBot AhrefsBot
AiHitBot AiHitBot
Alexibot
Alligator
AlphaBot
Anarchie Anarchie
Anarchy99 Anarchy99
Ankit
Aspiegel
Atomseobot Atomseobot
Attach AwarioRssBot
AwarioSmartBot
BBBike
BDCbot
BDFetch
BackDoorBot BackDoorBot
BackStreet BackStreet
BackWeb BackWeb
Backlink-Ceck BacklinkCrawler
Bandit Badass
Bigfoot Battleztar Bazinga
BetaBot
Black Hole
BlackWidow
Blow Blow
Brandprotect Boardreader
Brandwatch Bolt
Buck BotALot
Bullseye Bullseye
BuzzSumo
CATExplorador
CCBot
Calculon Calculon
Cegbfeieh Cegbfeieh
CensysInspect CensysInspect
Claritybot Copier
Cliqzbot
Cogentbot
CopyRightCheck CopyRightCheck
Cosmos Cosmos
CrazyWebCrawler Crawling at Home Project
Crescent Curious
DBLBot Custo
DatabaseDriverMysqli DIIbot
Demon DataCha0s
DigitalPebble Dirbuster
Disco Discoverybot
Discobot
Dispatch
DittoSpyder DittoSpyder
DnBCrawler-Analytics
DnyzBot
DomainAppender DomainAppender
DomainSigmaCrawler DomainSigmaCrawler
Dotbot Dotbot
Download Wonder Download Wonder
Dragonfly EMail Siphon
ECCP/1.0
EMail Wolf
EasyDL EasyDL
Ecxi Ebingbong
EroCrawler EirGrabber
Evil Evil
Exabot Exabot
Extractor
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
FemtosearchBot FemtosearchBot
Fimap FlashGet
Foobot Fuzz
Freeuploader GalaxyBot
FrontPage Go!Zilla
Fyrebot
Getintent
Go-Ahead-Got-It
Gotit Gotit
GrabNet GrabNet
Grafula GrapeshotCrawler
GrapeFX
HEADMasterSEO HEADMasterSEO
HTMLparser HTTP::Lite
Haansoft
Harvest Harvest
Havij HonoluluBot
Humanlinks Humanlinks
IDBTE4M IDBot
Iblog Iblog
Id-search IlseBot
IndeedBot Image Fetch
InfoTekies
InternetSeer InternetSeer
IstellaBot JOC Web Spider
JamesBOT JikeSpider
Jbrofuzz JustView
JennyBot Lanshanbot
JetCar
Jetty
Keybot Translation-Search-Machine
LWP::Simple
Larbin Larbin
LeechFTP LeechFTP
LeechGet Lftp
LexiBot
LibWeb LibWeb
LieBaoFast Libwhisker
Linkbot Linkbot
LinkextractorPro LinkextractorPro
LinqiaRSSBot Litemage_walker
LinqiaScrapeBot
Lipperhey Spider
Lmspider
Ltx71 Ltx71
MIDown tool MFC_Tear_Sample
MJ12bot MIIxpc
Mag-Net MSFrontPage
Majestic-SEO MSIECrawler
Magnet
Mail.RU_Bot
Majestic SEO
Majestic12
MarkMonitor
MarkWatch MarkWatch
MauiBot Mass Downloader
Masscan
Mata Hari
MeanPath Bot MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri Metauri
MicroMessenger
Microsoft Data Access Microsoft Data Access
Microsoft URL Control
Mister PiX Mister PiX
Mojolicious Mojolicious
MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
NICErsPRO Musobot
Name Intelligence NPbot
Nameprotect
Needle Needle
Nessus Nessus
NetAnts NetAnts
NetLyzer
NetMechanic
NetSpider NetSpider
NextGenSearchBot NetZIP
Netcraft
Nettrack
Nibbler Nibbler
Niki-bot Nimbostratus
Nmap Not
Nuclei Nuclei
Nutch
Octopus Octopus
Offline Navigator Offline Navigator
OpenLinkProfiler
OpenVAS
Openfind Openfind
OrangeBot Openvas
PECL::HTTP OutclicksBot
PageScorer OutfoxBot
POE-Component-Client-HTTP
PageAnalyzer
PageGrabber
Panscient
Papa Foto
Pavuk
PeoplePal
Petalbot Petalbot
Pi-Monster Pimonster
PictureFinder Pixray
PleaseCrawl PleaseCrawl
Psbot ProPowerBot
PxBroker ProWebWalker
RankActive Pump
RankActiveLinkBot Quick-Crawler
Rankivabot RSSingBot
RankurBot RankFlex
RankingBot
Re-re Re-re
RealDownload
RebelMouse RebelMouse
RepoMonkey RepoMonkey
SEOkicks-Robot Ripper
SEOlyticsCrawler RocketCrawler
SISTRIX SEOkicks
ScanAlert SalesIntelligent
Scanbot
ScoutJet
ScrepyBot ScrepyBot
SearchmetricsBot
Seekport Seekport
SemanticJuice
Semrush Semrush
SemrushBot Shodan
SentiBot Siphon
Seomoz Site Sucker
SiteLockSpider SiteExplorer
SiteSnagger SiteSnagger
SiteSucker SiteSucker
Sitebeam Sitebeam
Siteimprove Siteimprove
Snapbot SlySearch
Sosospider Snake
SpaceBison Snoopy
SocialRankIOBot
Spammen Spammen
SpankBot
Spinn3r Spinn3r
Stripper Sqlmap
Surfbot Sqworm
Suzuran Sucker
T0PHackTeam SuperBot
T8Abot SuperHTTP
Swiftbot
Teleport
TeleportPro TeleportPro
Telesphoreo The Intraformant
Thumbor TheNomad
TightTwatBot Titan
Trendictionbot Tracemyfile
Turnitin Turingos
TurnitinBot URLy Warning
TwengaBot UnisterBot
Twice
Upflow
VB Project VB Project
VCI VCI
Vagabondo Vacuum
VelenPublicWebCrawler
Virusdie
Voil
Voltron
WASALive-Bot WASALive-Bot
WEBDAV
WISENutbot
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
Wallpapers
WallpapersHD WallpapersHD
Web Fetch Web Auto
Web Fuck Web Collage
Web Pix Web Enhancer
WebCopier WebAuto
WebBandit
WebEnhancer WebEnhancer
WebGo IS
WebImageCollector WebImageCollector
WebPix WebPix
WebSauger WebStripper
WebSucker WebWhacker
WebZIP WebZIP
WebmasterWorldForumBot Webster
WebsiteExtractor Whacker
WebsiteQuester Who.is Bot
Whack Widow
WiseGuys Robot WinHTTrack
Wonderbot Xaldon_WebSpider
Woobot
Wotbox
Zade Zade
Zermelo
ZmEu
ZoomBot
ZoominfoBot
ZyBorg
archive.org_bot archive.org_bot
arquivo.pt arquivo-web-crawler
autoemailspider
backlink-check
cah.io.community cah.io.community
clark-crawler
com.plumanalytics
crawler.feedback
dataforseo.com dataforseo.com
demandbase-bot
eCatch
evc-batch
gopher
heritrix heritrix
instabid pcBrowser
isitwp.com
iubenda-radar
magpie-crawler
meanpathbot
mediawords
netEstate NE Crawler
plumanalytics
polaris version polaris version
probe-image-size scan.lol
ripz
satoristudio.net
seobility seobility
seoscanners serpstatbot
sp_auditbot sitechecker.pro
trendiction.com siteripz
trendiction.de
webprosbot webprosbot
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
bvlgaribracelet.xyz adcash.com
dogclothing.store asiavirtualsolutions.com
forexunion.net balla-boo.se
mostantikor.ru filmidivx.com
online-templatestore.com ido3.com
pflexads.com pendelprognos.se
prizeestates.cricket podshipniki-ntn.ru
sbricur.com pozdrawleniya.com
siteheart.net s-forum.biz
torrent-newgames.com salut-camp.ru

926
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1001desert.com 00go.com
1688.com 00webcams.com
1bet.com 1001watch.com.ua
2728fb936f0.com 101billion.com
273-fz.ru 1flag.co.za
2itech.ru 23kw.ru
2x2fan.ru
34.gs
3dracergames.com
4inn.ru 4inn.ru
7fon.ru 4webmasters.com
7makemoneyonline.com 5ws.dating-app.ru
7minuteworkout.com 6128786.com
7yue.org 8kisses.eu
a14download.com aaa38852e886ac4af1a3cff9b47cab6272.com
a1security.com.ua aanapa.ru
abbanreddy.com abbanreddy.com
abcdeg.xyz abusora.com
academiacsmendoza.org acheterviagrafr24.com
adrenalin-stk.ru adamoads.com
ads-cool.pro addbin.men
advokateg.xyz adexprts.com
ahrntal.verymes.xyz adloads.net
alert-fdm.xyz adultwebhosting.info
alkoravto.ru advancedcleaner.com
all4invest.info advancedsoftwaresupport.com
android-style.com afftrack001.com
anlimebel.kiev.ua agysya.ru
apimountainbikei-a.akamaihd.net aibolita.com
apioutoboxnet-a.akamaihd.net aimayangzhi.com
apparel-offer.com airlandsea.info
applyneedy.xyz aksonural.ru
arendakvartir.kz allornamenti.com
aludecor.info
alvtank.se
amazon-adsystem.com
amt-k.ru
anabolics.shop
android-systems.ru
angry-fermi-7633.arukascloud.io
ap.senai.br
apifasterlightin-a.akamaihd.net
apimegabrowsebiz-a.akamaihd.net
apimossnetinfo-a.akamaihd.net
apiqualitinknet-a.akamaihd.net
appfixing.space
architecturebest.com
ardimobileinfo.ml
arquapetrarca.info arquapetrarca.info
arraty.altervista.org artdestshop.eu
asophoto.com arvut.org
asseenontv.ru ascat.porn
atley.eu.pn aussie-prizes.com
atofilms.com autogrep.ru
autoblogger24.info autovideobroadcast.com
autonew.biz aviav.eu
availit.weebly.com avtochehli.by
aversis.net babs.com.ua
azartmsl.com backlinkwatch.com
azbaseimages.net bagcionderlab.com
azbuka-mo.ru baixar-musicas-gratis.com
azinoofficial777.ru baksman.com
bag77.ru banksrf.ru
baikaleminer.com bashtime.ru
bankmib.ru beachtoday.ru
bannerswap.com bear.gotcher.us
besthoro.ru beatifullandscape.co
becuo.com
beetpics.pw
bestbrainprod.win
bestbuy.ca
bestmouthwash.club bestmouthwash.club
bestpriceninja.com betterscooter.com
bestssaker.com bezcmexa.ru
besttorrentknifta.weebly.com bfz.biz
bet-prognoz.com biglistofwebsites.com
biketank.ga bkgr.se
birzha-truda.eu blackcurranthumidifiers.site
bitcoinremote.com blobar.org
blackplanet.com blogporn.in
bloglag.com boazpower.com
boanonihaca.tk bolegapakistan.com
boltalko.xyz bongacams.com
bonux.nextview.ru brandehk.dk
browserprotecter.com brokergid.ru
btnativenav.com buckspar.gq
business-suggested.tk buehne-fuer-menschenrechte.de
cacheimages.com buscarfoto.com
call-of-duty.info buyantiviralwp.com
callawaygolfstore.online buyhoverboard.com
callstevens.com buytizanidineonlinenoprescription.blogspot.com
candypeople.se bzero1jewelry.net
canzoni.ru cah.io.community
cardul.ru calcularpagerank.com
cartierwatch.xyz callawaygolfoutlet.online
casinorewards.info carmuffler.net
cayado.snn.gr cartierlovebracelet.xyz
cdncash.com cartierlovestore.com
centre-indigo.org.ua cartierreplica.win
changde.58.com ccbill.com
cheapwebsitehoster.com cfjump.com
check-host.net cfsrating.sonicwall.com
check-this-out-now.online checktext.org
chidporn.com christianlouboutinoutlet.win
chinajerseyswholesalecoupons.com clickaider.com
choosecuisine.com cmhomestayagency.com
civilwartheater.com co.lumb.co
cl.s7.exct.net coderstate.com
clips.ua.ac.be comeondog.info
clmforexeu.com compliance-checker.info
club-musics.ru compliance-walter.top
com-onlinesupport.host compliance-willy.top
comissionka.net conboy.us
compliance-jane.top copro.pw
counterbot.com counterbot.com
cowblog.fr crest3d.ru
crafthubs.com cupday.com
custom-product-labels.com customergrowthsystems.com
cvety24.by customsua.com.ua
d2jsp.org dalavia.ru
dame-ns.kz danceuniverse.ru
ddpills.com decenttools.com
dealwifi.com delayreferat.ru
detmebel.su dentalpearls.com.au
deutschehobbyhuren.net detskie-konstruktory.ru
dfiles.me devochki.top
diesel-parts28.ru diamond-necklace.info
dikx.gdn divatraffic.com
dipstar.org
disruptingdinnerparties.com
diy-handmade-ideas.com
djekxa.ru djekxa.ru
dneprsvet.com.ua dogbreedspicture.net
dodlive.mil domain.webkeyit.com
donvito.unas.cz downloader12.ru
downloadme.life downtuptv.gq
downvids.net dyshagi.ru
druzhbany.ru easync.io
dtnlyss.com edwinkonijn.com.au
e-collantes.com
e90post.com
editors.choice6912650.hulfingtonpost.com
ekaterinburg.xrus.org
elektrischezi.canalblog.com elektrischezi.canalblog.com
elektrischezigaretten2.beeplog.com elektronischezi.livejournal.com
elektronischezigarettekaufen2.tumblr.com elektrozigarette1.dreamwidth.org
elitedollars.com elitedollars.com
elitepcgames.com elstal.com.pl
elmacho.xyz energydiet-info.ru
epicbrogaming.com enskedesquashclub.se
erotag.com ereko.ru
ero-advertising.com
erotikstories.ru
espaceinventoristes.com
europeanwatches.ru
evehemming.blogspot.com.au evehemming.blogspot.com.au
expertblog.info expertblog.info
extremal-blog.com extads.net
ezigarettenshop2.myblog.de extremepornos.net
facialporntube.com familienzahnaerzte.com
factorynightclub.com
falcoware.com
familyphysician.ru
fanpagerobot.com fanpagerobot.com
fapgon.com ferieboligkbh.dk
farm26.ru fetroshok.ru
fashion-stickers.ru fighrofacciufreesig.ga
fast-torrent.ru financehint.eu
fickenbumsen.net findanysex.com
fickkontaktehobbyhuren.com findpik.com
filmci.pro fitness-video.net
financepoints.eu fiuxy.com
finansov.info flex4launch.ru
firesub.pl foodcrafts.website
flooringinstallation-edmonton.com footbalive.org
floridahuntingfishingadventures.com fordsonmajbor.cf
floridamhca.org forzeronly.com
free-floating-buttons.com free-deals.faith
freeuploader.com free-social-buttons.com
ftns.ru freeseedsonline.com
fxgallery.com freza-sverlo.ru
fyxabomiw.ru fullfileaccess.com
gabeshop.ru gagrasector.ru
galeon.com gayxperience.com
gambarkatabaru.com gearcraft.us
generousdeal-a.akamaihd.net gg-arena.ru
get-free-traffic-now.com gilbertbanda.net
get-seo-domain.com girlsfuckdick.com
ggiaro.com
ghernnqr.skyrock.com
gifspics.com
gigapeta.com
glavtral.ru
globalsurfari.com globalsurfari.com
go2jump.org gobongo.info
gogalleryawesome.com googlefeud.com
gosreg.amchs.ru gorabagrata.ru
greatzip.com goroda-vsego-mira.ru
grizzlysgrill.com gotcher.us
gsasearchengineranker.xyz grtyi.com
gsbs.com.ua gsasearchenginerankerdiscount.com
gtopstats.com
guildebzh.info
gwhwpxbw.bloger.index.hr gwhwpxbw.bloger.index.hr
hackers-crackers.tk hackers-crackers.tk
hanwei.us hannasolution.ru
hatdc.org
hatedriveapart.com
hd-filmy.net hd-filmy.net
hdapp1008-a.akamaihd.net hdapp1008-a.akamaihd.net
hdseriale.pl hdfreeporno.net
hdwallpapers-free.com hdxnxxtube.mobi
helpmymacfaster.trade highland-homes.com
helvetia.com.ua highstairs-a.akamaihd.net
hentai-manga.porn himgaws.pw
hidefiles.org hit-men.men
hidemyass.com hitcpm.com
hitsbox.info homeart.space
hkladys.com homedecoguide.info
host-protection.com hoverboard360.es
host-tracker.com hundejo.com
hoverboard360.de hunterboots.online
howopen.ru hyhj.info
humanorightswatch.org igithab.com
husky-shop.cz ilo134uloh.com
hyiphunter.org imediadesk.com
i4track.net imicrovision.com
iblogpress.xyz incanto.in.ua
ideibiznesa2015.ru infodocsportal.com
ifmo.ru instabid.tech
iframe-toloka.com intermesh.net
iideaidekonkatu.info inzn.ru
iminent.com iopeninghours.co.uk
imitex-plus.ru jacago.com
insurple.com javlibrary.cc
int.search.tb.ask.com je7.us
integritylandscapeservices.com jerseychinabizwholesale.us
intellektmedia.at
interferencer.ru
intim-uslugi.info
isoveti.ru
istizanidinelikexanax.blogspot.com
iusstf.org
james13prix.info
jamiembrown.com
janettabridal.com
javatex.co.id
javstock.com
jo24news.com
jolic2.com
jrpmakati.com
kam-dom.ru
kamalsinha.com
kartiniresto.com kartiniresto.com
keywordsdoctor.com kellyonline.xyz
keywordsking.com keywordhut.com
keywordsuggests.com keywords-monitoring-your-success.com
kino-ecran.ru kfon.eu
kinostorm.net kiinomaniak.pl
kmgamex.cf kndxbkdx.bloger.index.hr
kollesa.ru komp-pomosch.ru
konica.kz kskjco.club
krasnodar.zrus.org kupit-adenu.ru
kreativperlen.ch kurbappeal.info
kretpicf.pw
kustanay.kz kustanay.kz
lacave.ntic.fr la-fa.ru
lacloop.info labplus.ru
larchik.net ladov.ru
lennatin.info lancheck.net
levaquin750mg.blogspot.com law-check-two.xyz
lexixxx.com leadn.pl
lesbian.xyz
letsart.ru
lflash.ru lflash.ru
life.biz.ua
lignofix.ua lignofix.ua
likesdesign.com linerdrilling.com
liky.co.ua lionking-1994.blogspot.com
link.ac liran-locks.com
linkbux.ru listiseltemournlan.gq
lirunet.ru livefixer.com
littlesunraiser.com lkbennettoutlet.online
liveinternet.ru loadopia.com
liveu.infoteka.hu lookingglassemb.com
lob.com.ru louisvuittonoutletstore.net
lonely-mature.com lovi-moment.com.ua
louboutinreplica.pw m-google.xyz
luciddiagnostics.in m3gadownload.pl
m292.info manve.info
macdamaged.space
magicplayer-s.acestream.net
manimpotence.com
mapquestz.us
marketingvici.com
marmitaco.cat marmitaco.cat
mathgym.com.au medkletki.ru
mearns-tractors.co.uk mega-bony2017.pl
mecash.ru meteocast.net
medicinacom.ru
medispainstitute.com.au
meetingrainstorm.bid
mega-polis.biz.ua
member-quiz.com
mexicotravelnet.com mexicotravelnet.com
michaelkorsoutletstore.net microsoftportal.net
microsearch.ru minecraft-neo.ru
migente.com mirzonru.net
minecraft-rus.org misandesign.se
minet.club mitrasound.ru
mmofreegames.online mobile-appster.ru
mnogabukaff.net mobilemedia.md
mobplayer.net moetomnenie.com
mojowhois.com moi-glazki.ru
monclercoats.xyz monclerboots.xyz
monetizationking.net moroccosurfadventures.com
money-every-day.com movpod.in
moomi-daeri.com muschisexbilder.com
moscow-region.ru musicdaddy.net
moxo.com muz-shoes.ru
mozello.ru myonlinepayday.co
muabancantho.info mysex21.com
mulberryoutletonlineeu.com mysexpics.ru
mundoaberrante.com najaden.no
mygameplus.ru nalogovyy-kodeks.ru
mymercy.info nbsproject.ru
mywallpaper.top nextbackgroundcheck.gq
mzdish.site ngps1.ru
napalm51.nut.cc nibbler.silktide.com
negociosdasha.com nodding-passion.tk
new-post.tk nordstar.pro
newfilmsonline.ru notebook-pro.ru
nfljerseyscheapbiz.us o-o-6-o-o.com
nhl17coins.exblog.jp o-o-8-o-o.ru
nippon-bearings.ru oballergiya.ru
normalegal.ru obiavo.kz
nuker.com odesproperty.com
o-o-11-o-o.com oecnhs.info
obiavo.su oil-td.ru
offer.wpsecurity.website omoikiri-japan.ru
ok.ru onblastblog.online
okroshki.ru onclickpredictiv.com
online7777.com one.net.in
onlinebay.ru oneminutesite.it
onlinefilmz.net online247.ml
onlyforemont.ru onlinedomains.ru
openfrost.net onlinewritingjobs17.blogspot.ru
openlibrary.org openfrost.com
orenburg-gsm.ru oren-cats.ru
origin-my.ru osagonline.ru
ovirus.ru oz-offers.com
paidonlinesites.com partner-stop.men
papasdelivery.ru partnerline.men
partner-print.men partnerworkroom.men
partnerads.men pay2me.pl
pdfprof.com pc-virus-d0l92j2.pw
perfection-pleasure.ru pdn-4.com
photokitchendesign.com pearlisland.ru
piccdata.com photochki.com
picsfair.com php-market.ru
picturesfrom.com pihl.se
pizza-imperia.com pills24h.com
pkr1hand.com pinkduck.ga
plastgranar.nu pio.polytopesexempt.com
pokemon-go-play.online pixshark.com
pokemongooo.ml ploenjitmedia.azurewebsites.net
popads.net podshipniki-nsk.ru
poppen-nw.net popup-hgd.xyz
pornogad.com porndl.org
pornokorol.com porno-dojki.net
pornoplen.com pornoblood.com
pornosmola.info pornoelita.info
pornoxxx.com.mx pornogratisdiario.com
portadd.men pornoklad.ru
preventheadacheguide.info pornolook.net
prodvigator.ua portal-eu.ru
proekt-mos.ru pozdrawleniya.ru
pronekut.com primedice.com
ptr.ruvds.com prizestohandle.club
puserving.com pro-tec.kz
quickbuck.com progonrumarket.ru
quickcashlimited.com promover.org
quizzitch.net prostitutki-kharkova.org
rada.ru prpops.com
rank-checker.online pussyspace.net
putevka24.ru
pyrodesigns.com.au
qwesa.ru
r-control.ru
ranking2017.ga ranking2017.ga
raschtextil.com.ua rcb101.ru
real-time-analytics.com
realizmobi.com
rebelmouse.com rebelmouse.com
reimageplus.com rednise.com
renecaovilla.online reelheroes.net
rethinkwasteni.info remontvsamare.su
reversing.cc research.ifmo.ru
rewardpoll.com resultshub-a.akamaihd.net
rff-cfal.info risparmiocasa.bz.it
ric.info ros-ctm.ru
rock-cafe.info rubbed.us
rumamba.com rucrypt.com
ruex.org.ua
ruspdd.com ruspdd.com
sale-japan.com russia-today-video.ru
salut-camp.ru rutor.group
satellite.maps.ilovevitaly.com rybalka-opt.ru
satoristudio.net sajatvelemeny.com
saveriopiazza.it sampleletters.net
scanner-donald.top sanjosestartups.com
scanner-george.top sape.top
searchengineranker.email sarafangel.ru
seeresultshub-a.akamaihd.net savetubevideo.com
seminarykansas904.ml sbornik-zakonov.ru
semprofile.com scanner-don.top
seo-prof1.xyz scanner-irvin.top
seonetwizard.com scanner-john.top
seorankinglinks.us scanner-willy.top
searchaddis.com
searchencrypt.com
sel-hoz.com
seoprofiler.com
seorankinglinks.com
seozoom.it
service-core.ru service-core.ru
seryeznie-znakomstva.ru serw.clicksor.com
sexgalleries.top sex-tracker.com
sexy-pings.com sexad.net
sexyebonyteen.com sexkrasivo.net
sharebutton.org sexpornotales.net
shemale-sex.net sexsaoy.com
shisha-swag.de sextracker.be
shitmovs.com sherlock.se
shodanhq.com shodanhq.com
shopsellcardsdumps.com sibvitr.ru
sideeffectsoftizanidine.blogspot.com silverage.ru
sildenafilcitratemed.com silvercash.com
slayerlife.com simon3.ru
slimcdn.com sims-sims.ru
small-game.com simul.co
soda.media sinel.info
softlinesolutions.me sinestesia.host.sk
sovinsteel.ru site-analyzer.com
spasswelt.xyz site-auditor.online
spb.afora.ru sizeplus.work
specialfinanceoffers.com skytraf.xyz
spinazdrav.ru smallseotools.com
sportbetfair.com smartpet.ru
spywarespy.com smichovbike.cz
stackthatbucks.com smsactivator.ru
staff.prairiesouth.ca snowplanes.com
stairliftsarea.com socialbutton.xyz
standardchartered-forex.com socialmonkee.com
stats-collector.org softomix.net
stpicks.com sortthemesitesby.com
sotechco.co
soviet-portal.do.am
spitfiremusic.com
spywarebegone.com
stopnarco.ru
stpolice.com stpolice.com
streamin.to strv.se
streha-metalko.si stuffpride.com
stretchmate.net
strfls.com
sucsesofinspiration.com
sukirgenk.dvrlists.com sukirgenk.dvrlists.com
summerlinhomes411.info superlist.biz
supermama.top
supermodni.com.ua
suture.co
swingerseiten.com
sygraem.com
syndicate.fun
tako3.com
tattooha.com tattooha.com
td-33.ru
tech4master.com tech4master.com
telefonsex-ohne0900.net techart24.com
teletype.in teesdaleflyballclub.co.uk
thedownloadfromwarez.blogspot.com telefonsexi.com
thefds.net
thegalerie.eu thegalerie.eu
thegamerznetwork.com thetattoohut.com
thetoiletpaper.com thewomenlife.com
threecolumnblogger.com tizanidineandhydrocodone.blogspot.com
timhost.ru tizanidineformigraines.blogspot.com
tizanidinedrugtest.blogspot.com tizanidineinfo.blogspot.com
tizanidinepain.blogspot.com tizanidineinteractions.blogspot.com
tizanidinestreetprice.blogspot.com tizanidineoral.blogspot.com
toolsky.com tizanidineorflexeril.blogspot.com
topmira.com tizanidinesleepaid.blogspot.com
torrent-newgames.com tomatis.gospartner.com
toonfamilies.net
tovaroboom.vast.ru tovaroboom.vast.ru
traffictrade.life toy-shop.top
toyota.7zap.com
trafficjunky.net
trafficnetzwerk.de
traffixer.com
traxdom.ru
tri-slona.org tri-slona.org
tubeoffline.com trucri.me
turbo-suslik.org tryrating.com
turizmus.us uggbootsoutletsale.us
turkeyreport.tk ukkala.xyz
twincitiescarservice.com unitexindia.com
twinderbella.com unlimitdocs.net
ublaze.ru uptime-delta.net
unrealcommander.biz uzungil.com
unrealcommander.com vapsy.com
uptime-as.net vchulkax.com
urdoot.win vedomstvo.net
vapomnoncri.tk visionwell.com.cn
vavilone.com vrnelectro.ru
vbtracker.net
vclicks.net
veselokloun.ru
vesnatehno.ru
via-energy-order.com
viagroid.ru
video-chat.love
vigrx-original.ru
villakohlanta.nu
vipromoffers.com
visa-china.ru
visa-pasport.ru
vizitki.net
vksaver-all.ru
vladimir.xrus.org
voltrknc1.com
volume-pills.biz
vtcdns.com vtcdns.com
waterefficiency.co vvpg.ru
welcomeauto.ru vykup-avto-krasnodar.ru
wemedinc.com vzlomsn.org
whiteelephantwellington.com wardreapptokone.tk
wieseversa.no wareseeker.com
wikes.20fr.com warningwar.ru
wineration.com wdfdocando.com
woman-h.ru webix.me
wechatdownload10.com
whereverdesperate.gq
wildnatureimages.com
wordpress-crew.net wordpress-crew.net
wsgames.ru wormix-cheats.ru
xblog.in woweb.com.ua
xdoza.com www.arenda-yeisk.ru
xev.ru x-diesel.org
xmnb.net x-stars.ru
xn----9sbubg3ambdfl1j.xn--p1ai xmlinde.com
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai xn--80atua3d.xn--p1ai
xn--80aikhbrhr.net xn--lifehacer-1rb.com
xvideosbay.com xn--oogle-wmc.com
xuki.us
xxx-cam.webcam xxx-cam.webcam
yhit.press yebocasino.co.za
yopoint.in youtuhe.com
youporn-ru.com yun56.co
zero1.it z-master.ru
zebradudka.com
zeroredirect2.com
zeroredirect6.com
zeroredirect8.com
zigarettenonlinekaufen1.bloog.pl
zone-kev717.info
zootravel.com
zytpirwai.net

488
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
256bit.by 1688.com
3-letter-domains.net 1qingdao.com
7zap.com 2000k.ru
adconscious.com 2pxg8bcf.top
addbin.men 55wmz.ru
adman.gr 9med.net
adobereader-free.ru a.pr-cy.ru
adservme.com aa14ab57a3339c4064bd9ae6fad7495b5f.com
aerogo.com addray.pro
airlandsea.info alice.it
airmaxshoes-2016.com all4wap.ru
aktivator-windows10.blogspot.com ananumous.ru
alert-jdh.xyz apper.de
alpharma.net appleid-verification.com
apiitsthirteende-a.akamaihd.net audiobangout.com
approved.su autolombard-krasnodar.ru
apps-analytics.net autoseo-traffic.com
asrvrep-a.akamaihd.net availit.weebly.com
asrvvv-a.akamaihd.net avitocars.ru
avon-ukraine.com avon-ukraine.com
azadnegar.com avtochehli.by
azinoofficial777.ru azartclub.org
azte.ch azazu.ru
baixar-musicas-gratis.com battleforupdating.site
balois.worldbreak.com best-coupon-offer.com
bestadbid.com bestplacetobuyeriacta.jw.lt
bestpornuha.com bet365.com
bioscorp.ru beyan.host.sk
bizlist.com.de bitcoinremote.com
bkgr.se
blackbabesporn.com blackbabesporn.com
blog-hits.com blpmovies.com
bum.com.ru bonky.biz
byme.se bpro1.top
canrioloadm.gq bumascloset.com
carsdined.org callmd5map.com
carsplate.com chatroulette.world
carstrends2015.com cheapjerseysap.com
cartiertrinity.xyz chiptuninger.com
casinorewards.info clickcash.com
caveavins.fr cmd.kz
cenokos.ru compliance-don.top
centrdebut.ru consorzioilmosaico.org
chatroulette.online continent-e.tv
chatroulette.video crd.clan.su
cheapmoncler.xyz daisye.top
cheapsergiorossi.online dayibiao.com
checkhit.com dear-diary.ru
cienum.fr
cjmc.info
clickbank.net
com-onlinesupport.host
conboy.us
construmac.com.mx
cookie-law-enforcement-ee.xyz
cookingmeat.ru
darmebel.com.ua
dengi-pod-zalog-nedvizhimosti.ru dengi-pod-zalog-nedvizhimosti.ru
destinationrealestate.com dfwu1019.info
diarioaconcagua.com dirtpics.pw
digitalfaq.com div.as
discovertreasurenow.com domainsatcost.com
dnepropetrovsk.mistr-x.org download-walpaperhd.blogspot.com
domcran.net downloadkakaotalk.com
dotnetdotcom.org droidlook.net
e-biznes.info drugs-no-rx.info
e2click.com elektronischezigarette2.mex.tl
eager-nash.188-93-233-196.plesk.page elektrozigaretten2.yn.lt
easyshoppermac.com elitedollars.com
ed-shop01.ru enews.tech
eleimgo.pw engines-usa.com
elektir.ru englishdictionaryfree.com
elektronischezigarettekaufen1.beeplog.com errorfixing.space
elektrozigarettekaufen1.devhub.com essayservicewriting.org
elektrozigaretten1.postbit.com etur.ru
essay-writing.work euronis-free.com
eu-cookie-law.blogspot.com exchanges-bet.com
extremal-blog.com f012.de
eyelike.com.ua faz99.com
fanoboi.com ferrotodo.com
faracontrol.ir fetlifeblog.com
fefo.gdn firesub.pl
fertilitetsradgivningen.se flauntyoursite.com
fickkontaktehobbyhuren.com floridahuntingfishingadventures.com
filmgo.ru freakycheats.com
fitness-video.net free-fb-traffic.com
forum.doctissimo.fr freenom.link
forums.d2jsp.org freeseedsonline.com
fototravel.eu fuel-gas.com
free-today.com fym.com.ru
fugarif.ga gelezki.com
g.starmoe.xyz getlamborghini.ga
galleryawesome.com goldadpremium.com
gamezblox.com goodprotein.ru
gaz-voshod.ru goodwriterssales.com
girlsatgames.ru googst2.ru
gne8.com goroda-vsego-mira.ru
gorodservis.ru gosreg.amchs.ru
hamilton.ca gotwebsite1.com
hao123.com gox.com.ua
hatdc.org greenidesign.co
heatpower.ru gsasearchenginerankerhelp.com
homedo.fabpage.com guidefs.ru
imagecoolpub.com heartofbeijing.blogspot.com
incolors.club hermesreplica.pw
iopeninghours.co.uk hmywwogw.bloger.index.hr
iron-age.info homedecoguide.info
itbc.kiev.ua hotkeys.com
j33x.com hotshoppymac.com
jongose.ninja husky-shop.cz
karting196.ru increasewwwtraffic.info
kinozapas.com infobabki.ru
kol-energo.ru intimshop-fantasy.ru
krasivoe-hd.net islandminingsupply.wordpress.com
landmania.ru itch.io
lazy-z.com je7.us
lenvred.org kidskunst.info
lezbiyanki.net kinozapas.org
life.biz.ua kladrus.ru
localmatchbook.com koleso24.com.ua
louboutinreplica.pw kostenloser-sex.com
lunamedia.co kosynka-games.ru
mabdoola.blogspot.com krafte.ru
mafcards.ru krasivye-devushki.net
master-muznachas.ru laptop-4-less.com
mebel-ekb.com li-er.ru
medicineseasybuy.com live-sexcam.tk
mega-bony2017.pl live-sexchat.ru
megagrabber.ru liveshoppersmac.com
mex-annushka.ru locksmith.jp
mixed-wrestling.ru long-beach-air-conditioning.com
mnogolok.info lostaruban.com
mobplayer.ru lostfilm-online.ru
morepoweronmac.trade lottospring.com
mouselink.co louisvuittonoutletstore.net
mrlmedia.net lsex.xyz
mydownloadengine.com ltvperf.com
myfreemp3.eu luxup.ru
napalm51.nut.cc machicon-ueno.info
netpics.org magnetpress.sk
nfljerseyscheapbiz.us manuscript.su
njkmznnb.ru mapquestz.us
no-fuel.org marcowebonyodziezowe.pl
oakridgemo.com marketingvici.com
oddamzadarmo.eu meble-bogart.info
ok-ua.info microsoftportal.net
onlineporno.site minneapoliscopiers.com
ooo-olni.ru misandesign.se
optibuymac.com mosrif.ru
pic2fly.com mrinsidesales.com
piulatte.cz myonigroup.com
newstraveller.ru
newyorkhotelsmotels.info
njpalletremoval.com
norththeface.store
o-o-11-o-o.com
obnal.org
oil-td.ru
one-gear.com
online-sbank.ru
osnova3.ru
outpersonals.com
p-business.ru
parajumpersstore.online
pcimforum.com
pechikamini.ru
pictures-and-images.net
pinsdaddy.com
pinup-casino1.ru
pizdopletka.club pizdopletka.club
platesauto.com pizza-imperia.com
polybuild.ru plaff-go.ru
pornhive.org play-movie.pl
porno-home365.com poem-paying.gq
porodasobak.net popmarker.com
poshiv-chehol.ru pornonik.com
posible.net prchecker.info
pro-tec.kz pretty-mart.com
pron.pro proekt-gaz.ru
psa48.ru promoheads.com
pssucai.info proxyelite.biz
pulseonclick.com pussysaga.com
putitin.me rankings-analytics.com
registrysweeper.com reining.lovasszovetseg.hu
remmling.de ringporno.com
remont-mobile-phones.ru robotixix.com
responsinator.com roznica.com.ua
rfid-locker.co samara.rosfirm.ru
rialp.getenjoyment.net sanyuprojects.com
rogervivierforsale.com sashagreyblog.ga
rumamba.com sbornik-zakonov.ru
running-line.ru scanhub.ru
samo-soznanie.ru scansafe.net
scanner-jessica.top search.pch.com
sdi-pme.com searchtooknow-a.akamaihd.net
searchimage.co seccioncontrabajo.com
searchwebknow-a.akamaihd.net seolab.top
securityallianceservices.com serviporno.com
semp.net sethrollins.net
semxiu.com sex-znakomstva.online
seo-platform.com shopcheermakeup.info
servingnotice.com shopvilleroyboch.com.ua
seryeznie-znakomstva.ru siteheart.net
sexkontakteao.info sixcooler.de
sexuria.net softtor.com
shtora66.ru sotechco.co
signoredom.com spb.ru
similardeals.net speechfoodie.com
sims-sims.ru studiofaca.com
smadihome.com sukarame.net
sodexo.com
songplanet.ru
sovetogorod.ru
space4updating.win
spidtest.space
sportbetfair.com
spymac.net
spywareit.com
stairliftsarea.com
staynplay.net
superkanpo.com
supermama.top supermama.top
surflinksmedical.com survival.betteroffers.review
svnuppsalaorebro.se taqywu51.soup.io
swiped.su tecnoteakviareggio.it
tamada69.com teenbbw.yopoint.in
thewebsitetemplate.info teencastingporn.com
tizanidineoral.blogspot.com the-usa-games.blogspot.com
tizanidinepain.blogspot.com theplacetoupdating.pw
tizanidinestreetvalue.blogspot.com thin.me.pn
tonerbox.kz time-japan.ru
top10-way.com tizanidineinteractions.blogspot.com
torontoplumbinggroup.com tooplay.com
torrent-to-magnet.com tovaroboom.vast.ru
trainoffend.ml toy-shop.top
tryrating.com treasuretrack-a.akamaihd.net
umekana.ru truck-land.life
uptime.com ukr-lit.com
urccvfmc.bloger.index.hr ultimatesetnewfreeallsoftupgradesystems.pw
urzedowski.eu unisexjewelry.org
vabasa.inwtrade.com ussearche.cf
uzporno.mobi
vertaform.com
video-woman.com
videogamesecrets.com
videooko.weebly.com videooko.weebly.com
visitcambridge.org vielporno.net
viteonlusarezzo.it vipsexfinders.com
vkmusics.ru visa-china.ru
vksex.ru web-analytics.date
volgograd.xrus.org website-datenbank.de
vse-pesni.com websiteexplorer.info
vykupavto-krasnodar.ru westum.se
w3data.co white-truck.life
wardreapptokone.tk winx-play.ru
webradiology.ru wiosenny-bon-1500.pl
weekes.biz.tc www.souvenirua.com
welcomeauto.ru wzgyyq.com
wheelchairliftsarea.com xn----7sbahjd3btneuw1joc.xn--p1ai
wordpresscore.com xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
worldis.me xtraffic.plus
xn----9sbubg3ambdfl1j.xn--p1ai yaminecraft.ru
xn--80aanaardaperhcem4a6i.com yogamatsexpert.com
xn--b1ag5cfn.xn--p1ai yourdesires.ru
xpresscare.ru yur-p.ru
yourmovies.pl zahvat.ru
yubikk.info zarabotat-na-sajte.ru
za-music.mymobiplanet.com zdesformula.ru
znakomstva-moskva77.ru zigarettenonlinekaufen1.bloog.pl
zone-kev717.info zigarettenonlinekaufen2.bloog.pl
zoomovies.org znaturaloriginal.com
zvooq.eu

55
dev-tools/test_units/ratelimittest-master.txt
View file

@ -8,30 +8,6 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -494,6 +470,27 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
@ -584,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.208 0; 10.1.1.31 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.06.3199 ### Version: V4.2022.06.3200
### Updated: Thu Jun 30 10:03:46 UTC 2022 ### Updated: Thu Jun 30 22:01:28 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.06.3199 ### Version: V4.2022.06.3200
### Updated: Thu Jun 30 10:03:46 UTC 2022 ### Updated: Thu Jun 30 22:01:28 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##