vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.10.2679 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-10-28 10:18:19 +00:00
parent ae93a81ec5
commit 690b9a69be
47 changed files with 70625 additions and 70615 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2678 #### Version: V4.2021.10.2679
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2678 #### Version: V4.2021.10.2679
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.10.2678 #### Version: V4.2021.10.2679
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8690
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.10.2678 #### Version: V4.2021.10.2679
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8694
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8694
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8694
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8694
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

8694
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

8694
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

8694
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2678 2679

8694
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8694
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.185.255.187 0; 20.98.99.82 0;

8694
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

8694
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

8694
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

8694
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.185.255.187 0; 20.98.99.82 0;

8694
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.185.255.187 0; 20.98.99.82 0;

8694
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Drip CensysInspect
EirGrabber Internet Ninja
Express WebPictures Keyword Density
Jbrofuzz Needle
JustView NimbleCrawler
Nuclei Offline Navigator
Semrush WebBandit
Swiftbot ZmEu
Vacuum autoemailspider
Zitebot webprosbot

300
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
AIBOT 360Spider
404enemy
80legs
ASPSeek
Aboundex Aboundex
Aboundexbot Aboundexbot
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
Aipbot Aipbot
Alexibot Alligator
AlphaBot
Anarchie Anarchie
Anthill
Apexoo Apexoo
Asterias Attach
AwarioRssBot BBBike
AwarioSmartBot BDCbot
BackDoorBot BackStreet
BackWeb
BacklinkCrawler
Badass Badass
Barkrowler Bandit
BlackWidow BetaBot
Bolt Bigfoot
Brandprotect Blackboard
Buddy BlowFish
BotALot
BuiltBotTough
BuiltWith BuiltWith
CATExplorador Bullseye
CODE87 CODE87
CazoodleBot Calculon
CensysInspect Cegbfeieh
CheTeam
CheeseBot CheeseBot
ChinaClaw ChinaClaw
Cliqzbot Cliqzbot
Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot Cogentbot
Collector Copier
CopyRightCheck CrazyWebCrawler
Copyscape CrunchBot
Crescent
Custo
CyotekWebCopy CyotekWebCopy
DatabaseDriverMysqli DBLBot
DIIbot
DataCha0s
Demon Demon
Deusu
Devil
Digincore Digincore
DigitalPebble
Dirbuster Dirbuster
Disco Disco
Discoverybot
Dispatch Dispatch
DittoSpyder
DnyzBot DnyzBot
DomCopBot
DomainAppender DomainAppender
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project
Dotbot Dotbot
EasyDL Drip
ECCP/1.0
Ebingbong Ebingbong
EirGrabber Ecxi
Evil
Exabot Exabot
ExtractorPro Express WebPictures
Ezooms
FHscan
FemtosearchBot FemtosearchBot
Fimap Fimap
Firefox/7.0
Flunky Flunky
Fyrebot Fyrebot
G-i-g-a-b-o-t
GT::WWW GT::WWW
GalaxyBot
GermCrawler GermCrawler
GetRight
Getintent Getintent
Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
Gotit Gotit
Grabber GrabNet
Grafula HEADMasterSEO
GrapeFX HMView
GridBot
HTMLparser
HTTP::Lite HTTP::Lite
HTTrack Harvest
Havij Havij
Humanlinks Hloader
IDBot IDBot
Iblog
Id-search Id-search
InfoNaviRobot Image Fetch
IndeedBot
Indy Library
Intelliseek Intelliseek
InternetSeer Iskanie
IstellaBot
JOC Web Spider JOC Web Spider
Jbrofuzz JetCar
Jetty Jetty
Joomla Joomla
Jorgee JustView
Jyxobot Jyxobot
Keyword Density Larbin
Lanshanbot Leap
Lftp Libwhisker
LieBaoFast LinkScan
Likse LinksManager
Linkbot
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey
Lipperhey Spider Lipperhey Spider
Litemage_walker
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool
MIIxpc
MJ12bot MJ12bot
MSFrontPage MQQBrowser
MSIECrawler MSIECrawler
Majestic-SEO MTRobot
MarkMonitor MarkWatch
Masscan
Mb2345Browser Mb2345Browser
Microsoft URL Control MeanPath Bot
Mister PiX Metauri
MicroMessenger
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mozlila
NPbot Mr.4x3
NearSite Msrabot
Nessus Musobot
NetMechanic Nameprotect
NetSpider Navroad
Netcraft NetAnts
Nettrack NetLyzer
NetZIP
NextGenSearchBot NextGenSearchBot
Nibbler
Niki-bot
NimbleCrawler NimbleCrawler
Nimbostratus
Ninja Ninja
Nmap Nuclei
Nutch Nutch
Offline Explorer Octopus
OrangeSpider Offline Navigator
OutclicksBot OnCrawl
OutfoxBot OpenLinkProfiler
PECL::HTTP Openvas
OrangeBot
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageScorer PageGrabber
Panscient Panscient
Papa Foto Papa Foto
Pavuk
PeoplePal
Petalbot
Pi-Monster
Picscout Picscout
Picsearch Pimonster
PictureFinder Pixray
PleaseCrawl
Pockey Pockey
Psbot
Pu_iN Pu_iN
PxBroker
PyCurl PyCurl
Quick-Crawler Quick-Crawler
RankActiveLinkBot
RankFlex
RankingBot
Rankivabot Rankivabot
RankurBot
Reaper
RebelMouse
Recorder
Ripper
SEOkicks SEOkicks
SEOkicks-Robot SEOlyticsCrawler
SEOprofiler
SMTBot
SalesIntelligent SalesIntelligent
Scrapy ScanAlert
Screaming SearchmetricsBot
ScrepyBot SemrushBot
Semrush
SentiBot SentiBot
Site Sucker Site Sucker
SiteSucker
Siteimprove
SmartDownload SmartDownload
Snapbot Snake
Sogou web spider SocialRankIOBot
Sosospider
SpaceBison SpaceBison
Spanner Spanner
SputnikBot Spbot
Stripper Steeler
Sucker
SuperBot SuperBot
Surfbot
SurveyBot SurveyBot
Suzuran
Swiftbot Swiftbot
T0PHackTeam
T8Abot
Teleport
TeleportPro TeleportPro
The Intraformant Telesoft
Telesphoreo
TheNomad
Thumbor Thumbor
TightTwatBot
Titan Titan
Toweyabot Toata
Tracemyfile Trendictionbot
Trendiction TwengaBot
True_Robot Twice
Turingos URLy.Warning
Typhoeus Upflow
UnisterBot VB Project
VCI
Vagabondo Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
Virusdie Virusdie
VoidEYE WISENutbot
Voltron WPScan
WBSearchBot
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
WWWOFFLE WWWOFFLE
Wallpapers/3.0 Wallpapers
WallpapersHD
WeSEE
Web Auto Web Auto
Web Collage Web Pix
Web Fuck Web Sucker
Web Sauger WebImageCollector
WebCollage
WebPix WebPix
WebReaper
WebSauger WebSauger
WebWhacker Webalta
WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster Whacker
Whatweb Whatweb
Widow WinHTTrack
Wotbox WiseGuys Robot
YoudaoBot Wonderbot
Zade Xaldon WebSpider
Xaldon_WebSpider
Zauba Zauba
Zeus Zermelo
Zitebot
ZmEu ZmEu
ZoomBot ZyBorg
ZumBot cah.io.community
archive.org_bot clark-crawler
arquivo.pt com.plumanalytics
backlink-check
check1.exe
crawl.sogou.com crawl.sogou.com
crawler4j domainsproject.org
dataforseo.com
eCatch eCatch
evc-batch evc-batch
facebookscraper
heritrix heritrix
instabid
internetVista monitor internetVista monitor
isitwp.com lwp-trivial
magpie-crawler magpie-crawler
netEstate NE Crawler meanpathbot
oBot oBot
page scorer page scorer
pcBrowser plumanalytics
probe-image-size scan.lol
satoristudio.net seostar
scalaj-http sitechecker.pro
siteripz siteripz
sogouspider sp_auditbot
spyfu tAkeOut
webpros.com webmeup-crawler
zauba.io xpymep1.exe

304
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,178 +1,171 @@
01h4x.com
404checker
AIBOT AIBOT
ASPSeek Aipbot
Abonti
Aboundexbot
AiHitBot
Alexibot Alexibot
AllSubmitter AllSubmitter
Anarchy Anthill
Ankit Asterias
Apexoo
Attach Attach
AwarioRssBot BDFetch
AwarioSmartBot BLEXBot
BackStreet
Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass Battleztar Bazinga
Bandit
Black Hole Black Hole
BlackWidow
Blackboard
Blow Blow
Boardreader BotALot
Brandprotect
Brandwatch Brandwatch
Buck BuiltWith
BunnySlippers
BuzzSumo
CATExplorador CATExplorador
CCBot CCBot
Calculon
CazoodleBot CazoodleBot
Cegbfeieh CheTeam
CensysInspect
CheeseBot
ChinaClaw
Chlooe Chlooe
Claritybot Cliqzbot
Cocolyzebot Cloud mapping
Cogentbot Cogentbot
Collector Collector
Copier
CopyRightCheck
Copyscape Copyscape
Cosmos
Craftbot Craftbot
Crawling at Home Project Custo
CrazyWebCrawler CyotekWebCopy
Crescent
DBLBot DBLBot
DIIbot Deusu
DTS Agent Dirbuster
DatabaseDriverMysqli Disco
Demon DomCopBot
Devil DomainStatsBot
DittoSpyder Domains Project
DomainAppender
Dotbot
Dragonfly Dragonfly
Drip Drip
ECCP/1.0 EasyDL
EMail Siphon EirGrabber
Ebingbong
EroCrawler EroCrawler
Express WebPictures Extractor
ExtLinksBot ExtractorPro
Extreme Picture Finder Extreme Picture Finder
Fimap FDM
Firefox/7.0 Firefox/7.0
Foobot Freeuploader
GT::WWW Fuzz
GetWeb FyberSpider
Genieo
Gigabot Gigabot
Go!Zilla
Go-Ahead-Got-It
GoZilla GoZilla
Grabber Grabber
Grafula GrapeFX
HTMLparser GrapeshotCrawler
GridBot
HMView
HTTrack HTTrack
Haansoft Harvest
Havij
Heritrix Heritrix
IDBot Hloader
IRLbot IRLbot
Iblog IlseBot
Id-search InfoNaviRobot
Image Fetch
Image Sucker
Intelliseek
InterGET
Internet Ninja Internet Ninja
InternetSeer InternetSeer
IstellaBot IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT JetCar
Jetty Jetty
JikeSpider JikeSpider
JustView Jorgee
Jyxobot LWP::Simple
Kenjin Spider
Kinza
LNSpiderguy
Lanshanbot Lanshanbot
LeechFTP LeechGet
Lftp LibWeb
Libwhisker
LieBaoFast LieBaoFast
Linkdexbot Likse
LinkextractorPro LinkWalker
LinkpadBot LinkpadBot
LinksManager LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
MFC_Tear_Sample Lipperhey
MIIxpc Lipperhey Spider
Majestic12 Litemage_walker
Masscan MIDown tool
MegaIndex.ru MQQBrowser
MicroMessenger Mag-Net
Mail.RU_Bot
MarkMonitor
MarkWatch
Mediatoolkitbot
Microsoft URL Control
Moblie Safari Moblie Safari
Mojolicious Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
NICErsPRO Mr.4x3
NPbot
Name Intelligence
NearSite
Nessus Nessus
Netcraft NetMechanic
Netvibes Niki-bot
NextGenSearchBot NimbleCrawler
Nimbostratus Nimbostratus
Nmap
Nutch Nutch
Octopus Octopus
OnCrawl OnCrawl
OpenLinkProfiler
OpenVAS
Openfind Openfind
OutclicksBot OrangeBot
OutfoxBot OutfoxBot
PHPCrawl PECL::HTTP
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageScorer PageGrabber
Pavuk PageThing.com
Pandalytics
PeoplePal PeoplePal
Petalbot Petalbot
Pi-Monster
Picscout
PictureFinder
Pimonster Pimonster
Pockey ProPowerBot
ProWebWalker
Psbot
Pu_iN Pu_iN
Pump PxBroker
PyCurl PyCurl
RankActive RSSingBot
RankActiveLinkBot
RankFlex RankFlex
ReGet RankingBot
Reaper Rankivabot
RedesScrapy Re-re
RepoMonkey RealDownload
Ripper RebelMouse
Recorder
RocketCrawler
Rogerbot Rogerbot
SEOkicks
SEOkicks-Robot
SEOstats SEOstats
SISTRIX SMTBot
SalesIntelligent
ScanAlert
ScoutJet ScoutJet
Scrapy Screaming
ScreenerBot
ScrepyBot
Searchestate
SearchmetricsBot SearchmetricsBot
SemrushBot SemrushBot
SeoSiteCheckup SeobilityBot
SiteSucker Seomoz
Shodan
SiteLockSpider
Sitebeam
Siteimprove Siteimprove
Sitevigil Sitevigil
Snoopy Snake
SocialRankIOBot Snapbot
Sociscraper Sociscraper
Sogou web spider Sottopop
SpaceBison Spammen
SpankBot
Spbot Spbot
SputnikBot
Sqlmap
Sqlworm Sqlworm
Steeler Steeler
Stripper Stripper
@ -180,71 +173,78 @@ Sucuri
SuperBot SuperBot
SuperHTTP SuperHTTP
Surfbot Surfbot
Szukacz
T0PHackTeam T0PHackTeam
Teleport
TeleportPro TeleportPro
Telesphoreo
The Intraformant The Intraformant
TheNomad TheNomad
TightTwatBot Titan
Tracemyfile Toata
Trendiction Trendiction
True_Robot
Turingos Turingos
Turnitin Turnitin
Twice TurnitinBot
Typhoeus Typhoeus
V-BOT URLy.Warning
Vacuum VB Project
VidibleScraper VCI
WEBDAV VeriCiteCrawler
WWW-Collector-E Virusdie
VoidEYE
Voltron
WASALive-Bot
WPScan
WWW-Mechanize WWW-Mechanize
WWW::Mechanize
WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD
WeSEE
Web Auto
Web Collage Web Collage
Web Fuck Web Fetch
Web Sauger WebAuto
WebFetch WebImageCollector
WebLeacher WebStripper
WebSucker
WebWhacker WebWhacker
Webalta WebZIP
WebmasterWorldForumBot
Webshag
WebsiteQuester
Webster Webster
Whack
Whatweb
WinHTTrack WinHTTrack
Wonderbot Xaldon_WebSpider
Wotbox Xenu
Zade YoudaoBot
Zeus
ZoomBot ZoomBot
ZoominfoBot ZyBorg
ZumBot
archive.org_bot archive.org_bot
arquivo.pt arquivo.pt
autoemailspider clark-crawler
cah.io.community cognitiveseo
com.plumanalytics
dataforseo.com dataforseo.com
demandbase-bot demandbase-bot
eCatch
evc-batch evc-batch
instabid gopher
isitwp.com internetVista monitor
meanpathbot ips-agent
lwp-request
magpie-crawler
mediawords mediawords
netEstate NE Crawler muhstik-scan
s1z.ru page scorer
seostar polaris version
siteripz probe-image-size
ripz
satoristudio.net
scalaj-http
scan.lol
seobility
sogouspider
spyfu spyfu
tAkeOut tAkeOut
trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
webmeup-crawler voyagerx.com
webprosbot webprosbot
x09Mozilla
xpymep1.exe xpymep1.exe
zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
egvar.net 4pp13.com
hopeonthestreet.co.uk buron.pw
kuzinsp.ru compliance-ivan.xyz
matsdale.com homeandhealth.ru
resultshub-a.akamaihd.net intellego.info
sdrescher.net partybunny.ru
slavia.info seansonline24.pl
tecnoteakviareggio.it smailik.org
topsiteminecraft.com social-vestnik.ru
zytpirwai.net w3javascript.com

950
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com 0c47f8422d3f.com
13tabs.com 101lesbian.xyz
178evakuator178.ru 123any.com
1millionusd.xyz 2ads.co.uk
2020iscoming.info 2drittel.de
214jaluwobapef.cf 3rbseyes.com
2rich4bitches.com 3xforum.ro
2x2fan.ru 4webmasters.com
4kepics.com 5-steps-to-start-business.com
7a2rnuey1tw9ar.ru 70casino.online
91abcw.com 76brighton.co.uk
98oi.ru
aad1f4acb0a373420d9b0c4202d38d94fa.com
absugars.com absugars.com
adclickthru.net acmebtn.ml
addblueoff.com.ua adclickservice.com
addray.pro add-add.men
admanaerofoil.com advertex.info
adsland.men advokateg.xyz
adsloads.com aero2.ru
adult3dgames.com afmuseum.com
advancedsoftwaresupport.com
adviceforum.info
adzerg.com
adzpower.com
agroeconom.kz
ahmedabadwebs.com ahmedabadwebs.com
air-edem.ru aimayangzhi.com
allforminecraft.ru alfa9.com
allornamenti.com allfinweb.com
altamayoztourism.com alphahoverboards.com
ameblo.jp aludecor.info
analytics-ads.xyz amyfoxfitness.com
apidigidocketnet-a.akamaihd.net anaksma.info
appiq.mobi android-vsem.org
applyneedy.xyz apartamentwroclaw.eu
approved.su apper.de
aproposde.com appmsr.org
artel116.ru arabsexxxtube.com
arvut.org arcteryxstore.online
autowebmarket.com.ua artefakct.com
avkzarabotok.com astana.xxxkaz.org
azarttoto.com atout-energie-69.com
baikaleminer.com audiobangout.com
banki76.ru aussie-prizes.com
bannerpower.com autobudpostach.club
bannerswap.com autoloans.com
beachfix.co autovideobroadcast.com
beachtoday.ru aviav.co
becuo.com badavit.com.ua
bestadbid.com balla-boo.se
bestfortraders.com bashtime.ru
bestofupload.info berdasovivan.ru
besttorrentknifta.weebly.com best-games.today
bet-prognoz.com betshuckclean.com
bezlimitko.xyz bgviagrachrx.com
beztuberkuleza.ru bif-ru.info
bibys.com bio-optomarket.ru
bigcities.org bjgugu.net.ua
bildsuche.ru blog-hits.com
bingoporno.com blog.f00kclan.de
bizfly.info bolegapakistan.com
bkns.vn boo-arts.com
boanonihaca.tk bouda.kvalitne.cz
board.f00d.de brainzpod.win
bookmark4you.com brandimensions.com
braintobe.win brateg.xyz
bravegirlsclub.com breget74.ru
brg8.com briomotor.co
bristolhotel.com.ua bronzeaid-a.akamaihd.net
brus-vsem.ru buehne-fuer-menschenrechte.de
burger-tycoon.com bugshoot.cn
business-made-fun.com buqayy0.livejournal.com
buttons-for-website.com buyessayonline19.blogspot.ru
buypuppies.ca
buzzurl.jp
bvlgariwallet.xyz
byme.se byme.se
cartierbraceletsreplica.pw camdolls.com
cartierreplica.xyz carabela.com.do
cdn.walkme.com cardul.ru
cdncash.com casinos4dummies.co.uk
cfacarrosserie74.com cb.iphantom.com
cheapjerseysa.com cdncash.net
cheapwebsitehoster.com cenokos.ru
chimiver.info cenoval.ru
chocolateslim-original.com cheatcode-lita12.rhcloud.com
cleanersoft.com ciekawinki.pl
clickhype.com cinemaenergy-hd.ru
clickso.com ciproandtizanidine.blogspot.com
cmhomestayagency.com classiquebijoux.ru
cobaltpro.ru clipartnew.com
coldfilm.ru com-secure.download
com-onlinesupport.site
companies-catalog.com companies-catalog.com
compliance-ivan.top compliance-elena.top
compliance-josh.top compliance-fred.top
compliance-olga.top compliance-irvin.xyz
compliance-walter.top compliance-walter.top
compliance-willy.top compucelunlock.net
converse.ddsoldes.fr computernetworksonline.com
credit-online.ws cookie-law-enforcement-ff.xyz
czeshop.info cookie-law-enforcement-hh.xyz
danceuniverse.ru coolwallpapers-hd.com
darmebel.com.ua coolwallpapers4k.info
dating-time-now.com creams.makeforum.eu
deanmoore.ie csgo4.win
delayreferat.ru custom-product-labels.com
delfin-aqua.com.ua customsua.com.ua
delta-line.men datsun-do.com
diarioaconcagua.com dbmkfhqk.bloger.index.hr
dickssportinggoods.com defenderxtactical.com
diminishedvalueoforegon.com demon-tweeks.com
dimkino.ru deniven.1bb.ru
divku.ru dikx.gdn
documentbase.net djekxa.ru
documentsite.net docstoc.com
domcran.net dodlive.mil
downloadmefiranaratb1972.xpg.com.br dogclothing.store
dprtb.com dostavimvdom.ru
drugspowerstore.com dostavka-v-ukrainu.ru
duplicashapp.com download-of-the-warez.blogspot.com
ed-shop01.ru driving.kiev.ua
edudocs.net e-avon.ru
elektrischezigaretten1.blogse.nl earthmagic.info
elektronischezi.livejournal.com ebooknovel.club
elektronischezigarettekaufen2.tumblr.com efnor-ac.com
elektrozigarette2.wordpressy.pl ekto.ee
elexies.info el-nation.com
elidelcream.weebly.com electrik-avenue.com
embedle.com encodable.com
energydiet24.ru
eng-lyrics.com eng-lyrics.com
enge-fotzen.info enternet.ee
erotik0049.com enthuse.computernetworksonline.com
escortslet.net ereko.ru
esfchat.tk etenininrade.ga
eu-cookie-law.info euronis-free.com
executehosting.com expdom.com
expertnaya-ocenka.ru ezigarettenshop2.myblog.de
eyelike.com.ua f5mtrack.com
eyessurgery.ru facialporntube.com
ezigarettenkaufen2.dreamwidth.org falcoware.com
ezigaretteshop2.vefblog.net falllow.gq
ezrvrentals.com fba-mexico.com
f1nder.org fbdownloader.com
failingmarriege.blogspot.com fearcrow.com
familienzahnaerzte.com felizporno.com
fatmaelgarny.com financepoints.eu
feargames.ru findacheaplawyers.com
femdom.twiclub.in firstsiteguide.com
ff30236ddef1465f88547e760973d70a.com flipper.top
filesdatabase.net
finansov.info
findanysex.com
flytourisme.org
food.dtu.dk
fordsonmajbor.cf fordsonmajbor.cf
forexunion.net formulaantiuban.com
forum20.smailik.org forum.doctissimo.fr
foxinsocks.ru forums.d2jsp.org
foxtechfpv.com foto-weinberger.at
frankofficial.ru free-floating-buttons.com
free-video-tool.com free-gluten.ru
freenom.link free-social-buttons.com
freewebs.com freesitetest.com
freshsuperbloop.com fuck-paid-share-buttons.xyz
frighteningremain.cf funnel.co.za
frvo.alptandem.ru galaxys6manual.info
fsakhalin.ru gambarxkata.co
fuel-gas.com gameonasia.com
funnymama.com gamesprite.me
fxgallery.com gateway.zscalertwo.net
gazobeton-p.com.ua gay.adultgalls.com
ge0ip.org gayxperience.com
geile-lelly.eu geilehausfrauen.net
gemara.com germetiki.com.ua
gettpromos.com getridofstretchmarks.org
getyourimage.club ghostvisitor.com
gg-arena.ru gk-atlant.info
gigapeta.com gobongo.info
glavtral.ru golaya.pw
glicol.kz googglet.com
goatse.ru googlsucks.com
gojiberriess.apishops.ru grand-chlen.ru
gomusix.com groupmoney.ru
goodnightjournal.com growboxbl.ru
gov.yanao.ru gsasearchengineranker.site
greenzaim.ru gsasearchenginerankerdiscount.com
gsasearchengineranker.xyz guigyverpo.cf
gta-top.ru gwebtools.com
gymi.name handicapvantoday.com
habermetre.com hccoder.info
handicapbathtubarea.com hdimagegallery.net
hao123.com hdpixion.com
hatdc.org historichometeam.com
hatedriveapart.com hmywwogw.bloger.index.hr
havepussy.com holidaypics.org
hchha.com homeart.space
hd-film.pl hotgirlhdwallpaper.com
hermesreplica.win hotloans.ru
heroz.fr housekuba.org
hexpilot.com houston-vikings.com
himazin.info hulfingtonpost.com
histats.com hundejo.com
hjaoopoa.top hunthillfarmtrust.org
hledejvshopech.cz hustoon.over-blog.com
horoshieokna.com hystersister.com
hosting-tracker.com ico.re
hostsshop.ru
hotcore.info
hotenergy.ru
housediz.com
houseofrose.com
i-service.kz
ideawheel.com
ideibiznesa2015.ru ideibiznesa2015.ru
igru-xbox.net ilovevitaly.ru
igtools.club imagefinder.site
impisr.ru imagerydatabase.com
inboundlinks.win indo-export.ru
inet-traffic.com instakink.com
infodocsportal.com
interesnie-faktu.ru interesnie-faktu.ru
interior-stickers.ru internet-apteka.ru
invitefashion.com iopeninghours.co.uk
iqoption.com iscblog.info
istanbulit.com iusstf.org
itis4you.com ivanovo.zrus.org
ixora.pro ivoiretechnocom.ci
izhevsk.zrus.org iweblist.info
izismile.com jav-idol.com
janettabridal.com joingames.org
japfm.com jongose.ninja
jav-way.site jrpmakati.com
javstock.com juliadiets.com
jerseyswholesalechinalimited.com kaac.ru
jestr.org kambasoft.com
jetsli.de
jpcycles.com
jwss.cc
karadene.com
karpun-iris.ru
kasino-money.pw
katadhin.co
kazan.zrus.org kazan.zrus.org
keywordsdoctor.com ketoanhanoi.info
keywordsuggest.org keywords-monitoring-your-success.com
keywordsuggests.com kino-fun.ru
kihi.gdn kino2018.cc
kinoduh.ru kinobaks.com
kinohit1.ru kinozapas.com
kinoplen.ru kiprinform.com
kiskinhouse.com
kliksaya.com kliksaya.com
kmd-pto.ru kmd-pto.ru
kol-energo.ru kmgamex.cf
konpax.com kolotiloff.ru
kotaku.com komputers-best.ru
kovesszucs.atw.hu koptims.tiu.ru
krasivye-devushki.net kupiproday.com.ua
ku6.com ladov.ru
kvartiry-remont.ucoz.ru lancheck.net
lacasamorett.com larger.io
ladsup.com laxdrills.com
lanadelreyfans.us ledis.top
languagecode.com lerporn.info
lapitec.eu letmacwork.world
larose.jb2c.me life-instyle.com
lawyers.cafe liupis.com
lechtaczka.net
leftofcentrist.com
letolove.ru
life.biz.ua
liffa.ru
lion.misba.us
littlecity.ch
live-sexcam.tk live-sexcam.tk
love-baby.cz liveinternet.ru
localmatchbook.com
louisvuittonoutletstore.net
lsp-awak-perikanan.com lsp-awak-perikanan.com
lulea-auktionsverk.se lubetube.com
lumb.co luciddiagnostics.in
macfix.life lunamedia.co
makenahartlin.com m292.info
malwareremovalcenter.com magazintiande.ru
mansparskats.com map028.com
marcoislandvacations.net marcowebonyodziezowe.pl
marketingvici.com marwer.info
massage-info.nl mebelcomplekt.ru
masterseek.com meble-bogart.info
masturbate.co.uk medicovi.com
matematikus.info meetingrainstorm.bid
medanestesia.ru mirzonru.net
meds-online24.com misandesign.se
mega-polis.biz.ua mobicover.com.ua
menhealed.net mobile-appster.ru
mere.host.sk moinozhki.com
merryhouse.co.uk mojaocena.com
metarip.ru moje-recenze.cz
milblueprint.com mojpregled.com
mindbox.co.za moneymaster.ru
minneapoliscopiers.com moneyteam24.com
missis.top moonci.ru
mlf.hordo.win mozello.ru
mmostrike.ru mp3films.ru
mnogolok.info mp3ritm.top
moesen-ficken.com my-aladin.com
mompussy.net myseoconsultant.com
moomi-daeri.com nakozhe.com
morpicert.pw namecrumilchlet.tk
mouselink.co navalwiki.info
movpod.in negociosdasha.com
moyakuhnia.ru net-radar.com
msk-diplomat.com newstudio.tv
mydirtystuff.com
nailsimg.com
nash-krym.info
nastydollars.com
netoil.no
netpics.org
newsperuse.com
nfvsz.com
nhl17coins.exblog.jp nhl17coins.exblog.jp
oakridgemo.com nina.az
obsessionphrases.com noclegonline.info
ochistka-stokov.ru notasprensa.info
oivcvx.website notfastfood.ru
ok-ua.info okel.co
okuos.com old-rock.com
olvanto.ru oneshotdate.com
olympescort.com onload.pw
one-gear.com ontargetseo.us
onemantrip.com origin-my.ru
onlinefilmz.net paceform.com
ops.picscout.com pagesense.com
oralsexfilme.net paradontozanet.ru
orel-reshka.net parajumpersjakkesalgnorge.info
orhonit.com patol01.pw
ownshop.cf pb-dv.ru
oynat.info pc-test.net
paparazzistudios.com.au pdamods.ru
pavlodar.xkaz.org phpdevops.com
pay2me.pl picsearch.com
pcads.ru picturesify.com
penisvergrotendepillennl.ovh pierrehardysale.online
perl.dp.ua pills24h.com
phrcialiled.com pinstake.com
picscout.com
picture-group.com
pijoto.net
piulatte.cz piulatte.cz
piuminiita.com pk-pomosch.ru
pl-top.pl plastgranar.nu
pl-vouchers.com playlott.com
plugingeorgia.com pokemongooo.ml
polybuild.ru poquoson.org
pooleroadmedicalcentre.co.uk porngalleries.top
porno-raskazy.ru pornodojd.ru
pornokajf.com pornoinn.com
pornoklad.net
pornokorol.com
pornoplen.com
pornoslive.net
poshiv-chehol.ru poshiv-chehol.ru
postclass.com posible.net
pretty-mart.com potoideas.us
printingpeach.com predominant-invent.tk
prizestohandle.club preparevideosafesystem4unow.site
prod2016.com preparevideosafesystem4unow.space
profolan.pl pricheskaonline.ru
ptr.ruvds.com privat-girl.net
pyramidlitho.webs.com professionaldieselcare.com
qitt.ru progress-upakovka.ru
qld10000.net prohoster.info
ragecash.com projectforte.ru
refads.pro prosmibank.ru
registry-cleaner.net prosperent.com
remorcicomerciale.ro prpops.com
rff-cfal.info putitin.me
rixpix.ru qpypcx.com
rn-to-bsn.com quit-smoking.ga
roleforum.ru rangjued.com
rubanners.com rapidgator-porn.ga
runofilms.ru rasteniya-vs-zombi.ru
runstocks.com razorweb-a.akamaihd.net
saratov.xrus.org razyboard.com
sarm.tk realtytimes.com
sashagreyblog.ga redhotfreebies.co.uk
sc-specialhost.com rep-am.com
scanner-jess.top responsinator.com
scanner-josh.top rfid-locker.co
scanner-julianna.top robertefuller.com
scrapy.org rubbed.us
sdelai-prosto.ru ruclicks.com
securesmrt-dt.com rus-teh.narod.ru
seeingmeerkat.com rybalka-opt.ru
servethis.com s.lollypopgaming.com
service-core.ru sady-urala.ru
servicecenter.co.ua saugatuck.com
seryeznie-znakomstva.ru sbwealthsolutions.ca
sex-pr.net scanmyphones.com
sexad.net scanner-marwin.top
sexblog.pw schalke04fc.info
share-buttons-for-free.com searchengineranker.email
skanninge.se searchimpression.com
skypasss.com seojokes.net
social-buttons.com sex.hotblog.top
sohoindia.net sexuria.net
sonata-arctica.wz.cz sexvideo-sex.com
spacash.com sexxdate.net
space4updating.win shemale-sex.net
speeddream.xyz shivafurnishings.com
srgwebmail.nl shopvilleroyboch.com.ua
stard.shop simplepooltips.com
statoutlook.info sinel.info
stauga.altervista.org skylta.com
steamoff.net sms2x2.ru
stmassage.ru snomer1.ru
stpicks.com snowplanes.com
stroy-portal22.ru snts.shell-pmr.ru
stroymonolit.su soaksoak.ru
studworks.org social-buttons.xyz
svtrd.com socialmediasuggest.com
sweepstakes.rewardit.com softlinesolutions.me
szamponrevita.pl sortthemesitesby.com
t-bygg.com soundfrost.org
talant-factory.ru speechfoodie.com
tasteidea.com spidtest.org
tastyfoodideas.com stair.registrydomainservices.com
tdsing.ru stardevine.com
teastory.co stjamesschool.info
theporndude.com strag-invest.ru
thewomenlife.com stroicol.net
thexart.club stroiminsk.org
thin.me.pn stroyhelp-dv.ru
timdreby.com svensk-poesi.com
titslove.yopoint.in tako3.com
tizanidinehclsideeffects.blogspot.com tarad.com
tizanidinerestlesslegsyndrome.blogspot.com tecspb.ru
tmtrck.com templates.radiodigital.co
tobeyouday.win tengohydar.tk
traffic2cash.org thebluenoodle.com
traffictrade.life thedownloadfreeonlinegames.blogspot.com
thepornsex.org
timer4web.com
tizanidinehcl4mgtablet.blogspot.com
tizanidinehydrochloride4mgstreetvalue.blogspot.com
tizanidineonline.blogspot.com
tomck.com
tonerbox.kz
toon-families.com
topkarkas.com
torrentgamer.net
tour-line.net
toxicwap.com
trafficjunky.com
trafficshaper.com
trichizobswiv.agddns.net
trion.od.ua trion.od.ua
trumpetedextremes.com tripper.de
trustedmaccleaner.com trudogolik.net
turizmus.us truemfilelj.gq
uchetunet.su trustl.life
ugguk.online ttsq.fr
ukrtextbook.com twelvevisionspartyofcolorado.com
uptime-delta.net twitlinks.com
uroki.net twodollarshows.com
usadacha.net u17795.netangels.ru
v-doc.co u555u.info
valmetrundan.se uhodzalijami.ru
valsalud.com ukrup.com
vapsy.com umg-stroy.ru
velen.io unlimitdocs.net
veles.shop unpredictable.ga
vesnatehno.ru usacasino.com
viagraneggrx.com usiad.net
vip-dom.in validdomain.xyz
vip.51.la valkiria-tk.ru
virtualbb.com valuado.com
varikoz24.com
velobikestock.com
vereo.eu
versaut.xxx-cam.webcam
vesnatehno.com
via-energy-cumpara.com
video-chat.cn
video-chat.in
videochat.cafe
videochat.mx
vipps.com.my
vipromoffers.com
visionwell.com.cn
visitcambridge.org
vitoriacabos.com
vkontaktemusic.ru
vladhistory.com vladhistory.com
vmnmvzsmn.over-blog.com vrnelectro.ru
vodaodessa.com wareseeker.com
vseigry.fun waysbetter.cn
vzlomsn.org websiteaccountant.de
watracker.net
webalan.ru
weblibrary.win
website-speed-up.site
wemarketing.se wemarketing.se
wildworld.site weprik.ru
wineration.com wesharepics.info
winwotgold.pl westum.se
woman-orgasm.ru westvilletowingservices.co.za
wowas31.ucoz.ru wholesalejerseychinaoutlet.com
wufak.com wikes.20fr.com
www.labves.ru wildnatureimages.com
xchangetrak.com witherrom55.eklablog.fr
xjrul.com wleuaprpxuvr.ga
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai woman-tampon.ru
woodyguthrie.se
works.if.ua
www.solartek.ru
wygraj-teraz.com
wzgyyq.com
xcombear.ru
xmronta.com
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai
xn--80ab4aa2g.xn--p1ai
xn--80ahdheogk5l.xn--p1ai xn--80ahdheogk5l.xn--p1ai
xn--80aodinpgi.xn--p1ai xn--80aikhbrhr.net
xn--b1ag5cfn.xn--p1ai xn--80ajbshivpvn2i.xn--p1ai
xn--d1acah0c.xn--p1ai xvideosbay.com
xn--h1ahbi.com.ua xxxasianporn.net
xrus.org yeartwit.com
xxxguitars.com
xxxhdvideo.site
yadro.ru
yaoguangdj.com
yogamatsexpert.com
yopoint.in
youandcredit.ru youandcredit.ru
youghbould.wordpress.com youradexchange.com
youradulthosting.com yuweng.info
zaapplesales.blogspot.com
zarabotat-na-sajte.ru zarabotat-na-sajte.ru
zdesoboi.com
zeroredirect10.com zeroredirect10.com
zeroredirect11.com zeroredirect11.com
zhongwenlink.com zhacker.net
zhuravlev.info znakomstvaonlain.ru
zmoda.hostreo.com zog.link
zygophyceous.womanstars.site zoomovies.org
zootoplist.com
zrelaya.pw
zreloeporno.tv
zscalerone.net
zvezdagedon.ru
zytpirwai.net

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
12bet.com 000free.us
178evakuator178.ru 101billion.com
1hwy.com
1xbet4you.com
24x7-server-support.site 24x7-server-support.site
55wmz.ru 2girls1cup-free.com
76brighton.co.uk 2itech.ru
999webdesign.com acortarurl.es
aasoldes.fr addbin.men
aghanyna.com adf.ly
adidas.frwebs.fr
adman.gr
ads-seo.men
adventureparkcostarica.com
ahmedabadwebs.com
akuhni.by
alarmobninsk.ru
alfapro.ru
algerianembassy.co.in
alibestsale.com
allboard.xobor.de allboard.xobor.de
amateurgalls.com anal-acrobats.com
animal36.com ananumous.ru
appiq.mobi androids-store.com
apps-analytics.net apiluckyleapnet-a.akamaihd.net
architecturebest.com appfastplay.com
arendakvartir.kz appsaurus.com
arkkivoltti.net articlesdirectoryme.info
arraty.altervista.org asiengirls.net
autorn.ru atelielembrancaqueencanta.com.br
availit.weebly.com avtolombard-krasnodar.ru
aviav.eu azartclub.org
azartmsl.com azbukafree.com
babosas.com babieca.com
balois.worldbreak.com baikaleminer.com
betteroffers.review bannerads.de
beachpics.com
bensbargains.net
bestmarriages.com
betterscooter.com
beyan.host.sk
bigames.online bigames.online
bioca.org blobar.org
bluejays-jerseys.us bonux.nextview.ru
bobba.dzaba.com buy-loft.ru
bobinoz.com buzzonclick.com
bocoarchives.org bytimedance.ru
bookhome.info carapics.com
business-suggested.tk carstrends2015.com
bvlgariring.xyz cascadelink.org
cartierlove2u.com censys.io
castingbank.ru cfjump.com
cbox.ws chatroulette.si
cdncash.net cheapjerseysa.com
certifywebsite.win christianlouboutinsaleoutletonline.us
cfsrating.sonicwall.com cmhomestayagency.com
cgi2.nintendo.co.jp cntravelre.com
cheap-trusted-backlinks.com coderstate.com
cheapjerseysap.com comeondog.info
checktext.org compliance-checker-7.info
clickzzs.nl compliance-josh.top
comissionka.net
compliance-jack.top
compliance-jess.top
connectingsingles.com
cpabegins.ru
creditmoney.com.ua
cypernhuset.se cypernhuset.se
danashop.ru dailyfinancefix.com
date-withme.com daretodonate.co
decorazilla.com delitime.info
delfin-aqua.com.ua dentfidemountpreach.tk
descargar-musica-gratis.net destinationrealestate.com
directrev.com deutschehobbyhuren.net
diy-handmade-ideas.com diegolopezcastan.com
doctormakarova.ru dieswaene.com
dostavimvdom.ru dildofotzen.net
downloader12.ru dms-sw.ru
dyshagi.ru dojki-hd.com
ekspertmed.com domainroam.win
elektroniksigaraankara.info domainsphoto.com
elite-sex-finders.com downtuptv.gq
erotikstories.ru drupa.com
esopini.com econom.co
excitacao.com eorogo.top
eywords-monitoring-your-success.com esfchat.tk
ezaz.info everypony.ru
fastcrawl.com eyelike.com.ua
fickenprivat.info fearcrow.com
flirt4free.com ferrotodo.com
flytourisme.org findacheaplawyers.com
focalink.com finemanteam.com
forensicpsychiatry.ru franecki.net
forumprofi.de frbizlist.com
free-deals.faith freecamdollars.com
free-traffic.xyz freenode.info
fullfileaccess.com freshdz.com
gallerily.com galaxy-family.ru
gamerextra.com geoads.com
gaygalls.net gg-arena.ru
ghernnqr.skyrock.com ghernnqr.skyrock.com
gidonline.one gkvector.ru
goatse.ru glall.ru
golden-praga.ru globalsurfari.com
goodwinmetals.co globetrotting-culture.ru
grupografico-pilar.com.ar gombita.info
hamacapty.com gribokstop.com
hchha.com helicalpile.us
hebr.myddns-flir.com hitcpm.com
hem.passagen.se hully.altervista.org
hobbyhuren24.net iamsport.org
hut1.ru idc.com.ua
hyipmanager.in ido3.com
ideashome.id inbabes.sexushost.com
ilovevitaly.com incanto.in.ua
imagui.eu innoslicon.com
indiasourcemart.in intervsem.ru
infobanks.ru iqoption.com
infospot.pt istizanidinelikexanax.blogspot.com
infostatsvc.com izhevsk.zrus.org
investingclub.ru japan-bearings.ru
istizanidineanarcoticdrug.blogspot.com jav-idol.com
jabimgo.pw javxxx18.com
janavibekken.no journalhome.com
jav-fetish.site jrpmakati.com
jestr.org jurajskie.info
jillepille.com jus0wil.pp.ua
joessmogtestonly.com keywords-monitoring-success.com
joy-penguin.com kinohit1.ru
kaac.ru kiskinhouse.com
kabbalah-red-bracelets.com klitimg.pw
karpun-iris.ru krasivoe-hd.com
kinky-fetishes.com kredit-pod-zalog-krasnodar.ru
kinostorm.net lanadelreyfans.us
kiwi237au.tk laxdrills.com
kollekcioner.ru link.ac
kosova.de linkbux.ru
koversite.info liveinternet.ro
krasnodar.ru m1media.net
kredit-blog.ru macresource.co.uk
ktotut.net
kunstaktien.info
kw21.org
lakomka.com.ua
larutti.ru
libertybilisim.com
life.biz.ua
livejasmin.com
liveshoppersmac.com
louisvuittonoutletstore.net
luciddiagnostics.in
lyrics.home-task.com
macrotek.ru macrotek.ru
magento-crew.net magento-crew.net
mansparskats.com magicalfind-a.akamaihd.net
manuscript.su maranbrinfo.com.br
mature.free-websites.com menetie.ru
microsoftportal.net missis.top
mielec.pl mobilemedia.md
miniature.io mobot.site
mnogolok.info moomi-daeri.com
mobicover.com.ua mydoctorok.ru
modabutik.ru myindospace.com
moivestiy.biz myiptest.com
moncleroutletonline.win narkologiya-voronezh.ru
moneyzzz.ru newstudio.tv
mossmesi.com nmrk.ru
mrcsa.com.au noumeda.com
mttwtrack.com okayimage.com
myanyone.net olgacvetmet.com
mylesosibirsk.ru online-hit.info
mzdish.site orion-code-access.net
najaden.no paperwritingservice17.blogspot.ru
nalogovyy-kodeks.ru partybunny.ru
narkologiya-orel.ru pateaswing.com
nash-krym.info perederni.net
netoil.no pipki.r.acdnpro.com
netvouz.com piski.top
nhl09.ru pladform.ru
obiavo.su pornofiljmi.com
omgtnc.com pornstartits.xblog.in
onblastblog.online privatefx.all4invest.info
onload.pw projefrio.com.br
owathemes.com prosmibank.ru
ownshop.cf prostitutki-astana.org
pammik.ru prpops.com
partner-print.men pshare.biz
partner-trustworthy.men putitin.me
partnerads.men queerspace.com
pics-group.com quickbuck.com
pihl.se rangjued.com
pinsdaddy.com ranksonic.org
pogodnyyeavarii.gq rating-bestcasino.com
pornolook.net razleton.com
pourvous.info reckonstat.info
preparevideosafesystem4unow.space registry-clean-up.net
prlog.ru rightenergysolutions.com.au
professionalwritingservices15.blogspot.ru risparmiocasa.bz.it
pskcijdc.bloger.index.hr ruf777.com
rabotaetvse.ru s-iwantyou.com
reelheroes.net sbdl.no
remont-comp-pomosh.ru scanner-jack.top
renecaovillasale.online scrapinghub.com
rentehno.ru screpy.com
rospromtest.ru seonetwizard.com
rucrypt.com sextracker.de
rvi.biz
sa-rewards.co.za
sack.net
safe-app.net
sajatvelemeny.com
saletool.ru
santasgift.ml
scanner-alex.top
scanner-julia.top
scanner-willy.top
scanspyware.net
sdelai-prosto.ru
seccioncontrabajo.com
service-core.ru
sexxdate.net sexxdate.net
shemale-sex.net shiksabd.com
shikiso.info shinikiev.com.ua
sientalyric.co sigmund-freud.co.uk
site3.free-share-buttons.com sildenafilcitratemed.com
skyway24.ru slavia.info
snts.shell-pmr.ru smartshoppymac.com
spb.afora.ru snow.nvr163.com
stauga.altervista.org soc-proof.su
stmassage.ru softonicads.com
stopnarco.ru sonnikforme.ru
stpicks.com stair.registrydomainservices.com
tizanidinestreetprice.blogspot.com streetfooduncovered.com
tongkatmadura.info stroy-portal22.ru
tula.mdverey.ru studiofmp.com
turist-strani.ru success-seo.com
txxx.com superlist.biz
ussearche.cf supervesti.ru
v24s.net swagbucks.com
vchulkah.net talant-factory.ru
viagra-soft.ru taxi-v-eisk.ru
videochat.cafe thetoiletpaper.com
videochat.mx titan-cloud.life
videochat.ph tizanidinehcl4mgisitanarcotic.blogspot.com
vizitki.net tizanidinestreetvalue.blogspot.com
vkontaktemusic.ru tmtrck.com
voprosotvet24.ru tnaionline.org
warning.or.kr toon-families.com
webalan.ru topkarkas.com
weprik.ru trafficjunky.net
westen-v.life tri-slona.org
whatistizanidinehclusedfor.blogspot.com tt-ipd.info
wieseversa.no tuberkulezanet.ru
wovis.site tupper-shop.ru
ww2awards.info twinderbella.com
www.bookmaker-bets.com upstore.me
x-mix.info uptime-as.net
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai userequip.com
xtube.com varbergsvind.se
yourporngay.com vbabule.net
ytmnd.com vezuviy.su
yugk.net vipsiterip.org
zarenica.net virtuagirl.com
zdesoboi.com virtualbb.com
zigarettenonlinekaufen2.bloog.pl vitanail.ru
vzlom-na-zakaz.com
windowssearch-exp.com
winwotgold.pl
woodyguthrie.se
wormix-cheats.ru
woweb.com.ua
www.event-tracking.com
www.kabbalah-red-bracelets.com
x-true.info
xn--80aanaardaperhcem4a6i.com
xn--e1afanlbnfckd7c3d.xn--p1ai
xportvusbdriver8i.snack.ws
xxxkaz.org
yhit.press
yogamatsexpert.com
youbloodyripper.com
zebradudka.com
znaturaloriginal.com
zophim.me

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.196 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2678 ### Version: V4.2021.10.2679
### Updated: Wed Oct 27 22:03:11 UTC 2021 ### Updated: Thu Oct 28 10:12:31 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2678 ### Version: V4.2021.10.2679
### Updated: Wed Oct 27 22:03:11 UTC 2021 ### Updated: Thu Oct 28 10:12:31 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##