vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 11:10:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.07.3235 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-07-18 22:06:56 +00:00
parent 9fbb7754eb
commit 085d4cff22
47 changed files with 106262 additions and 106272 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.07.3234 #### Version: V4.2022.07.3235
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.07.3234 #### Version: V4.2022.07.3235
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.07.3234 #### Version: V4.2022.07.3235
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

13146
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.07.3234 #### Version: V4.2022.07.3235
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

13150
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

13150
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

13150
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

13150
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 1; 10.1.1.126 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 0; 10.1.1.126 0;
127.0.0.1 0; 127.0.0.1 0;

13150
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 1; 10.1.1.126 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 0; 10.1.1.126 0;
127.0.0.1 0; 127.0.0.1 0;

13150
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 1; 10.1.1.126 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 0; 10.1.1.126 0;
127.0.0.1 0; 127.0.0.1 0;

13150
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3234 3235

13150
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

13150
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.114.45.213 0; 13.82.102.13 0;

13150
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 1; 10.1.1.126 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 0; 10.1.1.126 0;
127.0.0.1 0; 127.0.0.1 0;

13150
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 1; 10.1.1.126 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 0; 10.1.1.126 0;
127.0.0.1 0; 127.0.0.1 0;

13150
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 1; 10.1.1.126 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 0; 10.1.1.126 0;
127.0.0.1 0; 127.0.0.1 0;

13150
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.114.45.213 0; 13.82.102.13 0;

13150
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.114.45.213 0; 13.82.102.13 0;

13150
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 1; 10.1.1.126 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AwarioSmartBot Aspiegel
BBBike JennyBot
Dirbuster MeanPath Bot
FDM Nmap
JustView Openvas
Linkbot PeoplePal
Snake WebBandit
Sqlmap arquivo-web-crawler
instabid serpstatbot
sp_auditbot sogouspider

312
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
AIBOT 01h4x.com
404checker
ADmantX
ALittle Client ALittle Client
Abonti Aboundex
Aboundexbot Aboundexbot
Acunetix
Aipbot Aipbot
AllSubmitter AllSubmitter
Alligator AlphaBot
Anarchie
Anarchy Anarchy
Ankit Ankit
Anthill Asterias
Apexoo Atomseobot
Aspiegel Attach
AwarioRssBot
BBBike BBBike
BDFetch BDCbot
BLEXBot
BackDoorBot
Backlink-Ceck
BacklinkCrawler
Badass Badass
Bandit
BatchFTP
Battleztar Bazinga Battleztar Bazinga
Black Hole Bitacle
BlackWidow
Blow Blow
BlowFish
Boardreader
Bolt Bolt
BotALot Brandwatch
BuiltBotTough Buck
BuiltWith BuiltWith
BuzzSumo
CCBot
CODE87 CODE87
CSHttp CSHttp
Calculon CheTeam
Cegbfeieh Chlooe
ChinaClaw Citoid
Cliqzbot
Cloud mapping Cloud mapping
Cogentbot Cocolyzebot
Collector
Copyscape
Craftbot
Crescent Crescent
CrunchBot
Curious Curious
Discoverybot Custo
DittoSpyder DTS Agent
DnBCrawler-Analytics DataCha0s
Deusu
Dirbuster
Dispatch
DomainAppender DomainAppender
DomainCrawler DomainSigmaCrawler
Domains Project
Dotbot Dotbot
Download Wonder
Drip Drip
EroCrawler EMail Siphon
Exabot EMail Wolf
EasyDL
Ebingbong
ExtLinksBot ExtLinksBot
Extractor ExtractorPro
Extreme Picture Finder EyeNetIE
FDM Ezooms
FHscan FHscan
Fimap Flunky
Firefox/7.0 FrontPage
FlashGet
Foobot
Freeuploader
FyberSpider FyberSpider
G-i-g-a-b-o-t Getintent
GT::WWW Go!Zilla
GalaxyBot Go-Ahead-Got-It
GetRight GrapeFX
HMView
HTMLparser HTMLparser
Haansoft
HaosouSpider
Harvest
Havij
Heritrix
Hloader
HonoluluBot HonoluluBot
IDBot
IRLbot IRLbot
Iblog Id-search
Image Fetch Image Fetch
Image Sucker Image Sucker
IndeedBot IndeedBot
Indy Library Indy Library
InfoTekies InfoTekies
Intelliseek InternetSeer
Internet Ninja Iskanie
JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz
JennyBot JennyBot
Jetty JetCar
Jorgee Jorgee
Kenjin Spider Kenjin Spider
Keyword Density Keybot Translation-Search-Machine
Kozmosbot Kozmosbot
LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot Lanshanbot
Leap Larbin
LeechFTP
LeechGet LeechGet
Lftp Lftp
LinkWalker LibWeb
Linkbot Lightspeedsystems
LinksManager
LinqiaScrapeBot
Lipperhey
Litemage_walker Litemage_walker
Ltx71 Ltx71
MFC_Tear_Sample MQQBrowser
MSIECrawler
MTRobot MTRobot
Mag-Net
Mail.RU_Bot
Majestic SEO
MarkMonitor
MarkWatch MarkWatch
Masscan Mediatoolkitbot
Mata Hari
Meanpathbot
MicroMessenger
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Minefield
Mojeek
Mojolicious
MolokaiBot MolokaiBot
Morfeus Fucking Scanner
Mozlila Mozlila
Msrabot
Musobot
NICErsPRO NICErsPRO
NPbot
Navroad Navroad
Needle
Nessus Nessus
NetLyzer Net Vampire
NetZIP Nettrack
Nibbler NextGenSearchBot
Niki-bot Nimbostratus
Nikto Nmap
Ninja
Nutch Nutch
OnCrawl OutfoxBot
OpenLinkProfiler PageAnalyzer
OpenVAS
Openvas
OrangeBot
OutclicksBot
PECL::HTTP
PHPCrawl
PageGrabber
Pandalytics
PeoplePal PeoplePal
Pi-Monster Petalbot
Picscout Picscout
Picsearch
Pimonster Pimonster
PleaseCrawl Pockey
ProPowerBot
ProWebWalker
Probethenet
Psbot Psbot
Pu_iN QueryN Metasearch
Quick-Crawler
RSSingBot RSSingBot
RankActive RankingBot
RankActiveLinkBot
ReGet
RealDownload RealDownload
RebelMouse Reaper
Recorder
RedesScrapy
Ripper
RocketCrawler RocketCrawler
Rogerbot Rogerbot
SEOkicks-Robot SBIder
SEOlyticsCrawler SEOlyticsCrawler
SMTBot
ScanAlert ScanAlert
ScreenerBot Scrapy
Seekport ScrepyBot
Semrush SearchmetricsBot
SemanticJuice
SemrushBot SemrushBot
Seomoz SeoSiteCheckup
SiteCheckerBotCrawler Shodan
Site Sucker
SiteExplorer SiteExplorer
SiteLockSpider
SiteSnagger SiteSnagger
Sitebeam
Siteimprove Siteimprove
Snapbot SmartDownload
Snake
Snoopy Snoopy
SocialRankIOBot
Sociscraper Sociscraper
Sogou web spider
Sosospider
Sottopop Sottopop
Spammen SpankBot
Spanner Spbot
Spinn3r Sqlworm
Sqworm SuperBot
Steeler Teleport
Sucker
Sucuri
Surfbot
Suzuran
Telesoft Telesoft
Telesphoreo
Telesphorep
The Intraformant
TheNomad TheNomad
Titan Thumbor
TightTwatBot
Tracemyfile Tracemyfile
Trendictionbot Trendictionbot
True_Robot
Turnitin Turnitin
URLy Warning URLy.Warning
Voltron V-BOT
WEBDAV VelenPublicWebCrawler
VeriCiteCrawler
VoidEYE
Voil
WBSearchBot
WISENutbot WISENutbot
WPScan WPScan
WWW::Mechanize Wallpapers/3.0
WWWOFFLE
WallpapersHD WallpapersHD
WeSEE
Web Collage Web Collage
Web Pix
WebAuto
WebBandit
WebFetch
Web Fuck Web Fuck
WebImageCollector
WebLeacher
Web Pix Web Pix
WebReaper
Web Sauger Web Sauger
WebWhacker WebBandit
WebZIP WebEnhancer
Webshag WebFuck
Whacker WebGo IS
WinHTTrack WebStripper
WiseGuys Robot WebSucker
WebmasterWorldForumBot
WebsiteExtractor
Webster
Whack
Whatweb
Widow
Wonderbot
Wotbox
Wprecon Wprecon
Zitebot Xaldon_WebSpider
ZoominfoBot YoudaoBot
ZumBot Zade
Zauba
Zermelo
ZmEu
ZoomBot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt arquivo.pt
cah.io.community autoemailspider
backlink-check
check1.exe check1.exe
clark-crawler clark-crawler
coccocbot coccocbot
crawl.sogou.com crawler.feedback
dataforseo.com dataforseo.com
domainsproject.org domainsproject.org
eCatch
facebookscraper
heritrix heritrix
instabid ips-agent
internetVista monitor muhstik-scan
iubenda-radar
linkdexbot
lwp-trivial
magpie-crawler
oBot
pcBrowser
plumanalytics
polaris version polaris version
ripz s1z.ru
seocompany.store scan.lol
seostar seostar
siteripz sitechecker.pro
sogouspider sp_auditbot
sysscan sysscan
tAkeOut ubermetrics-technologies.com
voyagerx.com webmeup-crawler
webprosbot webprosbot
x09Mozilla xpymep1.exe
zauba.io zauba.io
zgrab

300
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404checker
404enemy 404enemy
AIBOT ADmantX
Abonti
Aboundex Aboundex
Aboundexbot
AhrefsBot AhrefsBot
AiHitBot
Alexibot Alexibot
AllSubmitter Alligator
Anarchy99 Anarchie
Aspiegel Anarchy
Attach Ankit
AwarioRssBot AwarioSmartBot
BBBike BackStreet
BLEXBot Badass
Backlink-Ceck Black Hole
Barkrowler
BatchFTP
BetaBot
BlackWidow BlackWidow
Blow
Boardreader Boardreader
BotALot Brandprotect
Brandwatch Brandwatch
Buck Buck
BuzzSumo Buddy
CATExplorador CATExplorador
CODE87
CSHttp CSHttp
Calculon
CazoodleBot
Cegbfeieh
CensysInspect CensysInspect
CherryPicker CheTeam
CheeseBot
ChinaClaw
Chlooe
Claritybot Claritybot
Cliqzbot Cocolyzebot
Cloud mapping
Cogentbot Cogentbot
Collector
CopyRightCheck
Cosmos
Craftbot Craftbot
Crescent Crescent
CrunchBot CrunchBot
Custo DBLBot
DSearch DSearch
DatabaseDriverMysqli
Deusu Deusu
Devil
Digincore Digincore
Disco DnBCrawler-Analytics
DomainAppender DomainStatsBot
DomainCrawler
Dotbot
Dragonfly Dragonfly
ECCP/1.0 ECCP/1.0
EMail Siphon EMail Siphon
EMail Wolf
Ecxi Ecxi
EirGrabber
Evil Evil
ExtLinksBot
ExtractorPro ExtractorPro
FemtosearchBot EyeNetIE
Foobot Ezooms
Fimap
FlashGet
Flunky
Freeuploader Freeuploader
FyberSpider FyberSpider
Fyrebot Fyrebot
GT::WWW
Genieo
GetRight GetRight
GetWeb
Getintent
Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
GrapeshotCrawler GoZilla
GrapeFX
GridBot GridBot
HEADMasterSEO HaosouSpider
HTTrack Harvest
Humanlinks Humanlinks
HybridBot Iblog
IRLbot
Id-search
IndeedBot
Indy Library
InfoNaviRobot
InfoTekies InfoTekies
Iria InterGET
InternetSeer
Iskanie Iskanie
IstellaBot JOC Web Spider
Jetty JamesBOT
Joomla Jbrofuzz
JennyBot
JetCar
JikeSpider
JustView
Jyxobot Jyxobot
Keyword Density Keyword Density
Kinza LWP::Simple
LeechFTP LeechFTP
Lightspeedsystems Lftp
Linkbot
LinkextractorPro
LinksManager LinksManager
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Litemage_walker Lipperhey
Lipperhey Spider
Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MJ12bot MIIxpc
MTRobot MTRobot
Mag-Net
Magnet
Mail.RU_Bot
Majestic SEO
Majestic-SEO
MarkMonitor
MarkWatch MarkWatch
Mata Hari
Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Moblie Safari Minefield
Msrabot Mister PiX
NICErsPRO Mojeek
Navroad Morfeus Fucking Scanner
Mozlila
NPbot
NearSite
Net Vampire
NetLyzer
NetSpider NetSpider
NetZIP
Nettrack
Netvibes Netvibes
NextGenSearchBot
Nibbler Nibbler
Niki-bot NimbleCrawler
Nimbostratus
Ninja
Nutch Nutch
Offline Explorer OpenLinkProfiler
Offline Navigator Openfind
OnCrawl
OrangeBot OrangeBot
POE-Component-Client-HTTP OrangeSpider
OutfoxBot
Page Analyzer Page Analyzer
PageScorer PageScorer
Panscient Petalbot
Picsearch Pockey
Piepmatz
Pimonster
Pixray
ProWebWalker ProWebWalker
Pump Pu_iN
QueryN Metasearch Quick-Crawler
RSSingBot
RankActive RankActive
RankActiveLinkBot RankActiveLinkBot
RankingBot RankFlex
RankingBot2 RankurBot
Rankivabot Re-re
Reaper ReGet
RebelMouse
Recorder Recorder
RedesScrapy
Ripper Ripper
SEOkicks SBIder
SEOkicks-Robot SEOprofiler
SMTBot ScanAlert
SalesIntelligent
Scanbot Scanbot
ScoutJet Scrapy
Screaming Screaming
ScreenerBot
Searchestate Searchestate
Seekport Semrush
SemanticJuice
SentiBot SentiBot
SeoSiteCheckup
SeobilityBot
Seomoz
Shodan
Siphon
SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider
Site Sucker Site Sucker
Siteimprove Sitevigil
SlySearch
SmartDownload SmartDownload
Snake
Snoopy Snoopy
SocialRankIOBot SocialRankIOBot
SpaceBison Sociscraper
Sosospider
Sottopop
SpankBot SpankBot
Steeler
Sucker
SuperBot
SuperHTTP SuperHTTP
Surfbot Surfbot
Suzuran Swiftbot
Szukacz
T0PHackTeam
T8Abot
Teleport Teleport
Telesoft TeleportPro
The Intraformant Telesphoreo
Thumbor TheNomad
TightTwatBot Tracemyfile
Trendiction Trendiction
Trendictionbot
True_Robot
Turnitin Turnitin
TurnitinBot TurnitinBot
URLy Warning TwengaBot
Twice
URLy.Warning
UnisterBot UnisterBot
V-BOT Vacuum
VB Project
VCI
VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper VidibleScraper
Voltron Virusdie
WBSearchBot Voil
WISENutbot WWW-Collector-E
WPScan WeSEE
WWWOFFLE
WallpapersHD
Web Enhancer Web Enhancer
Web Fetch
Web Fuck Web Fuck
Web Sauger
WebAuto WebAuto
WebBandit WebCopier
WebCollage
WebFuck WebFuck
WebGo IS WebLeacher
WebPix WebPix
WebStripper WebZIP
WebWhacker
Webshag
WebsiteExtractor WebsiteExtractor
WebsiteQuester Webster
WiseGuys Robot Whacker
Who.is Bot
Widow
WinHTTrack
Woobot Woobot
Xaldon WebSpider Wprecon
Xaldon_WebSpider Xaldon_WebSpider
Zade Xenu
Zauba
Zeus
ZoomBot ZoomBot
ZoominfoBot ZoominfoBot
adscanner ZumBot
ZyBorg
archive.org_bot archive.org_bot
arquivo-web-crawler
backlink-check
check1.exe check1.exe
clark-crawler
coccocbot coccocbot
cognitiveseo cognitiveseo
com.plumanalytics crawler4j
crawl.sogou.com
dataforseo.com dataforseo.com
domainsproject.org demandbase-bot
eCatch
evc-batch
facebookscraper
heritrix
instabid instabid
lwp-trivial iubenda-radar
mediawords linkdexbot
netEstate NE Crawler magpie-crawler
oBot muhstik-scan
plumanalytics page scorer
scalaj-http pcBrowser
ripz
scan.lol scan.lol
seoscanners
sexsearcher
sitechecker.pro sitechecker.pro
siteripz
sp_auditbot sp_auditbot
spyfu spyfu
sysscan trendiction.com
trendiction.de ubermetrics-technologies.com
voyagerx.com voyagerx.com
webmeup-crawler
webpros.com webpros.com
x09Mozilla
xpymep1.exe
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
belinvestdom.by cablecar.us
cheapmoncler.win canoncdriverq3.pen.io
compiko.info cfjump.com
estibot.com m292.info
littlesexdolls.com plastgranar.nu
macnewtech.com pskcijdc.bloger.index.hr
privacyassistant.net s-forum.biz
social-vestnik.ru shodanhq.com
viagraneggrx.com timer4web.com
wpsecurity.website wildcattube.com

936
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com 01apple.com
1001desert.com 101billion.com
100dollars-seo.com 202ch.com
12bet.com 2daytrendingnews.com
1bet.com 3weekdiet.com
1millionusd.xyz 76brighton.co.uk
2345.com 7fon.ru
24x7-server-support.site 7wind.ru
273-fz.ru 8558l.jobs.net
5tey7463.weebly.com 991mostfm.co.id
70casino.online 999webdesign.com
7xc4n.com aac500b7a15b2646968f6bd8c6305869d7.com
aa9bd78f328a6a41279d0fad0a88df1901.com abbanreddy.com
aad1f4acb0a373420d9b0c4202d38d94fa.com acmebtn.ml
aasoldes.fr adclickthru.net
abclauncher.com adman.se
abiente.ru adminshop.com
absolute-s.ru adobereader-free.ru
absolutelyanalog.com adrenalin-stk.ru
abwa.tk adservme.com
acads.net adsloads.com
activecampaign.dreamhosters.com agadelha.com.br
adimmix.com agardomains.com
ads-seo.men agecheckadult.com
adult-shop.com.ua alfabot.xyz
aero2.ru alibestsale.com
agapovdg.ru alienwheels.de
aksonural.ru alldezservice.kz
aktobe.xkaz.org allknow.info
alert-fjg.xyz allsilver925.co.il
alot.com ameblo.top
alvtank.se anatomiy.com
amazon-adsystem.com angigreene.com
amazon-seo-service.com animal-drawings.com
android-systems.ru animali.deagostinipassion.it
angel1777.kz ap.senai.br
aosexkontakte.net api.stathat.com
apartamentwroclaw.eu apiwoodensealcom-a.akamaihd.net
apilinkswiftco-a.akamaihd.net artdestshop.eu
appfastplay.com artpicso.com
approved.su asacopaco.tk
arabgirls.adultgalls.com auto-zapchasti.org
arabsexxxtube.com autoseo-traffic.com
arate.ru aversis.net
arcadeplayhouse.com avirasecureserver.com
arclk.net azartclub.org
arpelsreplica.xyz bablonow.ru
arraty.altervista.org bala.getenjoyment.net
artel116.ru bambi.ck.ua
asrvvv-a.akamaihd.net bang-hotties.com
auspolice.com basisches-wasser.net
aviav.ru bausparen.bz.it
azartmix.com bavsac.com
azartmsl.com bcmp.org
azbuka-mo.ru beachpics.com
azinoofficial777.ru beauty-clinic.ru
backlink4u.net berlininsl.com
balitouroffice.com bestmouthwash.club
battle.net besttorrentknifta.weebly.com
beauty-bracelet.com bioca.org
belinka.com.ua bitcoin-ua.top
berdasovivan.ru
bestdooz.com
bestwaystogetpaid.us
bet-prognoz.com
bikini-image.com
binomo.kz
biographiya.com
bitcoinremote.com
bizcheapjerseyswholesalechina.com
bjetjt.com bjetjt.com
bmsco.co bjgugu.net.ua
bochemit.com.ua bloggen.be
bolitgorlo.net bmw.afora.ru
bongacams.com
bookhome.info
bookmark4you.com
brainboostingsupplements.org
braindaily.xyz
brainxs.us brainxs.us
breget74.ru brakehawk.com
brendbutik.ru brianhenry.co
brg8.com brillianty.info
browserprotecter.com brothers-smaller.ru
burger-tycoon.com buttons-for-your-website.com
buyclomidonlaine.com buybest1.biz
buypuppies.ca buyessay3.blogspot.ru
buyviagraa.blogspot.com buytizanidineonline.blogspot.com
buzzonclick.com buzzurl.jp
bvlgariring.xyz calc-for-credit.ru
cacheimages.com callcafe.info
casablancamanor.co.za camdenmemorials.com
canoncdriverq3.pen.io
carmuffler.net
cars-modification.net
carsplate.com
cartierlovestore.com
cartierreplica.win
casite-513049.cloudaccess.net casite-513049.cloudaccess.net
cbb1smartlist12.click cbcseward.com
cellfun.mobi cfsrating.sonicwall.com
chatseo.com
chcu.net chcu.net
cheapestjerseysonwholesale.com cheapcigarettesc.info
cheapmoncler.xyz cheapwebsitehoster.com
chinaelitecheapjerseys.com cherrypointplace.ca
chocolateslim-u-srbiji.com chocolateslim-u-srbiji.com
chocoslim.pro chorus.terakeet.com
cialisovercounteratwalmartusa.com cialis-samples.com
claytransformations.info cialisndbrx.com
cleanersoft.com ciceros.co
clipartnew.com clickaider.com
coindirect.io clickcash.com
conocer-sanabria.com clknsee.com
cntravelre.com
coccoc.com
coin-hive.com
compliance-alex.xyz
compliance-alexa.xyz
compliance-irvin.xyz
compliance-josh.top
compliance-mary.top
compliance-nelson.top
compucelunlock.net
cookingmeat.ru
coolingoods.com
counter.yadro.ru counter.yadro.ru
cowblog.fr
craftinsta.ru
creatives.adbetclickin.pink creatives.adbetclickin.pink
cymbaltaandtizanidine.blogspot.com credit.co.ua
dalavia.ru creditnation.ru
daretodonate.co dailyfinancefix.com
datract.com daydream-studio.ru
dcj-nn.ru daymusam.com
delta-line.men deluxedumps.com
demian.kz dengi-pod-zalog-nedvizhimosti.ru
design-lands.ru detki-opt.ru
deutschehobbyhuren.net dfiles.me
deutschland123.de digitalassetmanagement.site
dnepr-avtospar.com.ua discountbarbour.online
domainsatcost.com discountliv.com
domik-derevne.ru djekxa.ru
dominateforex.ml dkvorota.ru
dood.live dmmspy.com
dostavka-v-ukrainu.ru doska-vsem.ru
dotmass.top dosugrostov.site
doublepimp.com dptaughtme.com
downloaddy.net drillsaw.ru
downloadkakaotalk.com druzhbany.ru
downvids.net dvrlists.com
drhomes.biz eandsgallery.com
dscaas.website eas-seo.com
earthmagic.info easyshoppermac.com
editmedios.com ecommerce-seo.org
elektronischezigarette2.mex.tl ednorxmed.com
eloconcream.blogspot.com eduserver.net
eluxer.net ekaterinburg.xrus.org
emerson-rus.ru eldiariodeguadalajara.com
enge-fotzen.info elektronischezigarettekaufen1.beeplog.com
envaseslotusama.com epngo.bz
eredijovon.com erotikstories.ru
escort-russian.com essay-edu.biz
esnm.ru
essaypro.com essaypro.com
estelight.ru eugenevaultstorage.com
excitacao.com evehemming.blogspot.com.au
expolicenciaslatam.co executehosting.com
extremez.net eyes-on-you.ga
f07.de
f0815.de
face.hostingx.eu face.hostingx.eu
facebook-mobile.xyz fanoboi.com
fakehandbags.xyz farmingworm.com
family1st.ca fdzone.org
farm26.ru
fashion.stellaconstance.co
fearcrow.com fearcrow.com
ff30236ddef1465f88547e760973d70a.com fetroshok.ru
fickenprivat.info filesmonster.porn
findthe.pet financehint.eu
fireads.men financetip.eu
fix-website-errors.com findacheaplawyers.com
flipper.top flipper.top
fodelsedagspresenter.nu forexunion.net
fortunejack.com forum.poker4life.ru
fortwosmartcar.pw forumprofi.de
free-fb-traffic.com fotopop.club
free-fbook-traffic.com fotosfotos.eu
free-share-buttons.blogspot.com free-traffic.xyz
fsalas.com freecamdollars.com
fugarif.ga freenom.link
fullfileaccess.com freshwallpapers.info
g.starmoe.xyz frustrated-favorable.gq
galaxys6manual.info frvo.alptandem.ru
gamerextra.com ftns.ru
garciniaxt.us fuck-paid-share-buttons.xyz
gay-file.com gael-s.ru
generalporn.org gambarkatabaru.com
genericviagrasildenafiled.net gamedayhouse.com
gg-arena.ru gameprimary.com
go2jump.org gamevalue7.weebly.com
goldenggames.com geileweiber.tk
gratis-sexkontakte.com go2album.com
greatzip.com gogps.me
gta-top.ru gorabagrata.ru
gwebtools.com.br graphics8.info
hannasolution.ru greatgrace.ru
hawaiisurf.com guidefs.ru
hd-filmy.net hao123.com
hd720kino.ru harmonyglen.us
hdhc.ru haveinc.xyz
hermesreplica.pw healthcarestore.info
hidemyass.com hermesbelts.xyz
hitsbox.info hidefiles.org
homik.pw hitcpm.com
hotblognetwork.com hledejvshopech.cz
hoterika.com hoholikik.club
hoverboardforsaledirect.com hornymatches.com
hreade.com housediz.com
ideawheel.com howmuchdoestizanidinecost.blogspot.com
ifirestarter.ru i-midias.net.br
igrovyeavtomaty777.ru idealtits.net
igru-xbox.net ilovevitaly.ru
iklysha.ml imadedinner.net
ilo134uloh.com immigrational.info
intellekt21.ru in-tandem.co
intermesh.net inbabes.sexushost.com
investzalog.ru infospot.pt
isitpaleo.info infowarcraft.ru
ispaniya-costa-blanca.ru insider.pro
j33x.com insurple.com
jbl-charge.info inzn.ru
jerseywholesalechinabiz.com iopeninghours.co.uk
joessmogtestonly.com ip-guide.com
johannesburgsingles.co.za iron-age.info
jpcycles.com irunfar.com
kaac.ru istanbulit.com
kakadu-interior.com.ua istizanidinelikexanax.blogspot.com
kamagragelusa.net itis4you.com
karadene.com izhevsk.xrus.org
jerseywholesalebizchina.com
justucalling32211123456789.tk
kellyonline.xyz kellyonline.xyz
khadastoafarde.tk kenaba.su
kiinomaniak.pl kerei.ru
kinky-fetishes.com keywords-monitoring-success.com
kinohall.ru kihi.gdn
kinopolet.net kinocccp.net
kinosed.net kinosed.net
kinostorm.net kiskinhouse.com
kinotorka.ru klejonka.info
klin3952.ru
kochanelli.com
kollesa.ru
komputers-best.ru komputers-best.ru
kongoultry.net konica.kz
koopilka.com koptims.tiu.ru
kovesszucs.atw.hu koronirealestate.gr
kraljeva-sutjeska.com kosynka-games.ru
kumuk.info kunstaktien.info
kurbappeal.info kurbappeal.info
kustanay.kz labafydjxa.su
labplus.ru landinez.co
landoftracking.com landmania.ru
languagecode.com laxdrills.com
lapolis.it leboard.ru
legionalpha.com ledis.top
lenvred.org legalrc.biz
lernur.net lego4x4.ru
letsrepair.in lennatin.info
letto.by letmacworkfaster.site
lignofix.ua letsart.ru
linkredirect.biz letslowbefast.site
littlesexdolls.com lexaprogeneric.link
lollypopgaming.com lidoradio.com
likrot.com
linkbux.ru
liran-locks.com
liveinternet.ro
livingroomdecor.info
louboutinbooties.xyz
lubetube.com lubetube.com
luckyshop.net.ua lumb.co
magazin-pics.ru lunamedia.co
makis.nu m0r0zk0-krava.ru
manifestation.betteroffers.review madot.onlinewebshop.net
manve.info maju.bluesalt.co
marcogrup.com manifestyourmillion.com
marketingtechniques.info masterseek.com
massage-info.nl masthopehomes.com
masserect.com meble-bogart.info
medicinacom.ru medicinacom.ru
medkritika.ru medkritika.ru
meduza-consult.ru memberty.com
metascephe.com mensandals.xyz
metallosajding.ru
microsearch.ru microsearch.ru
mir-business-24.ru
mirobuvi.com.ua mirobuvi.com.ua
misandesign.se misandesign.se
mixtapetorrent.com missis.top
mmgq.ru mlvc4zzw.space
mobioffertrck.com mnogabukaff.net
monetizer.com-01.site mobplayer.ru
monthlywinners.com mojpreskumanie.com
montredemarque.nl moncleronline.xyz
most.gov.iq moyakuhnia.ru
mouselink.co msk-diplomat.com
mrbitsandbytes.com multgo.ru
mrcsa.com.au mwtpludn.review
mrpornogratis.xxx myftpupload.com
mtmtv.info mygameplus.com
mundoaberrante.com myra.top
muznachas-service.ru mysexpics.ru
my-floor.in.ua nakrutka.cc
myhitmp3.club
myonlinepayday.co
namecrumilchlet.tk namecrumilchlet.tk
narkologiya-belgorod.ru nastydollars.com
nash-krym.info nero-us.com
natali-forex.com netfacet.net
negral.pluto.ro nfljerseyscheapbiz.us
netoil.no niki-mlt.ru
news-speaker.com njpalletremoval.com
nextbackgroundcheck.gq northfacestore.online
nextconseil.com nottyu.xyz
nfvsz.com novatech.vn
nippon-bearings.ru nvformula.ru
nofreezingmac.work oakridgemo.com
novodigs.com obuv-kupit.ru
ntic.fr oda.as
nullrefer.com
nyfinance.ml
nzfilecloud.weebly.com
o-o-8-o-o.com
offside2.5v.pl offside2.5v.pl
olympescort.com ok-ua.info
omsk.xrus.org ok.ru
onclickpredictiv.com oklogistic.ru
online-templatestore.com oneminutesite.it
orhonit.com ooomeru.ru
ozoz.it openmultipleurl.com
partner-trustworthy.men otbelivanie-zubov.com
pattersonsweb.com owohho.com
p-business.ru
panicatack.com
pavlodar.xkaz.org
pc-services.ru
pearlisland.ru pearlisland.ru
percin.biz.ly pierrehardysale.online
petedrummond.com piulatte.cz
picturesify.com plastweb.ru
pills24h.com playfortuna-play.ru
piluli.info polimga.pw
pisanieprac.info pons-presse.com
pizza-tycoon.com popugaychiki.com
pl-vouchers.com pornolook.net
platezhka.net pornosemki.info
ploenjitmedia.azurewebsites.net pornoslive.net
pobeiranie.pl powc.r.ca.d.sendibm2.com
popserve.adscpm.net pravoholding.ru
poptool.net printdirectforless.com
porn555.com privatefx.all4invest.info
pornogad.com prodvigator.ua
pornohd1080.online pronorm.fr
pornpost.in proxyradar.com
portadd.men pxhdwsm.com
portal-eu.ru quangcaons.com
praisong.net queerspace.com
projefrio.com.br quelle.ru
promoforum.ru quickcashlimited.com
proxyelite.biz ranksonic.com
rabotaetvse.ru rapevideosmovies.com
rankingchart.de real-time-analytics.com
realresultslist.com recipedays.com
redbottomheels.xyz reeyanaturopathy.com
registry-clean-up.net registry-clean-up.net
reklama-i-rabota.ru remont-komputerov-notebook.ru
renhacklids.tk remote-dba.de
reyel1985.webnode.fr renecaovilla.online
riralmolamsaca.tk rennlist.com
rockingclicks.com rentaremotecomputer.com
roma-kukareku.livejournal.com rentehno.ru
rosbalt.com.ua resellerclub.com
rossanasaavedra.net ros-ctm.ru
ruclicks.com rumamba.com
running-line.ru ruscoininvest.company
runtnc.net russia-today-video.ru
rusprostitute.com s.lollypopgaming.com
rustic-quiver.win sadaholding.com
rutor.vip saddiechoua.com
sammyweaver.com satellite.maps.ilovevitaly.com
samo-soznanie.ru scanhub.ru
sarf3omlat.com scanmyphones.com
saugatuck.com scanner-andrew.top
sbwealthsolutions.ca scanner-barak.top
scanner-don.top scanner-donald.top
scanner-olga.top scanner-george.top
scanner-viktor.top scanner-julianna.top
schalke04fc.info scanner-willy.top
searchengineranker.email scenicmissouri.us
seccioncontrabajo.com scmor.ilxc.cc
segol.tv sdi-pme.com
search.alot.com
searchimage.co
searchmywindow-a.akamaihd.net
secret.xn--oogle-wmc.com
security60-e.com
seeingmeerkat.com
seinterface.com seinterface.com
seocdvig.ru seminarygeorgia59.ga
serdcenebolit.com seomarketings.online
service-core.ru sex-watch.com
sexspornotub.com shell-pmr.ru
sextracker.be shodanhq.com
shareyards.com shop-electron.ru
shmetall.com.ua sibtest.ru
shopwme.ru site-speed-checker.site
signx.info sk.golden-praga.ru
siha.de slayerlife.com
sim-service.net slonechka.ru
similardeals.net slowmac.tech
simple-image.com.ua sluganarodu.ru
site3.free-share-buttons.com smartadserver.com
siteheart.net snip.to
siteonomy.com snjatie-geroinovoy-lomki.ru
sitevalued.com soc-econom-problems.ru
sixcooler.de social-search.me
skuteczna-dieta.co.pl
skyway24.ru
sledstvie-veli.net
snegozaderzhatel.ru
social-buttons.xyz
socialbookmarksubmission.org
socialseet.ru
soheavyblog.com
soviet-portal.do.am
spacash.com
space2019.top
splendorsearch-a.akamaihd.net splendorsearch-a.akamaihd.net
sports-supplements.us
spy-sts.com spy-sts.com
spylog.com squidoo.com
ssconstruction.co steamoff.net
start.myplaycity.com steebook.com
staticfs.host sterva.cc
stroicol.net strag-invest.ru
stroiminsk.com streamin.to
subj.ukr-lit.com streetfire.net
success-seo.com support.nopeas.sk
sukirgenk.dvrlists.com tacbibirfa.tk
summerlinhomes411.info tamada69.com
surfbuyermac.com tandvardshuset.net
svetodiodoff.ru thenetinfo.com
taaaak.com thesmartsearch.net
tds-advert005.info titelhelden.eu
tetracsaudi.com tizanidineinteractions.blogspot.com
thaisamkok.com tizanidinesideeffects.blogspot.com
theallgirlarcade.com tkanorganizma.ru
thebluenoodle.com top-study.work
thefarmergame.com topflownews.com
thin.me.pn topsiteminecraft.com
tilido.com torrent-to-magnet.com
titan-ads.life tracklead.net
tivolibasket.it trafficgenius.xyz
tizanidinehcl2mg.blogspot.com trahic.ru
tizanidinehcl2mgtablet.blogspot.com trion.od.ua
tnaionline.org truck-land.life
topsy.com trudogolik.net
toy-shop.top tversvet.ru
tracksz.co twodollarshows.com
tradedeals.biz um-razum.ru
traffmonster.info univerfiles.com
trailer.cinemaflix.website unrealcommander.biz
trainoffend.ml uptime-gamma.net
triplepanda.xyz us-america.ru
tuberkulezanet.ru uvozdeckych.info
tvand.ru
tvnewsclips.info
twittruth.com
twu.com.ua
upstore.me
urldelivery.com
valkiria-tk.ru valkiria-tk.ru
vandrie-ict.nl
varbergsvind.se
varikozdok.ru
vashsvet.com vashsvet.com
vchulkah.net vbabule.net
ventelnos.com vertaform.com
via-energy-acquistare.com video--production.com
viagengrarx.com video-production.com
videochat.bz videojam.tv
videochat.cafe videospornogratisx.net
videochat.mx virtualbb.com
viel.su visionwell.com.cn
viktoria-center.ru voditeltrezviy.ru
villacoloniale.com voloslove.ru
vipromoffers.com vzlomfb.com
visa-china.ru
vita.com.hr
vizag.kharkov.ua
vladhistory.com
voronezh.xrus.org
vostoktrade.info
vzlomtw.com
w3data.co
w3javascript.com w3javascript.com
wallpaperstock.net wallpapersist.com
webenlace.com.ar webextract.profound.net
webmasters.stackexchange.com webix.me
webtherapy.ru webjam.com
whatsappdownload10.com weekes.biz.tc
whipme.yopoint.in welck.octopis.com
wieseversa.no westen-z.life
wikes.20fr.com wholesalejerseysgaa.com
wordkeyhelper.com wholinkstome.com
worldoffiles.ru wildnatureimages.com
wosik-dach.service-for-web.de win-spy.com
www.arenda-yeisk.ru wladimirpayen.com
x-porno.video womensplay.net
xchangetrak.com woodyguthrie.se
xedserver.com woweb.com.ua
xingzi-vision.com writingservices17.blogspot.ru
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai wsgames.ru
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai www.timer4web.com
xn----7sbifcamovvfggw9d.xn--p1ai x-lime.net
xn--80aafb2a.xn--p1ai xbaboon.com
xn--80aeahghtf8ac5i.xn--p1ai xblognetwork.com
xn--80ajbshivpvn2i.xn--p1ai xn--80ak6aa92e.com
xtrafficplus.com xn--80aodinpgi.xn--p1ai
xtubeporno.net xn--b1agm2d.net
xwatt.ru xolodremont.ru
xxxasianporn.net xxart.ru
yellowstonevisitortours.com xxxsiterips.xyz
youbrainboost.asia y8games-free.com
youporn-ru.com yogamatsexpert.com
yourporngay.com yuweng.info
yourtemplatefinder.com zambini.ru
zaimhelp.ru zaobao.com.sg
zajm-zalog-krasnodar.ru zapiszto.pl
zdesformula.ru zed21.net
zhorapankratov7.blogspot.com
zirondelli.it zirondelli.it
zixizop.net.ru znakomstvaonlain.ru
znakomstva-moskva77.ru zoodrawings.com
znaturaloriginal.com zophim.me
zskdla.site zrizvtrnpale.tk
zverokruh-shop.cz

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
214jaluwobapef.cf 108shot.com
55wmz.ru 123cha.com
5elementov.ru 2020iscoming.info
8xv8.com 76brighton.co.uk
aa625d84f1587749c1ab011d6f269f7d64.com 999webdesign.com
aaa38852e886ac4af1a3cff9b47cab6272.com a.pr-cy.ru
abtasty.com aa08daf7e13b6345e09e92f771507fa5f4.com
activecampaign.dreamhosters.com abcguru.xyz
adelly.bg above.com
adidas.frwebs.fr academiacsmendoza.org
adsref.men
advokat-grodno.by advokat-grodno.by
agysya.ru ahhjf.com
allblogroll.com aksonural.ru
alldezservice.kz alibestsale.com
anal-acrobats.hol.es alphahoverboards.com
animaltoplist.com angkortours.vn
animenime.ru apelsinnik.website
aquarium-pro.ru apiallgeniusinfo-a.akamaihd.net
arabseyes.com app5.letmacworkfaster.world
arclk.net apple.com-cleaner.systems
arendatora.ru arquivo.pt
arendovalka.xyz astana.xxxkaz.org
asiavirtualsolutions.com availit.weebly.com
atlasvkusov.ru avirasecureserver.com
babyguns.ru avtoarenda.by
balans.shahterworld.org balkanfarma.ru
battle.net battleforupdating.site
besttorrentknifta.weebly.com bausparen.bz.it
betterscooter.com beachtoday.ru
biocn.dx.am beetpics.pw
bloglag.com bestfortraders.com
bolezniorganov.ru besthoro.ru
boole.onlinewebshop.net bestserials.com
bscodecs.com betshuckclean.com
budpost.com.ua bget.ru
buy-cheap-pills-order-online.com bitcoins-live.ru
buynorxx.com blog.koreadaily.com
buzzurl.jp blogos.kz
cdncash.com blpmovies.com
cfjump.com boanonihaca.tk
chatroulette.life bpro1.top
cheap-trusted-backlinks.com brainzpod.win
cheapmoncler.pw brakehawk.com
cialischstgerts.com brk-rti.ru
clean-virus-mac.com buy-meds24.com
clickhype.com buzzzg.men
clips.ua.ac.be canacopegdl.com
cognitiveseo.com cardsharp1.ru
compliance-irvin.xyz carrauterie.be
converse.ddsoldes.fr cartierlove.xyz
coolingoods.com cbb1smartlist12.click
coprofam.org celebsopics.com
counciltally.com chelnytruck.ru
customcollegeessays.net chelyabinsk.xrus.org
customsua.com.ua chinajerseyswholesalecoupons.com
cutalltheshit.com cialisndbrx.com
danschawbel.com cityadspix.com
dfiles.me compliance-alex.xyz
dfwu1013.info compliance-alexa.top
digest-project.ru compliance-barak.xyz
divatraffic.com compliance-julianna.top
djihispano.com compliance-mark.top
docstoc.com constantaservice.net
download-wallpaper.net dame-ns.kz
dvd-famille.com darmebel.com.ua
e-c.al data.vtc.pw
elektrozigarette2.webs.com dieswaene.com
elektrozigaretten1.tumblr.com documentsite.net
eluxer.net downloadkakaotalk.com
elvel.com.ua drev.biz
englishgamer.com drpornogratisx.xxx
enternet.ee dumuelave.xyz
erotik0049.com eachdayisagift.review
everflora.ru earl-brown.info
ezigarettekaufen2.mpbloggar.se edudocs.net
ezigaretteshop2.vefblog.net electronicadirect.com
ezofest.sk elite-sex-finders.com
fanrto.com emergencyneeds.org
filmi-v.online enginebay.ru
free-fb-traffic.com enpolis.ru
freshberry.com.ua enthuse.computernetworksonline.com
freshmac.space eu-cookie-law.info
gameonasia.com europeanwatches.ru
generalporn.org evehemming.blogspot.com.au
girlsfuckdick.com ezb.elvenmachine.com
gogalleryawesome.com ezigarettekaufen.myblog.de
gorodservis.ru ezigarettekaufen2.yolasite.com
gratuitbaise.com fanoboi.com
growmyfunds.ca financehint.eu
gsasearchengineranker.space financepoints.eu
hacktougroup.ru finuse.com
hamacapty.com flprog.com
hanink.biz.ly fm-upgrade.ru
hdimagelib.com forum20.smailik.org
healgastro.com freelotto.com
hermesbelts.xyz freewhatsappload.com
holidaypics.org friendflnder.com
homedecoguide.info galleryawesome.com
homedecorpicture.us golaya.pw
igru-xbox.net googleseo.com.tr
iklysha.ml greenidesign.co
images-graphics-pics.com grtyi.com
immobiliaremassaro.com gta-club.ru
int.search.tb.ask.com gta-top.ru
iphantom.com guiadeserraazul.com
isotoner.com habermetre.com
istock-mebel.ru hermesbirkinhandbagoutlets.com
jbl-charge.info hotel-mkad.ru
joyceblog.top howmuchdoestizanidinecost.blogspot.com
kicknights.gq iamsport.org
id-forex.com
ilbe.club
imagine-ex.co
img.wallpaperstock.net
infobabki.ru
instasexyblog.com
intellektmedia.at
isitwp.com
itag.pw
japfm.com
javlibrary.cc
jmat.cn
jyvopys.com
kamen-e.ru
kartiniresto.com
kchaxton.com
ketrzyn.pl
kino-key.info kino-key.info
kino2018.cc kinogonew.ru
kit-opt.ru kinostorm.net
klumba55.ru kostenlos-sexvideos.com
kozhniebolezni.com krasula.pp.ua
kskjco.club kreativperlen.ch
laudit.ru letslowbefast.site
lexaprogeneric.link lexaprogeneric.link
lezbiyanki.net lirunet.ru
libertybilisim.com liveinternet.ro
liquimondo.com lob.com.ru
littlesunraiser.com loginduepunti.it
lostaruban.com lol-smurfs.com
mall.uk lordzfilmz.me
lostfilm-online.ru
louboutinbooties.xyz
lutherstable.org
m0r0zk0-krava.ru
mac-shield.com
magento-crew.net
make-money-online.com
mansparskats.com mansparskats.com
manyresultshub-a.akamaihd.net maranbrinfo.com.br
marblestyle.ru massage-info.nl
masthopehomes.com max-p.men
mistr-x.org merryhouse.co.uk
mncrftpcs.com mirzonru.net
moi-glazki.ru mitrasound.ru
morlat.altervista.org mmoguider.ru
mp3ringtone.info moviemail-online.co.uk
mvpicton.co.uk msfsaar.de
narco24.me musflashtv.com
narkologiya-voronezh.ru musicpro.monster
narosty.com myhealthcare.com
nationalbreakdown.com myliveblog.ru
net-radar.com naughtyconnect.com
nfljerseysforsalewholesaler.com
nomuos.it nomuos.it
noumeda.com norththeface.store
nucia.biz.ly obiavo.in
nullrefer.com offtime.ru
obiavo.net ons-add.men
odesproperty.com partner-host.men
ofanda.com pb-dv.ru
okeinfo.online picsearch.com
oxotl.com platinumdeals.gr
parajumpersjakkesalgnorge.info play-movie.pl
playlott.com playlott.com
porndairy.in plohaya-kreditnaya-istoriya.ru
pornoforadult.com polybuild.ru
pornosok.ru porn-w.org
producm.ru pornohd1080.online
progonrumarket.ru pornoklad.ru
putevka24.ru privetsochi.ru
quickcashlimited.com proekt-gaz.ru
rankexperience.com rankscanner.com
ranksonic.info rating-casino2021.ru
real-time-analytics.com realizmobi.com
redbottomheels.xyz relatodelpresente.com.ar
reklama-i-rabota.ru resant.ru
relayblog.com resultshub-a.akamaihd.net
rep-am.com
research.ifmo.ru
rfserial.net
rixpix.ru rixpix.ru
rocketchange.ru russintv.fr
rome2rio.com screpy.com
rubbed.us searchencrypt.com
sakhboard.ru servethis.com
sayyoethe.blogspot.co.za sex-foto.pw
sbprabooks.com sexs-foto.top
scanner-barak.top shitmovs.com
scanner-irvin.top silverage.ru
se-welding.ru siteonomy.com
securesmrt-dt.com skanninge.se
seminarykansas904.ml sklad-24.ru
semp.net soheavyblog.com
seogadget.ru spinazdrav.ru
seoholding.com starwars.wikia.com
seorankinglinks.us strongsignal-a.akamaihd.net
service-core.ru success-seo.com
servingnotice.com svetodiodoff.ru
serw.clicksor.com t3chtonic.com
sex-watch.com thedownloadfreeonlinegames.blogspot.com
sexcamamateurchat.com tizanidine4mgstreetprice.blogspot.com
sexkrasivo.net tizanidineandcipro.blogspot.com
shama-rc.net tizanidineformigraines.blogspot.com
shariki-zuma-lines.ru tizanidinestreetvalue.blogspot.com
sindragosa.comxa.com top-l2.com
sisi-go.ru topflownews.com
smartadserver.com topquality.cf
socas.pluto.ro torrentred.games
social-button.xyz toyota.7zap.com
socialmonkee.com tramadolandtizanidine.blogspot.com
softxaker.ru trustedmaccleaner.com
soietvousmaime.fr tt-ipd.info
spb-plitka.ru tupper-posuda.ru
steamoff.net tversvet.ru
steebook.com twodollarshows.com
streetfire.net uk-zheu20.ru
suture.co ukr-lit.com
swimpool.ca upproar.com
talant-factory.ru uptime.com
thebitcoincode.com uptimebot.net
theguardlan.com uroffer.link
thenews-today.info vellings.info
ticketsys.inetwd.com verymes.xyz
tizanidineingredients.blogspot.com visa-china.ru
top250movies.ru visitcambridge.org
topbestgames.com vizag.kharkov.ua
track112.site volgograd.xrus.org
triumf-realty.ru voloslove.ru
turist-strani.ru vseigry.fun
tvory.predmety.in.ua watsonrealtycorp.com
ukrobstep.com webs.com
universals.com.ua wladimirpayen.com
vancleefreplica.pw woman-h.ru
vitanail.ru wrz0iuebwhp5fg.freeddns.com
vmnmvzsmn.over-blog.com www.pinnacle-bets.com
vod.com.ua x-stars.ru
vostoktrade.info xaijo.com
vroze.com xclicks.net
vvpg.ru xn----dtbndd4ae7eub.top
wallpaperaccess.com xxxnatelefon.ru
wallpapersist.com zapatosenventa.info
watchdogs-2.ru zocaparj.kz
watchmygf.net zoodrawings.com
webads.co.nz
webcamtalk.net
weburok.com
wedding0venues.tk
wejdz-tu.pl
whatistizanidinehclusedfor.blogspot.com
wheelchairliftsarea.com
wildnatureimages.com
withstandingheartwarming.com
wosik-dach.service-for-web.de
www.888.com
xn----ctbigni3aj4h.xn--p1ai
xn--b1agm2d.net
xn--d1abj0abs9d.in.ua
yebocasino.com
yginekologa.com
yourdesires.ru
zeroredirect10.com
zynax.ua

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.41 0; 10.1.1.126 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.07.3234 ### Version: V4.2022.07.3235
### Updated: Mon Jul 18 10:03:25 UTC 2022 ### Updated: Mon Jul 18 22:01:13 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.07.3234 ### Version: V4.2022.07.3235
### Updated: Mon Jul 18 10:03:25 UTC 2022 ### Updated: Mon Jul 18 22:01:13 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##