vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.02.3567 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-02-03 22:07:13 +00:00
parent baf2563be7
commit 0f2f80f975
47 changed files with 101546 additions and 101539 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.02.3566 #### Version: V4.2023.02.3567
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.02.3566 #### Version: V4.2023.02.3567
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2023.02.3566 #### Version: V4.2023.02.3567
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12546
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2023.02.3566 #### Version: V4.2023.02.3567
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12550
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12550
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12550
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12550
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 1; 10.1.0.130 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 0; 10.1.0.130 0;
127.0.0.1 0; 127.0.0.1 0;

12550
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 1; 10.1.0.130 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 0; 10.1.0.130 0;
127.0.0.1 0; 127.0.0.1 0;

12550
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 1; 10.1.0.130 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 0; 10.1.0.130 0;
127.0.0.1 0; 127.0.0.1 0;

12550
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3566 3567

12550
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12550
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.88.61.211 0; 40.77.123.114 0;

12550
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 1; 10.1.0.130 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 0; 10.1.0.130 0;
127.0.0.1 0; 127.0.0.1 0;

12550
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 1; 10.1.0.130 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 0; 10.1.0.130 0;
127.0.0.1 0; 127.0.0.1 0;

12550
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 1; 10.1.0.130 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 0; 10.1.0.130 0;
127.0.0.1 0; 127.0.0.1 0;

12550
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.88.61.211 0; 40.77.123.114 0;

12550
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.88.61.211 0; 40.77.123.114 0;

12550
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 1; 10.1.0.130 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AiHitBot CensysInspect
Apexoo Cloud mapping
EasyDL Go-Ahead-Got-It
GetRight MSFrontPage
Picscout Mister PiX
Sottopop NearSite
VCI Page Analyzer
WebSucker Sucker
plumanalytics Titan
scalaj-http WebImageCollector

304
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
AIBOT 01h4x.com
Abonti 404enemy
Acunetix ADmantX
AiHitBot Aboundex
Alexibot
Anarchie Anarchie
Anarchy Anarchy
Anarchy99 Anarchy99
Ankit Apexoo
Anthill
Asterias Asterias
Attach AwarioRssBot
AwarioSmartBot BDFetch
BDCbot BLEXBot
BackDoorBot
BackStreet BackStreet
BacklinkCrawler Backlink-Ceck
Badass Badass
Bandit
Barkrowler Barkrowler
Battleztar Bazinga Black Hole
Bitacle
BlackWidow BlackWidow
Blow Blackboard
Boardreader Boardreader
Bolt
BotALot BotALot
Buddy Brandprotect
Bullseye CODE87
CATExplorador CSHttp
Calculon Calculon
Cegbfeieh CheTeam
CherryPicker
ChinaClaw
Chlooe
Cliqzbot Cliqzbot
Cocolyzebot Cloud mapping
Copier CopyRightCheck
Copyscape
Cosmos
Craftbot Craftbot
CrunchBot Crawling at Home Project
DIIbot CyotekWebCopy
DataCha0s Demon
Deusu
Devil
Digincore Digincore
Dirbuster DigitalPebble
Disco Discoverybot
Dispatch
DittoSpyder
DnyzBot DnyzBot
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
Domains Project Dotbot
Download Wonder
Dragonfly Dragonfly
Drip
EMail Wolf EMail Wolf
EasyDL
Ebingbong
Ecxi Ecxi
Exabot EirGrabber
Express WebPictures Express WebPictures
ExtLinksBot
Extreme Picture Finder Extreme Picture Finder
Ezooms EyeNetIE
FHscan FemtosearchBot
Fimap Flunky
Foobot Foobot
Freeuploader Fuzz
FrontPage
FyberSpider FyberSpider
Genieo Fyrebot
Gotit GT::WWW
Grabber Go!Zilla
GrabNet
GrapeFX GrapeFX
HEADMasterSEO HEADMasterSEO
HTTrack HMView
HaosouSpider HTMLparser
Harvest Haansoft
Havij Heritrix
Hloader Hloader
HonoluluBot HonoluluBot
Humanlinks Humanlinks
HybridBot
IDBTE4M IDBTE4M
IDBot
IRLbot
Iblog Iblog
Id-search
Image Sucker
Indy Library Indy Library
InfoNaviRobot
Intelliseek
InterGET InterGET
InternetSeer InternetSeer
IstellaBot IstellaBot
Jbrofuzz JennyBot
JetCar
JikeSpider JikeSpider
JustView
Keyword Density Keyword Density
Kinza Larbin
Kozmosbot LibWeb
LWP::Simple LinkWalker
LexiBot
LinkextractorPro LinkextractorPro
LinkpadBot LinkpadBot
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey Lipperhey
Litemage_walker Litemage_walker
MIDown tool Ltx71
MFC_Tear_Sample
MIIxpc
MJ12bot MJ12bot
MTRobot
Mag-Net
Mail.RU_Bot Mail.RU_Bot
MarkWatch Majestic-SEO
Masscan MarkMonitor
Mata Hari Mass Downloader
MauiBot
Mb2345Browser
MeanPath Bot MeanPath Bot
MicroMessenger Metauri
Mister PiX Microsoft Data Access
Moblie Safari Mojeek
Mr.4x3
Musobot Musobot
NICErsPRO
NPbot
Name Intelligence Name Intelligence
Nameprotect
Navroad Navroad
NearSite
Needle Needle
NetAnts Nessus
NetLyzer Net Vampire
NetSpider Nettrack
Netcraft Netvibes
Niki-bot Nibbler
Nikto Nikto
Nimbostratus Ninja
Nuclei Nmap
Not
Nutch Nutch
Octopus Octopus
Openfind Offline Explorer
OrangeSpider Offline Navigator
OpenLinkProfiler
PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageAnalyzer
Pandalytics Pandalytics
Papa Foto Pavuk
PeoplePal PeoplePal
Petalbot
Pi-Monster Pi-Monster
Picscout
Picsearch
Piepmatz
Pimonster Pimonster
Pixray Pockey
Probethenet Pump
Psbot
PyCurl PyCurl
Quick-Crawler
RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankingBot
RankingBot2 RankingBot2
Rankivabot Rankivabot
RankurBot ReGet
Re-re RealDownload
Reaper Reaper
RedesScrapy RebelMouse
Ripper
RocketCrawler RocketCrawler
SEOkicks SEOkicks
SEOstats SEOkicks-Robot
SISTRIX SEOlyticsCrawler
ScanAlert ScanAlert
Scrapy Scrapy
ScreenerBot SearchmetricsBot
ScrepyBot
Searchestate
Seekport Seekport
SemanticJuice
Semrush
SentiBot SentiBot
SeoSiteCheckup SeobilityBot
Seomoz Site Sucker
SiteSnagger SiteExplorer
SiteLockSpider
Sitebeam
Siteimprove Siteimprove
SlySearch Sitevigil
SmartDownload
Snapbot Snapbot
SocialRankIOBot
Sosospider
Sottopop Sottopop
SpaceBison
Spammen Spammen
SpankBot
Spanner Spanner
Spbot
Spinn3r Spinn3r
SputnikBot Sqlmap
Sqlworm Sqlworm
Sqworm
Steeler
Sucker Sucker
SuperBot Surfbot
SuperHTTP Swiftbot
Szukacz T8Abot
TightTwatBot Telesoft
Telesphoreo
TheNomad
Thumbor
Titan
Toata
Tracemyfile
Trendiction
Trendictionbot Trendictionbot
True_Robot True_Robot
TwengaBot
Typhoeus Typhoeus
URLy Warning
URLy.Warning
Upflow Upflow
V-BOT VCI
VB Project
Vagabondo Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
Voil VoidEYE
Voltron WASALive-Bot
WWW::Mechanize WBSearchBot
WWWOFFLE WEBDAV
Wallpapers WWW-Collector-E
WallpapersHD WallpapersHD
Web Collage
Web Fetch
Web Fuck
Web Pix
Web Sauger Web Sauger
Web Sucker Web Sucker
WebCollage WebEnhancer
WebCopier
WebFetch WebFetch
WebPix WebFuck
WebSauger
WebStripper WebStripper
WebWhacker WebSucker
Webalta
Webshag
Website Quester
WebsiteExtractor WebsiteExtractor
Webster
Widow
WiseGuys Robot
Woobot Woobot
YoudaoBot Xaldon_WebSpider
Zade Xenu
Zauba
Zeus
Zitebot Zitebot
ZoominfoBot ZoominfoBot
ZyBorg arquivo-web-crawler
archive.org_bot arquivo.pt
backlink-check backlink-check
coccocbot cah.io.community
clark-crawler
cognitiveseo cognitiveseo
com.plumanalytics crawl.sogou.com
crawler.feedback
demandbase-bot
domainsproject.org domainsproject.org
eCatch eCatch
evc-batch gopher
heritrix heritrix
instabid iubenda-radar
ips-agent netEstate NE Crawler
linkdexbot
magpie-crawler
mediawords
oBot
page scorer page scorer
polaris version pcBrowser
plumanalytics
probe-image-size
ripz
satoristudio.net satoristudio.net
scalaj-http
seoscanners
sexsearcher sexsearcher
sogouspider tAkeOut
spyfu trendiction.com
ubermetrics-technologies.com
webgains-bot
webmeup-crawler
webprosbot
x22Mozilla x22Mozilla

306
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider 360Spider
80legs 80legs
ALittle Client ASPSeek
Abonti
Aboundexbot Aboundexbot
AfD-Verbotsverfahren Acunetix
AhrefsBot
Alexibot
AllSubmitter AllSubmitter
Anarchie
Ankit Ankit
Anthill Anthill
Aspiegel Asterias
Atomseobot Atomseobot
Attach BBBike
AwarioRssBot BLEXBot
BDFetch
BackStreet BackStreet
BackWeb
Backlink-Ceck Backlink-Ceck
Badass BacklinkCrawler
Bandit Barkrowler
BatchFTP
Battleztar Bazinga
Bitacle Bitacle
Blackboard BlackWidow
Blow
Boardreader Boardreader
Bolt
Brandwatch Brandwatch
BuiltWith Buck
Buddy
Bullseye Bullseye
CSHttp CATExplorador
CCBot
CazoodleBot CazoodleBot
CensysInspect CheeseBot
CherryPicker
ChinaClaw ChinaClaw
Chlooe Citoid
Craftbot Copyscape
CyotekWebCopy CyotekWebCopy
DSearch
DTS Agent DTS Agent
DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Demon
Deusu Deusu
Digincore Digincore
Dirbuster DigitalPebble
DittoSpyder Disco
Discobot
Discoverybot
Dispatch
DnyzBot DnyzBot
DomainAppender DomCopBot
DomainSigmaCrawler EroCrawler
Domains Project Exabot
Dragonfly
Drip
EMail Siphon
Ecxi
Express WebPictures
ExtLinksBot
Extractor
Extreme Picture Finder Extreme Picture Finder
FemtosearchBot EyeNetIE
Ezooms
FDM
Fimap Fimap
Firefox/7.0
FlashGet
Flunky Flunky
Freeuploader
Fyrebot
G-i-g-a-b-o-t
GT::WWW GT::WWW
GalaxyBot
Genieo
GetRight GetRight
Getintent
Gigabot Gigabot
GoZilla
GrabNet GrabNet
HaosouSpider Grabber
GrapeshotCrawler
HEADMasterSEO
HTTrack
Haansoft
Harvest
Havij Havij
Heritrix
Hloader Hloader
HonoluluBot
Humanlinks Humanlinks
HybridBot HybridBot
IDBTE4M IDBTE4M
IRLbot IRLbot
Iblog
Id-search
Image Fetch
Image Sucker
IndeedBot IndeedBot
InfoNaviRobot Internet Ninja
InternetSeer InternetSeer
JetCar JOC Web Spider
Jorgee Jbrofuzz
Jetty
JustView JustView
Jyxobot Jyxobot
Kenjin Spider Kozmosbot
Keybot Translation-Search-Machine
Kinza
LWP::Simple
Lanshanbot Lanshanbot
Larbin
LeechFTP LeechFTP
LeechGet LexiBot
Libwhisker
Lightspeedsystems Lightspeedsystems
Likse Likse
LinkextractorPro
LinkpadBot
LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot Litemage_walker
Ltx71 Lmspider
MFC_Tear_Sample
MQQBrowser
MSFrontPage MSFrontPage
MarkWatch Mail.RU_Bot
Majestic-SEO
Majestic12
Mass Downloader Mass Downloader
MauiBot
MeanPath Bot MeanPath Bot
Mediatoolkitbot
Metauri Metauri
Minefield
Mojolicious Mojolicious
MolokaiBot MolokaiBot
Morfeus Fucking Scanner
Musobot
Nameprotect Nameprotect
Navroad NearSite
Nessus Needle
NetLyzer NetAnts
NetMechanic NetMechanic
NetSpider NetSpider
Netcraft
Netvibes Netvibes
Nibbler Nibbler
NimbleCrawler Nimbostratus
Ninja Ninja
Nmap
Not Not
Nuclei
Nutch Nutch
Offline Explorer OpenLinkProfiler
Offline Navigator OpenVAS
OnCrawl
Openfind Openfind
Openvas
OrangeBot OrangeBot
OutfoxBot OrangeSpider
PECL::HTTP
Page Analyzer
PageAnalyzer
PageGrabber
PageScorer
Pandalytics
Panscient Panscient
Picscout Papa Foto
Pavuk
PeoplePal
Picsearch
PictureFinder PictureFinder
Pimonster Pimonster
PleaseCrawl Pockey
ProWebWalker ProWebWalker
Psbot Probethenet
Pu_iN Pu_iN
Pump PxBroker
RSSingBot RankActive
RankActiveLinkBot RankActiveLinkBot
RankFlex RankingBot2
RankingBot
RankurBot RankurBot
ReGet Re-re
Reaper Reaper
RebelMouse Recorder
RedesScrapy RepoMonkey
Ripper SBIder
Rogerbot SEOkicks
SEOprofiler
SEOstats SEOstats
SISTRIX SISTRIX
SalesIntelligent SMTBot
ScanAlert Scanbot
Scrapy Scrapy
ScreenerBot
ScrepyBot
SearchmetricsBot
Seekport Seekport
SemanticJuice
Semrush Semrush
SentiBot SemrushBot
SeoSiteCheckup Siphon
SeobilityBot Site Sucker
Seomoz SiteCheckerBotCrawler
Shodan SiteExplorer
SiteLockSpider Sitebeam
SiteSnagger Sosospider
Siteimprove SpaceBison
SlySearch Spammen
Snapbot Spanner
SocialRankIOBot
Sociscraper
SpankBot
Spbot
Spinn3r Spinn3r
Sqlmap Sqlmap
Steeler Sqlworm
Sucuri SuperBot
SurveyBot SurveyBot
Suzuran
Swiftbot
Szukacz
T0PHackTeam
Teleport
Telesphoreo Telesphoreo
Telesphorep Thumbor
The Intraformant
TheNomad
Titan
Tracemyfile
Trendiction Trendiction
Trendictionbot
Turingos Turingos
TurnitinBot TurnitinBot
TwengaBot TwengaBot
URLy.Warning Twice
Upflow Typhoeus
V-BOT URLy Warning
Vagabondo UnisterBot
VCI
VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie Virusdie
Voltron VoidEYE
WASALive-Bot WISENutbot
WPScan WWWOFFLE
WWW::Mechanize Web Collage
Wallpapers WebCollage
WallpapersHD WebCopier
Web Auto
Web Pix
Web Sucker
WebEnhancer WebEnhancer
WebStripper WebImageCollector
WebWhacker WebLeacher
WebZIP WebPix
Webalta WebSauger
Webshag WebmasterWorldForumBot
Website Quester Website Quester
WebsiteExtractor
Webster Webster
Who.is Bot Widow
WiseGuys Robot Woobot
Wotbox Wotbox
Wprecon Wprecon
Xaldon WebSpider Xaldon_WebSpider
Zade YoudaoBot
Zauba
Zeus
ZmEu
ZoomBot ZoomBot
ZoominfoBot
ZumBot
ZyBorg
arquivo-web-crawler arquivo-web-crawler
check1.exe crawl.sogou.com
clark-crawler crawler.feedback
cognitiveseo
crawler4j
dataforseo.com
evc-batch
facebookscraper
gopher gopher
internetVista monitor iubenda-radar
linkdexbot
lwp-request lwp-request
lwp-trivial
mediawords mediawords
netEstate NE Crawler oBot
page scorer page scorer
pcBrowser pcBrowser
polaris version
satoristudio.net satoristudio.net
scan.lol scalaj-http
seocompany.store
seoscanners
seostar
sitechecker.pro
siteripz
sp_auditbot
spyfu spyfu
webmeup-crawler sysscan
webpros.com ubermetrics-technologies.com
webprosbot x22Mozilla
xpymep1.exe
zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
getresponse.com amazon-seo-service.com
ghostvisitor.com blog.koorg.ru
larose.jb2c.me blogig.org
medtherapy.ru felizporno.com
mistr-x.org iqoption.pro
moscow-region.ru onthemarch.co
owohho.com prizeestates.cricket
play.leadzupc.com regdefense.com
virtualbb.com sack.net
woweb.com.ua thebestphotos.eu

934
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
178evakuator178.ru 1001watch.com.ua
2728fb936f0.com 108shot.com
273-fz.ru 12bet.com
45en.ru 1j7740kd.website
4k-player.pl 1stat.ru
4ur.click 2015god.org
5-steps-to-start-business.com 2x2fan.ru
40cg.com
4kepics.com
5000-cotydzien.com 5000-cotydzien.com
5forex.ru 91abcw.com
76brighton.co.uk 9med.net
7fon.ru aa08daf7e13b6345e09e92f771507fa5f4.com
7minuteworkout.com aa8780bb28a1de4eb5bff33c28a218a930.com
888.com aa9bd78f328a6a41279d0fad0a88df1901.com
8si.ru adexprts.com
a.pr-cy.ru adminshop.com
aasoldes.fr ads.tremorhub.com
abcdeg.xyz adultmeetonline.info
abctoppictures.net affiliate-fr.com
acortarurl.es afslankpillen2017nl.eu
actulite.com agadelha.com.br
adbetclickin.pink agreda.pluto.ro
addray.pro
adviceforum.info
afora.ru
aihelen.net aihelen.net
all4wap.ru air-edem.ru
alltheviews.com alborzan.com
alphaforum.ru albuteroli.com
an-donut.com algarveglobal.com
animalrank.com allblogroll.com
anmysite.com alldezservice.kz
apiitsthirteende-a.akamaihd.net allforminecraft.ru
apimossnetinfo-a.akamaihd.net allpdfmags.net
apiportalmorecom-a.akamaihd.net allstatesugarbowl.org
app-ready.xyz aludecor.info
apper.de alyeskaresort.com
apple.com-webbrowsing-security.science amazon-adsystem.com
approved.su anal-acrobats.com
arcadepages.com animali.deagostinipassion.it
arenda-avtoprokat-krasnodar.ru apiwebwebgetcom-a.akamaihd.net
approvedlocal.co.za
apxeo.info
arcteryxsale.online
arewater.com arewater.com
atley.eu.pn artavenuegardenstudios.com
atmovs.com articlesdirectoryme.info
automobile-spec.com arvut.org
aviav.eu as5000.com
avtolombard-krasnodar.com atmagroup.ru
azazaporn.com autoblogger24.info
azte.ch bamo.xsl.pt
b00kmarks.com bankmib.ru
bablonow.ru barbourjackets.online
babyguns.ru
bag77.ru
bala.getenjoyment.net
bayareaaeroclub.org bayareaaeroclub.org
beachpics.com begalka.xyz
beauby.ru best-businessman.ru
becuo.com best-seo-solution.com
berrymall.ru bestplacetobuyeriacta.jw.lt
best-games.today bestwebsiteawards.com
bestdooz.com bet-prognoz.com
bestdraws.com beztuberkuleza.ru
bestfortraders.com bigcities.org
bestwebsitesawards.com bio-market.kz
bidbuy.co.kr blackle.com
bigpictures.club blackplanet.com
bikini-image.com blog.axant.it
bitcoinremote.com blogsfunky672.weebly.com
bkgr.se bmw-ark.ru
blog.koreadaily.com board.f00d.de
blogqpot.com boltalko.xyz
blogs.rediff.com bombla.org
bluesman.nu boole.onlinewebshop.net
bobba.dzaba.com bosman.pluto.ro
book-bets.com brandedleadgeneration.com
bookmaker-bet.com briomotor.co
brewdom.ru bristolhotel.com
brus-vsem.ru brokergid.ru
bum.com.ru browserprotecter.com
business-suggested.tk bumascloset.com
buqyxa.rincian.info
buypillsorderonline.com
buytizanidineonlinenoprescription.blogspot.com
buzzzg.men
bwlx.prepedu.cn bwlx.prepedu.cn
callawaygolfstore.online cablecar.us
canzoni.ru candy-glam-hp.com
capture-room.com canrioloadm.gq
carmuffler.net cartechnic.ru
carrauterie.be cartierreplica.xyz
carsdined.org cdncash.net
carstrends2015.com cerev.info
cdn.walkme.com chatroulette.si
celebsopics.com chinese-amezon.com
cellfun.mobi chlooe.com
cercacamion.it christianlouboutinreplica.win
cfacarrosserie74.com
cheap-trusted-backlinks.com
chocolateslim-original.com
christianlouboutinshoes.xyz christianlouboutinshoes.xyz
cienum.fr clicksor.com
cinemaflix.website club-samodelkin.ru
clarithromycin500mg.com
clothesforcash.com
clubfashionista.com clubfashionista.com
collegeessay19.blogspot.ru coaching-netz.info
compliance-barak.xyz codq.info
compliance-brian.top cognitiveseo.com
compliance-nelson.top compliance-checker-7.info
consorzioilmosaico.org comsysnet.com
cornomase.win connectionstrenth.com
cpajunkies.com continent-e.tv
csgo4.win cool-wedding.net
customergrowthsystems.com covadhosting.biz
czeshop.info cowlmash.com
dalavia.ru crd.clan.su
danschawbel.com creams.makeforum.eu
daretodonate.co crest3d.ru
data1.scopich.com custodieva.ru
deshevo-nedorogo.ru dawlenie.com
digilander.libero.it dekorkeramik.ru
dolohen.com deniven.1bb.ru
downloadkakaotalk.com design-ideas.info
doxysexy.com digest-project.ru
dscaas.website dikx.gdn
dyshagi.ru dildofotzen.net
easycommerce.cf djekxa.ru
eduinfosite.com dnsrsearch.com
ekosmetyki.net.pl dogoimage.com
elektrozigaretten1.webs.com domik-derevne.ru
elektrozigaretten2.yn.lt donna7753191.ru
elmifarhangi.com dumpsmania24.com
elvel.com.ua dynainbox.com
en.home-task.com e-biznes.info
escortslet.net e-commerce-seo.com
essayservicewriting.org eaplay.ru
ezigarettenkaufen1.pagina.gr edwinkonijn.com.au
ezofest.sk egvar.net
f00kclan.de elektrozigarette1.dreamwidth.org
factorynightclub.com elektrozigaretten1.tumblr.com
famix.xyz elitesportsadvisor.com
fanoboi.com erotik-kostenlos.net
fanrto.com errorfixing.space
fashion.stellaconstance.co eshop4u.jp
fba-mexico.com ezb.elvenmachine.com
fbdownloader.com falcon-images.blogspot.com
ferrotodo.com falopicm.pw
filesdatabase.net fearcrow.com
findthe.pet financepoints.eu
footbalive.org fix-website-errors.com
forum.tvmir.org flytourisme.org
food.dtu.dk
forex-indextop20.ru
forodvd.com
foxweber.com
frbizlist.com
frcls.fr frcls.fr
free-share-buttons.com free-deals.faith
freelotto.com freenom.link
frequiry.com freeseedsonline.com
freshmac.space funcrushgames.com
fuckingawesome.com gael-s.ru
funnymama.com galeon.com
fx-brokers-review.com galleryawesome.com
fyl.com.ru gamingspark.com
fym.com.ru gate5.co.za
gameonasia.com gdzkurokam.ru
gardene.ru ge0ip.net
gaygalls.net geile-lelly.eu
gemara.com gidonline.one
ghostvisitor.com glass-msk.ru
gigapeta.com
gktt.ru
glavprofit.ru
goldandcard.ru
golfresa.lucania.se golfresa.lucania.se
gosmeb.ru googlsucks.com
gox.com.ua googst2.ru
greendream.com.ua gorodservis.ru
growshop.es gotwebsite1.com
gta-top.ru gq-catalog.gq
gwebtools.com.br greatzip.com
hatedriveapart.com growboxbl.ru
hdimagelib.com growmyfunds.ca
hdseriale.pl guidefs.ru
hdxnxxtube.mobi gyffu.com
hit-kino.ru gz2.bbsoldes.fr
hitcpm.com hamacapty.com
homedecoguide.info hamilton.ca
howmuchdoestizanidinecost.blogspot.com hasfun.com
howopen.ru hd720kino.ru
hdhc.ru
hentai-manga.porn
hkladys.com
hornymatches.com
hotblognetwork.com
hotelcrocenzi.sm
housemilan.ru
howlongdoestizanidinestayinyoursystem.blogspot.com
howtowhitenteethfast.xyz howtowhitenteethfast.xyz
hunterboots.online hspline.com
hut1.ru huimin764128.com
hydropump.su hvd-store.com
idegenvezeto.eu igrovyeavtomaty777.ru
igadgetsworld.com ilmexico.com
imediadesk.com ilo134uloh.com
infektsii.com imagefinder.site
infobanks.ru images.gyffu.com
intellekt21.ru import-sales.com
interferencer.ru incanto.in.ua
invivo.hu infospot.pt
iphantom.com inome.com.ua
iqs.biz.ua int.search.tb.ask.com
itbc.kiev.ua iqupdatetmz.win
itrevolution.cf irkutsk.online-podarki.com
isitwp.com
istmira.ru
istock-mebel.ru
istripper.com
ixora.pro ixora.pro
izhevsk.xrus.org iyasimasennka.com
izhevsk.zrus.org j33x.com
jav-fetish.site janavibekken.no
jaxcube.info japfm.com
jikoman.info jasonpartington.com
jerseychinabizwholesale.us
jobmarket.com.ua
johnrobertsoninc.com johnrobertsoninc.com
journalhome.com joingames.org
justucalling32211123456789.tk joyceblog.top
jwcialislrt.com justbcause.com
jwss.cc kanimage.com
kaac.ru kazinogames.lv
kashubadesign.ru keyword-suggestions.com
kenaba.su kicknights.gq
kerch.site kino-filmi.com
kerwinandcariza.com kino2018.cc
ketrzyn.pl kinohit1.ru
keywordspay.com klejonka.info
khadastoafarde.tk kmd-pto.ru
kihi.gdn kndxbkdx.bloger.index.hr
kiinomaniak.pl komp-pomosch.ru
koleso24.com.ua konoplisemena.com
kometars.xyz konturkrasoty.ru
kupit-adenu.ru koptims.tiu.ru
kursy-ege.ru kumuk.info
kvartiry-remont.ucoz.ru labafydjxa.su
kwzf.net
labvis.host.sk labvis.host.sk
lanadelreyfans.us
lawyersinfo.org lawyersinfo.org
laxdrills.com lenvred.org
layola.biz.tc lexixxx.com
lenpipet.ru lignofix.ua
lernur.net link.ac
lflash.ru linkbux.ru
lifehacĸer.com littlecity.ch
lightinghomes.net lmrauction.com
likesdesign.com locatellicorretor.com.br
lion.misba.us lockerz.com
littlesexdolls.com loginduepunti.it
liupis.com lsp-awak-perikanan.com
liveinternet.ro luckyday.world
liverpool.gsofootball.com m292.info
livetsomudvekslingstudent.bloggersdelight.dk macresource.co.uk
loftdigital.eu
lonfon.xyz
lovelycraftyhome.com
lutherstable.org
lyrster.com
m1media.net
m4ever.net
mafa-free.com
manifestyourmillion.com manifestyourmillion.com
maranbrinfo.com.br manualterap.roleforum.ru
marblestyle.ru mararoom.ru
marcoislandvacations.net masterseek.com
mebeldekor.com.ua max-eclat.men
mediafresh.online mecash.ru
meltwater.com medanestesia.ru
member-quiz.com medictube.ru
mielec.pl meetingrainstorm.bid
mikrobiologies.ru mejoresfotos.eu
milblueprint.com menstennisforums.com
metarip.ru
mhtr.be
microsearch.ru
millionare.com millionare.com
minecraft-rus.org mindbox.co.za
miniads.ca minegam.com
mmstat.com misterjtbarbers.com
mncrftpcs.com mobplayer.net
mobifunapp.weebly.com
mobilemedia.md
mobilierland.com
mojaocena.com
moncleroutletonline.pw moncleroutletonline.pw
montredemarque.nl moncleroutletonline.win
morepoweronmac.trade moneyviking-a.akamaihd.net
moroccosurfadventures.com mrmoneymustache.com
mp3s.club mygameplus.com
mrbitsandbytes.com mymercy.info
mtmtv.info mzdish.site
musezone.su na15.ru
musflashtv.com nakrutka.cc
music.utrolive.ru nbsproject.ru
muz-tracker.net nippon-bearings.ru
myblogregistercm.tk nsatc.net
myhitmp3.club o-o-6-o-o.ru
myra.top obuv-kupit.ru
nacap.ru ochistka-stokov.ru
nakozhe.com oconto.ru
naturalpharm.com.ua onblastblog.online
neobux-bg.info onclkads.com
nerudlogistik.ru onload.pw
nina.az
no-rx.info
nonameread45.live
nottyu.xyz
nzfilecloud.weebly.com
o-o-11-o-o.com
obiavo.com
ofanda.com
offers.bycontext.com
offf.info
olvanto.ru
onescreen.cc
onetravelguides.com
onlinadverts.com
online.ktc45.ru
online7777.com
onlyforemont.ru
ooo-gotovie.ru
openfrost.com
ops.picscout.com ops.picscout.com
orion-code-access.net oralsexfilme.net
osoznanie-narkotikam.net orgasmatrix.com
parlament.biz oxford-book.com.ua
pendelprognos.se parajumpersoutlet.online
perm-profnastil.ru pathwhelp.org
perosan.com pcads.ru
picsforkeywordsuggestion.com pdn-4.com
picturesboss.com pechikamini.ru
piluli.info photo-clip.ru
pinstake.com photorepair.ru
pinup-casino1.ru piccdata.com
piski.top picphotos.net
picturesfrom.com
pijoto.net
pillscheap24h.com
piter.xrus.org piter.xrus.org
pizdeishn.com
pk-pomosch.ru pk-pomosch.ru
playlott.com plohaya-kreditnaya-istoriya.ru
pod-muzyku.club pod-muzyku.club
poptool.net pontiacsolstice.info
poquoson.org porn-w.org
pornobrazzers.biz porndroids.com
pornoklad.ru porngalleries.top
posible.net porno-dojki.net
pretty-mart.com porno-transsexuals.ru
pornogad.com
pornokajf.com
pornoplen.com
porodasobak.net
pr-ten.de
printie.com printie.com
privacyassistant.net privatamateure.com
produkto.net privatbank46.ru
proekt-gaz.ru prohoster.info
prom23.ru projectforte.ru
promodj.com promover.org
prosmibank.ru
prospekt-st.ru
prostitutki-almata.org
prostitutki-novgoroda.org
prostitutki-tyumeni.org prostitutki-tyumeni.org
putevka24.ru purplestats.com
py100.ru
qld10000.net
qualitymarketzone.com
quickloanbank.com quickloanbank.com
rada.ru rankings-analytics.com
razyboard.com ranksays.com
replica-watch.ru ranksonic.com
resultshub-a.akamaihd.net rczhan.com
rixpix.ru rebelmouse.com
rock-cafe.info remontvsamare.su
ru-dety.ru renecaovilla.online
rubanners.com rezeptiblud.ru
rusenvironmental.net rff-cfal.info
rustic-quiver.win ricorsogiustizia.org
rockma.se
royalads.net
rrutw.com
rubbed.us
rucrypt.com
ruscoininvest.company
ruscopybook.com
rvi.biz rvi.biz
s-forum.biz sady-urala.ru
sale-japan.com salmonfishingsacramentoriver.com
samolet.fr sandhillsonline.com
sampleletters.net saneitconsulting.com
sbf441.com sankt-peterburg.nodup.ru
sbprabooks.com sarf3omlat.com
scanner-jack.top scanner-alexa.top
scanner-olga.top scanner-jane.top
scanner-walter.top scanner-mark.top
scanspyware.net scenicmissouri.us
schlampen-treffen.com sdi-pme.com
scopich.com
sdelai-prosto.ru
search.xtconnect.com search.xtconnect.com
searchimpression.com searchaddis.com
semaltmedia.com searchmywindow-a.akamaihd.net
seoexperimenty.ru seccioncontrabajo.com
seorankinglinks.us seeresultshub-a.akamaihd.net
seorankinglinks.xyz seojokes.net
sergiorossistore.online seoprofiler.com
servicecenter.co.ua serialsway.ucoz.ru
sex-sex-sex5.com sex-watch.com
sex-tracker.com sexxdate.net
sexblog.pw sexy-pings.com
sexiporno.net shijian.ac.cn
sexkontakte-seite.com shillyourcoins.com
sexsaoy.com shmetall.com.ua
sexy-screen-savers.com shohanb.com
sexystrippe.info shopsellcardsdumps.com
sfj-ror.no silverage.ru
shemale-sex.net silvermature.net
shiksabd.com sinestesia.host.sk
shitmovs.com site3.free-share-buttons.com
shitting.pro sitechecker.pro
si-unique.com slkrm.ru
sibecoprom.ru slonechka.ru
signoredom.com sluganarodu.ru
sildenafil-tadalafil.info smokewithrabbits.com
simon3.ru soblaznu.net
site-speed-check.site softtor.com
site.ru spaceshipad.com
sizeplus.work spravka-medosmotr.ru
sklad-24.ru spymac.net
skuteczna-dieta.co.pl
skyway24.ru
slavia.info
slomm.ru
slowmac.tech
social-search.me
socialbookmarksubmission.org
softomix.com
songplanet.ru
sortthemesitesby.com
sotechco.co
spasswelt.xyz
splendorsearch-a.akamaihd.net
spywareit.com spywareit.com
stats-public.grammarly.io sterva.cc
steame.ru stromerrealty.com
strana-krasoty.ru stroyhelp-dv.ru
studentguide.ru svbur.ru
studiofaca.com svensk-poesi.com
studiokamyk.com.pl svetlotorg.ru
sucsesofinspiration.com sweepstakes.rewardit.com
suggest-keywords.com
surffoundation.nl
tacbelarus.ru
tam-gde-more.ru
taqplayer.info taqplayer.info
tattooha.com tech4master.com
tds-advert005.info tehngr.ru
the-usa-games.blogspot.com tengohydar.tk
thegamerznetwork.com tfxiq.com
thin.me.pn thebluffs.com
thedownloadfromwarez.blogspot.com
theendivechronicles.com
theprofitsmaker.net
threecolumnblogger.com threecolumnblogger.com
tizanidine4mg.blogspot.com timdreby.com
tizanidine4mgstreetvalue.blogspot.com titangel-vietnam.com
tizanidine4mgtablets.blogspot.com tizanidinecost.blogspot.com
tizanidinehcl4mgtab.blogspot.com tizanidinehcl4mgisitanarcotic.blogspot.com
tizanidinehydrochloride2mg.blogspot.com tizanidinezanaflex.blogspot.com
tizanidinesideeffectsweightloss.blogspot.com topcasinoratings.ru
tobeyouday.win topsiteminecraft.com
toys.erolove.in torrents.cd
traffic100.com totu.us
traffmonster.info trafficstars.com
trkur.com traxdom.ru
tula.mdverey.ru trustedhealthtips.com
twittruth.com turizm.bz
tytoona.com twinderbella.com
u-cheats.ru typimga.pw
ua-company.ru ugogo.info
uasb.ru umityangin.net
ublaze.ru unlimitdocs.net
um-razum.ru uprour.com
urll.eu uptimebot.net
urlopener.com us-america.ru
uroffer.link usadacha.net
v-doc.co vashsvet.com
valmetrundan.se vavilone.com
vandrie-ict.nl vektorpress.ru
vbabule.net vepad.com
vbtracker.net vgoloveboli.net
vduplo.ru viagra-soft.ru
via-energy-order.com video-chat.in
viberdownload10.com videtubs.pl
videos-for-your-business.com vietimgy.pw
vipms.ru
virtuagirl.com
viteonlusarezzo.it viteonlusarezzo.it
voronezh.xrus.org vkonche.com
vroze.com voloo.ru
watchinf.com voltrknc1.com
watchmyfb.pl vrotike.ru
wdfdocando.com wallpapersist.com
webads.co.nz webmasters.stackexchange.com
webadvance.club webnode.me
westvilletowingservices.co.za weboptimizes.com
whengirlsgowild.com webpromotion.ae
whereverdesperate.gq webscouter.net
white-truck.life website-analyzer.info
wildworld.site weburlopener.com
whatistizanidinehclusedfor.blogspot.com
wikes.20fr.com
wildnatureimages.com
wiosenny-bon-1500.pl wiosenny-bon-1500.pl
witclub.info womensterritory.ru
woodyguthrie.se
worldlovers.ru
wormix-cheats.ru wormix-cheats.ru
www.ehscloud.cn wowas31.ucoz.ru
x-diesel.biz wrz0iuebwhp5fg.freeddns.com
xblog.in wyniki-lista.pl
xmlinde.com xgames-04.com
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai xn----7sbahjd3btneuw1joc.xn--p1ai
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai xn----9sbebi2bvzr7h.xn--p1ai
xn--lifehacer-1rb.com xn----itbeirbjbi7bc6bh2d.xn--p1ai
xnxx699.com xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xtremeeagles.net xn--80ajbshivpvn2i.xn--p1ai
xxx-treker.ru xn--80ak6aa92e.com
xxxmania.top xn--b1adccaf1bzj.xn--p1ai
xxxtube69.com xpresscare.ru
yaoguangdj.com xvideosporn.biz
youbrainboost.asia xz618.com
yougotanewdomain.com yatrk.xyz
youporn-forum.ga yellocloud.be
youporn-ru.com youdao.com
yur-p.ru yougame.biz
zaimite.ru youghbould.wordpress.com
zaobao.com.sg youjizz.vc
zeroredirect12.com yourporn.com
zeroredirect9.com yurgorod.ru
zhorapankratov7.blogspot.com zagadki.in.ua
zigarettenonlinekaufen1.bloog.pl zaidia.xhost.ro
zambini.ru
zarabiaj-dzis.pl
zeg-distribution.com
zigarettenonl.canalblog.com
zolotoy-lis.ru
zone-kev717.info zone-kev717.info
zoosexart.com
zzlgxh.com

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1-free-share-buttons.com 1-free-share-buttons.com
101lesbian.xyz 12masterov.com
11pikachu.ru 1hwy.com
123any.com 24x7-server-support.site
4senses.co 4replicawatch.net
7xc4n.com 5elementov.ru
adultfriendfinder.com 7search.com
advertisingtag.net 91abcw.com
afftrack001.com aarbur.com
aghanyna.com abctoppictures.net
agysya.ru admatic.com.tr
ahrefs.com adrunnr.com
alborzan.com ads.gold
allpdfmags.net advertex.info
allstatesugarbowl.org advokateg.ru
alot.com afmuseum.com
alphahoverboards.com aibolita.com
amateurlivechat.org alive-ua.com
ampower.me allmarketsnewdayli.gdn
anal-acrobats.hol.es alltheviews.com
android-systems.ru aloofly.com
apiwebwebgetcom-a.akamaihd.net angel1777.kz
appartement-stumm.at apartamentwroclaw.eu
atmagroup.ru apibatbrowsecom-a.akamaihd.net
audiofree.ru apiluckyleapnet-a.akamaihd.net
azte.ch appsaurus.com
back.dog arendovalka.xyz
bannerswap.com artparquet.ru
battlecarnival.su asiavirtualsolutions.com
battleforupdating.site asophoto.com
bavsac.com ausmepa.org.au
bear.gotcher.us autoseo-traffic.com
bfz.biz avkzarabotok.info
bg6s0.com avon-severozapad.ru
bgviagrachrx.com ayanaspa.com
bio-market.kz backlink4u.net
bizru.info bagcionderlab.com
blavia.00author.com bankofthewext.com
bloggerads.net battle.net
blogos.kz beauty-things.com
bookmaker-bet.com best-seo-offer.com
bradleylive.xyz bestbrainprod.win
breastaugmentation.co.za bmsco.co
bristolhotel.com.ua brainxs.us
btc4u.ru brandimensions.com
buron.pw brus.city
c-gainsbourg.com buypuppies.ca
cakemediahq.com.au camdenmemorials.com
cdnanalytics.xyz camel-beach.com
cementaresearch.se casablancamanor.co.za
cheap-pills-norx.com cdn.walkme.com
cheapestjerseys-wholesale.com centrumcoachingu.com
cfacarrosserie74.com
charmstroy.info
cheapestjerseysonwholesale.com
cheapjerseysap.com
cheapmarmot.online
checkm8.com checkm8.com
chiblackhawks-jerseys.com chinese-amezon.com
christianlouboutinoutlet.win compliance-andrew.top
cialgenisrx.com compliance-donald.xyz
cialisovercounteratwalmartusa.com compliance-irvin.xyz
club-musics.ru cookie-law-enforcement-ff.xyz
coinsspb.com counter.yadro.ru
compliance-alexa.top covadhosting.biz
compliance-jessica.top crest3d.ru
comsysnet.com
coop-gamers.ru
cpajunkies.com
cyclobenzaprinevstizanidine.blogspot.com
daooda.com
daretodonate.co
daydream-studio.ru
ddospanel.com ddospanel.com
ddsoldes.fr decenttools.com
deadlinkchecker.com detmebel.su
deutschland123.de diarioaconcagua.com
digital-video-processing.com dnepropetrovsk.mistr-x.org
discovertreasure-a.akamaihd.net dnsrsearch.com
disruptingdinnerparties.com dogoimage.com
diusyawiga.tk
divisioncore.com
dms-sw.ru
dollartree.info dollartree.info
domaincdn.xyz dostavimvdom.ru
dumuelave.xyz dtm-spain.com
eandsgallery.com ege-essay.ru
erectile.bid ekto.ee
espaceinventoristes.com elektrischeziga.livejournal.com
etm-consult.de elektroniksigaraankara.info
evaashop.ru etur.ru
exlarseva.webblog.es ezofest.sk
fickenbumsen.net factorynightclub.com
filmfanatic.com fakehandbags.xyz
financepoints.eu fanrto.com
findpics.pw fast-torrent.ru
fingerprintjs.com fast-wordpress-start.com
fireads.men femdom.twiclub.in
fickenprivat.info
filerockstar298.weebly.com
filmetricsasia.com
filmi-onlain.info
financeloan.us
findclan.org
fivedwld.cf
flprog.com
flytourisme.org flytourisme.org
forms-mtm.ru fm-upgrade.ru
foxjuegos.com forum.poker4life.ru
frcls.fr foxweber.com
free-share-buttons.blogspot.com freenode.info
free-social-buttons.com gdcentre.ru
galeria-zdjec.com gentamicineyedrops.blogspot.com
gearcraft.us germes-trans.com
genetworx.com getlamborghini.ga
girlsfuckdick.com gilsonchiro.xyz
gkvector.ru glicol.kz
gloverid.site goldandcard.ru
guarrasdelporno.xxx greatzip.com
healgastro.com gsou.cf
h2monline.com
hamilton.ca
hdpixion.com
hitcpm.com
hjaoopoa.top
hobild.net
homik.pw homik.pw
htmlcorner.com hostnow.men
ideoworld.org hotelcrocenzi.sm
igru-xbox.net hotgirlhdwallpaper.com
imgata.com howlongdoestizanidinestayinyoursystem.blogspot.com
inclk.com images.gyffu.com
indianmedicaltourismshop.com imgarit.pw
instasexyblog.com imk.com.ua
invest-pamm.ru infobabki.ru
ispac.org int.search.mywebsearch.com
itservicesthatworkforyou.com istizanidineacontrolledsubstance.blogspot.com
ivanovo.zrus.org kaz.kz
joessmogtestonly.com kichenaid.ru
jose.mulinohouse.co klosetkitten.com
jurajskie.info kol-energo.ru
justdating.online kreativperlen.ch
ketoanhanoi.info kvartir-remont.biz
keywordsdoctor.com labplus.ru
keywordsuggests.com lacasamorett.com
keywordteam.net law-check-eight.xyz
kinoduh.ru legalrc.biz
kinofak.net lennatin.info
lafourmiliaire.com likesdesign.com
lakomka.com.ua lion.bolegapakistan.com
laptoper.net livetsomudvekslingstudent.bloggersdelight.dk
lasvegaslockandsafe.com longlifelomilomi.info
lemon-ade.site low-format.ru
lightinghomes.net matb3aa.com
linkarena.com michaelkorsoutlet.store
lomb.co missvietnam.org
lombardfinder.ru mitrasound.ru
luxmagazine.cf
mabdoola.blogspot.com
macdamaged.space
malwareremovalcenter.com
mebelcomplekt.ru
mjchamonix.org
mmofreegames.online
mobile.ok.ru mobile.ok.ru
mundoaberrante.com monarhs.info
music7s.me monsterdivx.tv
musicktab.com mrsdalloways.com
my-floor.in.ua my-big-family.com
myperiod.club mysexpics.ru
myplaycity.com natali-forex.com
myshopmatemac.com netcheckcdn.xyz
nalogovyykodeks.ru nisuturnetdgu.tk
nextconseil.com nonews.co
noclegonline.info northfacestore.online
okeinfo.online o-dachnik.ru
olvanto.ru obiavo.ru
omoikiri-japan.ru obnal.org
omsk.xrus.org omgtnc.com
oren-cats.ru onlinadverts.com
ostroike.org outrageousdeal-a.akamaihd.net
ourtherapy.ru parvezmia.xyz
par-fallen.ga pdn-4.com
pc-virus-d0l92j2.pw perl.dp.ua
pc4download.co phpdevops.com
pcboa.se picurams.pw
penisvergrotendepillennl.ovh pinapchik.com
perfectplanned.com plaff-go.ru
petedrummond.com plastgranar.nu
pinup-casino1.ru pliks.pl
pio.polytopesexempt.com porn-w.org
pl.id-forex.com porndl.org
polybuild.ru pornzone.tv
polytopesexempt.com pussyspace.net
pons-presse.com realnye-otzyvy.info
porndairy.in rentaremotecomputer.com
pornogig.com resellerclub.com
postclass.com rusvideos.su
prizeestates.cricket rutor.group
pukaporn.com ryetaw.com
q-moto.ru s.lollypopgaming.com
qld10000.net sarafangel.ru
relatodelpresente.com.ar savememoney.co.za
remybutler.fr scanner-willy.top
replicalouboutin.xyz sdrescher.net
retailwith.com seasaltwithfood.com
retreatia.com seeingmeerkat.com
rialp.getenjoyment.net seozoom.it
russintv.fr sexgalleries.top
samo-soznanie.ru sexs-foto.top
satoristudio.net sexyteens.hol.es
savefrom.com shoplvlv.us
scanhub.ru siha.de
scanner-barak.top sindragosa.comxa.com
searchimage.co sisi-go.ru
searchinterneat-a.akamaihd.net site-speed-check.site
semaltmedia.com siteimprove.com
seomarketings.online slonechka.ru
sexpornotales.com snegozaderzhatel.ru
shitmovs.com songplanet.ru
sledstvie-veli.net souvenir.cc
social-buttons.com spidtest.org
stroicol.net sptslmtrafms.com
surfbuyermac.com stats-collector.org
tasteidea.com store-rx.com
the-usa-games.blogspot.com stroyhelp-dv.ru
ticketsys.inetwd.com studiofaca.com
timhost.ru surffoundation.nl
tizanidinesideeffects.blogspot.com surflinksmedical.com
topflownews.com susanholtphotography.com
torrent-newgames.com swagbucks.com
traffictrade.life threecolumnblogger.com
tubeline.biz tizanidinehydrochloride4mgstreetvalue.blogspot.com
tvand.ru tizanidineinteractions.blogspot.com
typer.one topcar-krasnodar.ru
upproar.com trafficcentr.xyz
uvozdeckych.info trahic.ru
valuado.com trkdf.com
via-energy-order.com trustedhealthtips.com
video-chat.cn tsyndicate.com
vielporno.net tupper-posuda.ru
vmnmvzsmn.over-blog.com tv-spoty.info
vriel.batcave.net ucsol.ru
wallpapersist.com uggbootsoutletsale.us
warezaccess.com ukrtextbook.com
waterefficiency.co ultramart.biz
waycash.net unrealcommander.biz
web-analytics.date unrealcommander.org
webjam.com videochat.world
website-speed-checker.site videospornogratisx.net
whatzmyip.net virtualbb.com
winwotgold.pl vvpg.ru
wladimirpayen.com vzlomtw.com
xaylapdiendanang.com webs.com
xchangetrak.com website-speed-up.top
xep.info wleuaprpxuvr.ga
xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai wtsindia.in
xn--80aeahghtf8ac5i.xn--p1ai www.ehscloud.cn
xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai x-lime.com
xnxx699.com xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
yellowstonesafaritours.com xn--1-8sbcpb0bdm8k6a.xn--p1ai
yougotanewdomain.com xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
yourserverisdown.com xn--d1abj0abs9d.in.ua
yourtemplatefinder.com zionstar.net
zaim-pod-zalog-krasnodar.ru zocaparj.kz
zarabotok--doma.ru zvetki.ru
zeroredirect12.com zzbroya.com.ua
zigarettenonlinekaufen2.bloog.pl
zoompegs.com

167
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -8,86 +8,6 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -494,6 +414,83 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
@ -584,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.219 0; 10.1.0.130 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.02.3566 ### Version: V4.2023.02.3567
### Updated: Fri Feb 3 10:03:26 UTC 2023 ### Updated: Fri Feb 3 22:01:12 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.02.3566 ### Version: V4.2023.02.3567
### Updated: Fri Feb 3 10:03:26 UTC 2023 ### Updated: Fri Feb 3 22:01:12 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##