vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-05 04:00:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.02.2916 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-02-13 22:07:22 +00:00
parent 314b9e0dfc
commit 0524e1620e
48 changed files with 81532 additions and 81522 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2915 #### Version: V4.2022.02.2916
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2915 #### Version: V4.2022.02.2916
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.02.2915 #### Version: V4.2022.02.2916
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10054
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.02.2915 #### Version: V4.2022.02.2916
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10058
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10058
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10058
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10058
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

10058
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

10058
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

10058
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2915 2916

10058
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10058
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.247.201.92 0; 20.127.56.26 0;

10058
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

10058
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

10058
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

10058
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.247.201.92 0; 20.127.56.26 0;

10058
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.247.201.92 0; 20.127.56.26 0;

10058
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Bigfoot Aboundexbot
BlowFish Anthill
CCBot BBBike
Larbin CheTeam
NPbot GetWeb
Niki-bot GrapeFX
autoemailspider PHPCrawl
dataforseo.com Seomoz
isitwp.com SiteCheckerBotCrawler
oBot iubenda-radar

286
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
ALittle Client 404checker
404enemy
80legs
ADmantX
AIBOT
Aboundex Aboundex
Acunetix Aboundexbot
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot AiHitBot
Alexibot
AllSubmitter AllSubmitter
Alligator Alligator
AlphaBot AlphaBot
Anarchie
Anarchy99 Anarchy99
Apexoo Ankit
Atomseobot Aspiegel
Attach Attach
AwarioSmartBot BDCbot
BDFetch
BackDoorBot
BackWeb BackWeb
Backlink-Ceck Bandit
BatchFTP
Battleztar Bazinga Battleztar Bazinga
BetaBot Blow
Buck Buck
Buddy Buddy
BuiltWith BuiltBotTough
BunnySlippers
BuzzSumo BuzzSumo
CATExplorador CATExplorador
CODE87
CheTeam
CheeseBot CheeseBot
Chlooe CherryPicker
Citoid Cloud mapping
Claritybot Copyscape
Cliqzbot
Copier
Cosmos
Craftbot
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
CrunchBot CrunchBot
Custo Custo
DBLBot DIIbot
DTS Agent
Deusu Deusu
Digincore DigitalPebble
Disco
Discoverybot Discoverybot
DittoSpyder Dispatch
DnBCrawler-Analytics
DnyzBot DnyzBot
DomainCrawler DomCopBot
Dragonfly Domains Project
Download Wonder
Drip Drip
EMail Siphon Ecxi
EMail Wolf Express WebPictures
EasyDL Extractor
EirGrabber
EroCrawler
ExtLinksBot
Extreme Picture Finder Extreme Picture Finder
Ezooms Ezooms
FDM
FemtosearchBot FemtosearchBot
Fimap
Firefox/7.0
FlashGet FlashGet
Flunky Flunky
Foobot Foobot
Fuzz FrontPage
GT::WWW
GalaxyBot GalaxyBot
GetWeb Genieo
Getintent GermCrawler
Gigabot GetRight
GrabNet Go!Zilla
Go-Ahead-Got-It
GoZilla
Grafula Grafula
GrapeshotCrawler GrapeFX
GridBot GridBot
HMView HMView
HTTP::Lite HTMLparser
HTTrack
Haansoft Haansoft
Havij Hloader
HonoluluBot HonoluluBot
IDBot IDBTE4M
Id-search IRLbot
IlseBot Image Fetch
IndeedBot
InfoTekies InfoTekies
InterGET InterGET
Iria Iria
JamesBOT
Jetty Jetty
Jorgee JikeSpider
Kinza
Kozmosbot
Larbin
Leap
LeechFTP LeechFTP
LeechGet LexiBot
Lftp
LibWeb
Libwhisker
Lightspeedsystems Lightspeedsystems
Likse LinkWalker
Linkbot
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider Lipperhey Spider
Lmspider Lmspider
MIDown tool Ltx71
MFC_Tear_Sample
MJ12bot MJ12bot
Mail.RU_Bot MSFrontPage
Majestic SEO MSIECrawler
MarkWatch Magnet
Mass Downloader MarkMonitor
Masscan
Mata Hari
MauiBot MauiBot
Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot Meanpathbot
Microsoft Data Access Metauri
Mister PiX Mojolicious
Morfeus Fucking Scanner MolokaiBot
Mr.4x3 Mozlila
Musobot Musobot
Navroad NPbot
Name Intelligence
Nameprotect
NearSite
NetLyzer NetLyzer
Nettrack Netcraft
Netvibes Netvibes
NextGenSearchBot
Niki-bot Niki-bot
Nimbostratus Nmap
Nuclei OpenLinkProfiler
Octopus Openvas
Offline Navigator OrangeBot
OpenVAS PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageGrabber PageAnalyzer
Panscient Panscient
Papa Foto Papa Foto
Pavuk PeoplePal
Petalbot Petalbot
PictureFinder Picsearch
Piepmatz Pockey
ProPowerBot ProPowerBot
ProWebWalker Probethenet
Pu_iN PxBroker
Pump
Quick-Crawler Quick-Crawler
RSSingBot RankActive
RankFlex RankActiveLinkBot
RankingBot RankingBot
Rankivabot RankingBot2
RankurBot
Re-re
RealDownload RealDownload
Reaper RebelMouse
Recorder
RedesScrapy RedesScrapy
RepoMonkey Ripper
Rogerbot Rogerbot
SBIder
SEOprofiler SEOprofiler
SEOstats SEOstats
SMTBot SISTRIX
SalesIntelligent SalesIntelligent
Scrapy ScanAlert
ScreenerBot ScreenerBot
ScrepyBot
Searchestate
SearchmetricsBot SearchmetricsBot
Seekport
SemanticJuice
Semrush Semrush
SemrushBot
SentiBot SentiBot
SeobilityBot SeobilityBot
Seomoz
Shodan
Siphon Siphon
SiteCheckerBotCrawler Site Sucker
SiteExplorer SiteExplorer
SiteLockSpider SmartDownload
SiteSnagger
Snapbot
Snoopy Snoopy
SocialRankIOBot Sosospider
Sottopop SpaceBison
Spanner Spammen
SpankBot
Spbot Spbot
Spinn3r
SputnikBot SputnikBot
Sqlmap
Sqlworm Sqlworm
Sqworm
Stripper Stripper
Sucker Sucuri
SurveyBot SuperBot
Swiftbot Surfbot
Szukacz Titan
TheNomad Toata
Thumbor Toweyabot
TightTwatBot
Trendiction
Trendictionbot Trendictionbot
Turingos
TurnitinBot TurnitinBot
Twice
URLy Warning URLy Warning
UnisterBot UnisterBot
Upflow VB Project
Vacuum VeriCiteCrawler
VoidEYE Voil
WBSearchBot Voltron
WISENutbot WEBDAV
WPScan WPScan
WWW-Mechanize WWWOFFLE
Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD WallpapersHD
Web Collage Web Pix
Web Enhancer
Web Sauger Web Sauger
Web Sucker Web Sucker
WebGo IS WebAuto
WebPix WebFuck
WebImageCollector
WebReaper WebReaper
WebStripper
WebSucker
WebZIP WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Webshag
Website Quester
WebsiteExtractor WebsiteExtractor
Webster Whack
Whacker Whatweb
Wonderbot Who.is Bot
WinHTTrack
Wotbox Wotbox
Xaldon_WebSpider Xaldon_WebSpider
Xenu Xenu
YoudaoBot
Zeus
Zitebot Zitebot
ZmEu adscanner
backlink-check arquivo.pt
cah.io.community coccocbot-web
clark-crawler
cognitiveseo cognitiveseo
crawl.sogou.com crawler4j
facebookscraper domainsproject.org
heritrix internetVista monitor
instabid
iubenda-radar
magpie-crawler magpie-crawler
meanpathbot
muhstik-scan muhstik-scan
page scorer page scorer
polaris version pcBrowser
probe-image-size s1z.ru
satoristudio.net
scalaj-http
scan.lol scan.lol
seocompany.store seocompany.store
seoscanners
seostar
serpstatbot serpstatbot
siteripz
sp_auditbot sp_auditbot
ubermetrics-technologies.com spyfu
webmeup-crawler tAkeOut
webprosbot trendiction.com
webgains-bot
zauba.io zauba.io
zgrab

292
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
80legs 404checker
AIBOT AIBOT
ALittle Client ASPSeek
Abonti
Aboundexbot
AiHitBot AiHitBot
Aipbot AllSubmitter
Alexibot
AlphaBot AlphaBot
Anarchy Asterias
Anarchy99
Ankit
Anthill
Aspiegel
Atomseobot Atomseobot
Attach Attach
AwarioSmartBot AwarioSmartBot
BLEXBot
BackDoorBot
BackStreet BackStreet
Badass BacklinkCrawler
Barkrowler Bandit
Battleztar Bazinga
Bigfoot
Bitacle
BlackWidow
Blackboard Blackboard
Blow
BlowFish BlowFish
Bolt Buck
BotALot
Brandprotect
Bullseye Bullseye
CODE87
Calculon Calculon
CazoodleBot CazoodleBot
CheTeam CheTeam
ChinaClaw CherryPicker
Citoid
Cliqzbot Cliqzbot
Cloud mapping Cocolyzebot
Cogentbot
Copier Copier
Copyscape
Cosmos
Craftbot Craftbot
Crawling at Home Project
CrazyWebCrawler
Crescent Crescent
CrunchBot Curious
CyotekWebCopy Custo
DBLBot
DIIbot DIIbot
Digincore DSearch
DigitalPebble DnyzBot
Discobot DomainAppender
Discoverybot
Dispatch
DomainCrawler
Domains Project
Dotbot Dotbot
Download Wonder Download Wonder
Dragonfly Drip
EMail Siphon ECCP/1.0
EasyDL
Ebingbong Ebingbong
EroCrawler EirGrabber
Express WebPictures Extractor
ExtractorPro
Extreme Picture Finder
Ezooms Ezooms
FHscan FHscan
FemtosearchBot Flunky
Firefox/7.0 Freeuploader
FrontPage Fuzz
FyberSpider
G-i-g-a-b-o-t
GT::WWW
GalaxyBot
Genieo
GermCrawler GermCrawler
GetWeb Gigabot
Getintent Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
Gotit
GrabNet
Grabber Grabber
Grafula Grafula
GrapeFX
HEADMasterSEO HEADMasterSEO
HTMLparser HTMLparser
HTTrack HTTrack
Havij HaosouSpider
Harvest
Heritrix
Hloader Hloader
IDBot
IRLbot IRLbot
Id-search
Image Fetch Image Fetch
IndeedBot IndeedBot
Intelliseek
Iskanie Iskanie
JOC Web Spider IstellaBot
JamesBOT Jbrofuzz
JikeSpider
Jorgee Jorgee
Jyxobot
Kinza Kinza
Kozmosbot
Lanshanbot Lanshanbot
LeechGet Leap
LeechFTP
LibWeb LibWeb
Lightspeedsystems Libwhisker
Likse LieBaoFast
LinkScan LinkScan
Linkbot Linkbot
LinqiaScrapeBot LinksManager
Litemage_walker LinqiaMetadataDownloaderBot
Lipperhey Spider
Lmspider Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc MQQBrowser
Magnet MSFrontPage
Mail.RU_Bot MSIECrawler
Majestic-SEO Majestic SEO
Majestic12 MarkWatch
MarkMonitor
Mass Downloader Mass Downloader
MauiBot Mb2345Browser
MegaIndex.ru Mediatoolkitbot
Metauri Metauri
Minefield Microsoft URL Control
Mister PiX Mister PiX
Mojeek
Mojolicious
MolokaiBot
Morfeus Fucking Scanner
Mozlila Mozlila
NPbot Mr.4x3
Musobot
NICErsPRO
Name Intelligence Name Intelligence
Nameprotect Needle
Navroad
NearSite
Net Vampire Net Vampire
NetAnts NetAnts
NetLyzer NetLyzer
NetZIP
Netcraft Netcraft
Netvibes Nettrack
NextGenSearchBot NextGenSearchBot
Nibbler Niki-bot
NimbleCrawler
Nimbostratus Nimbostratus
Ninja
Nmap Nmap
Offline Explorer Offline Explorer
Offline Navigator OnCrawl
OpenLinkProfiler
Openfind Openfind
Page Analyzer OrangeBot
OutclicksBot
PECL::HTTP
PageGrabber
Panscient Panscient
Papa Foto Papa Foto
Petalbot
Picsearch Picsearch
PictureFinder Piepmatz
Pimonster Pockey
Pixray
ProWebWalker
Psbot
Pump Pump
PxBroker RSSingBot
QueryN Metasearch RankActive
RankActiveLinkBot RankingBot
Re-re RealDownload
ReGet
RebelMouse RebelMouse
Recorder
Rogerbot Rogerbot
SBIder SBIder
SEOlyticsCrawler SEOstats
Scanbot SalesIntelligent
Screaming Seekport
ScreenerBot SemanticJuice
Semrush
SentiBot SentiBot
SeoSiteCheckup SeobilityBot
Site Sucker Site Sucker
SiteExplorer SiteExplorer
SiteLockSpider
SiteSnagger
SiteSucker
Sitebeam Sitebeam
Sitevigil
SlySearch SlySearch
Snoopy SmartDownload
SocialRankIOBot Snapbot
Sottopop SpankBot
Spanner
Spbot Spbot
Spinn3r
SputnikBot SputnikBot
Sqlworm Steeler
Sqworm Stripper
SuperBot Sucuri
SuperHTTP SuperHTTP
Szukacz Surfbot
Suzuran
T0PHackTeam T0PHackTeam
T8Abot T8Abot
Teleport
TeleportPro TeleportPro
Telesphoreo
The Intraformant The Intraformant
TheNomad
Thumbor Thumbor
Tracemyfile Tracemyfile
Trendiction Trendiction
True_Robot
Turingos Turingos
Turnitin
Twice
Typhoeus Typhoeus
UnisterBot
Upflow
VB Project
VCI VCI
Vacuum Vagabondo
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
Voltron Voltron
WISENutbot WBSearchBot
WPScan WEBDAV
WWW-Collector-E
Wallpapers/3.0 Wallpapers/3.0
Web Auto
Web Collage Web Collage
Web Fuck
Web Sucker
WebAuto
WebCollage WebCollage
WebEnhancer
WebFetch WebFetch
WebFuck
WebGo IS
WebLeacher WebLeacher
WebPix WebStripper
WebReaper WebWhacker
WebZIP WebZIP
WebmasterWorldForumBot
Webshag
Website Quester
WebsiteExtractor
WebsiteQuester
Webster Webster
Whacker Whack
Widow
WinHTTrack
WiseGuys Robot WiseGuys Robot
Woobot Wonderbot
Zermelo Wotbox
Zeus Xaldon_WebSpider
Zitebot Xenu
YoudaoBot
Zade
ZoomBot ZoomBot
ZyBorg ZyBorg
adscanner arquivo-web-crawler
coccocbot-web cah.io.community
com.plumanalytics check1.exe
cognitiveseo
crawl.sogou.com crawl.sogou.com
crawler.feedback dataforseo.com
demandbase-bot gopher
evc-batch
instabid
internetVista monitor internetVista monitor
lwp-trivial ips-agent
magpie-crawler lwp-request
meanpathbot muhstik-scan
netEstate NE Crawler netEstate NE Crawler
oBot probe-image-size
page scorer
plumanalytics
ripz ripz
s1z.ru
satoristudio.net
scalaj-http
scan.lol scan.lol
seobility
seocompany.store seocompany.store
seoscanners seoscanners
seostar seostar
siteripz sitechecker.pro
sogouspider
sysscan sysscan
tAkeOut trendiction.com
ubermetrics-technologies.com ubermetrics-technologies.com
x22Mozilla voyagerx.com
webgains-bot
webprosbot
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
ahrefs.com 7zap.com
cornomase.win erectile.bid
flavors.me evehemming.blogspot.com.au
geoads.com goodhousekeeping.com
ifirestarter.ru listiseltemournlan.gq
larger.io manuscript.su
magazintiande.ru subj.ukr-lit.com
stuffpride.com track-rankings.online
toyota.7zap.com unrealcommander.biz
xtrafficplus.com youdesigner.kz

944
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
08800.top 00author.com
100dollars-seo.com 00it.com
123any.com 00webcams.com
123cha.com 178evakuator178.ru
24-ak.ru 1millionusd.xyz
3xforum.ro 1pamm.ru
4inn.ru 1stat.ru
4rent.ru 2728fb936f0.com
5tey7463.weebly.com 34.gs
6hopping.com 45en.ru
8xv8.com 4webmasters.org
991mostfm.co.id 66cpwgln.space
aa14ab57a3339c4064bd9ae6fad7495b5f.com 70casino.online
absugars.com 72-news.com
addbin.men 7search.com
adhome.biz absolutelyanalog.com
adnotbad.com actionnooz.com
adobereader-free.ru adbetclickin.pink
adsloads.com adtech.us
adtech.de ageofclones.com
adverster.com alert-fjg.xyz
adzpower.com alfa9.com
alfa-medosmotr.ru allnews.md
alfabot.xyz
allesohnegirls.net
alot.com
alyeskaresort.com
am-se.com
amazingpic.net
amazon-seo-service.com
amung.us
anabolics.shop
ananumous.ru ananumous.ru
angry-fermi-7633.arukascloud.io android-systems.ru
animalia-life.club androids-store.com
anonymous-redirect.com apifasterlightin-a.akamaihd.net
anticrawler.org apispringsmartne-a.akamaihd.net
apibrowsesmartne-a.akamaihd.net appfixing.space
apiqualitinknet-a.akamaihd.net arabseyes.com
appsaurus.com artdestshop.eu
apxeo.info asacopaco.tk
asia-forum.ru asmxsatadriverin.aircus.com
avtolombard-krasnodar.ru asseenontv.ru
axbocz.net autogrep.ru
b2b-lounge.com aviav.co
babs.com.ua azazu.ru
back.dog backlinkwatch.com
backgroundpictures.net balla-boo.se
bayareaaeroclub.org bannerbank.ru
beachfix.co barnfurnituremart.com
bensbargains.net bcmp.org
bestdooz.com beachtoday.ru
bestserials.com besthomemadepornsites.com
beta.hotkeys.com bestsexyblog.com
betterhdporn.com betterscooter.com
beyan.host.sk beztuberkuleza.ru
big-boards.info bharatdefencekavach.com
bioca.org bimmerpost.com
biographiya.com bizcheapjerseyswholesalechina.com
bioscorp.ru bkns.vn
blog.remote-computer.de bnt-team.com
bloke.com bookmark4you.com
bmw.afora.ru bouda.kvalitne.cz
bocoarchives.org braintobe.win
boostmyppc.com brandbucket.com
breastaugmentation.co.za brandedleadgeneration.com
brianhenry.co brandehk.dk
brokergid.ru brateg.xyz
buy-meds24.com bristolhotel.com.ua
buyclomidonlaine.com bronzeaid-a.akamaihd.net
buynorxx.com browserprotecter.com
c-english.ru btc4u.ru
carloans.com btnativenav.com
carson.getenjoyment.net bum.com.ru
cdncash.org buypillsorderonline.com
cellfun.mobi buyviagraa.blogspot.com
changde.58.com buzzonclick.com
cactussoft.biz
camel-beach.com
candelluxsklep.pl
canrole.com
cartechnic.ru
caveavins.fr
cbb1smartlist12.click
cbcseward.com
cbox.ws
cementaresearch.se
centrumcoachingu.com
cfcl.co.uk
chatroulette.world
chcu.net chcu.net
cheapbarbour.online cheapcigarettesc.info
cheapbelstaff.online chocolateslim-original.com
cialisovercounteratwalmartusa.com christianlouboutinsaleoutletonline.us
cigarpass.com cialgenisrx.com
cinemaenergy-hd.ru cialisndbrx.com
circlesl.com cleanersoft.com
citizenclsdriveri7.pen.io clickbank.net
clips.ua.ac.be clips.ua.ac.be
coaching-netz.info coaching-netz.info
compliance-alexa.top cocyq.inwtrade.com
concordexoticrentals.com com-secure.download
continent-e.tv compliance-barak.xyz
converse.ddsoldes.fr compucelunlock.net
coolwallpapers4k.info confib.ifmo.ru
corta.co contentlook.co
cottageofgrace.com cookie-law-enforcement-aa.xyz
cowblog.fr cordstrap.cc
craftburg.ru crafthubs.com
credit.co.ua crazyprotocol.com
customcatchcan.com custom-product-labels.com
d2jsp.org dangphoto.trade
dame-ns.kz darkages.info
darkbooks.org dellalimov.com
data.vtc.pw descargar-musica-gratis.net
degerlund.net dev-seo.blog
delayreferat.ru diegolopezcastan.com
demenageur.com diesel-parts28.ru
demian.kz
dev78.dioniqlabb.se
dichvuvesinhhanoi.com
dieswaene.com dieswaene.com
dimaka.info dimaka.info
documentsite.net dimfour.com
domaineaneblanc.com dinkolove.ya.ru
dpihatinh.gov.vn directivepub.com
drhomes.biz discovertreasurenow.com
duplicashapp.com divatraffic.com
dividendo.ru
documentbase.net
domainsphoto.com
dominterior.org
dosug-lux.ru
doxysexy.com
dptaughtme.com
drev.biz
drpornogratisx.xxx
drugs-no-rx.info
drunkenstepfather.com
e-commerce-seo.com
e-poker-2005.com
eaplay.ru eaplay.ru
edudocs.net earn-from-articles.com
edwinkonijn.com.au earncash.com.ua
election.interferencer.ru ekobata.ru
electricwheelchairsarea.com electronicadirect.com
eleimgo.pw elektrischezigaretten1.blogse.nl
elektrozigarettekaufen1.devhub.com elektrozigarette2.webs.com
elektrozigaretten1.postbit.com elektrozigarettekaufen2.blogse.nl
emailaccountlogin.co empis.magix.net
emergencyneeds.org en.home-task.com
englate.com enpolis.ru
eonpal.com ereko.ru
erotikstories.ru ero-advertising.com
es5.com etur.ru
everflora.ru eu-cookie-law.info
evrotekhservis.ru evidencecleanergold.com
faithe.top exmasters.com
familyphysician.ru exoneration-project.us
fast-wordpress-start.com ezigarettenkaufen1.deviantart.com
feel-planet.com facialporntube.com
feeriaclub.ru fashion-stickers.ru
fettefrauen.net fenoyl.batcave.net
fickenprivat.info ferretsoft.com
filesvine.com filkhbr.com
fillmewithhappiness.com florida-tourism.net
fiverr.com formseo.com
flirt4free.com forum.doctissimo.fr
floating-share-buttons.com forum.tvmir.org
flprog.com franch.info
foodgid.net free-fbook-traffic.com
forex.osobye.ru freelinkbuilding.website.tk
forsex.info freeuploader.com
forums.d2jsp.org freevpn.space
frameimage.org fuckbuddybestgilf.info
franklinfire.co fugarif.ga
freejabs.com fx-brokers-review.com
freenode.info galeon.com
freenom.link galeria-zdjec.com
freshmac.space games.kolossale.ru
froggytube.com geckoandfly.com
fuck-paid-share-buttons.xyz gembird.com
funnypica.com getlaid-xxxhookupdirect.com
fxund.us gg-arena.ru
fym.com.ru gifspics.com
garciniaxt.us glass-msk.ru
gateway.zscalertwo.net
gemgrab-a.akamaihd.net
genetworx.com
getoutofdebtfree.org
gettpromos.com
ggiaro.com
gilbertbanda.net
girlporn.ru
girlsatgames.ru
gkvector.ru
globalsurfari.com
goldandcard.ru
goldenggames.com goldenggames.com
googlemare.com gsasearchenginerankerhelp.com
gourcy.altervista.org guod.me
graphicwe.org handicapvantoday.com
greenzaim.ru handsandlegs.ru
gsasearchenginerankerdiscount.com hcate.com
guigyverpo.cf hd-film.pl
hamptonoaks.ca hdapp1008-a.akamaihd.net
happy.new.yeartwit.com heroz.fr
hard-porn.mobi hikesearch.net
hdpixion.com home-task.com
health-medical-portal.info homeart.space
hentaiheroes.com homeinns.com
hitcpm.com hostcritique.com
hotdl.in hostsshop.ru
howlongdoestizanidinestayinyoursystem.blogspot.com hoterika.com
hreade.com huimin764128.com
htmlcorner.com humanelydrew.com
husky-shop.cz i4track.net
iccornacircri.cf ico.re
ictizanidinehcl4mg.blogspot.com ilmexico.com
ifmo.ru imabase.com
iideaidekonkatu.info imagui.eu
ilikevitaly.com img.wallpaperstock.net
imgarit.pw importchinacoach-teach.com
imk.com.ua infospot.pt
inboxdollars.com intermesh.net
incolors.club ip-guide.com
inet-traffic.com itservicesthatworkforyou.com
infoupdate.org
iqoption.com
irkutsk.online-podarki.com
iyasimasennka.com
jacago.com jacago.com
jerseychinabizwholesale.com jackpotchances.com
jerseychinabizwholesale.us janerikholst.se
jerseywholesaleelitestore.com janettabridal.com
jewelryandfiligree.com jestr.org
jobmarket.com.ua jimmychoosale.online
joy-penguin.com jongose.ninja
kaac.ru junglenet-a.akamaihd.net
kadashihotel.com kalandranis.gr
karaganda.xkaz.org kamalsinha.com
kareliatobacco.ru katadhin.co
kazan.xrus.org keywords-monitoring-success.com
kazinogames.lv keywordsdoctor.com
keywordsuggests.com kihi.gdn
khafre.us kimcurlrvsms.com
kino-filmi.com kinomaniatv.pl
kinoflux.net
kinohall.ru
kinosed.net
kometars.xyz
komputernaya-pomosh-moscow.ru
konturkrasoty.ru
kostenlos-sexvideos.com
kostenloser-sex.com
krafte.ru
krasivoe-hd.net
kredit-blog.ru kredit-blog.ru
kupiproday.com.ua kretpicf.pw
kvartira-sutochno.com ktotut.net
kupit-adenu.ru
kutikomi.net
lakomka.com.ua lakomka.com.ua
landscapebackgrounds.blogspot.com landmania.ru
laxdrills.com landoftracking.com
laxob.com landscaping.center
le-clos-des-alouettes.com layola.biz.tc
lecbter-relationships.ga lego4x4.ru
lechenie-gemorroya.com letmacworkfaster.world
lennatin.info letslowbefast.site
lflash.ru
li-er.ru
lineavabit.it lineavabit.it
linkbuddies.com liquimondo.com
lisque.batcave.net littleberry.ru
livejasmin.com liver-chintai.org
loadingpages.me liverpool.gsofootball.com
locatellicorretor.com.br livesex-amateure.info
lonerangergames.com loadopia.com
lonfon.xyz lombia.co
longgreen.info louboutinshoes.xyz
louboutinbooties.xyz luckyshop.net.ua
low-format.ru lyrster.com
lyngdalhudterapi.no mackeeper-land-672695126.us-east-1.elb.amazonaws.com
macfix.life macrotek.ru
madot.onlinewebshop.net magicalfind-a.akamaihd.net
malls.com mail.allnews24.in
mansparskats.com map028.com
marketingvici.com marcowebonyodziezowe.pl
marmitaco.cat marmotstore.online
mathgym.com.au marwer.info
maximilitary.ru matpre.top
mazda-roadsters.com mcadamssupplyco.com
medanestesia.ru me-ke.com
meds-online24.com mediaoffers.click
medtherapy.ru medizinreisen.de
meetlocalchicks.com meendo-free-traffic.ga
metalonly.info mesto-x.com
metaxalonevstizanidine.blogspot.com mir-limuzinov.ru
meteocast.net missclub.info
meteostate.com missvietnam.org
michaelkorssaleoutletonline.net mobile.ok.ru
michellblog.online mobilemedia.md
mobile-appster.ru moesexy.com
mobsfun.net monclerjacketsoutlet.pw
moivestiy.biz monetizer.com-01.site
moncleroutletonline.win
monitorwebsitespeed.com monitorwebsitespeed.com
moskva.nodup.ru moscow-region.ru
mp3films.ru mrbitsandbytes.com
mp3ritm.top mriyadh.com
mp3s.club musicdaddy.net
muznachas-service.ru muz-tracker.net
mydoctorok.ru mydirtyhobby.com
mygameplus.ru myindospace.com
myhealthcare.com myonlinepayday.co
myonigroup.com myseoconsultant.com
naj-filmy24.pl mystats.xyz
napalm51.nut.cc myxdate.info
naughtyconnect.com myyour.eu
nbsproject.ru namecrumilchlet.tk
nelc.edu.eg namenectar.com
net-radar.com narkologiya-voronezh.ru
netvouz.com neks.info
networkad.net neodownload.webcam
nfljerseys.online newhairstylesformen2014.com
nfvsz.com nextbackgroundcheck.gq
nibbler.silktide.com nextlnk12.com
nippon-bearings.ru nikhilbahl.com
nonews.co ningessaybe.me
nrjmobile.fr noclegonline.info
o-o-8-o-o.com nodding-passion.tk
odywpjtw.bloger.index.hr nordstar.pro
ok.ru novodigs.com
okuos.com novosti-hi-tech.ru
onemactrckr.com nyfinance.ml
onlyforemont.ru o333o.com
optitrade24.com obiavo.net
ordernorxx.com obiavo.su
orenburg-gsm.ru ochistka-stokov.ru
offergroup.info
ons-add.men
onthemarch.co
ooomeru.ru
orgasmatrix.com
osoznanie-narkotikam.net osoznanie-narkotikam.net
oyster-green.com page2rss.com
panouri-solare-acoperis.com partner-stop.men
paperwritingservice17.blogspot.ru pathwhelp.org
partnerline.men pekori.to
pdns.cz perfectplanned.com
pensplan.com petsblogroll.com
percin.biz.ly photosaga.info
perfectpracticeweb.com picmoonco.pw
petitions.whitehouse.gov picture-group.com
phrcialiled.com pipki.r.acdnpro.com
picsearch.com pizdopletka.club
pio.polytopesexempt.com
piulatte.cz
piuminiita.com
pivka.xyz
placid-rounded-coast.glitch.me placid-rounded-coast.glitch.me
play-movie.pl popmarker.com
poisk-zakona.ru popup-fdm.xyz
popander.mobi pornhub-ru.com
poptool.net porno-play.net
porndl.org porno-video-chati.ru
porngalleries.top
pornhive.org
pornofeuer.com
pornohd1080.online
pornokorol.com pornokorol.com
pornotubs.com pozdravleniya-c.ru
posible.net pozdrawleniya.com
preconnubial.usuby.site pr-ten.de
privetsochi.ru praisong.net
prodess.ru presleycollectibles.com
pts163.ru primedice.com
purchasepillsnorx.com privatefx-in.ru
puzo2arbuza.ru privatov-zapisi.ru
pyramidlitho.webs.com privhosting.com
randki-sex.com pro-tec.kz
rarbg.to produkto.net
raschtextil.com.ua profolan.pl
rating-bestcasino.com prom23.ru
realizmobi.com promalp-universal.ru
rebuildermedical.com promgirldresses.xyz
redirectme.net pronekut.com
redirlock.com proposal-engine.com
reeyanaturopathy.com prostitutki-kharkova.org
renecaovilla.online psvita.ru
renhacklids.tk py100.ru
resant.ru queerspace.com
rethinkwasteni.info rainbowice.ru
rfid-locker.co registry-cleaner.net
roleforum.ru remont-comp-pomosh.ru
ros-ctm.ru resellerclub.com
sack.net restorator-msk.ru
sammlungfotos.online richinvestmonitor.com
sbornik-zakonov.ru rostov.xrus.org
sdi-pme.com ru-mediaget.ru
searchengineranker.email rukino.org
searchlock.com rusoft-zone.ru
seasaltwithfood.com rusvideos.su
secret.xn--oogle-wmc.com rutor.group
semxiu.com sad-torg.com.ua
seo-platform.com sbdl.no
seoboxes.com scanner-nelson.top
seotoolsagency.com scripted.com
serptehnika.ru se.bnt-team.com
serw.clicksor.com searchimpression.com
sex-znakomstva.online seeingmeerkat.com
sexreliz.com sell-fb-group-here.com
shareyards.com semalt.com
shemale-sex.net seoprofiler.com
seozoom.it
sergiorossistore.online
serialsway.ucoz.ru
sex-dating.co
sex-pr.net
sexs-foto.com
sexystrippe.info
shakhtar-doneck.ru
share-buttons-for-free.com
sharebutton.to
shijian.ac.cn
shiksabd.com
shitting.pro
shmetall.com.ua
shohanb.com shohanb.com
shopwme.ru shopvilleroyboch.com.ua
shymkent.xkaz.org si-unique.com
sibvitr.ru sindragosa.comxa.com
signal03.ru sitebeam.net
signoredom.com siteonomy.com
simpoed.ufop.br skanninge.se
site3.free-share-buttons.com sladkoevideo.com
sitiz.club slavic-magic.ru
sitopreferito.it smart-scripts.com
smartbalanceworld.com smartadserver.com
smsactivator.ru smsactivator.ru
snts.shell-pmr.ru social-vestnik.ru
socialbookmarksubmission.org sockshare.net
socialbuttons.xyz softomix.com
sortthemesitesby.com sohoindia.net
soserfis.com spitfiremusic.com
spamnuker.com splendorsearch-a.akamaihd.net
sports-supplements.us
spravka130.ru spravka130.ru
stackthatbucks.com spywarenuker.com
stard.shop srdrvp.com
stockspmb.info srecorder.com
stroicol.net staff.prairiesouth.ca
sugarkun.com startwp.org
sugarlyflex.pw stathat.com
sumo.com staynplay.net
superiends.org stroyplus.ru
supermesta.ru success-seo.com
surffoundation.nl supermama.top
swingerseiten.com svarkagid.com
t-bygg.com taaaak.com
tecspb.ru tabletkinaodchudzanie.com.pl
teencastingporn.com takethatad.com
teesdaleflyballclub.co.uk tandvardshuset.net
telegraf.by taqplayer.info
telegramdownload10.com taranerymagesswa.blogspot.com
telemetryverification.net taximytishi.ru
tech4master.com
testbotprocessor44.com testbotprocessor44.com
testingads.pro thetattoohut.com
texbaza.by tiandeural.ru
tfxiq.com tizanidine4mgtablets.blogspot.com
thebitcoincode.com tizanidine6mg.blogspot.com
thewebsitetemplate.info tizanidinehcl4mgtab.blogspot.com
titslove.yopoint.in tizanidinehydrochloride2mg.blogspot.com
tizanidine4mgstreetvalue.blogspot.com tizanidineinfo.blogspot.com
tizanidinecapsules.blogspot.com
tizanidinemusclerelaxant.blogspot.com tizanidinemusclerelaxant.blogspot.com
top1-seo-service.com tmm-kurs.ru
tooplay.com
topkarkas.com topkarkas.com
torontoplumbinggroup.com topshef.ru
torrentdownloadhub.com tourcroatia.co.uk
totu.info toxicwap.com
truck-addzilla.life toys.erolove.in
trudogolik.net track.deriv.com
ttrraacckkrr.com traffic2money.com
tupper-posuda.ru traffique.net
turist-strani.ru treasuretrack-a.akamaihd.net
twodollarshows.com umaseh.com
twu.com.ua urll.eu
tyumen.xrus.org uzporno.mobi
ucoz.ru vacances-voyages.info
ucsol.ru
unionmarkt.de
unisexjewelry.org
upproar.com
urlopener.blogspot.com.au
utiblog.fr
validdomain.xyz
vclicks.net
velen.io velen.io
videonsk.com viagra.pp.ua
videos-for-your-business.com video--production.com
vip-dom.in video-woman.com
vipromoffers.com videotuber.ru
vipsexfinders.com vinylvault.co.uk
vk-mus.ru vip-parfumeria.ru
voprosotvet24.ru vipsiterip.org
vriel.batcave.net virus-schutzmasken.de
vrnelectro.ru vitalads.net
wallpaperaccess.com vkmusics.ru
waterpurifier.club vkonche.com
web-betting.ru vladimir.xrus.org
webads.co.nz vsdshnik.com
wejdz-tu.pl vykupavto-krasnodar.ru
whatsappdownload10.com wait3sec.org
wheelchairliftsarea.com walkme.com
while.cheapwebsitehoster.com warningzscaler.heraeus.com
waysbetter.cn
wdrake.com
webix.me
webradiology.ru
website-stealer.nufaq.com
wesharepics.info
westen-v.life
white-truck.life
wholesalejerseychinashop.com wholesalejerseychinashop.com
wildcattube.com wildcattube.com
wingsofrefuge.net windowssearch-exp.com
wma-x.com wineration.com
wnhjavlhezp.gq winwotgold.pl
woman-tampon.ru womensplay.net
wonderfulflowers.biz wordpresscore.com
worldhistory.biz wstroika.ru
ww1943.ru xaijo.com
wzgyyq.com xbaboon.com
xblognetwork.com xblognetwork.com
xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks xcc24.pl
xn----9sbubg3ambdfl1j.xn--p1ai xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
xn--80aanaardaperhcem4a6i.com
xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai
xn--hxazdsfy.blogspot.com xn--hxazdsfy.blogspot.com
xvideosbay.com xrus.org
xxxtube69.com xtrafficplus.com
yatrk.xyz y8games-free.com
yellowstonevisitortours.com yachts-cruise.info
youdesigner.kz yebocasino.co.za
yourporngay.com yourmovies.pl
youtube-downloader.savetubevideo.com
ytmnd.com ytmnd.com
zambini.ru z-master.ru
zarabotat-na-sajte.ru zarajbuilders.com
zastenchivosti.net zebradudka.com
zigarettenonlinekaufen2.bloog.pl zeroredirect1.com
zixizop.net.ru zigarettenonlinekaufen1.bloog.pl
znakomstva-moskva77.ru zoolubimets.ru
znakomstvaonlain.ru zskdla.site
zrelaya.pw zvooq.eu
zrus.org

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
256bit.by 03e.info
273-fz.ru 100dollars-seo.com
2nt.ru 1bet.com
55wmz.ru 28n2gl3wfyb0.ru
7minuteworkout.com 883zy.com
91abcw.com 91abcw.com
aasoldes.fr aa14ab57a3339c4064bd9ae6fad7495b5f.com
abcdefh.xyz adnotbad.com
adanih.com ads-seo.men
addtoadd.men adserver-e7.com
adinterax.com adsfresh.men
admanaerofoil.com adtology.com
adultnet.in afftrack001.com
advertex.info akuhni.by
agecheckadult.com allfinweb.com
aghanyna.com
alf-img.com
allnews.md
alloysteel.ru
allwomen.info
alpharma.net alpharma.net
alyeskaresort.com amateurmatch.com
angry-fermi-7633.arukascloud.io anime.dougasouko.com
apiportalmorecom-a.akamaihd.net arate.ru
arquivo.pt arenda-avtoprokat-krasnodar.ru
asophoto.com arendatora.ru
atlasvkusov.ru
atout-energie-69.com
aucoinhomes.com
auto-moto-elektronika.cz
aviav.org aviav.org
ayanaspa.com aviav.ru.com
azazaporn.com avirasecureserver.com
b00kmarks.com azbaseimages.net
bad-stars.net baersaratov.ru
belsetirehafi.tk bamo.xsl.pt
best-businessman.ru bazaronline24.ru
betteroffers.review best-way.men
big-cash.net bestofupload.info
bikini-image.com big-boards.info
bimatoprost-careprost.com bjetjt.com
bonvillan.com blackwitchcraft.ru
brainboosting.club bouda.kvalitne.cz
brains2.biz brandehk.dk
bronzeaid-a.akamaihd.net brillianty.info
buyessay3.blogspot.ru buy-cheap-online.info
camel-beach.com cacheimages.com
calcularpagerank.com
cannibalcheerleader.com cannibalcheerleader.com
cascadelink.org cartechnic.ru
ciproandtizanidine.blogspot.com cartierlovebracelet.xyz
constantaservice.net cartierring.xyz
cottageofgrace.com cdncash.net
crest-poloski.ru cheapjerseysa.com
danschawbel.com cialis-samples.com
design-lands.ru clodo.ru
diminishedvalueoforegon.com cocolyze.com
divci-hry.info com-secure.download
dogoimage.com comeondog.info
dojki365.online commoncrawl.org
domaineaneblanc.com compliance-alex.xyz
domcran.net construmac.com.mx
downloadkakaotalk.com copro.pw
dyt.net countbertwistdisp26.soup.io
e-commerce-seo1.com creams.makeforum.eu
eachdayisagift.review creativehutindia.com
cutt.us
cwetochki.ru
dandiyabeats.in
dickssportinggoods.com
djstools.com
donvito.unas.cz
drugstoreforyou.com
easycommerce.cf
easyukraine.com easyukraine.com
electricwheelchairsarea.com education-cz.ru
en.altezza.travel elektrischezigarette2.onsugar.com
enbersoft.com emergencyneeds.org
eropornosex.ru englishgamer.com
exoclick.com erotik-kostenlos.net
extractorandburner.com europeanwatches.ru
eyelike.com.ua everflora.ru
ezigarettenshop1.yolasite.com expdom.com
familienzahnaerzte.com facecup.top
familyholiday.ml falcon-images.blogspot.com
falcoware.com
fast-wordpress-start.com
fastfixing.tech fastfixing.tech
findsexguide.com filesclub.net
fingerprintjs.com filunika.com.ru
flashahead.info firstdrugmall.ru
flashlarevista.com flashlarevista.com
floridahuntingfishingadventures.com florida-tourism.net
forpostlock.ru g7m.pl
fotzen-ficken.com gallerily.com
fr.netlog.com go2jump.org
freemags.cc gq-catalog.gq
gg-arena.ru gratuitbaise.com
gifspics.com greenzaim.ru
gigixo.com gsmlab.pl
googlefeud.com guigyverpo.cf
googst2.ru heatpower.ru
graphicwe.org hikesearch.net
greatgrace.ru hkladys.com
guge.io hollywoodactress.info
hannasolution.ru horoshieokna.com
hatedriveapart.com host-tracker.com
highspeed5.net ido3.com
hitsbox.info iflycapetown.co.za
hoste.octopis.com igrovyeavtomaty777.ru
hosting-tracker.com infospot.pt
idealtits.net instabid.tech
ilovevitaly.ru integritylandscapeservices.com
imperia31.ru irkutsk.zrus.org
importchinacoach-teach.com iyasimasennka.com
iqs.biz.ua jabimgo.pw
iradiology.ru jamiembrown.com
istizanidineanarcoticdrug.blogspot.com jasonpartington.com
istock-mebel.ru javatex.co.id
itch.io jo24news.com
izoll.ru kellyonline.xyz
jyvopys.com kerei.ru
kareliatobacco.ru keywordbasket.com
kaz.kz kiwi237au.tk
keyword-suggestions.com kliksaya.com
keywordhut.com knigonosha.net
kiinomaniak.pl koleso24.com.ua
kiprinform.com ks1234.com
knowyournextmove.com kskjco.club
konturkrasoty.ru labvis.host.sk
ktotut.net ladsblue.com
ledpolice.ru landinez.co
lignofix.ua landreferat.ru
liquimondo.com lecbter-relationships.ga
live-sexcam.tk lion.misba.us
locatellicorretor.com.br livingroomdecor.info
lomb.co lollypopgaming.com
magicdiet.gq lrac.info
marktforschung-stuttgart.com lsp-awak-perikanan.com
matpre.top lyrics.home-task.com
mcadamssupplyco.com marketingvici.com
mebel-ekb.com matb3aa.com
mejoresfotos.eu mbiologi.ru
mnogabukaff.net meccadumps.net
modenamebel.ru medtherapy.ru
modnie-futbolki.net mirmedinfo.ru
morefastermac.trade mixtapetorrent.com
mylovelibrabry.com mjchamonix.org
mymobilemoneypages.com mobioffertrck.com
mywallpaper.top moneyzzz.ru
nakozhe.com musicktab.com
nextconseil.com nastroyke.net
nextlnk12.com needtosellmyhousefast.com
nikhilbahl.com nordstar.pro
noumeda.com nucia.biz.ly
oballergiya.ru office-windows.ru
ochistka-stokov.ru
offtime.ru offtime.ru
okuos.com oivcvx.website
one.net.in onlinewritingjobs17.blogspot.ru
oralsexfilme.net optom-deshevo.ru
ozoz.it oxotl.com
paccohichetoti.ml panchro.xyz
page2rss.com partner-trustworthy.men
pechikamini.ru pila.pl
pewit.pw pornohub.me
phrcialiled.com pornokajf.com
poem-paying.gq pornonik.com
pogodnyyeavarii.gq pornosee.info
pornoblood.com pornowarp.info
pornoklad.net pourvous.info
porto.abuilder.net privat-girl.net
professionaldieselcare.com pro-poly.ru
profitsport.club prokotov.com
prombudpostach.com.ua promoheads.com
pronorm.fr promover.org
psoriasis-file.trade psvita.ru
pulseonclick.com pyramidlitho.webs.com
responsive-test.net r-e-f-e-r-e-r.com
roma-kukareku.livejournal.com rapidsites.pro
rucrypt.com razvratnoe.org
rusprostitute.com realmonte.net
rvi.biz redhotfreebies.co.uk
sajatvelemeny.com rff-cfal.info
salutmontreal.com rome2rio.com
sammlungfotos.online romhacking.ru
sape.top royalads.net
sbf441.com rrutw.com
sbwealthsolutions.ca rumamba.com
scanner-mark.top rusenvironmental.net
scanner-walter.top sarahmilne.top
scansafe.net scanner-alex.top
scanspyware.net scanner-brian.top
searchinquire.com scanner-don.top
seasonvar.ru scanner-fred.top
seo-smm.kz schalke04fc.info
seo-tools-optimizing.com screpy.com
service-core.ru seasaltwithfood.com
sexs-foto.top
sharebutton.to sharebutton.to
shemalegalls.blogporn.in skypasss.com
shopsellcardsdumps.com sloopyjoes.com
simul.co smadihome.com
site5.com smartbalanceworld.com
sogimlecal.tk softlinesolutions.me
spaceshipad.com spidtest.space
storehouse.ua steelmaster.lv
strongsignal-a.akamaihd.net streetfire.net
svetlotorg.ru stuff-about-money.com
tacbibirfa.tk sucsesofinspiration.com
tattomedia.com svetodiodoff.ru
td-l-market.ru szucs.ru
techart24.com telegramdownload10.com
teenfuck.tv textads.men
telefonsexsofort.tk the-trader.net
telesvoboda.ru thecounter.com
theautoprofit.ml thenews-today.info
thecoolimages.net thepokertimer.com
thegioixekhach.com tivolibasket.it
tizanidine4mg.blogspot.com tizanidineformigraines.blogspot.com
tizanidinesideeffectsweightloss.blogspot.com tizanidinehcl4mgisitanarcotic.blogspot.com
tk-assortiment.ru todohr.com
tomck.com torrents.life
tooplay.com tozup.com
topflownews.com trion.od.ua
tracksz.co tsyndicate.com
tracxn.com turbabitload.weebly.com
triumf-realty.ru tvorozhnaja-zapekanka-recept.ru
trubywriting.com tvory.predmety.in.ua
tube8.com twelvevisionspartyofcolorado.com
tvteleport.ru typimga.pw
unlimitdocs.net ukrup.com
uogonline.com us-america.ru
ustion.ru utrolive.ru
vereo.eu uzporno.mobi
vezuviy.su vacuumcleanerguru.com
viagra.pp.ua vasileostrovsky-rayon.ru
video-production.com vektorpress.ru
vip-dom.in venta-prom.ru
visitcambridge.org via-energy-acquistare.com
vitalads.net videochaty.ru
voprosotvet24.ru videospornogratisx.net
webextract.profound.net vkontaktemusic.ru
webpromotion.ae vonradio.com
website-speed-up.top vtc.pw
welcomeauto.ru vzubah.com
wineitudes.wordpress.com wait3sec.org
wufak.com watch-movies.ru
x-mix.info wnhjavlhezp.gq
x-true.info worldmusicfests.com
xblog.in x-rates.ru
xchat26.myfreecams.com xxxhdvideo.site
xn--80aafb2a.xn--p1ai xxxmania.top
xn--80ak6aa92e.com yachts-cruise.info
xn--d1aifoe0a9a.top yur-p.ru
yaderenergy.ru zawyna.ua
youjizz.vc zero1.it
yurgorod.ru zryydi.com
za-music.mymobiplanet.com zs2vm.top
zeroredirect.com zscalertwo.net
zeroredirect11.com
zygophyceous.womanstars.site

24
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -1,21 +1,11 @@
<!DOCTYPE html> <html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html> ml><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>
@ -581,4 +571,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.31 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2915 ### Version: V4.2022.02.2916
### Updated: Sun Feb 13 10:04:14 UTC 2022 ### Updated: Sun Feb 13 22:01:28 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2915 ### Version: V4.2022.02.2916
### Updated: Sun Feb 13 10:04:14 UTC 2022 ### Updated: Sun Feb 13 22:01:28 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##