vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 11:10:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.11.2687 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-11-01 10:18:29 +00:00
parent 78a0d8cca3
commit 82887ca400
48 changed files with 70031 additions and 70031 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2686 #### Version: V4.2021.11.2687
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2686 #### Version: V4.2021.11.2687
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.10.2686 #### Version: V4.2021.11.2687
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8618
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.10.2686 #### Version: V4.2021.11.2687
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8622
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8622
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8622
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8622
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.48 0;
127.0.0.1 0; 127.0.0.1 0;

8622
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.48 0;
127.0.0.1 0; 127.0.0.1 0;

8622
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.48 0;
127.0.0.1 0; 127.0.0.1 0;

8622
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2686 2687

8622
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8622
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.106.161.109 0; 20.120.11.95 0;

8622
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.48 0;
127.0.0.1 0; 127.0.0.1 0;

8622
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.48 0;
127.0.0.1 0; 127.0.0.1 0;

8622
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.48 0;
127.0.0.1 0; 127.0.0.1 0;

8622
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.106.161.109 0; 20.120.11.95 0;

8622
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.106.161.109 0; 20.120.11.95 0;

8622
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 1; 10.1.1.48 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
HMView CensysInspect
IDBTE4M EyeNetIE
Morfeus Fucking Scanner FHscan
OpenLinkProfiler LinkWalker
PageGrabber RSSingBot
WebsiteExtractor TheNomad
WinHTTrack WASALive-Bot
coccocbot-web Web Collage
pcBrowser ZumBot
satoristudio.net mediawords

306
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
404checker
404enemy
ADmantX
ALittle Client
Abonti Abonti
Aboundex Aboundex
Aboundexbot Aboundexbot
AiHitBot AhrefsBot
Alexibot
Alligator Alligator
Anarchie AlphaBot
Anarchy Ankit
Asterias Asterias
Attach
AwarioSmartBot AwarioSmartBot
BBBike BBBike
BackDoorBot BDCbot
BackStreet BDFetch
BLEXBot
Backlink-Ceck Backlink-Ceck
BacklinkCrawler
Badass Badass
Bandit Bandit
Barkrowler Barkrowler
BetaBot BetaBot
Bigfoot Blackboard
Bitacle Brandprotect
Blow
Boardreader
Bolt
BotALot
BuiltBotTough BuiltBotTough
BunnySlippers Bullseye
CATExplorador CATExplorador
Calculon CazoodleBot
Cegbfeieh
CensysInspect
CheTeam
CheeseBot CheeseBot
CherryPicker
Claritybot Claritybot
Cliqzbot Cocolyzebot
Cloud mapping Copier
Cogentbot Cosmos
Craftbot Crawling at Home Project
Crescent
CrunchBot CrunchBot
Curious Curious
CyotekWebCopy Custo
DBLBot DBLBot
DIIbot DataCha0s
DTS Agent DatabaseDriverMysqli
DigitalPebble Deusu
DnBCrawler-Analytics Devil
DomCopBot Disco
Download Wonder DnyzBot
Dragonfly DomainCrawler
EasyDL DomainSigmaCrawler
EMail Wolf
Ebingbong
EroCrawler
Evil Evil
ExtLinksBot
ExtractorPro ExtractorPro
FDM EyeNetIE
FHscan FHscan
Fimap Fimap
FlashGet FlashGet
Fuzz Flunky
FyberSpider FrontPage
G-i-g-a-b-o-t Fyrebot
GermCrawler GermCrawler
GetWeb GetRight
Getintent Go!Zilla
Go-Ahead-Got-It Grafula
GoZilla GridBot
GrapeFX
GrapeshotCrawler
HEADMasterSEO
HMView HMView
HTTrack HTTP::Lite
Harvest Harvest
Hloader Humanlinks
HybridBot
IDBTE4M IDBTE4M
IDBot IDBot
IRLbot
IlseBot IlseBot
Image Fetch
Image Sucker Image Sucker
InfoNaviRobot
Intelliseek
InterGET InterGET
Internet Ninja Internet Ninja
InternetSeer
IstellaBot IstellaBot
JOC Web Spider JOC Web Spider
JennyBot JamesBOT
JetCar Jbrofuzz
JikeSpider
Jorgee Jorgee
Jyxobot Jyxobot
Kozmosbot Keyword Density
Kinza
LWP::Simple
Lanshanbot
Larbin Larbin
LeechFTP LeechFTP
LeechGet LeechGet
LibWeb LibWeb
Libwhisker
LieBaoFast
Lightspeedsystems Lightspeedsystems
LinkScan
LinkWalker LinkWalker
Linkbot
Linkdexbot Linkdexbot
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Spider
Ltx71
MIDown tool
MQQBrowser MQQBrowser
MSFrontPage MSIECrawler
MTRobot
Mag-Net Mag-Net
Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic-SEO Mediatoolkitbot
MarkMonitor MicroMessenger
Mass Downloader Mister PiX
Masscan Mojolicious
Mata Hari
MauiBot
Mojeek
Morfeus Fucking Scanner Morfeus Fucking Scanner
Musobot Mozlila
NPbot Mr.4x3
NICErsPRO
Nameprotect
Navroad
NearSite NearSite
Needle Needle
Nessus
Net Vampire
NetLyzer NetLyzer
Netvibes NextGenSearchBot
Nibbler Nikto
Nimbostratus Nmap
Nutch OnCrawl
Octopus
OpenLinkProfiler
OpenVAS OpenVAS
Openfind
OrangeSpider
OutfoxBot OutfoxBot
PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageAnalyzer PageGrabber
Papa Foto PageScorer
PeoplePal Pavuk
Pi-Monster Pi-Monster
Picscout Picsearch
Piepmatz Piepmatz
Pimonster
Pixray Pixray
PleaseCrawl Pockey
Pump
PxBroker PxBroker
PyCurl PyCurl
QueryN Metasearch QueryN Metasearch
Quick-Crawler RankActiveLinkBot
RankingBot RankingBot
RankingBot2 Recorder
Rankivabot RedesScrapy
Re-re
Reaper
RocketCrawler RocketCrawler
SBIder SBIder
SalesIntelligent SMTBot
Scrapy ScanAlert
Semrush Screaming
ScrepyBot
SemrushBot SemrushBot
SentiBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz
Shodan
Site Sucker
SiteCheckerBotCrawler
SiteSnagger SiteSnagger
SmartDownload Sitevigil
Snake Snake
Snapbot Snapbot
SocialRankIOBot Snoopy
Sogou web spider Sociscraper
Sottopop Sosospider
Spammen SpaceBison
SpankBot
Spbot Spbot
SputnikBot
Sqlmap Sqlmap
Sqworm
Steeler Steeler
Stripper Stripper
Sucker
Sucuri
SuperHTTP SuperHTTP
Surfbot Surfbot
SurveyBot
Suzuran
TeleportPro TeleportPro
Telesphorep
Thumbor
TightTwatBot TightTwatBot
Titan
Toweyabot Toweyabot
Tracemyfile
Trendiction Trendiction
Trendictionbot Trendictionbot
True_Robot Turingos
Turnitin Turnitin
Upflow TwengaBot
VCI URLy Warning
Vacuum VB Project
Vagabondo VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
Virusdie VidibleScraper
VoidEYE WASALive-Bot
Voltron WWW-Collector-E
WBSearchBot WWWOFFLE
WEBDAV
WISENutbot
Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WeSEE WallpapersHD
Web Auto Web Pix
Web Collage Web Sucker
Web Enhancer WebCollage
Web Sauger WebFetch
WebAuto WebImageCollector
WebCollage WebLeacher
WebFuck WebSauger
WebSauger
WebStripper
WebSucker WebSucker
WebWhacker
Webalta Webalta
WebmasterWorldForumBot WebsiteQuester
WebsiteExtractor Whack
Webster WinHTTrack
Whacker Wonderbot
Whatweb
Who.is Bot
Widow
Woobot Woobot
Xaldon_WebSpider Wotbox
ZmEu Wprecon
arquivo-web-crawler Xaldon WebSpider
arquivo.pt Zade
autoemailspider Zermelo
check1.exe ZoomBot
ZoominfoBot
ZumBot
ZyBorg
backlink-check
cah.io.community
clark-crawler
cognitiveseo cognitiveseo
crawler.feedback com.plumanalytics
domainsproject.org domainsproject.org
eCatch
evc-batch evc-batch
gopher facebookscraper
heritrix heritrix
instabid
ips-agent ips-agent
lwp-request lwp-request
polaris version meanpathbot
ripz mediawords
s1z.ru muhstik-scan
netEstate NE Crawler
oBot
pcBrowser
plumanalytics
satoristudio.net
scalaj-http scalaj-http
seostar scan.lol
seobility
sexsearcher sexsearcher
sitechecker.pro siteripz
sp_auditbot
spyfu spyfu
trendiction.com sysscan
webprosbot tAkeOut
webmeup-crawler
x09Mozilla x09Mozilla
zauba.io
zgrab

312
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404checker
80legs 80legs
ADmantX ALittle Client
ASPSeek ASPSeek
Abonti
Aboundex
Aboundexbot Aboundexbot
AfD-Verbotsverfahren AhrefsBot
AllSubmitter
Alligator Alligator
AlphaBot
Anarchie Anarchie
Anarchy
Anarchy99 Anarchy99
Ankit
Anthill Anthill
Aspiegel Asterias
BBBike AwarioSmartBot
BDCbot
BDFetch BDFetch
BackStreet
BackWeb BackWeb
BatchFTP BacklinkCrawler
Bitacle Badass
Black Hole Bandit
BlackWidow Battleztar Bazinga
Blow BetaBot
BotALot Bigfoot
Buck Blackboard
Boardreader
Bolt
Brandprotect
Brandwatch
BuiltWith
Bullseye Bullseye
BunnySlippers CSHttp
BuzzSumo Cegbfeieh
CATExplorador CheTeam
Calculon CherryPicker
CensysInspect Cliqzbot
Claritybot Copier
Cloud mapping Crescent
Cocolyzebot
CopyRightCheck
Copyscape
Craftbot
CrazyWebCrawler
CrunchBot
Curious Curious
CyotekWebCopy
DBLBot DBLBot
DTS Agent DTS Agent
Deusu Devil
Digincore
Dirbuster Dirbuster
Discobot
DittoSpyder DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DomCopBot DnyzBot
DomainAppender DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Dotbot Download Wonder
Drip
ECCP/1.0
EMail Siphon
EMail Wolf EMail Wolf
Ebingbong Extractor
Ecxi ExtractorPro
EroCrawler
Evil
Extreme Picture Finder
EyeNetIE EyeNetIE
Ezooms Ezooms
FHscan FDM
Firefox/7.0
Flunky
Freeuploader Freeuploader
FyberSpider Fuzz
G-i-g-a-b-o-t GetWeb
GermCrawler
Getintent Getintent
Gigabot Gigabot
GoZilla
Gotit Gotit
GrabNet
Grabber
Grafula Grafula
GrapeFX GridBot
HEADMasterSEO HEADMasterSEO
HMView
HTMLparser
Haansoft Haansoft
IDBTE4M HaosouSpider
Harvest
Havij
Humanlinks
IDBot IDBot
IRLbot IRLbot
IlseBot IndeedBot
Image Sucker
Indy Library
InfoNaviRobot
Intelliseek
InterGET InterGET
Internet Ninja
Iria Iria
Iskanie
IstellaBot
JOC Web Spider JOC Web Spider
Jorgee JikeSpider
JustView
Kenjin Spider
Keyword Density Keyword Density
Kozmosbot LNSpiderguy
LWP::Simple LieBaoFast
Lanshanbot
Leap
Lftp
LibWeb
Lightspeedsystems
Likse
LinkWalker
Linkbot
LinkextractorPro
LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot
Litemage_walker Litemage_walker
MIDown tool
MIIxpc MIIxpc
MSFrontPage MQQBrowser
Majestic SEO MSIECrawler
MTRobot
Mag-Net
Mail.RU_Bot
Majestic-SEO Majestic-SEO
MarkMonitor
MarkWatch MarkWatch
Mass Downloader Mata Hari
MauiBot MauiBot
Mb2345Browser Mb2345Browser
Meanpathbot Mediatoolkitbot
MegaIndex.ru
Metauri Metauri
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Mister PiX Moblie Safari
Musobot Mojeek
NPbot Mojolicious
Needle Morfeus Fucking Scanner
Net Vampire Msrabot
NetSpider Nameprotect
Navroad
NetLyzer
NetMechanic
NetZIP
Netcraft Netcraft
Nettrack Nibbler
NextGenSearchBot Ninja
Nimbostratus
Nmap Nmap
OnCrawl Nutch
PageScorer Offline Explorer
PageThing.com OpenLinkProfiler
Pavuk Openfind
Petalbot Openvas
OutclicksBot
PECL::HTTP
POE-Component-Client-HTTP
PageAnalyzer
Pandalytics
Papa Foto
PictureFinder PictureFinder
Piepmatz Pimonster
Pixray Pixray
Pockey ProPowerBot
ProWebWalker Pu_iN
Probethenet
Psbot
PxBroker
Quick-Crawler Quick-Crawler
RankActiveLinkBot
RankFlex RankFlex
RankingBot RankingBot
RankurBot Rankivabot
Re-re
RealDownload RealDownload
Reaper Reaper
RebelMouse
Recorder Recorder
Ripper RepoMonkey
RocketCrawler
SBIder SBIder
SEOprofiler
SEOstats SEOstats
SalesIntelligent SMTBot
Scanbot Scanbot
Scrapy ScoutJet
Screaming Screaming
ScreenerBot ScreenerBot
ScrepyBot
Searchestate
SearchmetricsBot SearchmetricsBot
SeobilityBot SentiBot
Seomoz
Shodan Shodan
SiteSnagger Site Sucker
SiteLockSpider
Siteimprove
SlySearch
SmartDownload SmartDownload
Snake SocialRankIOBot
Snapbot
Snoopy
Sociscraper
Sogou web spider Sogou web spider
SpaceBison SpaceBison
SputnikBot Spammen
Sqlmap
Sqworm
Steeler Steeler
Stripper Stripper
Sucker SurveyBot
Sucuri Swiftbot
SuperBot
Surfbot
Suzuran
Szukacz Szukacz
T0PHackTeam
T8Abot
Teleport
Telesoft
Telesphoreo Telesphoreo
The Intraformant Telesphorep
TheNomad TheNomad
TightTwatBot Thumbor
Toweyabot Toata
TurnitinBot Tracemyfile
V-BOT Trendictionbot
VCI TwengaBot
Vacuum Typhoeus
URLy.Warning
UnisterBot
VB Project
Vagabondo Vagabondo
VoidEYE VeriCiteCrawler
VidibleScraper
Virusdie
Voltron Voltron
WEBDAV
WWW-Collector-E WWW-Collector-E
WWWOFFLE WWW-Mechanize
Wallpapers/3.0 WWW::Mechanize
Web Auto Web Collage
Web Fetch
Web Pix Web Pix
Web Sauger WebAuto
Web Sucker
WebBandit WebBandit
WebCollage
WebCopier WebCopier
WebImageCollector
WebLeacher WebLeacher
WebStripper WebSucker
Webshag WebWhacker
WebsiteQuester Webster
Whatweb Whacker
Who.is Bot
Widow Widow
Wonderbot WinHTTrack
Woobot WiseGuys Robot
Xaldon WebSpider YoudaoBot
Zeus Zermelo
ZmEu
ZoomBot
ZoominfoBot
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt
check1.exe check1.exe
clark-crawler
coccocbot-web
cognitiveseo
com.plumanalytics com.plumanalytics
crawl.sogou.com
crawler.feedback
dataforseo.com dataforseo.com
demandbase-bot demandbase-bot
domainsproject.org
eCatch eCatch
evc-batch
facebookscraper facebookscraper
instabid heritrix
internetVista monitor
ips-agent
isitwp.com
lwp-request lwp-request
mediawords meanpathbot
muhstik-scan plumanalytics
netEstate NE Crawler
page scorer
probe-image-size probe-image-size
ripz s1z.ru
satoristudio.net satoristudio.net
scalaj-http scalaj-http
seostar
sexsearcher sexsearcher
sysscan tAkeOut
trendiction.de trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
webmeup-crawler
webpros.com webpros.com
webprosbot
x09Mozilla
x22Mozilla x22Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
7wind.ru appmsr.org
ad-words.ru babyfactory.fr
buypillsorderonline.com blockety.co
ezigarettenshop2.postbit.com cellfun.mobi
memberty.com goatse.ru
music7s.me hop.clickbank.net
sampleletters.net lawyersinfo.org
scanner-marwin.top remont-mobile-phones.ru
zastenchivosti.net wordpress-crew.net
zetgie.com.pl yhit.press

910
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1001desert.com 00author.com
123locker.com 0c47f8422d3f.com
1pamm.ru 101billion.com
1stat.ru 123movies.love
2itech.ru 256bit.by
5ws.dating-app.ru 2drittel.de
8kisses.eu 2rich4bitches.com
aa14ab57a3339c4064bd9ae6fad7495b5f.com 4rent.ru
aasoldes.fr 55wmz.ru
activepr.ru 70casino.online
actulite.com 76brighton.co.uk
adloads.net 7a2rnuey1tw9ar.ru
adman.gr 7fon.ru
adserver-e7.com 7search.com
adsfresh.men 8gold.com
adtech.us a14download.com
adtiger.tk aa08daf7e13b6345e09e92f771507fa5f4.com
ahhjf.com abclauncher.com
alborzan.com absolutelyanalog.com
ald2014.org actionnooz.com
alert.scansafe.net adsref.men
all4invest.ru advertex.info
allornamenti.com advokat-grodno.by
allsilver925.co.il afora.ru
amanda-porn.ga akuhni.by
amigobulls.com albamargroup.com
angigreene.com alevinefesleri.com
anonymous-redirect.com alibestsale.com
anticrawler.org alldezservice.kz
apartmentbay.ru alldubai.biz
apartmentratings.com allkrim.com
apisecretsaucebi-a.akamaihd.net allnews.md
alltheviews.com
allwidewallpapers.com
allwomen.info
alpharma.net
amazingpic.net
androids-store.com
anlimebel.kiev.ua
api.stathat.com
apiadanaknet-a.akamaihd.net
apiwebwebgetcom-a.akamaihd.net apiwebwebgetcom-a.akamaihd.net
appleid-verification.com appearance-cool.com
aquarium-pro.ru apper.de
artavenuegardenstudios.com apxeo.info
asociatia-tipografilor-transilvania.ro arabgirls.adultgalls.com
asrvrep-a.akamaihd.net ardimobileinfo.ml
atlasvkusov.ru artdeko.info
atovh.local-finders.com artdestshop.eu
ausmepa.org.au artparquet.ru
aviav.co asseenontvonline.ru
aviav.org atofilms.com
ayeartoforget.com audiofree.ru
azinoofficial777.ru aviapanda.ru
azlex.uz avitocars.ru
backlink4u.net avramstroy.ru
backuperwebcam.weebly.com avto-oligarh.ru
bannerconnect.net backiee.com
barbourjackets.online baditri.com
bedandbreakfast.com balitouroffice.com
best-seo-solution.com bangla-chat-uk-paralud.ga
betonka.pro banksrf.ru
bitcoins-live.ru bdsmgalls.net
boanonihaca.tk beauty-b0x.pl
bonanza-fortune.men beauty-clinic.ru
boo-arts.com becuo.com
boobsimge.com bestpornuha.com
bidbuy.co.kr
bif-ru.info
big-boards.info
binomo.kz
biteg.xyz
bizlist.com.de
bkgr.se
blog-hits.com
bluerobot.info
bmsco.co
bosman.pluto.ro
brains2.biz
breastaugmentation.co.za breastaugmentation.co.za
brendbutik.ru bumble.cheapwebsitehoster.com
brimstonehillfortress.org buzzsumo.com
bronzeaid-a.akamaihd.net cam-kontakte.org
brothers-smaller.ru
burkesales.com
businesn.men
buypanicdisorderpill.com
buytizanidineonlinenoprescription.blogspot.com
bzero1jewelry.net
callawaygolfoutlet.online
canacopegdl.com
candiceloves.us
capture-room.com
carsdined.org
carson.getenjoyment.net carson.getenjoyment.net
cartechnic.ru cartierjusteunclou.xyz
cartiertrinity.xyz cartierlove2u.com
cashonads.com cartierlovebracelet.xyz
celejihad.info cartierloveringreplica.xyz
cheapjerseysfootballshop.com cartujano-pre.de
cb.iphantom.com
censys.io
ceotrk.com
cfacarrosserie74.com
chatroulette.online
cheapmoncler.win
cheatcode-lita12.rhcloud.com cheatcode-lita12.rhcloud.com
choosecuisine.com chelnytruck.ru
chinese-amezon.com
chorus.terakeet.com chorus.terakeet.com
christianlouboutinreplica.win christianlouboutinoutlet.win
cialisndbrx.com cialischmrx.com
cienum.fr cialischsrx.com
cityadspix.com circlesl.com
clicksor.com cjmc.info
collegeessay19.blogspot.ru classicakuhni.ru
collegerentals.com cleanersoft.com
com-supportcenter.website cleanmypc.com
compliance-julia.top clickbanksites.info
compliance-mark.top clickso.com
clipartnew.com
clksite.com
cocolyze.com
com-secure.download
compliance-andrew.top
compliance-ivan.xyz
compliance-julianna.top
compliance-olga.top compliance-olga.top
compliance-willy.top counciltally.com
computernetworksonline.com country-chic.ru
copblock.org courtshipgift.com
covadhosting.biz cpabegins.ru
credit-online.ws creativehutindia.com
customcollegeessays.net customsua.com.ua
cutt.us dailyrank.net
cyse.tk dandiyabeats.in
czat.wp.pl daptravel.com
damasarenai.info datsun-do.com
davebestdeals.com daydream-studio.ru
dcj-nn.ru decoratinghomes.ga
ddpills.com demenageur.com
dekoration.us
deniven.1bb.ru deniven.1bb.ru
destinationrealestate.com
detmebel.su detmebel.su
digitalassetmanagement.site dipstar.org
digitalfaq.com directrev.com
dimfour.com diusyawiga.tk
diy-handmade-ideas.com divisioncore.com
dnsrsearch.com djonwatch.ru
dokumentalkino.net docsportal.net
donvito.unas.cz dodlive.mil
dood.live domain-submit.info
downloaddy.net dorratex.tn
drupa.com downloader12.ru
dynainbox.com downvids.net
doyouknowtheword-flummox.ml
dtm-spain.com
duawitchrarato.tk
e-poker-2005.com
earncash.com.ua
easync.io
easytuningshop.ru
ecommerce-seo.org ecommerce-seo.org
electro-prom.com edge.sharethis.com
eleimgo.pw educontest.net
elektrischezigarettekaufen2.cowblog.fr eets.net
elitepcgames.com efkt.jp
eonpal.com ege-russian.ru
eropho.net electronicadirect.com
es5.com elektronischezigarettekaufen1.beeplog.com
eshop4u.jp elektronischezigarettekaufen2.tumblr.com
esopini.com elektrozigarette2.wordpressy.pl
exchangeit.gq eloconcream.blogspot.com
extads.net en.home-task.com
f1nder.org energydiet24.ru
f5mtrack.com englishgamer.com
fable.in.ua escortplius.com
faithe.top esnm.ru
faptitans.com evidencecleanergold.com
exlarseva.webblog.es
eyes-on-you.ga
ez8motelseaworldsandiego.com
factorynightclub.com
fanrto.com
farm26.ru
fashion.stellaconstance.co fashion.stellaconstance.co
fashionindeed.ml favornews.com
fellowshipoftheminds.com femmesdenudees.com
fetroshok.ru filesclub.net
filerockstar298.weebly.com folowsite.com
film-one.ru foodcrafts.website
filmfanatic.com forex.osobye.ru
finemanteam.com fortunejack.com
fivedwld.cf fotzen-ficken.com
flash4fun.com freeuploader.ml
floridamobilebillboards.com front.to
flytourisme.org funnel.co.za
fotoxxxru.com galeria-zdjec.com
foxtechfpv.com gamblingnerd.com
freshdz.com gamerscorps.com
frivgame250.com games.kolossale.ru
fuckbuddybestgilf.info genericviagrasildenafiled.net
gameplexcity.com get-free-traffic-now.com
gearcraft.us ghernnqr.skyrock.com
geckoandfly.com gilbertbanda.net
gerhardhealer.com globalsurfari.com
get-seo-domain.com gne8.com
getaclueamerica.com gnuetella.com
gidonline.one goodprotein.ru
gilsonchiro.xyz goodwinmetals.co
glass-msk.ru googleseo.com.tr
glcomputers.ru gopixdatabase.com
gojiberriess.apishops.ru gorabagrata.ru
gok-kasten.net gov.yanao.ru
google-liar.ru gpms.org.my
googlsucks.com growmyfunds.ca
growshop.es gta-club.ru
gsmlab.pl gyffu.com
gta-top.ru h2monline.com
guge.io hasshe.com
handicapvantoday.com hatdc.org
hanink.biz.ly hdpixion.com
hannasolution.ru healgastro.com
hauleddes.com highland-homes.com
havepussy.com highspeed5.net
heatpower.ru histats.com
helpmymacfaster.trade histock.info
hentaiheroes.com homeandhealth.ru
hkladys.com homegardenlova.com
hobbyhuren-datenbank.com hotblognetwork.com
horoshieokna.com hotkeys.com
hostcritique.com houseofgaga.ru
hotshoppymac.com howopen.ru
hoverboard360.at hully.altervista.org
htmlcorner.com hvd-store.com
hydropump.su idc.com.ua
ibmdatamanagement.co igru-xbox.net
ideoworld.org ilte.info
ifmo.ru imagerydatabase.com
igadgetsworld.com images-free.net
igrovyeavtomaty777.ru imgarcade.com
ihc2015.info impotentik.com
ilmexico.com increasewwwtraffic.info
imgata.com infostatsvc.com
imperia31.ru instakink.com
import-sales.com intellektmedia.at
incolors.club interferencer.ru
indetiske.ya.ru intervsem.ru
indo-export.ru iradiology.ru
interesnie-faktu.ru ispac.org
internetproviderstucson.com ivanovo.zrus.org
iron-age.info jacago.com
iskalko.ru jbl-charge.info
itservicesthatworkforyou.com jeremyeaton.co
ix20.ru jerseychinabizwholesale.us
janavibekken.no joingames.org
jasonpartington.com
jerseyssportsshop.com
job.icivil.ir
jrcigars.com jrcigars.com
juliadiets.com kalb.ru
kakadu-interior.com.ua
kambasoft.com
karadene.com karadene.com
kareliatobacco.ru kevblog.top
karting196.ru keywordspay.com
kidd.reunionwatch.com kino-filmi.com
kihi.gdn kinohall.ru
kinofak.net klikbonus.com
kinozapas.org kochanelli.com
kliksaya.com kol-energo.ru
kreativperlen.ch kongruan.com
kursy-ege.ru kunstaktien.info
labvis.host.sk kuzinsp.ru
laexotic.com ladsup.com
landscapebackgrounds.blogspot.com laserpen.club
letmacwork.world layola.biz.tc
life-instyle.com leadwayau.com
linkbolic.com lidoradio.com
lirunet.ru likesdesign.com
live-sexcam.tk ljusihus.se
locatellicorretor.com.br lockerz.com
lollypopgaming.com
lovi-moment.com.ua
lowephotos.info
lsp-awak-perikanan.com
luchshie-topcasino.ru
lunamedia.co lunamedia.co
lutherstable.org m.b00kmarks.com
m-google.xyz m5home.ru
magnetpress.sk macdamaged.tech
manipulyator-peterburg.ru mackeeper-land-672695126.us-east-1.elb.amazonaws.com
matrixalchemy.com makemoneyonline.com
mcar.in.ua manyresultshub-a.akamaihd.net
medtherapy.ru map028.com
meltwater.com mararoom.ru
member-quiz.com marinetraffic.com
menetie.ru mcnamaratech.com
metarip.ru meendo-free-traffic.ga
meteostate.com mere.host.sk
message-warning.net
michaelkorssaleoutletonline.net
minecraft-rus.org minecraft-rus.org
mitrasound.ru mjchamonix.org
mobstarr.com mnimmigrantrights.net
moesonce.com moesonce.com
money-for-placing-articles.com monsterdivx.com
monthlywinners.com monthlywinners.com
mowser.com moscow.online-podarki.com
mp3downloadhq.com mosrif.ru
mp3s.club movies-in-theaters.net
mrcsa.com.au mrpornogratis.xxx
musezone.su msk-diplomat.com
muznachas-service.ru mxgetcode.com
mygameplus.ru mydoctorok.ru
myra.top mymobilemoneypages.com
mywallpaper.top naturalpharm.com.ua
nalogovyy-kodeks.ru
narutonaruto.ru
naval.jislaaik.com naval.jislaaik.com
neodownload.webcam navalwiki.info
networkad.net needtosellmyhousefast.com
news-speaker.com news-speaker.com
nextlnk12.com nisuturnetdgu.tk
nfljerseys.online noclegonline.info
nootrino.com notebook-pro.ru
nosecret.com.ua novatech.vn
novgorod.xrus.org november-lax.com
nudepatch.net
nufaq.com nufaq.com
o00.in nzfilecloud.weebly.com
oakleyglassesonline.us oconto.ru
oddamzadarmo.eu offtime.ru
odoratus.net oklogistic.ru
okmedia.sk onetravelguides.com
olvanto.ru online-templatestore.com
omgtnc.com onlinemeetingnow.com
onemantrip.com
onlineporno.site onlineporno.site
ontargetseo.us
optikremont.ru
oralsexfilme.net oralsexfilme.net
ostroike.org ostroike.org
ourville.info otbelivanie-zubov.com
owohho.com pagesense.com
ozas.net paidonlinesites.com
panouri-solare-acoperis.com partner-cdn.men
paperwritingservice17.blogspot.ru partnerworkroom.men
paramountmarble.co.uk pcboa.se
pensplan4u.com pcimforum.com
perso.wanadoo.es pdns.download
petedrummond.com peugeot-club.org
petrovka-online.com picturesboss.com
phelissota.xyz piulatte.cz
phormchina.com piuminiita.com
picturesfrom.com placid-rounded-coast.glitch.me
pio.polytopesexempt.com platinumdeals.gr
pizza-tycoon.com ploenjitmedia.azurewebsites.net
pl-vouchers.com plusnetwork.com
plohaya-kreditnaya-istoriya.ru podshipniki-nsk.ru
poem-paying.gq
polcin.de polcin.de
polska-poezja.com poquoson.org
popup-fdm.xyz porndairy.in
pornofiljmi.com pornmania.pl
pornogratisdiario.com porno-dojki.net
pornoklad.ru porno-play.net
porto.abuilder.net pornosexrolik.com
portal-eu.ru
predominant-invent.tk
printdirectforless.com
privatefx-in.ru privatefx-in.ru
privatefx.all4invest.info privatov-zapisi.ru
pro-poly.ru proekt-mos.ru
prodvigator.ua prokotov.com
profit-opportunity.com promalp-universal.ru
promgirldresses.xyz
pron.pro
pronekut.com
prostitutki-novgoroda.org prostitutki-novgoroda.org
pssucai.info pshare.biz
push-ad.com pussyfleet.com
qualitymarketzone.com qwarckoine.com
rangjued.com qwertty.net
r-e-f-e-r-e-r.com
randki-sex.com
ratemodels.net
razleton.com razleton.com
rcb101.ru
realtytimes.com
recipedays.com
registrationdomainsite.com
reimageplus.com reimageplus.com
rentalcarnavi.info resellerclub.com
restaurantlescampi.com
rialp.getenjoyment.net rialp.getenjoyment.net
rukino.org rock-cafe.info
ruspdd.com rossmark.ru
rutor.vip royalcar-ufa.ru
s-forum.biz ru-dety.ru
sampleletters.net rucrypt.com
sayyoethe.blogspot.co.za ruscoininvest.company
sbprabooks.com rusexy.xyz
sbt-aqua.ru rusprostitute.com
score-ads.men s1z.ru
sdelai-prosto.ru s8-nowy-wygraj.comli.com
se.bnt-team.com sarahmilne.top
seccioncontrabajo.com savememoney.co.za
seeresultshub-a.akamaihd.net scalerite.co.za
selectads.men scanner-marwin.top
seo-2-0.com schalke04fc.info
seo-tools-optimizing.com scripted.com
seocdvig.ru seasonvar.ru
seocheckupx.com see-your-website-here.com
seoheap.com seonetwizard.com
seoholding.com sexkontakte-seite.com
seorank.info
sex.hotblog.top
sexkontakteao.info
sexs-foto.com
sexspornotub.com sexspornotub.com
sexyteens.hol.es sherlock.se
shisha-swag.de shinikiev.com.ua
shodanhq.com shoesonlinebuy.cn
shopsellcardsdumps.com shoppytoolmac.com
shopwme.ru sindragosa.comxa.com
similardeals.net sispe.com.br
site.ru skuteczna-dieta.co.pl
sitevalued.com slujbauborki.ru
smstraf.ru small-game.com
snowplanes.com snowplanes.com
socialmonkee.com socialbookmarksubmission.org
socialseet.ru spb.afora.ru
softtor.com spin2016.cf
soietvousmaime.fr
spacash.com
space-worry.ml
splendorsearch-a.akamaihd.net splendorsearch-a.akamaihd.net
sports-supplements.us
spywarebegone.com spywarebegone.com
spywarespy.com spywareit.com
standardchartered-forex.com squidoo.com
steame.ru stairliftsarea.com
stjamesschool.info star61.de
stoki.ru startufa.ru
storehouse.ua stathat.com
strag-invest.ru stpicks.com
stroyplus.ru strana-krasoty.ru
studiofaca.com superoboi.com.ua
sukarame.net svbur.ru
sundrugstore.com
supermama.top
superstats.com
surintech.ac.th
swagbucks.com swagbucks.com
t3chtonic.com swimpool.ca
tacbelarus.ru tattomedia.com
tackletarts.co tecspb.ru
taqywu51.soup.io teenforporn.com
tds-advert002.info teesdaleflyballclub.co.uk
technika-remont.ru telefonsexsofort.tk
tedxrj.com texbaza.by
tehngr.ru tgtclick.com
terrafootwear.us theallgirlarcade.com
testingads.pro thecoral.com.br
thefarmergame.com thegioixekhach.com
titan-ads.life theplacetoupdating.pw
titslove.yopoint.in tiens2010.ru
tizanidine4mgstreetprice.blogspot.com tizanidinefibromyalgia.blogspot.com
tizanidineandhydrocodone.blogspot.com tizanidineoral.blogspot.com
tizanidinecost.blogspot.com tizanidinepain.blogspot.com
tizanidinehcl2mgsideeffects.blogspot.com tk-assortiment.ru
tizanidinehcl4mgisitanarcotic.blogspot.com
tizanidineinteractions.blogspot.com
tmm-kurs.ru tmm-kurs.ru
tmtrck.com topcar-krasnodar.ru
tn811.us torrentgamer.net
topflownews.com
torrentred.games
torrents.life torrents.life
tracfone.com tourismvictoria.com
tracklead.net toyota.7zap.com
tracxn.com trackzapper.com
trafficgenius.xyz
trubywriting.com
truck-addzilla.life
truebeauty.cc truebeauty.cc
truemfilelj.gq tx41tclega.ru
turn-up-life.life uchil.net
tvand.ru ukr-lit.com
twodollarshows.com umaseh.com
u555u.info upproar.com
ua-company.ru
uptime-delta.net uptime-delta.net
upupa.net uptimechecker.com
urzedowski.eu urccvfmc.bloger.index.hr
usacasino.com urlcut.ru
vabasa.inwtrade.com urll.eu
usbggettwku.ga
utiblog.fr
vacuumcleanerguru.com vacuumcleanerguru.com
valoresito.com vchulkah.net
vasileostrovsky-rayon.ru vchulkax.com
vbtracker.net vepad.com
viagraneggrx.com via-energy-acquistare.com
video-production.com
videochat.mx
videochat.tv.br videochat.tv.br
videogamesecrets.com videonsk.com
virus-schutzmasken.de videooko.weebly.com
vkgaleria.com vids18.site
vkontarkte.com virus-respirators.com
vod.com.ua vremya.eu
voloo.ru walkme.com
vrnelectro.ru warningzscaler.heraeus.com
vzlomsn.org watch-movies.ru
watchmyfb.pl watchmyfb.pl
watchmygf.net
waterefficiency.co waterefficiency.co
website-analyzer.info weburlopener.com
wechatdownload10.com wfb.hatedriveapart.com
westvilletowingservices.co.za whatsappbot.flyland.ru
wheelchairliftsarea.com wheelchairliftsarea.com
where-toget.com where-toget.com
whereiskentoday.com wholesalecheapjerseysfree.com
whosonmyserver.com wiosenny-bon-1500.pl
wildworld.site word-vorlagen.net
wineration.com worldoffiles.ru
worldinternetauthority.com ww1943.ru
writersgroup580.web.fc2.com www.888.com
wufak.com www.souvenirua.com
www.ehscloud.cn www.wohnkabinen-shop.de
x-diesel.com wygraj-skiny.win
x69ty.ru xcc24.pl
xitjw.info xdoza.com
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai xgames-04.com
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai xn--80ajbshivpvn2i.xn--p1ai
xn----9sbubg3ambdfl1j.xn--p1ai xpresscare.ru
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai xtraffic.plus
xn--b1addnj3cah.xn--p1ai xxx-treker.ru
xsfetish.org xxxsiterips.xyz
xz618.com
yes-com.com yes-com.com
ynymnwbm.bloger.index.hr youandcredit.ru
yourserverisdown.com yourothersite.com
youtubologia.it yuweng.info
zaapplesales.blogspot.com zaidia.xhost.ro
zaimhelp.ru zawyna.ua
zakon-ob-obrazovanii.ru zerocash.msk.ru
zambini.ru
zapiszto.pl
zebradudka.com
zeroredirect11.com
zeroredirect2.com
zeroredirect8.com
zhongwenlink.com zhongwenlink.com
zog.link znakom.sibtest.ru
zolotoy-lis.ru zrelaya.pw
zs2vm.top zvooq.eu
zskdla.site zx6.ru
zygophyceous.womanstars.site zzlgxh.com

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
101raccoon.ru 4istoshop.com
3w24.com 4pp13.com
9med.net 4replicawatch.net
adinterax.com 51unlim.ru
7zap.com
absolutelyanalog.com
abwa.tk
adloads.com adloads.com
advokat-grodno.by adsloads.com
advokateg.xyz algerianembassy.co.in
ahrefs.com allboard.xobor.de
aibolita.com ananumous.ru
alphahoverboards.com anmysite.com
amateurmatch.com anonymousfox.co
animalia-life.club aosexkontakte.net
anonymous-redirect.com apibrowsesmartne-a.akamaihd.net
apelsinnik.website apioutoboxnet-a.akamaihd.net
apidiamondatanet-a.akamaihd.net arcadepages.com
asophoto.com arcteryxstore.online
arendovalka.xyz
as5000.com
asia-forum.ru
auto-zapchasti.org auto-zapchasti.org
autobudpostach.club autoua.top
awaybird.ru avtochehli.by
azartmix.com bankofthewext.com
babyfactory.fr bayareaaeroclub.org
babyguns.ru bbtec.net
back.dog beachtoday.ru
bag77.ru best-mam.ru
baladur.ru bigpictures.club
bannerpower.com binaryoptionscops.info
bashtime.ru bitcoins-live.ru
bcmp.org bjorkbacken.nu
belinvestdom.by bluejays-jerseys.us
bestfortraders.com buypuppies.ca
bikini-image.com calc-for-credit.ru
blackcurranthumidifiers.site californianews.cf
bmw.afora.ru callejondelpozo.es
boeuklad.com carloans.com
book-bets.com cartierlovebraceletreplica.xyz
boole.onlinewebshop.net cercacamion.it
builtwith.com certifywebsite.win
buyk.host.sk cheapmoncler.win
candcstuccoandstone.com cherubinimobili.it
catalogs-parts.com civilwartheater.com
cdncash.org colehaanoutlet.store
cezartabac.ro com-onlinesupport.host
cheapbarbour.online compliance-willy.top
cialischstgerts.com concordexoticrentals.com
citetick.com coolwallpapers4k.info
classicakuhni.ru
clickcash.com
codysbbq.com
companies-catalog.com
compliance-alexa.top
compliance-andrew.xyz
compliance-barak.xyz
compliance-jack.top
cornerstone-countertops.com
cranly.net cranly.net
ddospanel.com credit-respect.ru
dealwifi.com daisye.top
dekoration.us dawlenie.com
demian.kz
derevesendeco.com
designdevise.com designdevise.com
djihispano.com dfwu1013.info
domainsphoto.com dimfour.com
dosug-lux.ru diusyawiga.tk
dverimegapolis.ru dojki365.online
earn-from-articles.com domainsatcost.com
earthmagic.info dosugrostov.site
ecookna.com.ua drugstoreforyou.com
edelstahlschornstein-123.de e-collantes.com
edge.sharethis.com electro-prom.com
egvar.net elektroniksigaraankara.info
elite-sex-finders.com elite-sex-finders.com
empis.magix.net enbersoft.com
ereko.ru
ero-advertising.com ero-advertising.com
etotupo.ru essay-edu.biz
eyemagination.com euromasterclass.ru
ezigarettenkaufen1.deviantart.com exmasters.com
fast-torrent.ru ezigarettenkaufen2.dreamwidth.org
fighrofacciufreesig.ga factorynightclub.com
financehint.eu financeloan.us
forex.osobye.ru fishtauto.ru
forsex.info frankofficial.ru
forum20.smailik.org freewhatsappload.com
fotosfotos.eu gabeshop.ru
free-fb-traffic.com galeon.com
fsalas.com gallery.rennlist.com
funnel.co.za gamblingnerd.com
funnymama.com gerhardhealer.com
fyl.com.ru get-your-social-buttons.info
gamerextra.com girlsfuckdick.com
get-free-traffic-now.com goldenggames.com
getpopunder.com golfresa.lucania.se
glass-msk.ru gourcy.altervista.org
golaya.pw grand-chlen.ru
graphics8.info greenidesign.co
hdpixent.com guge.io
heatpower.ru h2monline.com
hikesearch.net hatdc.org
historichometeam.com healbio.ru
hjaoopoa.top hop.clickbank.net
home-task.com inet-traffic.com
hornymatches.com infostatsvc.com
hosting-tracker.com infowarcraft.ru
hotblognetwork.com integritylandscapeservices.com
hotgirlhdwallpaper.com invitefashion.com
iamsport.org junglenet-a.akamaihd.net
ibb.com.ua kazinogames.lv
icaseclub.ru keywordsking.com
iccornacircri.cf keywordspay.com
ilte.info kihi.gdn
imperiafilm.ru kinobaks.com
internetartfair.com kinohit1.ru
investzalog.ru klikbonus.com
io9.com labafydjxa.su
jobgirl24.ru lapolis.it
jpcycles.com lechenie-gemorroya.com
jumptap.com lenvred.org
jurajskie.info lineshops.biz
justkillingti.me linkbolic.com
kambasoft.com linksharingt.com
kinozapas.com livingroomdecor.info
klitimg.pw
kollekcioner.ru
lacapilla.info
landmania.ru
legionalpha.com
linkredirect.biz
ljusihus.se
losangeles-ads.com losangeles-ads.com
luchshie-topcasino.ru lostaruban.com
madot.onlinewebshop.net lotto6888.com
magento-crew.net louboutinbooties.xyz
make-money-online.com lumb.co
manimpotence.com macfix.life
maranbrinfo.com.br makis.nu
mega-polis.biz.ua marmitaco.cat
microsoftportal.net mcnamaratech.com
minneapoliscopiers.com minharevisao.com
mirmedinfo.ru missis.top
moje-recenze.cz mmoguider.ru
moneyteam24.com monclerjacketsoutlet.win
my-floor.in.ua moneymaster.ru
myfreemp3.eu moroccosurfadventures.com
mymercy.info movpod.in
naperehresti.info mycouponizemac.com
nbsproject.ru myshopmatemac.com
nisuturnetdgu.tk nextconseil.com
notaria-desalas.com ngps1.ru
nudejapan.net no-rx.info
nuup.info noumeda.com
online-hd.pl ntic.fr
onlinefilmz.net oballergiya.ru
osoznanie-narkotikam.net offergroup.info
pandarastore.top onlineserialy.ru
paparazzistudios.com.au openlibrary.org
parajumpersstore.online ops.picscout.com
parfusale.se panchro.xyz
pawli.eu partner-host.men
pc-virus-d0l92j2.pw phrcialiled.com
pc4download.co pk-pomosch.ru
photo.houseofgaga.ru plastgranar.nu
picquery.com playboyfiles.xblog.in
picturesmania.com plusnetwork.com
pipki.r.acdnpro.com pontiacsolstice.info
pivka.xyz pp-budpostach.com.ua
pixshark.com pr-ten.de
plastweb.ru projectforte.ru
pony-business.com purplesphere.in
porno-play.net quizzitch.net
postclass.com
potoideas.us
pozdrawleniya.ru
prizefestival.mobi
prostitutki-novgoroda.org
pulse33.ru
rankia.com
ranksays.com ranksays.com
ranksignals.com recipedays.ru
ratemodels.net redhotfreebies.co.uk
remont-ustanovka-tehniki.ru redirlock.com
remontbiz.ru reimageplus.com
rentaremotecomputer.com remont-fridge-tv.ru
repeatlogo.co.uk renecaovilla.online
royal-investments.net replicalouboutin.xyz
ruclicks.com rewardpoll.com
ruscoininvest.company ritlweb.com
satellite.maps.ilovevitaly.com rniaeba.ga
sbricur.com rocis.site
scanner-irvin.top rockma.se
schoolfiles.net roleforum.ru
seo18.su samo-soznanie.ru
seorank.info scenicmissouri.us
shakhtar-doneck.ru screpy.com
sharebutton.to searchaddis.com
shell-pmr.ru searchimpression.com
shoesonlinebuy.xyz seasaltwithfood.com
simpoed.ufop.br security60-e.com
smartadserver.com seoholding.com
smsactivator.ru service-core.ru
socialmediasuggest.com sex.hotblog.top
sohoindia.net sexyteens.hol.es
stockspmb.info share-buttons-for-free.com
stroilka.info shlyahten.ru
stroiminsk.com shtora66.ru
sugvant.ru sibtest.ru
superiends.org silverage.ru
surfbuyermac.com sim-service.net
susanholtphotography.com sisiynas.ru
t3chtonic.com sitopreferito.it
slavkokacunko.de
slayerlife.com
smart-scripts.com
sneakyboy.com
social-button.xyz
social-s-ggg.xyz
socialsignals24.com
solicita.info
sophang8.com
sotechco.co
sport7777.net
srdrvp.com
tabakur77.com
tagil.zrus.org
talant-factory.ru
tanieaukcje.com.pl tanieaukcje.com.pl
taraz.xkaz.org taxi-v-eisk.ru
tattoo33.ru testbotprocessor44.com
template-kid.com thefotosgratis.eu
thewomenlife.com tizanidinedrugtest.blogspot.com
tizanidinevscyclobenzaprine.blogspot.com tizanidinehcl2mgtablet.blogspot.com
tk-assortiment.ru tizanidineingredients.blogspot.com
toproadrunner5.info tizanidinewithdrawal.blogspot.com
torrentdownloadhub.com turbabitload.weebly.com
trk-4.net uhodzalijami.ru
twojebook.pl vchulkah.net
unrealcommander.com viagengrarx.com
urengoy.pro videtubs.pl
v-doc.co
vashsvet.com
vclicks.net
versaut.xxx-cam.webcam
video-chat.love
videochat.world
vinsit.ru
vipms.ru
viven.host.sk viven.host.sk
vkonche.com wallpaperaccess.com
vriel.batcave.net warezkeeper.com
vrnelectro.ru webmasters.stackexchange.com
vzlomtw.com website-speed-up.top
w-journal.ru woman-h.ru
waterefficiency.co womensplay.net
weblibrary.win worldinternetauthority.com
weboptimizes.com www.jbetting.com
website-speed-check.site
wieseversa.no
www.kabbalah-red-bracelets.com www.kabbalah-red-bracelets.com
x-stars.ru wzgyyq.com
x-true.info xedserver.com
xblognetwork.com xn----itbkqkfiq.xn--p1ai
xn----dtbndd4ae7eub.top xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xrus.org xn--80aikhbrhr.net
xxart.ru xnxx699.com
xxxasianporn.net xxx-cam.webcam
yourads.website xxxkaz.org
zigarettenonlinekaufen2.bloog.pl xxxtube69.com
zynax.ua yachts-cruise.info
yellowfootprints.com
yoshkarola.zrus.org
zajm-zalog-krasnodar.ru
zeroredirect.com
zeroredirect11.com
zone-kev717.info
zootoplist.com
zscaler.net
zvezdagedon.ru

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.88 0; 10.1.1.48 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2686 ### Version: V4.2021.11.2687
### Updated: Sun Oct 31 22:02:50 UTC 2021 ### Updated: Mon Nov 1 10:12:47 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2686 ### Version: V4.2021.11.2687
### Updated: Sun Oct 31 22:02:50 UTC 2021 ### Updated: Mon Nov 1 10:12:47 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##