vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 06:15:06 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.10.3439 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-10-29 10:09:19 +00:00
parent a28ca4670f
commit 5d57a15e5b
47 changed files with 94836 additions and 94826 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3438 #### Version: V4.2022.10.3439
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3438 #### Version: V4.2022.10.3439
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.10.3438 #### Version: V4.2022.10.3439
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11718
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.10.3438 #### Version: V4.2022.10.3439
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11722
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11722
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11722
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11722
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.1.120 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.1.120 0;
127.0.0.1 0; 127.0.0.1 0;

11722
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.1.120 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.1.120 0;
127.0.0.1 0; 127.0.0.1 0;

11722
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.1.120 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.1.120 0;
127.0.0.1 0; 127.0.0.1 0;

11722
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3438 3439

11722
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11722
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.253.205.195 0; 20.49.46.2 0;

11722
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.1.120 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.1.120 0;
127.0.0.1 0; 127.0.0.1 0;

11722
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.1.120 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.1.120 0;
127.0.0.1 0; 127.0.0.1 0;

11722
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.1.120 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.1.120 0;
127.0.0.1 0; 127.0.0.1 0;

11722
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.253.205.195 0; 20.49.46.2 0;

11722
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.253.205.195 0; 20.49.46.2 0;

11722
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 1; 10.1.1.120 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Exabot CrunchBot
NetSpider Deusu
Recorder Digincore
Voil ECCP/1.0
WallpapersHD Offline Navigator
arquivo-web-crawler WebGo IS
ips-agent WebLeacher
netEstate NE Crawler WiseGuys Robot
satoristudio.net isitwp.com
sogouspider ubermetrics-technologies.com

310
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
AIBOT 404checker
Aboundex AiHitBot
Aboundexbot Aipbot
AfD-Verbotsverfahren
Alligator
AlphaBot AlphaBot
Anarchie Apexoo
Anthill
Aspiegel Aspiegel
Asterias
Atomseobot
Attach Attach
Badass AwarioSmartBot
BBBike
BDCbot
BDFetch
BLEXBot
BackDoorBot
BackStreet
Backlink-Ceck
Bandit Bandit
Barkrowler Barkrowler
BatchFTP BatchFTP
Black Hole Battleztar Bazinga
BlackWidow BetaBot
Bigfoot
Blackboard Blackboard
BlowFish Blow
Boardreader
Bolt
BotALot BotALot
Buck Buddy
BuiltWith BuiltBotTough
Bullseye Calculon
BunnySlippers
CATExplorador
CCBot
CSHttp
CazoodleBot
CheTeam CheTeam
CheeseBot CheeseBot
ChinaClaw
Claritybot
Cliqzbot Cliqzbot
Cocolyzebot Cogentbot
Collector
CopyRightCheck CopyRightCheck
Craftbot Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
CrunchBot CrunchBot
DIIbot Custo
CyotekWebCopy
DBLBot
DatabaseDriverMysqli DatabaseDriverMysqli
Devil Deusu
Dirbuster DittoSpyder
DnyzBot
DomCopBot DomCopBot
DomainSigmaCrawler DomainAppender
DomainCrawler
DomainStatsBot DomainStatsBot
Domains Project
Download Wonder Download Wonder
Drip Ebingbong
EMail Siphon Express WebPictures
Ecxi ExtLinksBot
Evil
Exabot
Extreme Picture Finder Extreme Picture Finder
Ezooms EyeNetIE
FHscan FDM
Foobot FemtosearchBot
FrontPage FrontPage
FyberSpider
G-i-g-a-b-o-t
GT::WWW
Genieo
GetRight GetRight
Gigabot Gigabot
GoZilla Go!Zilla
Grafula Go-Ahead-Got-It
Gotit
GrapeFX GrapeFX
HTTP::Lite GridBot
Haansoft HTTrack
HaosouSpider HaosouSpider
Havij Havij
Heritrix Heritrix
HonoluluBot Humanlinks
HybridBot HybridBot
IDBTE4M IDBTE4M
IRLbot IRLbot
IlseBot Image Sucker
Indy Library
InfoNaviRobot
InterGET InterGET
Internet Ninja
InternetSeer InternetSeer
Iskanie Iria
JOC Web Spider
JennyBot JennyBot
JetCar
Jetty Jetty
Joomla
Jorgee Jorgee
Kinza Keybot Translation-Search-Machine
LNSpiderguy
Lanshanbot
Larbin Larbin
LibWeb LeechFTP
LinkWalker Likse
Linkbot
LinkpadBot LinkpadBot
LinksManager LinqiaRSSBot
LinqiaMetadataDownloaderBot Lipperhey
LinqiaScrapeBot Ltx71
Lipperhey Spider MJ12bot
MIIxpc MSFrontPage
MQQBrowser
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic SEO
Majestic-SEO Majestic-SEO
Masscan
Mediatoolkitbot
MegaIndex.ru
Metauri Metauri
MicroMessenger Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Minefield Mister PiX
Moblie Safari
Mojeek
MolokaiBot
Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3 Msrabot
NearSite NPbot
Nameprotect
Navroad
Needle
Nessus
Net Vampire Net Vampire
NetAnts Netvibes
NetMechanic
NetSpider
NetZIP
Nettrack
Nibbler Nibbler
Niki-bot Nikto
NimbleCrawler NimbleCrawler
Nimbostratus Nimbostratus
Not Nmap
Nuclei Nutch
Offline Navigator OpenLinkProfiler
OpenVAS Openfind
OrangeBot OrangeBot
OutclicksBot OrangeSpider
PECL::HTTP PHPCrawl
Page Analyzer POE-Component-Client-HTTP
PageScorer
PageThing.com PageThing.com
Pandalytics Pavuk
PeoplePal PeoplePal
PictureFinder Picscout
Piepmatz Picsearch
Pimonster
Pixray Pixray
PleaseCrawl PleaseCrawl
ProWebWalker Pockey
Psbot PxBroker
Pump Quick-Crawler
RankActiveLinkBot RankActiveLinkBot
RankingBot2 RankFlex
Re-re RankingBot
ReGet Rankivabot
RankurBot
Reaper Reaper
Recorder RocketCrawler
RedesScrapy SEOkicks
Ripper
Rogerbot
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler ScanAlert
Scanbot Scanbot
ScoutJet ScreenerBot
Scrapy
Semrush
SemrushBot
SentiBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot
Seomoz
Shodan Shodan
Site Sucker SiteExplorer
SiteCheckerBotCrawler SiteLockSpider
SiteSnagger
SiteSucker SiteSucker
Siteimprove Siteimprove
SlySearch
Snake Snake
SocialRankIOBot Snoopy
Spammen Sociscraper
Sogou web spider
Sosospider
Sottopop
SpankBot
Spbot Spbot
Sqlmap Spinn3r
Stripper Sqworm
Steeler
Sucker
Surfbot Surfbot
Suzuran SurveyBot
Szukacz
Teleport Teleport
TightTwatBot TeleportPro
Telesoft
Telesphoreo
Telesphorep
Titan Titan
Toata Toata
Toweyabot Tracemyfile
Trendiction
Trendictionbot Trendictionbot
True_Robot
Turnitin Turnitin
TurnitinBot TurnitinBot
TwengaBot
Typhoeus Typhoeus
UnisterBot
Upflow
V-BOT V-BOT
VCI Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler VidibleScraper
VoidEYE Virusdie
WBSearchBot Voil
WPScan WASALive-Bot
WWW-Collector-E WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers Wallpapers
WeSEE WallpapersHD
Web Auto Web Auto
Web Sauger Web Fetch
Web Sucker Web Fuck
WebCollage WebBandit
WebCopier WebCopier
WebEnhancer
WebFetch
WebFuck
WebGo IS WebGo IS
WebImageCollector
WebSauger WebSauger
WebStripper WebStripper
WebWhacker
WebZIP WebZIP
Webshag WebmasterWorldForumBot
WebsiteQuester
Whack
Whacker
Who.is Bot
Widow Widow
Wonderbot
Woobot
Wotbox Wotbox
Wprecon Xaldon WebSpider
Zade Xenu
Zeus
ZmEu ZmEu
adscanner
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt autoemailspider
backlink-check backlink-check
check1.exe cah.io.community
cognitiveseo com.plumanalytics
crawl.sogou.com demandbase-bot
dataforseo.com facebookscraper
evc-batch
heritrix
instabid
internetVista monitor internetVista monitor
ips-agent isitwp.com
lwp-trivial iubenda-radar
page scorer lwp-request
mediawords
muhstik-scan
oBot
pcBrowser pcBrowser
polaris version
s1z.ru s1z.ru
seobility satoristudio.net
scalaj-http
seocompany.store seocompany.store
seoscanners seoscanners
seostar
serpstatbot serpstatbot
sexsearcher
sp_auditbot sp_auditbot
sysscan voyagerx.com
webgains-bot
webmeup-crawler webmeup-crawler
x22Mozilla x09Mozilla
xpymep1.exe zauba.io

306
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 404checker
80legs 404enemy
ADmantX ADmantX
AIBOT Aboundexbot
Abonti
AfD-Verbotsverfahren AfD-Verbotsverfahren
AiHitBot
Aipbot Aipbot
AllSubmitter Alexibot
Alligator
AlphaBot
Anarchie Anarchie
Anarchy99 Anarchy
Ankit Ankit
Anthill
Apexoo
Atomseobot
Attach Attach
AwarioRssBot BDFetch
BBBike BLEXBot
BackDoorBot BackDoorBot
BackStreet
BackWeb BackWeb
Backlink-Ceck Barkrowler
Bandit BatchFTP
Battleztar Bazinga Bigfoot
BetaBot Bitacle
Black Hole
BlackWidow
Blow
BlowFish
Bolt Bolt
Brandwatch Brandwatch
Buck
BuiltWith BuiltWith
Bullseye Bullseye
CATExplorador BunnySlippers
CazoodleBot CCBot
CheTeam CODE87
CheeseBot Calculon
Cliqzbot Cegbfeieh
Cocolyzebot ChinaClaw
CopyRightCheck Claritybot
Cosmos Cogentbot
Craftbot Copier
Crawling at Home Project Copyscape
CrazyWebCrawler Curious
Crescent CyotekWebCopy
CrunchBot
DBLBot DBLBot
DIIbot
DSearch DSearch
DataCha0s Deusu
Demon Digincore
DigitalPebble DigitalPebble
Disco Disco
Discobot Discoverybot
DnyzBot
DomainAppender
DomainCrawler DomainCrawler
Domains Project DomainSigmaCrawler
Download Wonder Download Wonder
Dragonfly ECCP/1.0
Drip EasyDL
EirGrabber Ecxi
Evil
Express WebPictures
ExtLinksBot ExtLinksBot
ExtractorPro ExtractorPro
EyeNetIE EyeNetIE
Ezooms Ezooms
FHscan FemtosearchBot
FlashGet Fimap
Foobot
FrontPage FrontPage
Fuzz Fuzz
FyberSpider
G-i-g-a-b-o-t G-i-g-a-b-o-t
GalaxyBot GalaxyBot
Getintent GermCrawler
Gotit Gigabot
Go-Ahead-Got-It
GoZilla
GrabNet
Grabber Grabber
Grafula
GrapeFX GrapeFX
GridBot GrapeshotCrawler
HMView HEADMasterSEO
HTMLparser HTTP::Lite
Haansoft
Havij Havij
Heritrix HonoluluBot
Hloader Humanlinks
HybridBot
IDBot IDBot
Id-search Id-search
IlseBot
Image Sucker Image Sucker
IndeedBot
Indy Library Indy Library
InterGET InfoTekies
Internet Ninja InternetSeer
Iria
Iskanie
JOC Web Spider JOC Web Spider
JennyBot JamesBOT
JetCar
Jetty Jetty
JikeSpider Jorgee
JustView
Jyxobot Jyxobot
Lanshanbot Kenjin Spider
Keybot Translation-Search-Machine
Kinza
LNSpiderguy
LWP::Simple
Larbin Larbin
Leap LeechFTP
LibWeb Lftp
Libwhisker
LieBaoFast
Lightspeedsystems
LinkScan
Linkbot Linkbot
LinkextractorPro LinkpadBot
LinqiaMetadataDownloaderBot LinksManager
LinqiaScrapeBot LinqiaScrapeBot
Lmspider
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc Mag-Net
MTRobot Mb2345Browser
Magnet MegaIndex.ru
Mail.RU_Bot Mojeek
Majestic12 Msrabot
Mass Downloader Name Intelligence
Masscan
Mata Hari
Meanpathbot
Microsoft URL Control
MolokaiBot
Nessus Nessus
Net Vampire
NetAnts
NetZIP
Netvibes
Niki-bot Niki-bot
Nikto Nikto
NimbleCrawler
Nimbostratus Nimbostratus
Ninja Ninja
Nmap
Nuclei Nuclei
Offline Navigator Offline Navigator
OpenLinkProfiler
OpenVAS OpenVAS
Openfind Openfind
OrangeSpider
OutclicksBot
OutfoxBot
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageGrabber Page Analyzer
PageScorer Papa Foto
Pandalytics Picscout
Pavuk
Picsearch Picsearch
PictureFinder
Piepmatz
Pimonster Pimonster
Pockey ProPowerBot
Probethenet Probethenet
Pu_iN Pu_iN
PxBroker
QueryN Metasearch QueryN Metasearch
RankActiveLinkBot
RankFlex RankFlex
RankingBot RankingBot
RankurBot
ReGet
Reaper
Recorder
RepoMonkey RepoMonkey
Rogerbot Ripper
SEOkicks RocketCrawler
SalesIntelligent SMTBot
SemanticJuice ScanAlert
SemrushBot Scrapy
Seomoz Screaming
ScrepyBot
Searchestate
Seekport
Semrush
SeobilityBot
Shodan
Siphon Siphon
Site Sucker SiteSnagger
SiteCheckerBotCrawler Siteimprove
SiteSucker
SmartDownload SmartDownload
Snake Sosospider
Sottopop Sottopop
Spammen Spammen
SpankBot
Spanner Spanner
Spinn3r Spinn3r
SputnikBot Sqworm
Sqlmap Sucker
Sqlworm
Steeler
Stripper
Sucuri Sucuri
SuperBot SuperBot
SuperHTTP Surfbot
SurveyBot SurveyBot
Szukacz T0PHackTeam
Telesoft
Telesphoreo
Telesphorep Telesphorep
TheNomad
Thumbor Thumbor
TightTwatBot TightTwatBot
Toata
Toweyabot Toweyabot
Trendiction
True_Robot True_Robot
Turnitin Turnitin
TwengaBot TwengaBot
Twice
Typhoeus
V-BOT V-BOT
VCI VB Project
Voltron Vacuum
WASALive-Bot VelenPublicWebCrawler
WISENutbot VidibleScraper
Virusdie
VoidEYE
WEBDAV
WWW-Collector-E
WWW::Mechanize
WallpapersHD
WeSEE WeSEE
Web Enhancer Web Auto
Web Fetch
Web Fuck Web Fuck
Web Pix
Web Sucker Web Sucker
WebAuto
WebCollage
WebCopier
WebEnhancer WebEnhancer
WebFuck WebFetch
WebGo IS
WebImageCollector
WebReaper WebReaper
WebSauger
WebStripper WebStripper
WebSucker WebSucker
WebWhacker WebZIP
Webalta Webalta
WebmasterWorldForumBot
Webshag Webshag
Website Quester Website Quester
WebsiteQuester WebsiteExtractor
Whatweb Whacker
WiseGuys Robot Wonderbot
Xaldon_WebSpider YoudaoBot
Xenu Zauba
Zermelo Zermelo
Zeus Zeus
ZmEu ZoomBot
ZoominfoBot
ZumBot
ZyBorg ZyBorg
adscanner
archive.org_bot
cah.io.community cah.io.community
crawl.sogou.com clark-crawler
coccocbot
cognitiveseo
crawler.feedback crawler.feedback
crawler4j crawler4j
dataforseo.com demandbase-bot
facebookscraper eCatch
instabid heritrix
internetVista monitor
isitwp.com isitwp.com
iubenda-radar magpie-crawler
lwp-request muhstik-scan
lwp-trivial
meanpathbot
netEstate NE Crawler netEstate NE Crawler
page scorer oBot
plumanalytics pcBrowser
probe-image-size
ripz
s1z.ru
scalaj-http
scan.lol
seocompany.store seocompany.store
seostar seostar
sitechecker.pro sogouspider
siteripz ubermetrics-technologies.com
tAkeOut webmeup-crawler
trendiction.com webpros.com
voyagerx.com
x22Mozilla x22Mozilla
xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
alveris.ru air-edem.ru
haikuware.com eloconcream.blogspot.com
heartofpayne.xyz fbdownloader.com
hildinghr.se hamilton.ca
homedecorpicture.us kozhniebolezni.com
kino-rf.ru naj-filmy24.pl
kopihijau.info prizesbook.online
savetubevideo.com pronorm.fr
se.bnt-team.com tobeyouday.win
stuff-about-money.com yeartwit.com

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1001watch.com.ua 00go.com
1webmaster.ml 0912701309f8ce.com
2015god.org 1001desert.com
202ch.com 10bet.com
2728fb936f0.com 4pp13.com
2x2fan.ru 7wind.ru
4kepics.com 91abcw.com
51.la aad1f4acb0a373420d9b0c4202d38d94fa.com
51unlim.ru abiente.ru
5tey7463.weebly.com acgs.tk
72-news.com adanih.com
888.com add-add.men
a14download.com addblueoff.com.ua
aa14ab57a3339c4064bd9ae6fad7495b5f.com adidas.frwebs.fr
academia-nsk.org
acortarurl.es
adrenalin-stk.ru adrenalin-stk.ru
ads.gold adultwebhosting.info
adsref.men advokateg.xyz
adultfriendfinder.com afmuseum.com
adultgalls.com agapovdg.ru
advokateg.ru aihelen.net
ahmedabadwebs.com akama.com
aksonural.ru albamargroup.com
alert-fdm.xyz alfa-medosmotr.ru
alert.scansafe.net amung.us
alloysteel.ru anabolics.shop
aludecor.info angel1777.kz
anapa-inns.ru
anime.dougasouko.com
aosexkontakte.net
ap.senai.br
apelsinnik.website apelsinnik.website
apimegabrowsebiz-a.akamaihd.net apiwoodensealcom-a.akamaihd.net
apioutoboxnet-a.akamaihd.net apple.com-webbrowsing-security.review
appmsr.org arcteryxstore.online
aviav.ru.com arendatora.ru
avtolombard-krasnodar.ru arraty.altervista.org
azlex.uz aruplighting.com
babespcs.com asrvvv-a.akamaihd.net
balkanfarma.org asscenihotosidea.blogspot.co.za
bankmib.ru autogrep.ru
bannerads.de autotop.com.ua
bannerpower.com avek.ru
barbourjackets.online aviav.eu
battlecarnival.su azadnegar.com
berdasovivan.ru b2b-lounge.com
best-coupon-offer.com babyguns.ru
bet-prognoz.com backuperwebcam.weebly.com
bg6s0.com bazaronline24.ru
bitcoinpile.com beget.tech
blavia.00author.com belinka.com.ua
belinvestdom.by
benchmarkcommunications.co.uk
best-seo-software.xyz
bestbuy.ca
bestimagecoollibrary.com
bestprofits-there.com
betterhdporn.com
bidr.trellian.com
bingoporno.com
bio-market.kz
bioinnovate.co
biteg.xyz
blackcurranthumidifiers.site
blockworld.ru blockworld.ru
blog-hits.com blog.yam.com
blogrankers.com bloggen.be
blpmovies.com
bluerobot.info
bmwhighperformers.com
board.f00d.de board.f00d.de
bobinoz.com boleznikogi.com
bolegapakistan.com bolitgorlo.net
bosefux.esy.es bonzbuddy.com
brainxs.us braintobe.win
brg8.com
brus.city brus.city
businesn.men bum.com.ru
buyclomidonlaine.com burger-imperia.com
buyessaynow.biz buydissertation.net
buzz-porno.info buytizanidineonlinenoprescription.blogspot.com
bwlx.prepedu.cn cannibalcheerleader.com
bzero1jewelry.net carina-sy.de
c-gainsbourg.com cartierring.xyz
c1.onedmp.com cartujano-pre.de
canvas.gloverid.site casinorewards.info
cardiosport.com.ua catalogs-parts.com
casinopinup-wins.com cayado.snn.gr
caveavins.fr cbox.ws
ccbill.com centraletermice.us
ccjp.eu chatroulette.online
cdn.walkme.com chatroulette.world
cdncash.net check-this-out-now.online
cdncash.org cherrypointplace.ca
cellfun.mobi chocoslim.pro
ceotrk.com choice-credit.ru
cheapjerseysa.com christianlouboutinshoes.xyz
cheapmoncler.pw cialis-samples.com
check-host.net ciproandtizanidine.blogspot.com
chuckguilford.com cleanpcnow.com
clean-virus-mac.com compliance-checker.info
clickbanksites.info coolgramgoods.com
clickcash.com coolyarddecorations.com
clips.ua.ac.be coop-gamers.ru
clothesforcash.com cordstrap.cc
coinsspb.com cornomase.win
collegeessay19.blogspot.ru
com-cleaner.systems
com-onlinesupport.host
compliance-alexa.top
compliance-donald.xyz
compliance-julia.top
cool-wedding.net
coolgamechannel.com
costablue.xyz costablue.xyz
cougfan.info cranly.net
craftburg.ru crynet.cc
creditwell.ru cs-passion.pl
cyprusbuyproperties.com cubs-jerseys.us
czat.wp.pl culmimg.pw
d-black.bz custom-electric-guitar.com
daneshetabiat.com cutalltheshit.com
darodar.com cyclobenzaprinevstizanidine.blogspot.com
dichvuvesinhhanoi.com cypernhuset.se
dickssportinggoods.com data1.scopich.com
dikqlyremy.info dataloading.net
divku.ru dating-app.ru
djonwatch.ru datsun-do.com
docstoc.com davidovic.info
dogclothing.store dayibiao.com
dood.live deda-moroza-zakaz.ru
detskie-zabavi.ru
dildofotzen.net
dividendo.ru
documentbase.net
domain-submit.info
dostavka-v-krym.com
doublepimp.com doublepimp.com
dptaughtme.com drillsaw.ru
dscaas.website driving.kiev.ua
dstroy.su drunkmoms.net
dustyorate.com dvd-famille.com
dverimegapolis.ru dynainbox.com
e-commerce-seo.com earthmagic.info
e2click.com ebooknovel.club
eaplay.ru edudocs.net
earncash.com.ua edwinkonijn.com.au
easyukraine.com efnor-ac.com
editmedios.com egovaleo.it
eksprognoz.ru electrik-avenue.com
electro-prom.com elektir.ru
elektrozigarette1.dreamwidth.org elektrozigaretten1.tumblr.com
elektrozigaretten2.yn.lt elkacentr.ru
energydiet24.ru enbersoft.com
estibot.com encodable.com
evening-dating-club.info enginebay.ru
excitacao.com englishgamer.com
exportshop.us enthuse.computernetworksonline.com
ezigarettekaufen2.blox.pl eonpal.com
ezigaretteshop2.vefblog.net erectile.bid
fandlr.com esopini.com
fast-wordpress-start.com exdocsfiles.com
executehosting.com
extlinks.com
eyessurgery.ru
ezrvrentals.com
factorynightclub.com
familienzahnaerzte.com
fashionavenuegame.com
fastcrawl.com fastcrawl.com
fba-mexico.com fickenprivat.info
feeriaclub.ru figuringmoneyout.com
fellowshipoftheminds.com fillmewithhappiness.com
ferretsoft.com
filmci.pro filmci.pro
financeloan.us financehint.eu
finansov.info fingerprintjs.com
finteks.ru fitnesspiks.com
flooringinstallation-edmonton.com fivedwld.cf
floridahuntingfishingadventures.com flowersbazar.com
flytourisme.org
fodelsedagspresenter.nu
forex.osobye.ru forex.osobye.ru
fsakhalin.ru forexgb.ru
fuel-gas.com formulaf1results.blogspot.com
fxgallery.com fortwosmartcar.pw
gameprimary.com forum-engineering.ru
games.kolossale.ru forum.doctissimo.fr
gdzkurokam.ru foxjuegos.com
getlamborghini.ga free-fbook-traffic.com
gettpromos.com freelotto.com
gg.zzyjxs.com freshwallpapers.info
glall.ru freza-sverlo.ru
globatur.ru frivgame250.com
funnymama.com
gamesprite.me
gamezblox.com
gazporno.com
gdcentre.ru
genericviagrasildenafiled.net
gigixo.com
gk-atlant.info
globalscam.ga
gobongo.info gobongo.info
goosefishpost.bid goldadpremium.com
graphid.com gov.yanao.ru
greenshop.su graphicwe.org
gribkovye-zabolevaniya.com green-tea.tv
grtyi.com guildebzh.info
gsbs.com.ua gz2.bbsoldes.fr
gungamesz.com handicapvansarea.com
hawaiisurf.com hatedriveapart.com
hdseriale.pl havepussy.com
honyaku.yahoofs.jp hdxnxxtube.mobi
housemilan.ru heartofbeijing.blogspot.com
hspline.com hermesbracelets.xyz
hulfingtonpost.com hifidesign.ru
ikearugs.xyz highspeed5.net
ilbe.club highstairs-a.akamaihd.net
ilovevitaly.com hildinghr.se
hitsbox.info
hkdiiohi.skyrock.com
hledejvshopech.cz
homemature.net
hostcritique.com
hothot.ru
howmuchdoestizanidinecost.blogspot.com
howtoclean.club
hptwaakw.blog.fc2.com
hunthillfarmtrust.org
hydropump.su
hyipmanager.in
ideoworld.org
ido3.com
igru-xbox.net
ilte.info
images.gyffu.com
imagez.co
imagine-ex.co imagine-ex.co
imguramx.pw
imperiafilm.ru imperiafilm.ru
in-tandem.co
inboundlinks.win
incest-ru.com
inclk.com inclk.com
inmate-locator.us innodgfdriverhm.aircus.com
insomniagamingfestival.com
intellego.info
intermesh.net intermesh.net
investingclub.ru investingclub.ru
investpamm.ru investpamm.ru
invivo.hu iphantom.com
irunfar.com isitpaleo.info
isoveti.ru isoveti.ru
istizanidinelikexanax.blogspot.com
iwanttodeliver.com
jav-idol.com
jerseychinabizwholesale.com
jerseysbizwholesalecheap.com jerseysbizwholesalecheap.com
jmat.cn jongose.ninja
johnrobertsoninc.com journeydownthescale.info
joingames.org kabbalah-red-bracelets.com
jovencitas.gratis kaliningrad.zrus.org
juliadiets.com kellyonline.xyz
kam-dom.ru ketrzyn.pl
kambasoft.com keywordbasket.com
karpun-iris.ru keywordglobal.co.za
kasino-money.pw keywordsking.com
kenaba.su kino-key.info
keyhantercume.com kinozapas.org
keywordblocks.com knogg.net
khadastoafarde.tk kretpicf.pw
khafre.us kupiproday.com.ua
kino-filmi.com larger.io
kinopolet.net lenvred.org
kleine-titten.biz likrot.com
kndxbkdx.bloger.index.hr liver-chintai.org
kosmetyki.tm.pl livingcanarias.com
krasnodar.xrus.org lockerz.com
ku6.com lolnex.us
ladov.ru louboutinreplica.xyz
lasvegaslockandsafe.com louboutinshoes.xyz
law-check-eight.xyz lovelycraftyhome.com
lawyers.cafe
lazymanyoga.com
legionalpha.com
lemon-ade.site
lesbian.xyz
limtu.ifmo.ru
lineshops.biz
linkwithin.com
livefixer.com
lol-smurfs.com
lookingglassemb.com
lottospring.com
louboutinbooties.xyz
lrac.info
lsex.xyz
m1media.net
mac-shield.com mac-shield.com
machicon-akihabara.info macfix.life
manimpotence.com manyresultshub-a.akamaihd.net
mantramusic.ru masthopehomes.com
marcoislandvacations.net matematikus.info
mebel-vstroika.ru mature.free-websites.com
medicinacom.ru maximpartnerspr.com
medicine-4u.org mearns-tractors.co.uk
meltwater.com medi-fitt.hu
metafilter.com member-quiz.com
methodsmarketing.com metabar.ru
mexicosleevegastrectomy.com mezaruk.info
mexicotravelnet.com mielec.pl
mhi-systems.ru
misterjtbarbers.com misterjtbarbers.com
mlvc4zzw.space mixed-wrestling.ru
monetizationking.net mmostrike.ru
morlat.altervista.org mobioffertrck.com
morocco-nomad-excursions.com moncleronline.xyz
morpicert.pw morepoweronmac.trade
mountainstream.ms mp3s.club
moviezbonkerssk.cf mrbitsandbytes.com
moyakuhnia.ru mturkcontent.com
mrwhite.biz musicas.baixar-musicas-gratis.com
mvpicton.co.uk myanyone.net
mydoctorok.ru mydirtystuff.com
mygameplus.ru nastydollars.com
myhitmp3.club netanalytics.xyz
myiptest.com netpics.org
nagdak.ru newfilmsonline.ru
naj-filmy24.pl
narco24.me
neobux-bg.info
netcheckcdn.xyz
newtechspb.ru
nfljerseys.online
nibbler.silktide.com nibbler.silktide.com
no-fuel.org nikitsyringedrivelg.pen.io
nomuos.it nofreezingmac.work
november-lax.com nullrefer.com
nsatc.net nvformula.ru
o-o-6-o-o.ru oballergiya.ru
o333o.com
oakridgemo.com
obiavo.kz obiavo.kz
obsessionphrases.com
oda.as oda.as
old-rock.com offtopic.biz
onescreen.cc okayimage.com
onlinadverts.com olvanto.ru
online-hd.pl oneminutesite.it
online-x.ru online.ktc45.ru
onlinemeetingnow.com
onlineserialy.ru
onlineslotmaschine.com
ontargetseo.us
openlibrary.org openlibrary.org
optibuymac.com orhonit.com
osagonline.ru origin-my.ru
outclicks.net
paclitor.com
panchro.co.uk
paparazzistudios.com.au
paperwritingservice17.blogspot.ru paperwritingservice17.blogspot.ru
peugeot-club.org pawli.eu
paydayloanslocal.com
paydayonlinecom.com
pcboa.se
pensplan4u.com
pharmacyincity.com
phormchina.com phormchina.com
photosaga.info photosaga.info
picmoonco.pw pic2fly.com
pierrehardysale.online picquery.com
pigrafix.at picscout.com
pinwallpaper.top pictures-and-images.com
pomoc-drogowa.cba.pl pinup-casino1.ru
popup-fdm.xyz pistonclasico.com
pornhubforum.tk pixgood.com
porno-transsexuals.ru play-movie.pl
pornoblood.com play.leadzupc.com
pornoplen.com poems.com.ua
pornosemki.info polytopesexempt.com
potoideas.us popmarker.com
pr0fit-b0x.com popup-jdh.xyz
praisong.net porndairy.in
preparevideosafesystem4unow.site pornowarp.info
priora-2.com pornozhara.com
productarium.com priceg.com
projefrio.com.br privatbank46.ru
prosmibank.ru prizeestates.cricket
proxyelite.biz pro-okis.ru
ptr.ruvds.com produkto.net
putevka24.ru prospekt-st.ru
qcstrtvt.bloger.index.hr prostitutki-kharkova.org
qexyfu.bugs3.com pushdata.sendpulse.com
qld10000.net
quality-traffic.com
quickbuck.com
raisedseo.com raisedseo.com
rankexperience.com rankia.com
rankscanner.com rankscanner.com
ranksignals.com rarbg.to
refads.pro
registratciya-v-moskve.ru
registrationdomainsite.com
remedyotc.com
remont-mobile-phones.ru remont-mobile-phones.ru
remontgruzovik.ru repeatlogo.co.uk
rosbalt.com.ua replicalouboutin.xyz
resultshub-a.akamaihd.net
revistaindustria.com
rewardit.com
rfid-locker.co
ribieiendom.no
rocis.site
rospromtest.ru
rostov.xrus.org rostov.xrus.org
royalads.net ru-dety.ru
royalcar-ufa.ru running-line.ru
ru-mediaget.ru rusvideos.su
rumamba.com
russia-tao.ru
russkoe-zdorovie.ru
rvi.biz
s-iwantyou.com s-iwantyou.com
s8-nowy-wygraj.comli.com saltspray.ru
salutmontreal.com santasgift.ml
santechnik.jimdo.com sbornik-zakonov.ru
savememoney.co.za scanner-donald.top
sbprabooks.com scanner-mary.top
scalerite.co.za scanner-olga.top
scanner-jess.top scoopquest.com
scripted.com
seansonline24.pl seansonline24.pl
searchaddis.com searchimpression.com
searchimage.co searchmywindow-a.akamaihd.net
security60-e.com
selectads.men
senger.atspace.co.uk
seocdvig.ru seocdvig.ru
seonetwizard.com seoholding.com
seorank.info seolab.top
servicecenter.co.ua seozoom.it
servingnotice.com serving.adbetclickin.pink
servisural.ru serviporno.com
sex-dating.co sex-pr.net
sex-sex-sex5.com sex-tracker.com
sex-tracker.de sexflirtbook.com
sexcamamateurchat.com sexsaoy.com
sexfreepornoxxx.com sextracker.be
sexyteens.hol.es sexxdate.net
shareyards.com sexyebonyteen.com
sherlock.se shariki-zuma-lines.ru
shop.acim.org sharpchallenge.com
shoppytoolmac.com sheerseo.com
shoppingmiracles.co.uk
shtaketniki.kz
simon3.ru
simply.net simply.net
simul.co sims-sims.ru
sindragosa.comxa.com
sirpornogratis.xxx
site-speed-checker.site
sitevaluation.com sitevaluation.com
sitevaluation.org skincrate.net
sitiz.club skylta.com
slavia.info sladkoevideo.com
slim.sellany.ru
smartpet.ru smartpet.ru
sms2x2.ru smichovbike.cz
sodexo.com snjatie-geroinovoy-lomki.ru
softomix.com snowplanes.com
solicita.info sobecjvuwa.com.ru
songplanet.ru soc-econom-problems.ru
sotkal.lark.ru sovinsteel.ru
sovetogorod.ru spamnuker.com
spinazdrav.ru spanking.to
sports-supplements.us spl63.fr
sprttrack.com stackthatbucks.com
ssconstruction.co statoutlook.info
startufa.ru sterva.cc
steebook.com sticken.co
stockquotes.wooeb.com stmassage.ru
stop-gepatit.te.ua stroyhelp-dv.ru
stromerrealty.com studentguide.ru
stuff-about-money.com support.nopeas.sk
superiends.org suralink.com
surfbuyermac.com svetlotorg.ru
swagbucks.com tandvardshuset.net
taraz.xkaz.org taqywu51.soup.io
tedy.su tasteidea.com
teencastingporn.com tds-advert002.info
thedownloadfreeonlinegames.blogspot.com tecnoteakviareggio.it
thefds.net tengohydar.tk
thegioixekhach.com terraclicks.com
thetattoohut.com titan-cloud.life
thetoiletpaper.com tizanidinehcl2mg.blogspot.com
timer4web.com tizanidinesideeffects.blogspot.com
tizanidine6mg.blogspot.com tizanidinesideeffectsweightloss.blogspot.com
tizanidinehcl4mgisitanarcotic.blogspot.com tk-assortiment.ru
tizanidineonline.blogspot.com tobeyouday.win
tizanidinesleepaid.blogspot.com top-l2.com
tn811.us toxicwap.com
track.deriv.com track112.site
trafficfactory.biz traffmonster.info
trk-4.net traxdom.ru
truck-land.life treasuretrack-a.akamaihd.net
truebeauty.cc truck-rece.life
tvoystartup.ru trucri.me
tytoona.com trudogolik.net
tyumen.xrus.org trustedmaccleaner.com
u-cheats.ru try-rx.com
unitygame3d.com tt-ipd.info
univerfiles.com tuckpointingmasonrysystems.com
unlimitdocs.net tvand.ru
url2image.com ublaze.ru
uhodzalijami.ru
uk-zheu20.ru
unionmarkt.de
unpredictable.ga
urldelivery.com urldelivery.com
uroffer.link
userequip.com
uzungil.com uzungil.com
vacuumcleanerguru.com v-doc.co
validdomain.xyz vavilone.com
valoresito.com vegascosmetics.ru
varikozdok.ru venerologiya.com
vduplo.ru videochat.world
verymes.xyz videospornogratisx.net
viagra-soft.ru vigrx-original.ru
video--production.com
videochat.tv.br
videotuber.ru
viel.su
vipms.ru vipms.ru
vipsiterip.org virus-schutzmasken.de
virtualbb.com visa-pasport.ru
vltai.com voluumtracker1.com
volume-pills.biz vse-pesni.com
vpnmouse.com vsexkontakte.net
vremya.eu vykup-avto-krasnodar.ru
vsdshnik.com webradiology.ru
vzlom-na-zakaz.com webscouter.net
vzlomsn.org websitevaluebot.com
wallpaperstock.net welck.octopis.com
waysbetter.cn wesharepics.info
we-are-gamers.com westen-v.life
web.cvut.cz westen-z.life
website-speed-up.top whatsupinfoley.com
wedding0venues.tk wineitudes.wordpress.com
wellcome2slovenia.ru worldhistory.biz
wholesalejerseys-cheapest.com worldis.me
wholesalejerseyscheapjerseys.us.com www.jbetting.com
works.if.ua x69ty.ru
wpsecurity.website xlovecam.com
writersgroup580.web.fc2.com xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
wurr.voila.net xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
x-rates.ru xn----7sbahjd3btneuw1joc.xn--p1ai
xaijo.com xn----9sbubg3ambdfl1j.xn--p1ai
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai xn----dtbndd4ae7eub.top
xn----9sbebi2bvzr7h.xn--p1ai xn----itbkqkfiq.xn--p1ai
xn----btbdvdh4aafrfciljm6k.xn--p1ai xn--80ab4aa2g.xn--p1ai
xn--80abgj3a5acid6ghs.top xn--b1ag5cfn.xn--p1ai
xn--hxazdsfy.blogspot.com xn--lifehacer-1rb.com
xnxxandxvideos.com xn--oogle-wmc.com
youradulthosting.com xsfetish.org
yourtemplatefinder.com xuki.us
youtubologia.it yellowfootprints.com
z-master.ru yogamatsexpert.com
zaimhelp.ru you-shall-not-pass.is74.ru
zaimite.ru yourporngay.com
zakonobosago.ru za-fun-offer.com
zaobao.com.sg zaidia.xhost.ro
zeroredirect12.com zmoda.hostreo.com
zeroredirect7.com zoosexart.com
zigarettenonlinekaufen.tumblr.com zrizvtrnpale.tk
zixizop.net.ru zs2vm.top
znakom.sibtest.ru zscalerone.net
znakomstva-moskva77.ru
zzbroya.com.ua

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
123movies.love 2345.com
1pamm.ru 2pxg8bcf.top
1stat.ru abclauncher.com
3rbseyes.com adcash.com
3th.co.in adktrailmap.com
aa9bd78f328a6a41279d0fad0a88df1901.com admitad.com
adultfriendfinder.com adtech.de
afmuseum.com aktivator-windows10.blogspot.com
aghanyna.com allnews.md
all-number.com amateurlivechat.org
all4invest.info animenime.ru
allwomen.info arenda-avtoprokat-krasnodar.ru
ample-awards-today.us artdestshop.eu
anniemation.com asacopaco.tk
appmsr.org asia-forum.ru
arkartex.ru atleticpharm.org
artavenuegardenstudios.com autoplate.info
asseenontvonline.ru autovideobroadcast.com
audiofree.ru azartniy-bonus.com
aversis.net babosas.com
avkzarabotok.info belstaffstore.online
ayanaspa.com bestcalovebracelet.cn
balois.worldbreak.com bharatdefencekavach.com
basedecor.ru blog.remote-computer.de
beachdriveblog.com blogsfunky672.weebly.com
beauby.ru bluerobot.info
benchmarkcommunications.co.uk boostmyppc.com
bestbuy.ca brainxs.us
bestofferswalkmydogouteveryday.gq browserprotecter.com
bidbuy.co.kr buchananshardware.com
bingo8888.com burn-fat.ga
bitcoinremote.com buttons-for-your-website.com
blog.axant.it buyviagraa.blogspot.com
breget74.ru cardsharp1.ru
brg8.com carfax.com.ua
brillianty.info cartierlove2u.xyz
bumascloset.com cartierlovebraceletreplica.xyz
buntube.net cbb1smartlist12.click
buscarfoto.com ccbill.com
buyhoverboard.com chatroulette.video
buytizanidineonline.blogspot.com cheapestjerseys-wholesale.com
cartierreplica.xyz choosecuisine.com
caveavins.fr cibpenonptib.flu.cc
cialgenisrx.com ciproandtizanidine.blogspot.com
cindymatches.com circlesl.com
citetick.com citysecurity.nu
cleanpcnow.com clicksor.com
club-samodelkin.ru club-samodelkin.ru
communisave.co.za compliance-fred.xyz
compliance-george.xyz compliance-jane.top
confib.ifmo.ru coslab.club
credit-respect.ru cosmeticswomens-womensports.rhcloud.com
customerpromos-a.akamaihd.net crafthubs.com
cvety24.by
czat.wp.pl
datract.com datract.com
decorazilla.com dekorkeramik.ru
dev-seo.blog diminishedvalueoforegon.com
dikqlyremy.info docstoc.com
dojki365.online domik-derevne.ru
dptaughtme.com druzhininevgeniy63.blogspot.com
drivotracker.com
dscaas.website dscaas.website
dvd-famille.com dumpsmania24.com
ege-essay.ru dyt.net
elektronischezigarettekaufen2.tumblr.com electrik-avenue.com
elektrozigarette2.wordpressy.pl elektrozigarette1.dreamwidth.org
eugenevaultstorage.com elektrozigarettekaufen1.devhub.com
exoneration-project.us elektrozigarettekaufen2.blogse.nl
extremepornos.net elitesportsadvisor.com
ezigarettekaufen.myblog.de emerson-rus.ru
farmingworm.com epicbrogaming.com
fearcrow.com escort-russian.com
filkhbr.com este-line.com.ua
estelight.ru
eu-cookie-law.info
expolicenciaslatam.co
favorcosmetics.com
fbdownloader.com
fickblock18.com
financehint.eu financehint.eu
finemanteam.com flex4launch.ru
firstdrugmall.ru forex-indextop20.ru
flowwwers.com forum.doctissimo.fr
fok.nl foxjuegos.com
footballfarrago.com free-laptop-reward.com
forsex.info fsakhalin.ru
freeuploader.com fyxabomiw.ru
fun2cell.net gamedayassist.com
gallictures.com get-free-traffic-now.com
globetrotting-culture.ru gok-kasten.net
goatse.ru guigyverpo.cf
goodwriterssales.com guod.me
googlepositions.com habermetre.com
goosefishpost.bid hamilton.ca
gotomontenegro.net
gox.com.ua
gsmtlf.ru
haikuware.com
handicapbathtubarea.com
havepussy.com havepussy.com
herehloadibs.cf headpress.ru
hitcpm.com healing-dysplasia.ru
hustoon.over-blog.com helvetia.com.ua
ilikevitaly.com hoterika.com
ilo134uloh.com https-legalrc.biz
ilte.info huhn.altervista.org
imagerydatabase.com ifirestarter.ru
images-graphics-pics.com immigrational.info
inboxdollars.com importchinacoach-teach.com
ipchicken.com intellego.info
islandminingsupply.wordpress.com intellekt21.ru
ivoiretechnocom.ci iopeninghours.co.uk
james13prix.info iqoption-bin.com
janerikholst.se isitpaleo.info
javxxx18.com isitwp.com
jikoman.info job.icivil.ir
kaidalibor.de jumptap.com
katushka.net justprofit.xyz
kenaba.su kabbalah-red-bracelets.com
keywordhut.com kamalsinha.com
kirov.zrus.org kerwinandcariza.com
kleine-titten.biz kidskunst.info
komputers-best.ru kino-key.info
kopihijau.info klin3952.ru
kustanay.kz konkursov.net
ladov.ru krasnodar-avtolombards.ru
linkdebrideur.xyz lacave.ntic.fr
longgreen.info lashstudia.ru
lyrster.com lexaprogeneric.link
makis.nu lflash.ru
littlesexdolls.com
lizace.com
lodki-pvh.dp.ua
lottospring.com
macotool.com
mafcards.ru
magento-crew.net
maladot.com
mall.uk
marcowebonyodziezowe.pl marcowebonyodziezowe.pl
marketland.ml masturbate.co.uk
meget.co.za medi-fitt.hu
megaindex.ru
methodsmarketing.com methodsmarketing.com
mikrobiologies.ru mobile.ok.ru
mjchamonix.org mobilemedia.md
moesen-ficken.com moesonce.com
moscow.online-podarki.com mokrayakiska.com
moneyzzz.ru
montazhnic.ru
moomi-daeri.com
moyaterapiya.ru moyaterapiya.ru
musicktab.com mozello.ru
musicspire.online mp3ritm.top
musicvidz.ru mp3s.club
mvpicton.co.uk mrcsa.com.au
myxdate.info mrinsidesales.com
narco24.me mttwtrack.com
narkologiya-voronezh.ru muznachas-service.ru
mydearest.co
mylesosibirsk.ru
narutonaruto.ru
nastydollars.com
netoil.no netoil.no
newstaffadsshop.club nfljerseyscheapchinabiz.com
nibbler.silktide.com niroo.info
nuup.info o-dachnik.ru
obsessionphrases.com o00.in
okuos.com omoikiri-japan.ru
omgtnc.com onlinadverts.com
online247.ml openstat.com
ossmalta.com osnova3.ru
ozas.net ownshop.win
paighambot.com panicatack.com
paradontozanet.ru parlament.biz
partner-host.men
pathwhelp.org
pepperstyle.ru pepperstyle.ru
petitions.whitehouse.gov perl.dp.ua
petrovka-online.com phobia.us
picphotos.net piccshare.com
popander.mobi pijoto.net
popup-fdm.xyz pillscheap24h.com
porno-home365.com placid-rounded-coast.glitch.me
pornoelita.info polimga.pw
pornosexrolik.com pornolook.net
pregnant.guru pornosok.ru
pro-poly.ru productarium.com
profitsport.club progonrumarket.ru
prostitutki-odessa.org prohoster.info
psoriasis-file.trade prombudpostach.com.ua
qwarckoine.com promoforum.ru
rabot.host.sk pst2017.onlinewebshop.net
ragecash.com raisedseo.com
rapevideosmovies.com realitykings.com
resellerclub.com redirectingat.com
salut-camp.ru registry-cleaner.net
samoiedo.it replica-watch.ru
sanyuprojects.com rukino.org
sarafangel.ru rusexy.xyz
s8-nowy-wygraj.comli.com
sajatvelemeny.com
sarahmilne.top
scanner-irvin.top scanner-irvin.top
scanner-jess.top seorank.info
scripted.com servisural.ru
sdelai-prosto.ru sex-foto.pw
semaltmedia.com sex-pr.net
seorussian.ru sexkvartal.com
sexs-foto.top sitechecker.pro
sexspornotub.com siteheart.net
shariki-zuma-lines.ru sk.golden-praga.ru
ship-marvel.co.ua smart-scripts.com
sideeffectsoftizanidine.blogspot.com socas.pluto.ro
sildenafil-tadalafil.info sports-supplements.us
simplepooltips.com staynplay.net
site5.com sterva.cc
slutloadlive.com susanholtphotography.com
social-s-iii.xyz swinginwithme.ru
soft-program.com telefonsexi.com
solicita.info tfxiq.com
sonnikforme.ru thedownloadfreeonlinegames.blogspot.com
spidtest.space thepornsex.org
spymac.net tizanidineandhydrocodone.blogspot.com
sr-rekneskap.no tizanidineonline.blogspot.com
stillmiracle.com topsiteminecraft.com
stoki.ru trafaret74.ru
store-rx.com truck-rece.life
streamin.to truemfilelj.gq
svarbit.com tuckermktg.com
teastory.co tvnewsclips.info
tech4master.com twiclub.in
teenfuck.tv u-cheats.ru
tizanidine4mgtablets.blogspot.com ua.tc
tizanidinehcl4mgtabinfo.blogspot.com ucsol.ru
tizanidinehydrochloride2mg.blogspot.com uhodzalijami.ru
tizanidinehydrochloride4mgstreetvalue.blogspot.com uptime-as.net
tizanidinerecreationaluse.blogspot.com vavilone.com
tizanidinesideeffectsweightloss.blogspot.com vchulkax.com
tmtrck.com velpanex.ru
todohr.com video-chat.cn
topquality.cf vilingstore.net
tracklead.net virtuagirl.com
tradgardspartner.se vita.com.hr
trafficjunky.net weightatraining.com
twittrading.com wtsindia.in
umekana.ru wufak.com
underthesite.com x-diesel.com
us-america.ru xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
usadacha.net xzlive.com
uvozdeckych.info youandcredit.ru
valsalud.com zerocash.msk.ru
vepad.com zigzog.ru
versaut.xxx-cam.webcam zmoda.hostreo.com
vesnatehno.com znaniyapolza.ru
vgoloveboli.net zryydi.com
vkak.ru zscaler.net
vkonche.com
vkontarkte.com
vladhistory.com
vozbujdenie.com
vykupavto-krasnodar.ru
vzlomfb.com
waycash.net
wcb.su
windowssearch-exp.com
winx-play.ru
worldis.me
ww2awards.info
xgftnlrt.bloger.index.hr
xn----ctbigni3aj4h.xn--p1ai
xn--80abgj3a5acid6ghs.top
xn--80ahdheogk5l.xn--p1ai
xxxnatelefon.ru
xz618.com
yun56.co
zoomovies.org

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.47 0; 10.1.1.120 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3438 ### Version: V4.2022.10.3439
### Updated: Fri Oct 28 22:01:28 UTC 2022 ### Updated: Sat Oct 29 10:03:34 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3438 ### Version: V4.2022.10.3439
### Updated: Fri Oct 28 22:01:28 UTC 2022 ### Updated: Sat Oct 29 10:03:34 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##