vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 10:40:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.01.3553 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-01-21 10:09:18 +00:00
parent 7a52909efc
commit f43bbc8092
47 changed files with 97669 additions and 97659 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3552 #### Version: V4.2023.01.3553
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.01.3552 #### Version: V4.2023.01.3553
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2023.01.3552 #### Version: V4.2023.01.3553
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12072
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2023.01.3552 #### Version: V4.2023.01.3553
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12076
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12076
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12076
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12076
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

12076
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

12076
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

12076
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3552 3553

12076
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12076
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.88.61.192 0; 172.177.73.0 0;

12076
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

12076
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

12076
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

12076
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.88.61.192 0; 172.177.73.0 0;

12076
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.88.61.192 0; 172.177.73.0 0;

12076
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 1; 10.1.0.8 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Attach BuiltBotTough
BacklinkCrawler CSHttp
Harvest Gigabot
Linkbot Mr.4x3
RankActive Nibbler
Voil Reaper
Web Collage WebCollage
Zauba cognitiveseo
check1.exe facebookscraper
muhstik-scan x22Mozilla

298
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
AIBOT 404checker
ALittle Client 80legs
Aboundexbot ADmantX
AfD-Verbotsverfahren ASPSeek
Alexibot Acunetix
Alligator AllSubmitter
Anarchie Anarchy
Anarchy99
Ankit
Anthill Anthill
Aspiegel Atomseobot
BDCbot BBBike
BackDoorBot
BackWeb BackWeb
Bandit BacklinkCrawler
Barkrowler Badass
BetaBot Battleztar Bazinga
BlowFish Bigfoot
Bitacle
Black Hole
BlackWidow
Bolt Bolt
BotALot Brandprotect
BuiltBotTough Buck
BuiltWith
Bullseye
BunnySlippers
BuzzSumo BuzzSumo
CSHttp CATExplorador
CCBot
CensysInspect
CheeseBot CheeseBot
Chlooe CherryPicker
ChinaClaw
Citoid Citoid
Claritybot Claritybot
Cloud mapping
Copier Copier
Cosmos Copyscape
CyotekWebCopy CrunchBot
DBLBot DBLBot
DIIbot
DSearch DSearch
DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli
Demon Demon
Devil DigitalPebble
Disco DomCopBot
DittoSpyder
DomainAppender DomainAppender
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Dotbot Dotbot
Download Wonder Dragonfly
Drip Drip
EMail Siphon
EMail Wolf EMail Wolf
EasyDL Ebingbong
Ecxi Ecxi
EirGrabber
ExtLinksBot
Extractor Extractor
EyeNetIE ExtractorPro
Extreme Picture Finder
Ezooms Ezooms
Fimap FDM
FemtosearchBot
Flunky Flunky
Foobot Fyrebot
Fuzz Genieo
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla GoZilla
GrabNet
Grafula
GridBot GridBot
HTMLparser HEADMasterSEO
HTTP::Lite HMView
HaosouSpider
Harvest Harvest
Havij Havij
Hloader Hloader
HonoluluBot
Humanlinks Humanlinks
IRLbot IDBot
Id-search Id-search
Image Sucker Image Fetch
Intelliseek InfoNaviRobot
InfoTekies
Internet Ninja Internet Ninja
JOC Web Spider Iria
IstellaBot
Jbrofuzz Jbrofuzz
Jetty JennyBot
Joomla JikeSpider
JustView Jorgee
Kenjin Spider Keyword Density
Kinza
LWP::Simple LWP::Simple
Lanshanbot LeechFTP
Larbin
Leap
LexiBot LexiBot
LibWeb
Libwhisker
LieBaoFast LieBaoFast
Likse Linkbot
LinkWalker LinkextractorPro
LinkpadBot LinksManager
LinqiaMetadataDownloaderBot
Lipperhey
Lipperhey Spider Lipperhey Spider
MIDown tool Lmspider
MIIxpc MJ12bot
MTRobot MSIECrawler
Mail.RU_Bot
Majestic-SEO Majestic-SEO
Majestic12 MarkWatch
Mata Hari Masscan
Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot Meanpathbot
MegaIndex.ru
Metauri Metauri
MicroMessenger Moblie Safari
Mister PiX
Mojeek Mojeek
Mojolicious
MolokaiBot MolokaiBot
Morfeus Fucking Scanner Mr.4x3
Msrabot Net Vampire
Musobot
NPbot
Nessus
NetSpider NetSpider
NetZIP NetZIP
Netvibes Netcraft
NextGenSearchBot NextGenSearchBot
Niki-bot NimbleCrawler
Nuclei Nimbostratus
Nutch Nmap
Offline Explorer Not
Octopus
OnCrawl
OpenLinkProfiler
OpenVAS OpenVAS
Openvas Openfind
OrangeSpider OrangeSpider
OutclicksBot OutfoxBot
PECL::HTTP PageGrabber
PHPCrawl PageThing.com
POE-Component-Client-HTTP
Page Analyzer
PageAnalyzer
Pandalytics Pandalytics
Panscient Piepmatz
Pavuk
Picscout
PictureFinder
Pimonster Pimonster
Pixray PleaseCrawl
Pockey
ProPowerBot ProPowerBot
Probethenet Probethenet
Pu_iN Pu_iN
QueryN Metasearch Pump
RankFlex Quick-Crawler
RSSingBot
RankingBot2 RankingBot2
Rankivabot Re-re
RealDownload Reaper
RebelMouse RebelMouse
Recorder RepoMonkey
RedesScrapy
RocketCrawler
Rogerbot Rogerbot
SEOprofiler SBIder
SEOkicks
SEOstats SEOstats
ScanAlert SISTRIX
Scanbot Screaming
ScrepyBot
Searchestate Searchestate
SearchmetricsBot SearchmetricsBot
Semrush Seekport
SemrushBot SemrushBot
SeobilityBot SentiBot
SiteCheckerBotCrawler
Sitebeam Sitebeam
Siteimprove Siteimprove
Sitevigil Sitevigil
Snapbot Snake
Snoopy Snoopy
Sociscraper Sottopop
SpaceBison SpaceBison
Spammen
SpankBot
Spbot Spbot
Spinn3r SputnikBot
Sqlmap Steeler
Sqworm Sucuri
Stripper
SuperBot SuperBot
SurveyBot SuperHTTP
Suzuran Suzuran
Swiftbot Teleport
TeleportPro Telesphorep
Telesoft
Titan Titan
Tracemyfile Toata
Trendiction
Turingos
Turnitin Turnitin
Twice TurnitinBot
Typhoeus Typhoeus
URLy Warning URLy Warning
URLy.Warning
Upflow Upflow
VCI VCI
VelenPublicWebCrawler Vagabondo
VidibleScraper Virusdie
VoidEYE VoidEYE
Voil
WASALive-Bot
WBSearchBot WBSearchBot
WEBDAV
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWWOFFLE
WeSEE Wallpapers/3.0
WallpapersHD
Web Collage
Web Enhancer Web Enhancer
Web Fetch
Web Fuck
Web Pix
Web Sucker Web Sucker
WebAuto WebAuto
WebCollage WebCollage
WebCopier
WebFetch WebFetch
WebImageCollector WebFuck
WebGo IS
WebLeacher
WebSauger WebSauger
WebStripper
WebWhacker
WebmasterWorldForumBot
Webshag Webshag
Website Quester WebsiteQuester
WebsiteExtractor
Webster Webster
Whack Whacker
Who.is Bot
Widow Widow
WiseGuys Robot
Wonderbot Wonderbot
Woobot Woobot
Wprecon Wprecon
Xenu
YoudaoBot
Zauba
Zeus
Zitebot Zitebot
ZmEu ZmEu
ZumBot ZoomBot
ZyBorg
adscanner
archive.org_bot
arquivo-web-crawler
arquivo.pt arquivo.pt
backlink-check
cognitiveseo
crawl.sogou.com
crawler.feedback
crawler4j
demandbase-bot
facebookscraper facebookscraper
instabid linkdexbot
internetVista monitor meanpathbot
isitwp.com
lwp-trivial
magpie-crawler
mediawords mediawords
muhstik-scan netEstate NE Crawler
pcBrowser polaris version
plumanalytics ripz
s1z.ru s1z.ru
scalaj-http scalaj-http
scan.lol scan.lol
seobility seobility
seoscanners seoscanners
serpstatbot sp_auditbot
sexsearcher
sitechecker.pro
trendiction.com trendiction.com
trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com webgains-bot
x22Mozilla webprosbot
xpymep1.exe xpymep1.exe
zauba.io

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider
404checker
80legs
AIBOT
Abonti Abonti
Aboundex Aboundex
AfD-Verbotsverfahren Acunetix
AhrefsBot AhrefsBot
AiHitBot AiHitBot
Aipbot
Alexibot Alexibot
AllSubmitter
Alligator Alligator
Ankit AlphaBot
Anarchie
Anarchy99
Apexoo Apexoo
Aspiegel Aspiegel
Attach
AwarioRssBot
BDFetch BDFetch
BackWeb BLEXBot
Backlink-Ceck Backlink-Ceck
Badass Badass
Bandit BatchFTP
BetaBot
Bitacle
Black Hole Black Hole
Blackboard Blackboard
Blow
BlowFish BlowFish
Bolt Boardreader
BuiltBotTough BuiltBotTough
BunnySlippers Bullseye
CATExplorador
CCBot
CODE87
Cegbfeieh Cegbfeieh
CensysInspect
CheTeam
CherryPicker
Chlooe Chlooe
Claritybot Claritybot
Cocolyzebot Cloud mapping
Collector Collector
Copier Cosmos
CopyRightCheck
Craftbot
CrazyWebCrawler CrazyWebCrawler
Crescent CrunchBot
Custo Custo
CyotekWebCopy DBLBot
Demon
Devil Devil
DigitalPebble
Dirbuster Dirbuster
Disco Dispatch
DittoSpyder DittoSpyder
DnyzBot DnBCrawler-Analytics
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project Dotbot
Download Wonder Download Wonder
Drip Dragonfly
EMail Siphon EMail Siphon
EMail Wolf
Ebingbong Ebingbong
EroCrawler Ecxi
Express WebPictures EirGrabber
Evil
Exabot
ExtLinksBot
ExtractorPro
Ezooms Ezooms
FDM
FHscan
FemtosearchBot FemtosearchBot
Fimap Fimap
FlashGet FlashGet
Flunky Foobot
Freeuploader
FrontPage
FyberSpider FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t
GT::WWW GT::WWW
Genieo GalaxyBot
GetWeb GetRight
Getintent Go-Ahead-Got-It
GoZilla GoZilla
GrabNet
Grafula Grafula
GrapeFX
GrapeshotCrawler GrapeshotCrawler
HMView GridBot
HTTP::Lite Havij
Hloader Heritrix
Humanlinks
IDBot IDBot
IRLbot IRLbot
Iblog
Id-search Id-search
IlseBot IlseBot
Image Fetch Image Fetch
Indy Library
InfoNaviRobot
InfoTekies InfoTekies
Jbrofuzz Intelliseek
Jetty InterGET
InternetSeer
JOC Web Spider
JetCar
JikeSpider JikeSpider
Jorgee Jorgee
JustView Keyword Density
Kinza Kozmosbot
LWP::Simple Leap
LeechFTP LeechFTP
LeechGet LeechGet
LexiBot LexiBot
Lftp Lftp
LibWeb LibWeb
Libwhisker LieBaoFast
Lightspeedsystems LinkScan
LinkWalker
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lmspider Lipperhey
Ltx71 Litemage_walker
MIIxpc MJ12bot
MQQBrowser
MSFrontPage
MSIECrawler
MTRobot MTRobot
Majestic SEO Majestic SEO
Majestic12 Majestic-SEO
Masscan Mass Downloader
MauiBot Mata Hari
MeanPath Bot MeanPath Bot
Metauri Meanpathbot
Microsoft URL Control Mediatoolkitbot
Minefield Microsoft Data Access
Moblie Safari Mister PiX
Mojolicious Mojeek
MolokaiBot
Morfeus Fucking Scanner
Mozlila
Mr.4x3 Mr.4x3
Msrabot Msrabot
Musobot
NPbot
Name Intelligence Name Intelligence
Net Vampire Nameprotect
NetAnts NearSite
NetSpider Needle
NetZIP NetZIP
Nettrack Nettrack
Nibbler
Nimbostratus Nimbostratus
Octopus Ninja
OpenLinkProfiler Nuclei
Offline Explorer
OnCrawl
OpenVAS OpenVAS
Openvas OrangeSpider
PECL::HTTP PHPCrawl
PageGrabber POE-Component-Client-HTTP
PageScorer PageScorer
Pavuk Papa Foto
Pixray Petalbot
ProPowerBot Picsearch
Probethenet Piepmatz
Pockey
ProWebWalker
Pump
PxBroker PxBroker
PyCurl
QueryN Metasearch QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot
RankActive
RankActiveLinkBot
RankFlex RankFlex
RebelMouse Rankivabot
Recorder Re-re
RedesScrapy
Ripper
RocketCrawler RocketCrawler
SEOkicks SEOkicks
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SEOprofiler
SalesIntelligent SEOstats
ScanAlert Scanbot
ScoutJet Scrapy
Screaming
ScrepyBot ScrepyBot
Semrush
SemrushBot SemrushBot
SeoSiteCheckup SentiBot
Shodan Seomoz
Siphon Siphon
Site Sucker
SiteSnagger SiteSnagger
SiteSucker Sitevigil
Sitebeam Sociscraper
SlySearch Spanner
Snapbot Spbot
SocialRankIOBot Sqlmap
Sosospider Sqlworm
Spammen Stripper
SpankBot Sucker
SuperHTTP Suzuran
Swiftbot Szukacz
Teleport
TeleportPro TeleportPro
Telesphoreo Telesoft
Telesphorep Titan
Trendiction
Trendictionbot Trendictionbot
Turnitin True_Robot
TurnitinBot Turingos
URLy.Warning TwengaBot
UnisterBot
V-BOT
VB Project
VCI
Vacuum
VeriCiteCrawler VeriCiteCrawler
Virusdie
VoidEYE
Voil Voil
WEBDAV Voltron
WBSearchBot
WISENutbot
WPScan WPScan
WWW-Mechanize WWWOFFLE
WallpapersHD WeSEE
Web Collage Web Collage
Web Fetch Web Enhancer
Web Fuck Web Fuck
WebCollage Web Pix
WebFuck Web Sauger
WebImageCollector WebEnhancer
WebLeacher WebFetch
WebPix
WebSauger WebSauger
WebStripper WebWhacker
Webshag WebmasterWorldForumBot
WebsiteExtractor WebsiteExtractor
WebsiteQuester
Webster
Whacker Whacker
Whatweb
Widow
WiseGuys Robot
Woobot Woobot
Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Zauba YoudaoBot
Zade
ZmEu ZmEu
ZumBot arquivo.pt
adscanner cah.io.community
arquivo-web-crawler dataforseo.com
backlink-check
com.plumanalytics
demandbase-bot
eCatch eCatch
evc-batch evc-batch
heritrix
instabid
internetVista monitor internetVista monitor
ips-agent
linkdexbot linkdexbot
lwp-request lwp-request
lwp-trivial lwp-trivial
meanpathbot magpie-crawler
mediawords netEstate NE Crawler
plumanalytics oBot
page scorer
pcBrowser
polaris version polaris version
ripz
s1z.ru s1z.ru
seocompany.store
seoscanners seoscanners
seostar
serpstatbot serpstatbot
sogouspider siteripz
trendiction.de sp_auditbot
ubermetrics-technologies.com spyfu
voyagerx.com sysscan
webmeup-crawler webgains-bot
webpros.com webpros.com
x22Mozilla xpymep1.exe
zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
bgviagrachrx.com apps-analytics.net
bird1.ru druzhbany.ru
getyourimage.club dtm-spain.com
gsasearchengineranker.xyz free-fbook-traffic.com
gwebtools.com.br istock-mebel.ru
klikbonus.com pic2fly.com
mebelcomplekt.ru search-goo.com
panamaforbeginners.com topcasinoratings.ru
thenews-today.info vucms.com
topkarkas.com zeroredirect8.com

934
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
007angels.com 18ps.ru
00it.com 1kdailyprofit.me
1-99seo.com 1stat.ru
101billion.com 1webmaster.ml
101raccoon.ru 4istoshop.com
11235813.webzdarma.cz 4webmasters.com
1xbet4you.com 4webmasters.org
256bit.by 5-steps-to-start-business.com
3weekdiet.com 83net.jp
4kplayer.pl 91abcw.com
4replicawatch.net
5forex.ru
8lufu.com
a.frcls.fr
a96527gi.beget.tech
aa9d046aab36af4ff182f097f840430d51.com
aac500b7a15b2646968f6bd8c6305869d7.com
aarbur.com
abcdeg.xyz
abovetherivernc.com abovetherivernc.com
absolutelyanalog.com abtasty.com
acc.eu.org abwa.tk
activepr.ru acortarurl.es
addbin.men activecampaign.dreamhosters.com
acunetix-referrer.com
adamoads.com
addray.pro addray.pro
admanaerofoil.com adktrailmap.com
adtology.com ads.tremorhub.com
adultactioncam.com advancedmassagebysara.com
agecheckadult.com advocatemsk.ru
ageofclones.com adzpower.com
aktobe.xkaz.org agahinameh.com
alert-fjg.xyz ahrntal.verymes.xyz
algarveglobal.com alfa9.com
alkoravto.ru alienwheels.de
allbizne.co.ua all-number.com
altamayoztourism.com alta-realestate.com
aludecor.info ameblo.jp
amazon-seo-service.com amt-k.ru
amyfoxfitness.com amyfoxfitness.com
ananumous.ru anaseracresar.tk
animal-drawings.com andpolice.com
api.stathat.com android-style.com
apiappenableinfo-a.akamaihd.net anmysite.com
apibrowsesmartne-a.akamaihd.net apiholdingmypage-a.akamaihd.net
apimountainbikei-a.akamaihd.net apiqualitinknet-a.akamaihd.net
appfixing.space app-ready.xyz
approvedlocal.co.za appfastplay.com
arpelsreplica.xyz appmsr.org
ascat.porn arendakvartir.kz
asrvrep-a.akamaihd.net arkartex.ru
auspolice.com atelielembrancaqueencanta.com.br
auto-moto-elektronika.cz atlasvkusov.ru
autoblogger24.info australia-opening-times.com
autotop.com.ua autogrep.ru
azbukadiets.ru azazaporn.com
azbukafree.com babosas.com
balook.com balkanfarma.ru
baltgem.com
bankmib.ru bankmib.ru
bedandbreakfast.com bankofthewext.com
begalka.xyz bannerconnect.net
belstaffstore.online bannertesting.com
bestbrainprod.win barbourjackets.online
besthomemadepornsites.com bard-real.com.ua
bestmarriages.com basisches-wasser.net
betshuckclean.com battle.net
big-boards.info bazaronline24.ru
bigames.online beauty-things.com
bioscorp.ru berlininsl.com
bestempresas.es
bestimagecoollibrary.com
betune.onlinewebshop.net
biketank.ga
bizcheapjerseyswholesalechina.com bizcheapjerseyswholesalechina.com
blavia.00author.com biznesrealnost.ru
blog.f00kclan.de biznesschat.net
bloggerads.net bjanshee.ru
blue-square.biz bjorkbacken.nu
bmsco.co blacktwhite.com
board.f00d.de blog.koreadaily.com
bonvillan.com bodybuilding-shop.biz
brg8.com bonzbuddy.com
brus.city bosman.pluto.ro
btc4u.ru brandbucket.com
brateg.xyz
bumble.cheapwebsitehoster.com bumble.cheapwebsitehoster.com
buytizanidineonlinenoprescription.blogspot.com buqyxa.rincian.info
buyviagraa.blogspot.com buy-loft.ru
buzz-porno.info buyessaynow.biz
callawaygolfstore.online call-of-duty.info
callejondelpozo.es calvet.altervista.org
candlehandmade.com candiceloves.us
cartierjusteunclou.xyz cardiosport.com.ua
cartierreplica.win cartierlovebracelet.xyz
casablancamanor.co.za cdncash.com
casite-513049.cloudaccess.net cdncash.net
cb.iphantom.com cdncash.org
cdnanalytics.xyz centrumcoachingu.com
cete.ru chastnoeporno.com
cfacarrosserie74.com chatroulette.life
cfsrating.sonicwall.com cheapestjerseysonwholesale.com
chatroulette.world cheapsergiorossi.online
cheapmoncler.pw check-host.net
chimiver.info chinaelitecheapjerseys.com
chocolateslim-en-france.com choice-credit.ru
christianlouboutinreplica.win christianlouboutinreplica.pw
cindymatches.com christianlouboutinsaleoutletonline.us
citetick.com cialiswithoutadoctor.net
claytransformations.info click2pawn.com
clickprotects.com clickpapa.com
clippingphotoindia.com clothesforcash.com
codq.info club-musics.ru
coindirect.io collegeessay19.blogspot.ru
collegerentals.com compliance-don.top
compliance-elena.top construmac.com.mx
compliance-walter.top contentsexpress.com
conciergegroup.org cookie-law-enforcement-ff.xyz
coolingoods.com cordstrap.cc
copyrightclaims.org corta.co
customerguru.in courtshipgift.com
dailystrength.org cowblog.fr
dairyindia.in crafthubs.com
daisye.top credit.co.ua
damasarenai.info crest3d.ru
ddlmega.net custom-product-labels.com
ddsoldes.fr danceuniverse.ru
demian.kz daooda.com
digital-sale.su dayibiao.com
directivepub.com dealighted.com
disruptingdinnerparties.com design-ideas.info
detsky-nabytek.info
devochki.top
dimaka.info
dimfour.com
dnepr-avtospar.com.ua dnepr-avtospar.com.ua
dnepropetrovsk.mistr-x.org dokfilms.net
dscaas.website doktoronline.no
dustyorate.com domain.webkeyit.com
e90post.com domaincdn.xyz
education-cz.ru domainroam.win
empirepoker.com domini.cat
esopini.com dosugrostov.site
eugenevaultstorage.com download-of-the-warez.blogspot.com
europeanwatches.ru drugstoreforyou.com
exct.net dtm-spain.com
ezigarettekaufen.myblog.de e2click.com
ezigarettenshop2.postbit.com eandsgallery.com
facialporntube.com econom.co
familienzahnaerzte.com ecxtracking.com
familyholiday.ml educatemetv.com
fantasticpixcool.com edudocs.net
femmesdenudees.com eets.net
fenoyl.batcave.net efkt.jp
fetlifeblog.com elektronischezi.livejournal.com
fifa-coins.online elektrozigarette1.dreamwidth.org
filefilter.weebly.com elexies.info
enbersoft.com
enews.tech
epicbrogaming.com
eropho.com
eu-cookie-law.blogspot.com
excitacion.info
exoclick.com
fable.in.ua
familyphysician.ru
favorcosmetics.com
faz99.com
fbdownloader.com
fettefrauen.net
fickluder69.com
fil.ru
filmania-x.ru filmania-x.ru
fishtauto.ru filmci.pro
flash4fun.com findthe.pet
flashahead.info fireads.men
foxweber.com fiuxy.com
free-floating-buttons.com flex4launch.ru
freemaintenancesysforpcandmac.top flowersbazar.com
frequiry.com flowwwers.com
frustrated-favorable.gq fordsonmajbor.cf
fuckmill.com formulaf1results.blogspot.com
fun-mobi.pl forums.toucharcade.com
fxgallery.com free-stock-illustration.com
g7m.pl freevpn.space
gallery.rennlist.com fym.com.ru
gamerextra.com gagrasector.ru
garciniaxt.us galeon.com
gaz-voshod.ru gambarkatabaru.com
games.kolossale.ru
gazobeton-p.com.ua
ge0ip.org ge0ip.org
gearcraft.us gemgrab-a.akamaihd.net
generalporn.org generic-pills-online.com
geoads.com germes-trans.com
get-free-traffic-now.com getprismatic.com
getoutofdebtfree.org ghernnqr.skyrock.com
getridofstretchmarks.org girlsatgames.ru
gkvector.ru globalscam.ga
glastecfilms.com.my gne8.com
globetrotting-culture.ru gogalleryawesome.com
golaya.pw
golden-praga.ru
goodly.pro
googlefeud.com
gosarhivrt.ru
gov.yanao.ru
grand-chlen.ru
greatfind-a.akamaihd.net
greatzip.com
growmyfunds.ca
gstatey.net gstatey.net
guardlink.com hackers-crackers.tk
handsandlegs.ru hao123.com
hatedriveapart.com hazardky.net
hchha.com hd-filmy.net
health-medical-portal.info hebr.myddns-flir.com
heartofpayne.xyz helvetia.com.ua
hermesbracelets.xyz hermesbracelets.xyz
hitmuzik.ru hetmanship.xyz
hotelcrocenzi.sm hikesearch.net
hvd-store.com hitcpm.com
iamsport.org homedecoguide.info
iboss.com homemade.gq
imgarit.pw horoshieokna.com
hoste.octopis.com
hotblognetwork.com
hotgirlhdwallpaper.com
howtoclean.club
howtowhitenteethfast.xyz
i-hobot.ru
iblogpress.xyz
ico.re
ilikevitaly.com
ilmen.net
imadedinner.net
imediadesk.com
imicrovision.com imicrovision.com
impotentik.com inbabes.sexushost.com
insurple.com incanto.in.ua
interesnie-faktu.ru infodocsportal.com
intimshop-fantasy.ru investsuccess.org
invivo.hu iomoio.net
ipornox.xxx irkutsk.online-podarki.com
istizanidineacontrolledsubstance.blogspot.com istizanidineanopiate.blogspot.com
iusstf.org itrevolution.cf
janerikholst.se javstock.com
janettabridal.com justbcause.com
javspace.net justucalling32211123456789.tk
jerseywholesalechinabiz.com jwss.cc
karadene.com kalb.ru
keywordsdoctor.com kambasoft.com
kidskunst.info katjimej.blog.fc2.com
kino-rf.ru kevblog.top
kino2018.club keywordteam.net
klejonka.info klikbonus.com
knigonosha.net knowsitall.info
kometars.xyz koronirealestate.gr
konturkrasoty.ru kosova.de
krasivye-devushki.net kovrenok.ru
krasnodar-avtolombards.ru krasivoe-hd.net
krasula.pp.ua krasnodar.xrus.org
laapp.com krassh.ru
labelwater.se kupit-adenu.ru
lanasshop.ru kuzinsp.ru
languagecode.com kw21.org
lapitec.eu laptoper.net
lazy-z.com laserpen.club
lechenie-gemorroya.com lasvegaslockandsafe.com
linkbolic.com lawyersinfo.org
littleberry.ru lenpipet.ru
littlesunraiser.com lernur.net
livejasmin.com lezbiyanki.net
locksmith.jp life-instyle.com
locustdesign.co lifebyleese.com
lodki-pvh.dp.ua lifehacĸer.com
lolnex.us link.ac
lombia.com linkarena.com
lonerangergames.com lombardfinder.ru
louboutinreplica.xyz long-beach-air-conditioning.com
love-baby.cz longgreen.info
luxup.ru luciddiagnostics.in
magazin-pics.ru m3gadownload.pl
maju.bluesalt.co madot.onlinewebshop.net
make-money-online.com mafcards.ru
manipulyator-peterburg.ru magicplayer-s.acestream.net
mansiondelrio.co makedo.ru
mapquestz.us makenahartlin.com
massage-info.nl malls.com
marcogrup.com
masserect.com masserect.com
master-muznachas.ru masterseek.com
matb3aa.com matchpal-a.akamaihd.net
max-p.men mb140.ru
mebel-vstroika.ru mediafresh.online
mediaoffers.click medicineseasybuy.com
members.ghanaweb.com medispainstitute.com.au
memberty.com medizinreisen.de
michaelkorsoutlet.store mesbuta.info
millionare.com metabar.ru
mmostrike.ru migente.com
mobile-appster.ru mikozstop.com
mobsfun.net mikrobiologies.ru
moncleroutletonline.pw minecraft-neo.ru
monthlywinners.com mitrasound.ru
morlat.altervista.org mixx.com
mosrif.ru mlvc4zzw.space
moviezbonkerssk.cf mobilemedia.md
moyakuhnia.ru mobstarr.com
moetomnenie.com
moi-glazki.ru
mompussy.net
monclercoats.xyz
most-kerch.org
motherboard.vice.com
mrinsidesales.com
msk.afora.ru
multgo.ru
musicstock.me musicstock.me
my-bc.ru mxgetcode.com
nac-bearings.ru my-floor.in.ua
nastydollars.com mybinaryoptionsrobot.com
naturalpharm.com.ua myftpupload.com
naturalshair.site mylovelibrabry.com
netpics.org nacap.ru
nextlnk12.com naj-filmy24.pl
nfljerseyscheapbiz.us nakozhe.com
nina.az narutonaruto.ru
nastroyke.net
naturalbreakthroughsresearch.com
naturtreenspicerx.pw
naval.jislaaik.com
nextrent-crimea.ru
nicefloor.co.uk
nonews.co nonews.co
novgorod.xrus.org normalegal.ru
obnal.org novosibirsk.xrus.org
oconto.ru o00.in
obiavo.in
oda.as oda.as
office-windows.ru officedocuments.net
olympescort.com
omoikiri-japan.ru omoikiri-japan.ru
online-x.ru online.ktc45.ru
onlinebay.ru onlinebay.ru
onload.pw onlinedomains.ru
ontargetseo.us ordernorxx.com
openfrost.net orion-v.com
openmultipleurl.com otbelivanie-zubov.com
oranga.host.sk outclicks.net
owathemes.com page2rss.com
ownshop.cf pathwhelp.org
owohho.com pensplan4u.com
palocco.it perederni.net
pandroid.co photostudiolightings.com
parajumpersstore.online
partner-cdn.men
pavlodar.xkaz.org
pawli.eu
pc-test.net
php-market.ru php-market.ru
pihl.se phrcialiled.com
pivka.xyz picquery.com
pix-hd.com pics-group.com
pk-services.ru pictures-and-images.net
pl-top.pl pillscheap24h.com
poem-paying.gq pinsdaddy.com
polcin.de pinstake.com
popup-fdm.xyz popugauka.ru
poquoson.org porndairy.in
porno.simple-image.com.ua pornhive.org
pornoreino.com pornobrazzers.biz
pornosemki.info pornoplen.com
pornosexrolik.com
pornoslive.net pornoslive.net
pornosmola.info
pornstartits.xblog.in pornstartits.xblog.in
poshiv-chehol.ru pravoholding.ru
predominant-invent.tk privacylocationforloc.com
pro-poly.ru privatamateure.com
prod2016.com privetsochi.ru
profitfx.online prohoster.info
prom23.ru pts163.ru
promoforum.ru purchasepillsnorx.com
promover.org putitin.me
pst2017.onlinewebshop.net rabotaetvse.ru
qualitymarketzone.com rapevideosmovies.com
raavidesigns.com rapidsites.pro
radiodigital.co real-time-analytics.com
rangjued.com reckonstat.info
ranking2017.ga registry-cleaner.net
raschtextil.com.ua reklamuss.ru
razamicroelectronics.com resultshub-a.akamaihd.net
redirect.trafficreceiver.club rfid-locker.co
reeyanaturopathy.com ribieiendom.no
referencemoi.com rique.host.sk
reimageplus.com rn-to-bsn.com
reklama-i-rabota.ru ruex.org.ua
remont-fridge-tv.ru ruf777.com
rentaremotecomputer.com runtnc.net
replicaclub.ru russkie-gorki.ru
retailwith.com safe-app.net
ringtonepartner.com sanatorrii.ru
romhacking.ru
rossanasaavedra.net
royal-betting.net
ruscoininvest.company
s.lollypopgaming.com
s8-nowy-wygraj.comli.com
santasgift.ml
santechnik.jimdo.com santechnik.jimdo.com
sarahmilne.top saratov.xrus.org
sashagreyblog.ga
saveindex.xyz
sbf441.com
scalerite.co.za
scanmarine.info scanmarine.info
scanner-margo.top school-diplomat.ru
sdsjweb.com schoolfiles.net
seansonline24.pl sdi-pme.com
searchengineranker.email searchengineranker.email
secretscook.ru seksotur.ru
security60-e.com sell-fb-group-here.com
semxiu.com seokicks.de
seofirmreviewsus.info servethis.com
serialsway.ucoz.ru service-core.ru
sex-foto.pw sexkontakte-seite.com
sex-tracker.com shivafurnishings.com
sexflirtbook.com site-auditor.online
sexsaoy.com site5.com
shikiso.info sitebeam.net
shiksabd.com sitevalued.com
shop.xz618.com sitopreferito.it
shoppingmiracles.co.uk slomm.ru
silvermature.net smartpet.ru
sim-service.net sobecjvuwa.com.ru
simple-share-buttons.com social-s-ggg.xyz
simpoed.ufop.br softomix.com
siteexpress.co.il soserfis.com
skladvaz.ru souvenir.cc
sky-mine.ru
sladkoevideo.com
smadihome.com
smart-scripts.com
sneakyboy.com
social-fun.ru
sohoindia.net
sophang8.com
sortthemesitesby.com
sovinsteel.ru sovinsteel.ru
speedup-my.site space4updating.win
spinnerco.ca spamnuker.com
spanking.to
sprttrack.com
sptslmtrafms.com
spy-sts.com
stair.registrydomainservices.com stair.registrydomainservices.com
starwars.wikia.com star61.de
stromerrealty.com starpages.net
suhanpacktech.com stroymonolit.su
supermama.top supermesta.ru
svetodiodoff.ru surflinksmedical.com
svnuppsalaorebro.se swiped.su
swsociety.se syndicate.fun
tagil.zrus.org tackletarts.co
taqywu51.soup.io tam-gde-more.ru
tds-advert005.info tehngr.ru
tedy.su thebluenoodle.com
telegraf.by thecoral.com.br
telsis.com thegolfclub.info
tengohydar.tk theguardlan.com
the-usa-games.blogspot.com
theprofitsmaker.net
thruport.com
titan-ads.life
tivolibasket.it tivolibasket.it
tizanidineandgabapentin.blogspot.com
tizanidineduringpregnancy.blogspot.com tizanidineduringpregnancy.blogspot.com
tizanidinehcl2mgsideeffects.blogspot.com tizanidineforopiatewithdrawal.blogspot.com
tizanidinehcl4mgtab.blogspot.com tizanidinehcl4mgtabinfo.blogspot.com
tizanidinehcl4mgtablet.blogspot.com tizanidinehclsideeffects.blogspot.com
tizanidinehydrochloride2mg.blogspot.com tizanidinerestlesslegsyndrome.blogspot.com
tizanidineusedfor.blogspot.com tizanidinevssoma.blogspot.com
tkanorganizma.ru tizanidinezanaflex.blogspot.com
tmtrck.com tongkatmadura.info
toloka.hurtom.com topads.men
toondinsey.com topcasinoratings.ru
top10registrycleaners.com topsy.com
top250movies.ru tozup.com
topcar-krasnodar.ru tradgardspartner.se
topmira.com trafficmonetize.org
toursmaps.com tri-slona.org
tracksz.co tripper.de
tradedeals.biz truebeauty.cc
traderzplanet.in tula.howotorg.ru
trafficnetzwerk.de
trahvid.com
trainoffend.ml
trkur.com
truck-rece.life
trustl.life
tryrating.com
ttsq.fr
tupper-shop.ru
turvgori.ru
twinderbella.com
twittruth.com
twodollarshows.com
u-cheats.ru
u555u.info u555u.info
uhod-za-sobakoj.ru umnovocaminho.com
unacittaconte.org url2image.com
unimodemhalfduplefw.pen.io uzungil.com
unrealcommander.biz vavilone.com
upstore.me veerotech.com
usdx.us verymes.xyz
viagengrarx.com viagra.pp.ua
viagra-soft.ru viberdownload10.com
video-woman.com video-chat.love
videos-for-your-business.com videochat.bz
videosbox.ru videosbox.ru
videospornogratisx.net videositename.com
videtubs.pl
vielporno.net
vintontech.info
vipps.com.my vipps.com.my
viteonlusarezzo.it viteonlusarezzo.it
vladimir.xrus.org viveresaniesnelli.it
voditeltrezviy.ru vkontarkte.com
volume-pills.biz voloslove.ru
w7s.ru vonradio.com
wallpaperaccess.com vrnelectro.ru
wanker.us wardreapptokone.tk
watsonrealtycorp.com
wdrake.com wdrake.com
webradiology.ru webpromotion.ae
websites-reviews.com whiteproduct.com
whatsupinfoley.com whosonmyserver.com
whos.amung.us wildworld.site
windowssearch-exp.com wingsoffury2.com
wineitudes.wordpress.com woman-orgasm.ru
woodyguthrie.se woman-tampon.ru
word-vorlagen.net wordpress-crew.net
woweb.com.ua world-mmo.com
wrona.it www.kabbalah-red-bracelets.com
www.ehscloud.cn x-lime.com
www.labves.ru xaijo.com
x-true.info xn----9sbebi2bvzr7h.xn--p1ai
xblognetwork.com xn----ctbigni3aj4h.xn--p1ai
xerox-douglas.cf xn--oogle-wmc.com
xfluro.com xxxtubesafari.com
xgftnlrt.bloger.index.hr yoshkarola.zrus.org
xn--80aeahghtf8ac5i.xn--p1ai youporn-ru.com
xn--80ak6aa92e.com z-master.ru
xvideospornoru.com zaimite.ru
xwatt.ru zarabotat-na-sajte.ru
yebocasino.com zdesformula.ru
yourads.website zeroredirect10.com
yoursite.com zhacker.net
youtube-downloader.savetubevideo.com zhongwenlink.com
yuarra.pluto.ro zixizop.net.ru
zarenica.net zmoda.hostreo.com
zebradudka.com zoogdiesney.com
zigarettenonlinekaufen1.bloog.pl
zirondelli.it
zootoplist.com
zx6.ru

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1001desert.com 00webcams.com
214jaluwobapef.cf 1-99seo.com
5tey7463.weebly.com 12bet.com
6128786.com 3weekdiet.com
albumsuper.info 4pp13.com
angel1777.kz 55wmz.ru
animebox.com.ua 7fon.ru
apartmentratings.com 883zy.com
apimossnetinfo-a.akamaihd.net 8xv8.com
app-ready.xyz abwa.tk
aproposde.com adinterax.com
aquarium-pro.ru admeasures.com
atlasvkusov.ru ads.tremorhub.com
autoloans.com ahhjf.com
b2b-lounge.com akvamaster.dp.ua
alert-fjg.xyz
allboard.xobor.de
allstatesugarbowl.org
apartmentbay.ru
appfixing.space
apple.com-webbrowsing-security.science
architecturebest.com
arendakvartir.kz
artel116.ru
ascat.porn
astrochicks.com
aucoinhomes.com
aviapanda.ru
azte.ch
backlinkwatch.com
baikaleminer.com
balois.worldbreak.com balois.worldbreak.com
banan.tv beclean-nn.ru
bannerbank.ru blackcurranthumidifiers.site
berdasovivan.ru
best-businessman.ru
bestwaystogetpaid.us
bif-ru.info
bigpenisguide.com
bitnote.co
blackhatworth.com blackhatworth.com
blogs.rediff.com blogs.rediff.com
bmw-ark.ru bloke.com
bnt-team.com bracketsmackdown.com
board.f00d.de braip.com.br
boltalko.xyz brk-rti.ru
bombla.org bupropion-sr-150-mg.us
book-bets.com buyclomidonlaine.com
bookhome.info cacheimages.com
bradleylive.xyz calc-for-credit.ru
brandov.ru call-of-duty.info
bravegirlsclub.com callejondelpozo.es
breget74.ru camdenmemorials.com
burger-tycoon.com camdolls.com
buttons-for-website.com camsex.xxx-cam.webcam
candlehandmade.com casablancamanor.co.za
cete.ru cheapbarbour.online
cfcl.co.uk cheapmoncler.pw
cfjump.com cheapmoncler.win
classiquebijoux.ru chelyabinsk.xrus.org
clickintext.net chinaelitecheapjerseys.com
clicksor.com chocolateslim-en-espana.com
club-lukojl.ru citizenclsdriveri7.pen.io
coccoc.com collegerentals.com
comeondog.info com-onlinesupport.host
compliance-barak.top compliance-don.xyz
compliance-ivan.xyz compliance-walt.top
compliance-jack.top costablue.xyz
compliance-walter.top cottageofgrace.com
credit-online.ws cyse.tk
creditmoney.com.ua dailyfinancefix.com
creditnation.ru daptravel.com
cv.wallhade.co dekorkeramik.ru
czeshop.info delayreferat.ru
dating-time-now.com destinationrealestate.com
dipstar.org dfiles.me
dividendo.ru download-wallpaper.net
divisioncore.com drivotracker.com
djstools.com e-avon.ru
driving.kiev.ua ecommerce-seo.com
drupa.com educatemetv.com
easync.io ege-essay.ru
el-nation.com ekspertmed.com
elektrischezigarette1.onsugar.com ekspresihati.info
elektroniksigaraankara.info elektrozigarette2.webs.com
elektrozigarette2.wordpressy.pl enbersoft.com
epngo.bz ertelecom.ru
ereko.ru esfchat.tk
ezigarettenkaufen1.pagina.gr etotupo.ru
f1nder.org european-torches.ru
failingmarriege.blogspot.com evrotekhservis.ru
fickenprivat.info extremal-blog.com
film-one.ru fakehandbags.xyz
finansov.info falcoware.com
fireads.men firstdrugmall.ru
flex4launch.ru fok.nl
formulaantiuban.com folowsite.com
foto-telok.net fortevidyoze.net
frbizlist.com freakycheats.com
freenode.info free-share-buttons.com
fsakhalin.ru frustrated-favorable.gq
fuckmill.com futbolkisales.ru
games.kolossale.ru gallerily.com
gay-file.com gdebestkupit.ru
geckoandfly.com ge0ip.com
gepezz.info gepezz.info
girlsfuckdick.com getaclueamerica.com
gpirate.com getlaid-xxxhookupdirect.com
gstatey.net gheus.altervista.org
gtopstats.com glass-msk.ru
guidefs.ru glcomputers.ru
h2monline.com gotorussia.com
hausfrauensex18.com gsasearchenginerankerhelp.com
hchha.com guarrasdelporno.xxx
hdimagegallery.net gungamesz.com
hdseriale.pl headpress.ru
headpharmacy.com hentaiheroes.com
homedecorpicture.us homemature.net
hotloans.ru housemilan.ru
hoverboard360.nl ihc2015.info
ilmen.net imadedinner.net
immobiliaremassaro.com impresagaia.it
infospot.pt istanbulit.com
insta-add.pro jackpotchances.com
int.search.tb.ask.com kamen-e.ru
iplusbit.blogspot.co.za kinoflux.net
ispac.org ku6.com
itservicesthatworkforyou.com lacave.ntic.fr
ivoiretechnocom.ci landscaping.center
jerseychinabizwholesale.com laptoper.net
johannesburgsingles.co.za larger.io
karaganda.xkaz.org lescinq.com
kihi.gdn linkbolic.com
kinohall.ru louboutinreplica.pw
kmd-pto.ru lovasszovetseg.hu
konkursowo-24.pl luchshie-topcasino.ru
kretpicf.pw machicon-ueno.info
kung-fu-ru.com magazintiande.ru
lalalove.ru markjaybeefractal.com
ledpolice.ru masterseek.com
lemon-ade.site masthopehomes.com
lflash.ru mebel-vstroika.ru
limtu.ifmo.ru mebelcomplekt.ru
lineshops.biz micasainvest.com
liumimgx.pw microsoftportal.net
lottospring.com miniature.io
lunamedia.co moinozhki.com
m292.info money-every-day.com
maju.bluesalt.co mrvideospornogratis.xxx
makenahartlin.com muz-tracker.net
mamasuper.prom.ua myhitmp3.club
marmitaco.cat mymercy.info
mbiologi.ru mystats.xyz
mcnamaratech.com narutonaruto.ru
militarysale.pro nationalbreakdown.com
mindbox.co.za netanalytics.xyz
mirobuvi.com.ua nrv.co.za
mobot.site o-o-8-o-o.com
most.gov.iq obiavo.uz
mrwhite.biz ok-ua.info
muabancantho.info old-rock.com
muz-baza.net olympescort.com
mxgetcode.com omgtnc.com
my-aladin.com ons-add.men
my-bc.ru
mydeathspace.com
nailsimg.com
negociosdasha.com
nfljerseyscheapbiz.us
nicefloor.co.uk
noclegonline.info
nodding-passion.tk
nw-servis.ru
obiavo.ru
ofanda.com
offer.wpsecurity.website
offside2.5v.pl
ops.picscout.com
outrageousdeal-a.akamaihd.net
owathemes.com owathemes.com
panicatack.com partner-print.men
partybunny.ru partnersafe.men
perederni.net patterntrader-en.com
perso.wanadoo.es pechikamini.ru
physfunc.ru phormchina.com
piccshare.com photosaga.info
pl.id-forex.com piuminiita.com
polcin.de porn555.com
pornhub-ru.com pornhubforum.tk
pornotubexxx.name pornophoto.xyz
pr0fit-b0x.com portside.cc
preparevideosafesystem4unow.space prchecker.info
privatefx.all4invest.info quick-seeker.com
rank-checker.online quizzitch.net
rating-bestcasino.com raavidesigns.com
razyboard.com realizmobi.com
reelheroes.net repeatlogo.co.uk
regdefense.com rossanasaavedra.net
registrysweeper.com royalvegascasino.com
restorator-msk.ru ru-mediaget.ru
rockprogblog.com sa-live.com
roll123.com search.alot.com
running-line.ru serialsway.ucoz.ru
runstocks.com serptehnika.ru
samara.rosfirm.ru servicecenter.co.ua
sarf3omlat.com sindragosa.comxa.com
saugatuck.com siteexpress.co.il
screpy.com siteripz.net
sdelai-prosto.ru skylta.com
serdcenebolit.com slayerlife.com
servisural.ru snworks.com
sexgalleries.top staynplay.net
sexystrippe.info stop-zavisimost.com
shijian.ac.cn
shmetall.com.ua
shoesonlinebuy.cn
siha.de
sispe.com.br
siteimprove.com
socialbookmarksubmission.org
softonicads.com
solnplast.ru
strangeduckfilms.com strangeduckfilms.com
stuff-about-money.com strv.se
sumo.com suhanpacktech.com
supermodni.com.ua support.nopeas.sk
tacbibirfa.tk swinger-mobil.net
takethatad.com swingerseiten.com
tds-advert002.info tedy.su
thecounter.com telegraf.by
tizanidineandgabapentin.blogspot.com templates.radiodigital.co
tizanidineandhydrocodone.blogspot.com thebestphotos.eu
top-l2.com thepokertimer.com
topflownews.com tizanidinehydrochloride4mgstreetvalue.blogspot.com
topquality.cf tizanidinemusclerelaxant.blogspot.com
trafficshaper.com tmm-kurs.ru
topcar-krasnodar.ru
tour-line.net
toxicwap.com
tracxn.com
traffictrade.life
treasuretrack-a.akamaihd.net treasuretrack-a.akamaihd.net
twitlinks.com tsyndicate.com
tyumen.xrus.org tvteleport.ru
ua-company.ru typer.one
unimodemhalfduplefw.pen.io uhdtv.website
us-america.ru ukr-lit.com
ustion.ru urlcut.ru
vepad.com valsalud.com
vipps.com.my veopornogratis.xxx
vkgaleria.com viktoria-center.ru
voloomoney.com villakohlanta.nu
westen-v.life volume-pills.biz
worst-sites.online vseigru.one
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf vuclip.com
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai vzglyadriv.kg
xn--80aeahghtf8ac5i.xn--p1ai w-journal.ru
xn--d1acah0c.xn--p1ai w3javascript.com
xxxtube69.com wait3sec.org
yellowproxy.net wallpaperstock.net
yoluxuryevents.com warning.or.kr
yoopsie.com webix.me
yoshkarola.zrus.org weightatraining.com
youandcredit.ru wfb.hatedriveapart.com
yougetsignal.com williamrobsonproperty.com
youtubologia.it word-vorlagen.net
ypmuseum.ru www.get-free-traffic-now.com
zagadki.in.ua xjlottery.com
zawyna.ua xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
zvooq.eu xn----7sbaphztdjeboffeiof6c.xn--p1ai
zygophyceous.womanstars.site xn--80aikhbrhr.net
xtremeeagles.net
yourads.website
youtube-downloader.savetubevideo.com
yun56.co
yunque.pluto.ro
yurgorod.ru
zerocash.msk.ru
zeroredirect2.com
znakomstva-moskva77.ru

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.66 0; 10.1.0.8 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3552 ### Version: V4.2023.01.3553
### Updated: Fri Jan 20 22:01:22 UTC 2023 ### Updated: Sat Jan 21 10:03:24 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.01.3552 ### Version: V4.2023.01.3553
### Updated: Fri Jan 20 22:01:22 UTC 2023 ### Updated: Sat Jan 21 10:03:24 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##