vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.02.2891 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-02-01 10:09:47 +00:00
parent 33f0e239a6
commit 147e031567
47 changed files with 85190 additions and 85180 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2890 #### Version: V4.2022.02.2891
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2890 #### Version: V4.2022.02.2891
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.01.2890 #### Version: V4.2022.02.2891
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10512
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.01.2890 #### Version: V4.2022.02.2891
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10516
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10516
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10516
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10516
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

10516
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

10516
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

10516
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2890 2891

10516
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10516
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.168.143.100 0; 20.127.116.168 0;

10516
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

10516
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

10516
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

10516
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.168.143.100 0; 20.127.116.168 0;

10516
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.168.143.100 0; 20.127.116.168 0;

10516
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 1; 10.1.0.196 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
80legs BackWeb
Apexoo HTTP::Lite
Crawling at Home Project PageGrabber
Mediatoolkitbot PyCurl
NetAnts ScreenerBot
SearchmetricsBot ScrepyBot
Seekport Steeler
Siteimprove Tracemyfile
check1.exe Web Fuck
probe-image-size WebmasterWorldForumBot

312
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider
80legs 80legs
ADmantX
ASPSeek
Acunetix Acunetix
AfD-Verbotsverfahren AiHitBot
Alexibot AllSubmitter
AlphaBot Alligator
Anarchy99
Ankit Ankit
Asterias Apexoo
Attach AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike BacklinkCrawler
BackStreet
Backlink-Ceck
Badass Badass
BatchFTP Bandit
BlackWidow Bitacle
Blow BlowFish
Brandwatch Boardreader
Bolt
Buck Buck
BuiltWith
Bullseye Bullseye
CSHttp BuzzSumo
CATExplorador
CazoodleBot CazoodleBot
CheeseBot ChinaClaw
Cliqzbot Cloud mapping
Cocolyzebot Collector
Cogentbot Copier
Copyscape Cosmos
Crawling at Home Project Craftbot
DataCha0s CrazyWebCrawler
DBLBot
DTS Agent
DatabaseDriverMysqli
Demon Demon
Devil
DigitalPebble DigitalPebble
Dirbuster
Disco
Discoverybot
DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DomCopBot
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
Dotbot DomainStatsBot
Download Wonder EMail Wolf
Dragonfly Ebingbong
Drip Ecxi
EasyDL
EirGrabber
EroCrawler
Evil Evil
Express WebPictures Exabot
Extractor
ExtractorPro ExtractorPro
Extreme Picture Finder Extreme Picture Finder
EyeNetIE EyeNetIE
FDM FemtosearchBot
FHscan FlashGet
Foobot
Freeuploader Freeuploader
FrontPage
Fyrebot
GT::WWW GT::WWW
GalaxyBot
Genieo Genieo
GermCrawler
GetWeb GetWeb
Getintent
Gigabot
Go!Zilla Go!Zilla
Go-Ahead-Got-It GrabNet
GoZilla HEADMasterSEO
Grabber HTMLparser
Grafula
GrapeFX
GrapeshotCrawler
HMView
HTTP::Lite HTTP::Lite
HTTrack
Haansoft
Harvest Harvest
HybridBot Havij
IDBot Hloader
HonoluluBot
Iblog Iblog
Id-search Image Sucker
IndeedBot
Indy Library
InfoNaviRobot
Intelliseek Intelliseek
Internet Ninja JOC Web Spider
InternetSeer JamesBOT
Iskanie
IstellaBot
Jbrofuzz
JetCar JetCar
Jetty Jetty
Jorgee JikeSpider
Kenjin Spider Joomla
JustView
Keyword Density Keyword Density
LNSpiderguy LNSpiderguy
Larbin
Leap
LeechFTP LeechFTP
LeechGet LeechGet
LexiBot LexiBot
LibWeb
Libwhisker Libwhisker
LieBaoFast
LinkScan
LinkWalker
LinkextractorPro LinkextractorPro
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot Lipperhey Spider
Lipperhey Litemage_walker
Lmspider Lmspider
MIDown tool Ltx71
MJ12bot MFC_Tear_Sample
Mag-Net MIIxpc
Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic SEO
Majestic12 Majestic12
MegaIndex.ru MarkMonitor
Metauri MarkWatch
Microsoft URL Control MauiBot
Mister PiX Mb2345Browser
MicroMessenger
Microsoft Data Access
Minefield
Mojeek
Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3
Musobot Musobot
NPbot NICErsPRO
Nameprotect Needle
Navroad Net Vampire
Nessus
NetMechanic NetMechanic
NetSpider Netvibes
NextGenSearchBot NextGenSearchBot
Nibbler NimbleCrawler
Not Nutch
Offline Navigator Octopus
OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS
Openfind Openfind
OrangeSpider Openvas
OutfoxBot POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageAnalyzer PageAnalyzer
PageGrabber
PageThing.com PageThing.com
Pavuk Papa Foto
PeoplePal PeoplePal
Pi-Monster Pi-Monster
Picscout Picscout
Picsearch
PictureFinder PictureFinder
Pimonster
PleaseCrawl PleaseCrawl
Pockey Pockey
ProPowerBot
ProWebWalker
Psbot Psbot
Pu_iN PxBroker
Pump PyCurl
QueryN Metasearch QueryN Metasearch
Quick-Crawler
RankActiveLinkBot
RankingBot2 RankingBot2
Rankivabot Rankivabot
ReGet
RealDownload RealDownload
RebelMouse SBIder
RepoMonkey SEOkicks
RocketCrawler SEOlyticsCrawler
SEOprofiler
SEOstats SEOstats
SMTBot Screaming
ScanAlert Searchestate
ScoutJet Semrush
ScreenerBot
SemanticJuice
SemrushBot SemrushBot
SeoSiteCheckup
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
Siteimprove Siteimprove
SlySearch Snake
SmartDownload Snapbot
Snoopy
SocialRankIOBot
Sociscraper
Sottopop
Spammen
Spanner Spanner
Spbot Spinn3r
Sucuri SputnikBot
SuperBot Sqlmap
SuperHTTP Sqlworm
Surfbot Stripper
SurveyBot SurveyBot
T8Abot Swiftbot
Teleport Teleport
TeleportPro TeleportPro
Telesoft Telesoft
Telesphorep Telesphorep
Thumbor Thumbor
Toata
Toweyabot Toweyabot
Trendictionbot Tracemyfile
Turnitin Trendiction
TurnitinBot URLy Warning
Twice Upflow
UnisterBot V-BOT
VB Project VB Project
Vacuum Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
Virusdie VidibleScraper
VoidEYE Voltron
WASALive-Bot WBSearchBot
WEBDAV
WISENutbot
WPScan WPScan
WWW-Mechanize WWW::Mechanize
WWWOFFLE Web Enhancer
Wallpapers/3.0
WeSEE
Web Fetch Web Fetch
Web Fuck WebAuto
Web Sauger
Web Sucker
WebCollage WebCollage
WebCopier WebImageCollector
WebFetch WebLeacher
WebGo IS WebReaper
WebPix WebStripper
WebSauger WebSucker
WebmasterWorldForumBot WebsiteExtractor
Webshag WebsiteQuester
Whatweb Webster
Who.is Bot Whacker
Widow Wonderbot
WiseGuys Robot
Wotbox Wotbox
Wprecon Xaldon WebSpider
Xenu
YoudaoBot YoudaoBot
Zauba Zauba
ZmEu ZmEu
ZoominfoBot ZyBorg
ZumBot arquivo.pt
adscanner autoemailspider
archive.org_bot
clark-crawler clark-crawler
coccocbot-web coccocbot-web
cognitiveseo
crawl.sogou.com crawl.sogou.com
domainsproject.org crawler.feedback
facebookscraper dataforseo.com
eCatch
evc-batch
gopher gopher
heritrix instabid
lwp-request ips-agent
lwp-trivial iubenda-radar
magpie-crawler
meanpathbot meanpathbot
muhstik-scan
pcBrowser pcBrowser
polaris version
probe-image-size
ripz ripz
seocompany.store s1z.ru
seostar scan.lol
seobility
serpstatbot
sexsearcher sexsearcher
sp_auditbot sp_auditbot
spyfu sysscan
tAkeOut
trendiction.de
voyagerx.com voyagerx.com
webpros.com webprosbot
x09Mozilla x09Mozilla
x22Mozilla
xpymep1.exe xpymep1.exe
zauba.io zauba.io
zgrab

298
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 01h4x.com
404enemy 404enemy
80legs 80legs
ADmantX
ALittle Client ALittle Client
Acunetix Aboundex
AhrefsBot AiHitBot
AllSubmitter Alexibot
Anarchie Anarchie
Attach Ankit
AwarioRssBot Apexoo
BBBike Aspiegel
Atomseobot
BDCbot BDCbot
BackStreet BDFetch
BackWeb BLEXBot
Backlink-Ceck Backlink-Ceck
BacklinkCrawler Barkrowler
Bandit
BatchFTP
Black Hole Black Hole
Blackboard Blackboard
BotALot BlowFish
Brandwatch BuiltBotTough
Buddy
Bullseye Bullseye
BunnySlippers CATExplorador
BuzzSumo CODE87
CSHttp CSHttp
Calculon CazoodleBot
Cegbfeieh Cegbfeieh
CheTeam CheTeam
CheeseBot
CherryPicker CherryPicker
Chlooe ChinaClaw
Cogentbot Citoid
Collector Cocolyzebot
CopyRightCheck Copyscape
Cosmos Cosmos
CrazyWebCrawler
CrunchBot CrunchBot
Curious DIIbot
DTS Agent DSearch
Demon
Deusu Deusu
Devil
Digincore
Dirbuster Dirbuster
Discobot Discobot
DittoSpyder Discoverybot
Domains Project Domains Project
Drip Dotbot
Ebingbong Download Wonder
EirGrabber Dragonfly
EroCrawler Express WebPictures
Exabot Extreme Picture Finder
Extractor
ExtractorPro
Ezooms
FDM
FemtosearchBot
Firefox/7.0
Fuzz Fuzz
FyberSpider
Fyrebot
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW
GermCrawler GermCrawler
GetWeb
Gigabot
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla GoZilla
GrabNet Grabber
GrapeFX
HEADMasterSEO
HTMLparser HTMLparser
HTTP::Lite HTTP::Lite
HTTrack HTTrack
Haansoft Harvest
HaosouSpider
Heritrix
Hloader
Humanlinks Humanlinks
HybridBot
IDBTE4M IDBTE4M
IRLbot Iblog
Id-search
IlseBot IlseBot
Indy Library Image Sucker
InfoNaviRobot InfoNaviRobot
InfoTekies
InterGET InterGET
Internet Ninja Internet Ninja
InternetSeer
Iskanie Iskanie
JOC Web Spider IstellaBot
JamesBOT JennyBot
Jbrofuzz JikeSpider
JetCar
Joomla Joomla
JustView JustView
Jyxobot Jyxobot
Kenjin Spider
Kinza Kinza
LWP::Simple
LexiBot LexiBot
Lftp Lftp
LibWeb LibWeb
Lightspeedsystems Libwhisker
LieBaoFast
Likse Likse
LinkScan
LinkWalker LinkWalker
LinkpadBot Lipperhey Spider
LinqiaRSSBot Litemage_walker
Lmspider Lmspider
MFC_Tear_Sample MJ12bot
MIDown tool MSFrontPage
MIIxpc MTRobot
Mag-Net Mail.RU_Bot
Magnet MarkWatch
MarkMonitor Masscan
MauiBot Mata Hari
Mb2345Browser Mb2345Browser
MeanPath Bot Meanpathbot
Metauri Metauri
MicroMessenger MicroMessenger
Microsoft URL Control Microsoft Data Access
Minefield Msrabot
Moblie Safari Musobot
Mojeek NPbot
Mojolicious Name Intelligence
MolokaiBot Navroad
Mr.4x3 Net Vampire
NICErsPRO
NearSite
NetLyzer NetLyzer
NetMechanic NetZIP
Nettrack NextGenSearchBot
Netvibes Nibbler
Nikto Nikto
Nimbostratus
Ninja Ninja
Octopus Nmap
Nutch
Offline Explorer
Offline Navigator
OnCrawl
OpenVAS OpenVAS
Openvas
OrangeBot OrangeBot
PageScorer OrangeSpider
PageThing.com PECL::HTTP
Pandalytics PHPCrawl
Panscient POE-Component-Client-HTTP
Page Analyzer
PageAnalyzer
PageGrabber
Papa Foto
PeoplePal PeoplePal
Petalbot
Pi-Monster Pi-Monster
Picscout Picsearch
Pimonster PictureFinder
Piepmatz
Pockey Pockey
Psbot ProPowerBot
Pu_iN Probethenet
PxBroker
PyCurl
Quick-Crawler Quick-Crawler
RankActive RSSingBot
RankingBot RankFlex
RankingBot2 RankurBot
Re-re
ReGet ReGet
RebelMouse RebelMouse
RedesScrapy RepoMonkey
RocketCrawler RocketCrawler
SBIder SBIder
SEOkicks SEOkicks
SEOprofiler SEOkicks-Robot
SEOstats SEOstats
SISTRIX SISTRIX
ScanAlert
ScoutJet ScoutJet
Screaming
ScreenerBot ScreenerBot
ScrepyBot
Searchestate
Seekport
Semrush
SemrushBot SemrushBot
Shodan Shodan
Siphon Siphon
SiteCheckerBotCrawler SiteExplorer
SiteLockSpider SiteLockSpider
SiteSucker SiteSucker
Sitevigil Sitebeam
SocialRankIOBot Siteimprove
Sogou web spider Snoopy
Spanner Sosospider
Spbot
Spinn3r Spinn3r
Sqlworm Sqlmap
Sqworm Stripper
Sucker
Sucuri
SuperBot
SuperHTTP SuperHTTP
Suzuran Suzuran
Swiftbot Swiftbot
T8Abot Szukacz
TeleportPro T0PHackTeam
The Intraformant Teleport
TheNomad Telesphorep
Titan Titan
Toata Turingos
Trendiction
True_Robot
Turnitin Turnitin
TurnitinBot
TwengaBot
Twice Twice
V-BOT V-BOT
Vacuum VelenPublicWebCrawler
Vagabondo VeriCiteCrawler
Virusdie VidibleScraper
Voltron Voltron
WBSearchBot WASALive-Bot
WWW-Mechanize
Wallpapers Wallpapers
WallpapersHD Web Collage
WeSEE
Web Pix Web Pix
WebAuto Web Sauger
WebCollage WebCollage
WebEnhancer WebCopier
WebFetch WebFetch
WebImageCollector
WebReaper WebReaper
WebSauger WebSauger
WebmasterWorldForumBot WebStripper
Website Quester Website Quester
Whack
Whatweb Whatweb
Widow Who.is Bot
WinHTTrack WinHTTrack
Wonderbot Woobot
Xaldon_WebSpider Wotbox
Xenu Wprecon
Zade Zade
Zeus ZmEu
Zitebot
ZoomBot
ZoominfoBot
ZyBorg ZyBorg
adscanner arquivo.pt
archive.org_bot autoemailspider
arquivo-web-crawler
check1.exe check1.exe
coccocbot-web coccocbot-web
cognitiveseo cognitiveseo
com.plumanalytics com.plumanalytics
dataforseo.com crawl.sogou.com
crawler4j
domainsproject.org domainsproject.org
eCatch eCatch
evc-batch gopher
facebookscraper heritrix
instabid instabid
ips-agent
iubenda-radar iubenda-radar
meanpathbot
oBot
pcBrowser pcBrowser
plumanalytics
polaris version
probe-image-size probe-image-size
ripz s1z.ru
scan.lol satoristudio.net
sexsearcher seobility
sogouspider spyfu
sysscan
tAkeOut tAkeOut
trendiction.com ubermetrics-technologies.com
trendiction.de voyagerx.com
webgains-bot webpros.com
webprosbot
x22Mozilla
zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
3xforum.ro apimossnetinfo-a.akamaihd.net
bear.gotcher.us autogrep.ru
hdhc.ru best-seo-software.xyz
kaac.ru nfljerseysforsalewholesaler.com
mebel-ekb.com piski.top
netanalytics.xyz questionmarque.ch
predmety.in.ua visa-china.ru
psoriasis-file.trade wufak.com
seocheki.net xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
stuff-about-money.com xrus.org

920
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
101raccoon.ru 0daymusic.org
1stat.ru 1688.com
28n2gl3wfyb0.ru 20pascals.nl
2itech.ru 24-ak.ru
2x2fan.ru 2ads.co.uk
2daytrendingnews.com
2girls1cup-free.com
3xforum.ro 3xforum.ro
4kplayer.pl 4inn.ru
4webmasters.com 7search.com
5-steps-to-start-business.com 8si.ru
5000-cotydzien.com a14download.com
5elementov.ru aarbur.com
7makemoneyonline.com acortarurl.es
991mostfm.co.id acunetix-referrer.com
abbanreddy.com adclickthru.net
abcguru.xyz admitad.com
abiente.ru adservme.com
absolutelyanalog.com adsloads.com
actices.com adsref.men
advancedcleaner.com ahhjf.com
advocatemsk.ru
afmuseum.com
air-edem.ru
aktobe.xkaz.org
alborzan.com alborzan.com
alert-fjg.xyz alert-hgd.xyz
alfa-img.com alienwheel.es
allblogroll.com alive-ua.com
alldezservice.kz all4invest.ru
alltheviews.com allwidewallpapers.com
amateurgalls.com aludecor.info
am15.net
amehdaily.com amehdaily.com
amospalla.es anaksma.info
android-style.com anaseracresar.tk
android-vsem.org andpolice.com
angel1777.kz angry-fermi-7633.arukascloud.io
animal-drawings.com appiq.mobi
apibetweenlinesn-a.akamaihd.net appsaurus.com
apiholdingmypage-a.akamaihd.net arenda-avtoprokat-krasnodar.ru
appfastplay.com arenda-yeisk.ru
apptester.tk artavenuegardenstudios.com
arcteryxsale.online
arquivo.pt
asmxsatadriverin.aircus.com asmxsatadriverin.aircus.com
asophoto.com autolombard-krasnodar.ru
autobrennero.it automobile-spec.com
avtolombard-krasnodar.com autoseo-traffic.com
ayodhya.co availit.weebly.com
azartniy-bonus.com avirasecureserver.com
azinoofficial777.ru avto-oligarh.ru
bankofthewext.com axbocz.net
bannerpower.com azartclub.org
bashtime.ru babespcs.com
bbtec.net bablonow.ru
bdsmgalls.net babosas.com
beachpics.com balans.shahterworld.org
best-way.men balitouroffice.com
bestbuy.ca balkanfarma.ru
betterhdporn.com balla-boo.se
bitporno.sx banan.tv
blog.koorg.ru bankiem.pl
bloggen.be beauty-clinic.ru
bobba.dzaba.com bestserials.com
bosman.pluto.ro biketank.ga
bradleylive.xyz bingo8888.com
bio-market.kz
blacktwhite.com
blog.remote-computer.de
bloke.com
boeuklad.com
bonzibuddi.com
braip.com.br braip.com.br
brandov.ru brandov.ru
browsepulse-a.akamaihd.net brewdom.ru
bugshoot.cn brothers-smaller.ru
buketeg.xyz buketeg.xyz
bum.com.ru buyessay3.blogspot.ru
buntube.net bvlgariring.xyz
buyantiviralwp.com byme.se
buybest1.biz
buyessayonline19.blogspot.ru
buzzsumo.com
cacheimages.com cacheimages.com
calcularpagerank.com cakesplus.com.au
callstevens.com callawaygolfoutlet.online
candelluxsklep.pl capsderfudd.tk
canoncdriverq3.pen.io cartechnic.ru
capture-room.com cartierlovebracelet.xyz
carabela.com.do cartierreplica.top
carsplate.com cayado.snn.gr
ccjp.eu cdn.walkme.com
cete.ru cdncash.net
cfsrating.sonicwall.com celebsopics.com
chlooe.com cenokos.ru
christianlouboutinreplica.pw chatroulette.si
ci.ua cheapsergiorossi.online
cigarpass.com cialischsrx.com
cityadspix.com cialisndbrx.com
clevernt.com cleanersoft.com
click2pawn.com clickhype.com
clipartnew.com clubfashionista.com
coaching-netz.info collegerentals.com
cognitiveseo.com community.allhiphop.com
com-onlinesupport.host
com-supportcenter.website
commoncrawl.org
compiko.info compiko.info
compliance-alex.top compliance-alex.xyz
compliance-andrew.xyz compliance-barak.xyz
compliance-checker.info compliance-irvin.xyz
compliance-ivan.top compliance-jessica.top
compliance-jane.top contentsexpress.com
compliance-mary.top converse.ddsoldes.fr
cookie-law-enforcement-ff.xyz cookie-law-enforcement-aa.xyz
cosmeticswomens-womensports.rhcloud.com copyrightclaims.org
counter.yadro.ru counciltally.com
country-chic.ru
covid-schutzmasken.de
crd.clan.su
cryptoswap.biz cryptoswap.biz
custom-product-labels.com cs-passion.pl
cyprusbuyproperties.com csgo4.win
cyse.tk customerpromos-a.akamaihd.net
d2jsp.org cyclobenzaprinevstizanidine.blogspot.com
daretodonate.co czat.wp.pl
darkbooks.org dairyindia.in
datadepths.com daisye.top
db.speedup.gdn dangphoto.trade
decenttools.com daooda.com
designdevise.com dating-time-now.com
destinationrealestate.com ddpills.com
dgfitness.co dekoration.us
diegolopezcastan.com delitime.info
digilander.libero.it dentfidemountpreach.tk
digitalassetmanagement.site deutschehobbyhuren.net
dktr.ru dev33.dioniqlabb.se
dnepropetrovsk.mistr-x.org dieta-personalna.pl
domik-derevne.ru discountbarbour.online
doreenblog.online discovertreasurenow.com
drhomes.biz dispo.de
driving.kiev.ua djekxa.ru
drivotracker.com dolohen.com
dscaas.website domaincdn.xyz
dvr.biz.ua domainsphoto.com
e-biznes.info drpornogratisx.xxx
e-commerce-seo1.com dvrlists.com
eandsgallery.com ebooknovel.club
easyshoppermac.com ec-file.info
ege-essay.ru ecxtracking.com
el-nation.com editmedios.com
electro-prom.com editors.choice6912650.hulfingtonpost.com
elektrischezigaretten2.beeplog.com elektronischezi.livejournal.com
elektrozigaretten1.tumblr.com elektrozigaretten1.webs.com
empis.magix.net elvenar.com
envaseslotusama.com energydiet24.ru
erotikstories.ru esnm.ru
estibot.com
etur.ru
eurosamodelki.ru eurosamodelki.ru
ewebarticle.info evaashop.ru
exchangeit.gq
excitacao.com excitacao.com
exoclick.com
exoneration-project.us
expertblog.info expertblog.info
expresstoplivo.ru feel-planet.com
familienzahnaerzte.com fidelityfunding.com
fashion-mk.net filmfanatic.com
fatfasts-4tmz.com filmidivx.com
favornews.com
fealq.com
fearcrow.com
feargames.ru
fertilitetsradgivningen.se
filmi-v.online
financepoints.eu financepoints.eu
finuse.com fireads.men
firma-legion.ru fitnesspiks.com
flavors.me flashbannernow.com
focalink.com forexunion.net
fordsonmajbor.cf forpostlock.ru
forex21.ru free-video-tool.com
formularz-konkurs.tk freemags.cc
forodvd.com freemaintenancesysforpcandmac.top
freakycheats.com freewebs.com
free-stock-illustration.com freshwallpapers.info
free-traffic.xyz gambarkataku.co
freecamdollars.com gameplexcity.com
freelotto.com gamevalue7.weebly.com
freesitetest.com generalporn.org
freewlan.info getyourimage.club
frighteningremain.cf gigapeta.com
frivgame250.com given2.com
fun-mobi.pl gloverid.site
fun2cell.net gnuetella.com
fungamelands.com goodly.pro
funtoonez.com googlsucks.com
fyxabomiw.ru gosreg.amchs.ru
game-top.su gq-catalog.gq
gaypornmovie.net
gcup.ru
gearcraft.us
getlamborghini.ga
getridofstretchmarks.org
gettpromos.com
ghazel.ru
ghostvisitor.com
globalscam.ga
glopages.ru
goldpanningtools.com
googlepositions.com
gosmeb.ru
greatzip.com greatzip.com
gribkovye-zabolevaniya.com
grizzlysgrill.com
gta-top.ru
gunsvicceadadebt.tk
hahashka.ru
harmonyglen.us
hatedriveapart.com hatedriveapart.com
hccoder.info headpharmacy.com
hebr.myddns-flir.com
heliko.no heliko.no
help.tpu.ru hifidesign.ru
himazin.info high-speed1.net
histock.info histock.info
hitcpm.com
hoholikik.club hoholikik.club
homeart.space home-task.com
hoporno.com homedo.fabpage.com
hotblog.top honyaku.yahoofs.jp
hotel-mkad.ru hotsocialz.com
hothot.ru hoverboard360.de
hotkeys.com hoverboard360.es
hotloans.ru hoztorg-opt.ru
housediz.com hptwaakw.blog.fc2.com
housemilan.ru iflycapetown.co.za
hully.altervista.org ihtec2019.org
ifmo.ru imagefinder.site
igru-xbox.net imagine-ex.co
ilbe.club in-tandem.co
imabase.com innodgfdriverhm.aircus.com
impotentik.com investpamm.ru
infogame.name ip-guide.com
insta-add.pro
interferencer.ru
intervsem.ru
invitefashion.com
invivo.hu
iphantom.com iphantom.com
irkutsk.zrus.org iplogger.org
itrevolution.cf iqoption.pro
iusstf.org iron-age.info
jagg.info istripper.com
javatex.co.id itbc.kiev.ua
javstock.com izhevsk.xrus.org
jobmarket.com.ua izhevsk.zrus.org
johannesburgsingles.co.za izoll.ru
jyrxd.com javlibrary.cc
kamagragelusa.net javspace.net
kaz.kz javxxx18.com
khadastoafarde.tk karting196.ru
kino-rating.ru kemerovo.zrus.org
kino-rf.ru keywordsuggests.com
kol-energo.ru kiwe-analytics.com
kollesa.ru knowyournextmove.com
konteiner24.com koversite.info
kumuk.info krasnodar.ru
kupit-adenu.ru laparfumotec.com
landscaping.center larose.jb2c.me
lexiangwan.com law-check-two.xyz
lincolntheatre.com le-clos-des-alouettes.com
legionalpha.com
letsart.ru
letslowbefast.site
letto.by
levaquin750mg.blogspot.com
linerdrilling.com
linkdebrideur.xyz linkdebrideur.xyz
lion.playtap.us livefixer.com
liquimondo.com lizace.com
lkbennettstore.online localmatchbook.com
localflirtbuddies.com lonely-mature.com
long-beach-air-conditioning.com
louboutinbooties.xyz louboutinbooties.xyz
louboutinshoes.xyz
lsex.xyz
lubetube.com lubetube.com
luchshie-topcasino.ru mafa-free.com
lutherstable.org maju.bluesalt.co
macdamaged.tech makemoneyonline.com
mackeeper-center.club mapquestz.us
macnewtech.com marcowebonyodziezowe.pl
mailemedicinals.com marketland.ml
manifestyourmillion.com masterseek.com
manyresultshub-a.akamaihd.net matb3aa.com
maranbrinfo.com.br me-ke.com
marblestyle.ru medicine-4u.org
marwer.info megavolt.net.ua
mastershef.club metascephe.com
mb140.ru minneapoliscopiers.com
mcadamssupplyco.com mmgq.ru
mebel-vstroika.ru mncrftpcs.com
memberty.com moje-recenze.cz
mexicosleevegastrectomy.com monetizationking.net
mikrobiologies.ru money-for-placing-articles.com
militarysale.pro moneytop.ru
millionare.com monsterdivx.tv
mmostrike.ru montazhnic.ru
modnie-futbolki.net mosrif.ru
moesexy.com most.gov.iq
moetomnenie.com mrbojikobi4.biz
moivestiy.biz mrsdalloways.com
monclerboots.xyz muycerdas.xxx
moncleroutletonline.pw myfreemp3.eu
monthlywinners.com mygameplus.com
moonci.ru
moroccosurfadventures.com
moscow.online-podarki.com
moskva.nodup.ru
mtmtv.info
my-bc.ru
mydirtystuff.com
myonlinepayday.co
mypornfree.ru
nac-bearings.ru
nalogovyykodeks.ru nalogovyykodeks.ru
nastydollars.com narkologiya-voronezh.ru
netoil.no neks.info
news-readers.ru networkcheck.xyz
newstudio.tv new-apps.ru
nikkiewart.ru newyorkhotelsmotels.info
nina.az nfljerseyscheapchinabiz.com
northfacestore.online njkmznnb.ru
nudo.ca njpalletremoval.com
noclegonline.info
nrjmobile.fr
nw-servis.ru nw-servis.ru
o-dachnik.ru nzfilecloud.weebly.com
o.light.d0t.ru o333o.com
obesidadealgarve.com oballergiya.ru
obiavo.su odywpjtw.bloger.index.hr
okeinfo.online of-ireland.info
onescreen.cc onemactrckr.com
openmultipleurl.com onlinemegax.com
oren-cats.ru onlinewritingjobs17.blogspot.ru
osagonline.ru owathemes.com
osoznanie-narkotikam.net p-business.ru
oz-offers.com painting-planet.com
paccohichetoti.ml pandarastore.top
pagesense.com partner-cdn.men
partner-print.men partner-pop.men
pdns.download pcboa.se
perm-profnastil.ru pensplan4u.com
phormchina.com perederni.net
photostudiolightings.com perosan.com
phrcialiled.com pewit.pw
pijoto.net photorepair.ru
pixshark.com picsearch.com
pkr1hand.com picurams.pw
platesauto.com pinwallpaper.top
plohaya-kreditnaya-istoriya.ru pirateday.ru
popup-fdm.xyz pizdopletka.club
pornhub-forum.ga pizza-imperia.com
porno-raskazy.ru platezhka.net
pornobrazzers.biz playlott.com
pornoelita.info pobeiranie.pl
pornokajf.com polimga.pw
pornophoto.xyz pomoc-drogowa.cba.pl
pornpost.in pons-presse.com
porodasobak.net pontiacsolstice.info
portadd.men pooleroadmedicalcentre.co.uk
pozdrawleniya.com popmarker.com
privacyassistant.net popup.matchmaker.com
privatamateure.com porndairy.in
prod2016.com porno-dojki.net
promalp-universal.ru porno-video-chati.ru
pornohub.me
pornoklad.net
pornoklad.ru
pornotubs.com
portal-eu.ru
portside.cc
portside.xyz
poshiv-chehol.ru
preconnubial.usuby.site
privacylocationforloc.com
pronekut.com
prospekt-st.ru
prosperent.com
prostitutki-almata.org
psa48.ru
pskcijdc.bloger.index.hr
pssucai.info
psvita.ru
pussyspace.net pussyspace.net
puteshestvennik.com qitt.ru
py100.ru queerspace.com
pyrodesigns.com.au
questionmarque.ch questionmarque.ch
quizzitch.net quick-seeker.com
r-e-f-e-r-e-r.com qwertty.net
radiodigital.co
radiogambling.com
ragecash.com ragecash.com
rczhan.com rank-checker.online
realnye-otzyvy.info realresultslist.com
registry-clean-up.net reckonstat.info
remorcicomerciale.ro reining.lovasszovetseg.hu
reward-survey.net remontgruzovik.ru
rfid-locker.co remote-dba.de
renewablewealth.com
restaurantlescampi.com
richinvestmonitor.com richinvestmonitor.com
roll123.com rustic-quiver.win
royalads.net sabaapress.com
rubbed.us safe-app.net
runtnc.net sc-specialhost.com
s8-nowy-wygraj.comli.com scanner-jane.top
sack.net scanner-walter.top
sammlungfotos.online scenarii-1-sentyabrya.uroki.org.ua
satoristudio.net school-diplomat.ru
savetubevideo.com se-welding.ru
sbornik-zakonov.ru seeresultshub-a.akamaihd.net
scanner-fred.top seo-traffic-ranking.info
scanner-margo.top seoholding.com
schlampen-treffen.com seolab.top
scoopquest.com seorankinglinks.us
search-error.com sex-tracker.com
searchimage.co sexflirtbook.com
seminarykansas904.ml sexiporno.net
seoheap.com sexvporno.ru
sevendays.com.ua shiksabd.com
sexcamamateurchat.com shohanb.com
sexreliz.com shop-electron.ru
sexspornotub.com shymkent.xkaz.org
sexyteens.hol.es sims-sims.ru
shop.acim.org simul.co
sibvitr.ru sinestesia.host.sk
site3.free-share-buttons.com sisiynas.ru
sitopreferito.it sitechecker.pro
skachat-besplatno-obrazcy.ru
skuteczna-dieta.co.pl skuteczna-dieta.co.pl
slavia.info smartadserver.com
slimcdn.com smichovbike.cz
snow.nvr163.com snegozaderzhatel.ru
socas.pluto.ro soaksoak.ru
socialbookmarksubmission.org
socialseet.ru socialseet.ru
soft-terminal.ru
softonicads.com
softxaker.ru
solinf.co
solmarket.by
soundfrost.org soundfrost.org
soviet-portal.do.am speedup-my.site
space4updating.win spitfiremusic.com
spasswelt.net sptslmtrafms.com
splendorsearch-a.akamaihd.net stair.registrydomainservices.com
spy-sts.com statistici.ro
startwp.org stjamesschool.info
staynplay.net superkanpo.com
steame.ru support.nopeas.sk
steelmaster.lv susanholtphotography.com
stefanbakosab.se syndicate.fun
stoki.ru takeflyte.com
stroiminsk.com tattooha.com
stroymonolit.su taximytishi.ru
stuff-about-money.com
sundrugstore.com
sunflowerdrawingpaintings.blogspot.com
superiends.org
superinterstitial.com
surintech.ac.th
svtrd.com
tabletkinaodchudzanie.com.pl
tds-advert005.info tds-advert005.info
tecnoteakviareggio.it tdsing.ru
telefonsexsofort.tk
texbaza.by texbaza.by
thebitcoincode.com theguardlan.com
thecoral.com.br timecrimea.ru
tizanidineinfo.blogspot.com timer4web.com
tizanidinestreetvalue.blogspot.com tizanidine4mgstreetprice.blogspot.com
tmm-kurs.ru tizanidinedruginteractions.blogspot.com
tn811.us tizanidineduringpregnancy.blogspot.com
top-study.work tizanidinehcl4mgtab.blogspot.com
top10-way.com tizanidineoral.blogspot.com
toys.erolove.in tizanidinesideeffectsweightloss.blogspot.com
track112.site tizanidinevssoma.blogspot.com
trackmedia101.com tizanidinevsvalium.blogspot.com
trafficfactory.biz toondinsey.com
trafficjunky.net top-karkas.ru
trafficmonetize.org topads.men
u555u.info topanasex.com
topflownews.com
totu.us
traffictrade.life
triplepanda.xyz
tt-ipd.info
ttrraacckkrr.com
tversvet.ru
twojebook.pl
twu.com.ua
uggsale.online
ugogo.info ugogo.info
ukrtextbook.com uk-zheu20.ru
ultimatesetnewfreeallsoftupgradesystems.pw ukkala.xyz
umg-stroy.ru unmaroll.ya.ru
uroffer.link uptime-delta.net
v720hd.ru urengoy.pro
vadimkravtcov.ru urlopener.blogspot.com.au
valsalud.com usacasino.com
velobikestock.com utiblog.fr
veopornogratis.xxx valmetrundan.se
videochat.tv.br vasileostrovsky-rayon.ru
videooko.weebly.com vbtracker.net
visa-china.ru vegan-foods.us
vseigry.fun video-woman.com
vykupavto-krasnodar.ru viel.su
vysigy.su vikistars.com
wallpaperaccess.com viktoria-center.ru
wapsite.me vkgaleria.com
webcamdevochka.com vladimir.xrus.org
weclipart.com vltai.com
westermarkanjou.se vozbujdenie.com
wholesalejerseys-cheapest.com vse-pesni.com
wildcattube.com weblo.com
wildworld.site webnode.me
woman-h.ru website-stealer.nufaq.com
woman-orgasm.ru westvilletowingservices.co.za
worldhistory.biz whatsupinfoley.com
wrz0iuebwhp5fg.freeddns.com whereiskentoday.com
www.arenda-yeisk.ru wingsofrefuge.net
www.timer4web.com wiosenny-bon-1500.pl
x-diesel.com withstandingheartwarming.com
xmladserver.com wma-x.com
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai workle.website
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai wygraj-skiny.win
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf x5market.ru
xn----btbdvdh4aafrfciljm6k.xn--p1ai xingzi-vision.com
xlovecam.com
xn----itbkqkfiq.xn--p1ai
xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn--d1abj0abs9d.in.ua xn--d1abj0abs9d.in.ua
xtubeporno.net xn--e1aggki3c.xn--80adxhks
xn--h1aakne2ba.xn--p1ai
xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
xvideosbay.com xvideosbay.com
xwatt.ru xwatt.ru
xxxtubesafari.com xxxtube69.com
yellowstonesafaritours.com yellocloud.be
youbloodyripper.com yoopsie.com
youbrainboost.asia youjizz.vc
yoursearch.me yourporngay.com
yourtemplatefinder.com youtubologia.it
ytmnd.com zaapplesales.blogspot.com
yugo-star.ru zhuravlev.info
yur-p.ru
yuweng.info
zapiszto.pl
zeroredirect5.com
zigarettenonlinekaufen.tumblr.com zigarettenonlinekaufen.tumblr.com
zigarettenonlinekaufen2.bloog.pl
zigarettenonlinekaufen2.drupalgardens.com
zmoda.hostreo.com zmoda.hostreo.com
zverokruh-shop.cz znaniyapolza.ru
zvetki.ru zona-aqua.ru
zscalerone.net
zvuker.net

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
123locker.com 101raccoon.ru
18ps.ru
24videos.tv
2nt.ru 2nt.ru
991mostfm.co.id 8558l.jobs.net
aa81bf391151884adfa3dd677e41f94be1.com abctoppictures.net
adconscious.com adhome.biz
adserver-e7.com adrunnr.com
adsref.men alessandraleone.com
adtech.de alibestsale.com
adverster.com all-number.com
adzpower.com allboard.xobor.de
agysya.ru allstatesugarbowl.org
alborzan.com amazingninja.com
alfa9.com antons-transporte.de
allblogroll.com apple.com-webbrowsing-security.science
aludecor.info arendakvartir.kz
animalrank.com arquapetrarca.info
appartement-stumm.at artefakct.com
apxeo.info audiofree.ru
arendatora.ru autochoixspinelli.com
artavenuegardenstudios.com aversis.net
astana.xxxkaz.org best-seo-software.xyz
autowebmarket.com.ua bget.ru
avon-ukraine.com bildsuche.ru
azbukadiets.ru binomo.kz
bannerads.de bjgugu.net.ua
barbourjackets.online blackle.com
batanga.net blog-hits.com
belsetirehafi.tk blogporn.in
bestempresas.es braintobe.win
betterscooter.com brandov.ru
betwinservice.com bristolhotel.com
beztuberkuleza.ru
biglistofwebsites.com
blackbabesporn.com
blagovest-med.ru
blogos.kz
bonzibuddi.com
bosefux.esy.es
brk-rti.ru brk-rti.ru
brus-vsem.ru call-of-duty.info
bscodecs.com camdenmemorials.com
burkesales.com charmstroy.info
cah.io.community cheapmoncler.xyz
cannibalcheerleader.com cialischstgerts.com
canoncdriverq3.pen.io cicaki.net
cardsharp1.ru classiquebijoux.ru
cartierbracelet.xyz clean-virus-mac.com
cartierlovestore.xyz cleanallvirus.com
ccbill.com click2pawn.com
cheapcigarettesc.info clickprotects.com
checkm8.com companies-catalog.com
chidporn.com compliance-andrew.top
colobit.biz compliance-george.top
compliance-irvin.xyz
compliance-olga.top compliance-olga.top
conboy.us cookie-law-enforcement-aa.xyz
costablue.xyz cookie-law-enforcement-hh.xyz
countbertwistdisp26.soup.io custom-product-labels.com
cpabegins.ru daptravel.com
craftburg.ru daretodonate.co
credit.co.ua ddpills.com
danceuniverse.ru dolohen.com
dangphoto.trade domik-derevne.ru
dating-app.ru downloads-whatsapp.com
diesel-parts28.ru dptaughtme.com
domainroam.win dtnlyss.com
dorratex.tn educontest.net
download-of-the-warez.blogspot.com eropornosex.ru
downloadeer.net erotik0049.com
dscaas.website exoneration-project.us
dynainbox.com ezigarettekaufen.myblog.de
dyt.net fatmaelgarny.com
e-commerce-seo1.com feeriaclub.ru
ee77ee.com femdom.twiclub.in
efkt.jp fickenprivat.info
eleimgo.pw fidelityfunding.com
elektronischezigarettekaufen1.myblog.de financeloan.us
elektrozigarette2.webs.com findacheaplawyers.com
elitepcgames.com firstdrugmall.ru
elvenmachine.com fiuxy.com
eredijovon.com forodvd.com
everflora.ru foto-basa.com
exoclick.com frbizlist.com
ezigarettenkaufen1.deviantart.com freewlan.info
f-online.de fyl.com.ru
familyphysician.ru fym.com.ru
fantasticpixcool.com gael-s.ru
fashionavenuegame.com garciniaxt.us
fetishinside.com geckoandfly.com
figuringmoneyout.com generalporn.org
filerockstar298.weebly.com ghazel.ru
filesvine.com golden-praga.ru
findpics.pw gox.com.ua
fireads.men gratis-sexkontakte.com
fotzen-ficken.com gsasearchengineranker.top
free-gluten.ru guardlink.com
free-social-buttons.xyz hatedriveapart.com
front.ru heroz.fr
gazporno.com hjaoopoa.top
gigapeta.com homeandhealth.ru
goforexvps.com hosting-tracker.com
golmau.host.sk hoverboard360.at
greenshop.su hoverboardforsaledirect.com
happy.new.yeartwit.com hulfingtonpost.com
hd720kino.ru hunterboots.online
hdimagegallery.net husky-shop.cz
healgastro.com ibmdatamanagement.co
himgaws.pw ideoworld.org
hobbyhuren-datenbank.com iinstalll-fii1leis.jus0wil.pp.ua
humanorightswatch.org imabase.com
iboss.com inet-traffic.com
igtools.club infokonkurs.ru
ihc2015.info irunfar.com
imperia31.ru izoll.ru
infoupdate.org jerseychinabizwholesale.com
j-times.ru jerseyssportsshop.com
kerwinandcariza.com kerch.site
kinogolos.ru kinohit1.ru
komputernaya-pomosh-moscow.ru kiskinhouse.com
krasnodar-avtolombards.ru komputers-best.ru
krasnodar.xrus.org konturkrasoty.ru
kredit-pod-zalog-krasnodar.ru koronirealestate.gr
kutikomi.net labplus.ru
ladsblue.com lifebyleese.com
lampokrat.ws lirunet.ru
leadwayau.com magiadinamibia.blogspot.com
likrot.com magicplayer-s.acestream.net
lion.conboy.us mainlinehobby.net
liveinternet.ru managerpak204.weebly.com
lizace.com marinetraffic.com
lookingglassemb.com marwer.info
lsp-awak-perikanan.com maximpartnerspr.com
macdamaged.space meble-bogart.info
mafcards.ru meccadumps.net
manifestation.betteroffers.review mikozstop.com
massage-info.nl mindeyegames.com
minecraft-rus.org mobioffertrck.com
miniature.io mobstarr.com
missvietnam.org moscow.xrus.org
mncrftpcs.com mosdverka.ru
mokrayakiska.com mulberryoutletonlineeu.com
monarchfind-a.akamaihd.net muycerdas.xxx
moscow.online-podarki.com
mylesosibirsk.ru
mylida.org mylida.org
mylovelibrabry.com narosty.com
myonlinepayday.co naturalbreakthroughsresearch.com
mysex21.com nelc.edu.eg
narutonaruto.ru nevansk.ru
nastydollars.com nuker.com
nikitabuch.com o-o-8-o-o.ru
nudo.ca obiavo.ru
ochistka-stokov.ru obiavo.su
onblastblog.online obnal.org
optitrade24.com obsessionphrases.com
outshop.ru olgacvetmet.com
ovirus.ru olympescort.com
partnerline.men onlinebay.ru
pdfprof.com openlibrary.org
pic2fly.com outrageousdeal-a.akamaihd.net
pinkduck.ga paceform.com
poptool.net pacificair.com
pornhive.org parfusale.se
pornofeuer.com pdns.cz
pornogig.com pelfind.me
pornoslive.net pizza-tycoon.com
pornoxxx.com.mx pl.aasoldes.fr
pornzone.tv poem-paying.gq
pr0fit-b0x.com popugaychiki.com
privacylocationforloc.com porno-transsexuals.ru
prizeestates.cricket pornolook.net
pussyfleet.com posible.net
quangcaons.com powc.r.ca.d.sendibm2.com
quizzitch.net prodess.ru
razorweb-a.akamaihd.net produkto.net
rcpmda.ikan1080.xyz pussyspace.net
reelheroes.net putitin.me
rentalcarnavi.info r-control.ru
resultshub-a.akamaihd.net rasteniya-vs-zombi.ru
relatodelpresente.com.ar
rentehno.ru
retailwith.com retailwith.com
rialp.getenjoyment.net ribieiendom.no
ring4rhino.com robotixix.com
rn-to-bsn.com ruex.org.ua
rockingclicks.com
rp9.ru
ruscoininvest.company ruscoininvest.company
samara.rosfirm.ru sakhboard.ru
sayyoethe.blogspot.co.za sale-japan.com
sbprabooks.com sammyweaver.com
sell-fb-group-here.com sanyuprojects.com
seoprofiler.com scanner-george.top
serdcenebolit.com scanspyware.net
servicecenter.co.ua search.pch.com
sevendays.com.ua searchmywindow-a.akamaihd.net
sex-dating.co searchwebknow-a.akamaihd.net
sexiporno.net seemoreresultshu-a.akamaihd.net
shama-rc.net sexflirtbook.com
shillyourcoins.com sexy-pings.com
shop-electron.ru sexytrend.ru
silverage.ru skincrate.net
sitebeam.net snworks.com
smartbalanceworld.com social-button.xyz
snomer1.ru softomix.net
snow.nvr163.com sophang8.com
socialmadesimple.com sosdepotdebilan.com
softonicads.com spacash.com
ssconstruction.co spamnuker.com
starpages.net spinnerco.ca
stuff-about-money.com spitfiremusic.com
symphonyintegratedhealthcare.com stpolice.com
td-l-market.ru stroimajor.ru
tizanidinehcl4mgtabinfo.blogspot.com subj.ukr-lit.com
tizanidinehydrochloride2mg.blogspot.com summerlinhomes411.info
tizanidinemusclerelaxant.blogspot.com sundrugstore.com
topsiteminecraft.com swinon.site
torontoplumbinggroup.com t-bygg.com
torrents.cd tandvardshuset.net
tourcroatia.co.uk teesdaleflyballclub.co.uk
trackzapper.com telsis.com
tripper.de terraclicks.com
trudogolik.net tizanidine4mgstreetvalue.blogspot.com
tsan.net tizanidineduringpregnancy.blogspot.com
tubo360.com tizanidineinteractions.blogspot.com
twojebook.pl tizanidineorflexeril.blogspot.com
uptime-as.net tizanidinestreetprice.blogspot.com
ural-buldozer.ru tizanidinevscyclobenzaprine.blogspot.com
usiad.net tk-assortiment.ru
validdomain.xyz totu.info
veselokloun.ru trafficfactory.biz
vesnatehno.ru ttrraacckkrr.com
via-energy-order.com twinderbella.com
vpnhowto.info twitlinks.com
ucsol.ru
ussearche.cf
vacances-voyages.info
valsalud.com
vapsy.com
vduplo.ru
videochaty.ru
virtuagirl.com
vizag.kharkov.ua
vriel.batcave.net vriel.batcave.net
webcamdevochka.com vuclip.com
webcamtalk.net w7s.ru
webscouter.net wardreapptokone.tk
while.cheapwebsitehoster.com web.cvut.cz
wieseversa.no webextract.profound.net
wikes.20fr.com wesharepics.site
workle.website whengirlsgowild.com
worldoffiles.ru x-true.info
worst-sites.online xerox-douglas.cf
wsgames.ru xn----itbeirbjbi7bc6bh2d.xn--p1ai
xwatt.ru xn--80aafb2a.xn--p1ai
yoshkarola.zrus.org xn--oogle-wmc.com
yugo-star.ru xxxdatinglocal.us
zed21.net xxxguitars.com
znakom.sibtest.ru znaniyapolza.ru
znakomstva-moskva77.ru zoolubimets.ru
zynax.ua

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.22 0; 10.1.0.196 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2890 ### Version: V4.2022.02.2891
### Updated: Mon Jan 31 22:01:33 UTC 2022 ### Updated: Tue Feb 1 10:03:36 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2890 ### Version: V4.2022.02.2891
### Updated: Mon Jan 31 22:01:33 UTC 2022 ### Updated: Tue Feb 1 10:03:36 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##