vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-17 02:29:49 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.01.3534 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-01-02 10:09:39 +00:00
parent 5e24dd5303
commit e0489e32a7
48 changed files with 93604 additions and 93624 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3533 #### Version: V4.2023.01.3534
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3533 #### Version: V4.2023.01.3534
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.12.3533 #### Version: V4.2023.01.3534
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11564
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.12.3533 #### Version: V4.2023.01.3534
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11568
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11568
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11568
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11568
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 1; 10.1.0.166 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 0; 10.1.0.166 0;
127.0.0.1 0; 127.0.0.1 0;

11568
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 1; 10.1.0.166 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 0; 10.1.0.166 0;
127.0.0.1 0; 127.0.0.1 0;

11568
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 1; 10.1.0.166 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 0; 10.1.0.166 0;
127.0.0.1 0; 127.0.0.1 0;

11568
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3533 3534

11568
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11568
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.45.202.145 0; 138.91.71.18 0;

11568
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 1; 10.1.0.166 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 0; 10.1.0.166 0;
127.0.0.1 0; 127.0.0.1 0;

11568
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 1; 10.1.0.166 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 0; 10.1.0.166 0;
127.0.0.1 0; 127.0.0.1 0;

11568
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 1; 10.1.0.166 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 0; 10.1.0.166 0;
127.0.0.1 0; 127.0.0.1 0;

11568
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.45.202.145 0; 138.91.71.18 0;

11568
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.45.202.145 0; 138.91.71.18 0;

11568
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 1; 10.1.0.166 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BackWeb Badass
Calculon BuiltWith
Drip Demon
ProWebWalker Dirbuster
Siphon HTMLparser
Snapbot Ltx71
Surfbot NextGenSearchBot
TeleportPro PECL::HTTP
VB Project PageThing.com
iubenda-radar Pockey

316
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 404checker
404enemy
ALittle Client ALittle Client
Aboundex
Acunetix Acunetix
AfD-Verbotsverfahren AhrefsBot
AiHitBot AiHitBot
Aipbot Aipbot
AllSubmitter AllSubmitter
Anarchie AlphaBot
Ankit Anarchy
Apexoo Aspiegel
Atomseobot BLEXBot
AwarioRssBot BackDoorBot
BBBike BackStreet
BDCbot Backlink-Ceck
BDFetch
BacklinkCrawler BacklinkCrawler
BetaBot Badass
Black Hole Bandit
BlackWidow Bitacle
Blow
BlowFish
Boardreader
Bolt
BotALot BotALot
Brandwatch Buddy
BuiltBotTough BuiltWith
BunnySlippers BunnySlippers
CazoodleBot BuzzSumo
CensysInspect CheTeam
Citoid CherryPicker
Cloud mapping Cloud mapping
Collector Cogentbot
Copier
CopyRightCheck CopyRightCheck
Copyscape Craftbot
CrazyWebCrawler Crawling at Home Project
Crescent DSearch
CrunchBot Demon
DTS Agent
DatabaseDriverMysqli
Devil Devil
Digincore Dirbuster
Discobot Disco
Discoverybot Discoverybot
Dispatch DittoSpyder
DnyzBot
DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
Dotbot DomainStatsBot
Download Wonder Download Wonder
EMail Wolf Dragonfly
EasyDL ECCP/1.0
Ecxi Ecxi
EirGrabber EirGrabber
EroCrawler
Evil Evil
Ezooms Extractor
Extreme Picture Finder
EyeNetIE
FHscan FHscan
FemtosearchBot Fimap
FlashGet FlashGet
Fuzz Foobot
FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t GermCrawler
GT::WWW Getintent
GalaxyBot Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla
Gotit Gotit
Grabber
GrapeshotCrawler GrapeshotCrawler
HEADMasterSEO HEADMasterSEO
HTTP::Lite HMView
Haansoft Harvest
Havij
Hloader Hloader
HonoluluBot Humanlinks
HybridBot HybridBot
IDBot
Id-search Id-search
Indy Library IlseBot
Intelliseek Image Fetch
Internet Ninja InfoNaviRobot
JennyBot InfoTekies
InterGET
IstellaBot
JetCar JetCar
Jetty Jetty
JikeSpider JikeSpider
JustView Jorgee
Jyxobot Keybot Translation-Search-Machine
Keyword Density
Kinza Kinza
Kozmosbot LNSpiderguy
Lanshanbot LeechFTP
Larbin LexiBot
Leap
Lftp Lftp
Libwhisker
Lightspeedsystems Lightspeedsystems
LinkScan
LinkWalker LinkWalker
Linkbot Linkbot
LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Litemage_walker Lipperhey
Lmspider Lipperhey Spider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool
MIIxpc
MJ12bot
MSFrontPage
MSIECrawler
MTRobot MTRobot
Magnet Magnet
Majestic SEO
Majestic-SEO
Majestic12 Majestic12
Mass Downloader MarkMonitor
Mata Hari Masscan
Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
MicroMessenger MicroMessenger
Microsoft URL Control Microsoft Data Access
Minefield
Mister PiX
Mojeek Mojeek
Mojolicious Mojolicious
Mr.4x3 Mr.4x3
NICErsPRO Name Intelligence
Needle Navroad
NearSite
NetAnts NetAnts
NetMechanic NetMechanic
NetSpider NetSpider
NextGenSearchBot Nettrack
Netvibes
NimbleCrawler
Nimbostratus Nimbostratus
Ninja
Nutch Nutch
OrangeBot Offline Navigator
PECL::HTTP OpenVAS
Openvas
PHPCrawl PHPCrawl
POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageGrabber
PageScorer
Pandalytics Pandalytics
Panscient Petalbot
Papa Foto
Pavuk
Picscout Picscout
Picsearch
PictureFinder
Piepmatz Piepmatz
Pimonster
PleaseCrawl PleaseCrawl
Pu_iN Pockey
Psbot
Pump Pump
PxBroker
QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot RankFlex
RankingBot
Rankivabot Rankivabot
ReGet RankurBot
RebelMouse Re-re
RealDownload
Recorder
RedesScrapy
RocketCrawler
SEOkicks SEOkicks
SEOlyticsCrawler SEOkicks-Robot
SISTRIX SEOstats
ScoutJet SalesIntelligent
Scrapy Scrapy
Screaming Searchestate
ScreenerBot SemanticJuice
SearchmetricsBot SemrushBot
SentiBot
SeobilityBot
Seomoz
Siphon Siphon
SiteCheckerBotCrawler SiteCheckerBotCrawler
Sitevigil SiteExplorer
SiteLockSpider
SmartDownload SmartDownload
Snake Snake
Sociscraper Snoopy
Sogou web spider Sogou web spider
Sosospider SpaceBison
Sottopop SpankBot
Spanner Spbot
SputnikBot Spinn3r
Sqworm Steeler
Stripper Stripper
SuperHTTP Sucker
Sucuri
SuperBot
SurveyBot SurveyBot
Swiftbot T8Abot
T0PHackTeam Teleport
TeleportPro Telesphoreo
Toweyabot Telesphorep
Trendiction The Intraformant
Thumbor
TightTwatBot
Toata
Trendictionbot Trendictionbot
True_Robot Turnitin
TurnitinBot TurnitinBot
TwengaBot Twice
URLy Warning Typhoeus
URLy.Warning
V-BOT
Vagabondo Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VidibleScraper VeriCiteCrawler
Virusdie Voil
VoidEYE Voltron
WBSearchBot
WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers
WeSEE
Web Enhancer Web Enhancer
Web Pix WebCollage
Web Sauger
WebCopier WebCopier
WebEnhancer WebEnhancer
WebFuck WebLeacher
WebPix WebStripper
WebSauger
WebSucker WebSucker
WebZIP
Webalta Webalta
WebmasterWorldForumBot
Website Quester Website Quester
WebsiteQuester WebsiteExtractor
Whack Webster
Whacker Whatweb
Who.is Bot WinHTTrack
Widow Wonderbot
Woobot Woobot
Xaldon WebSpider Zitebot
Xenu
Zermelo
Zeus
ZmEu ZmEu
ZoominfoBot ZoomBot
ZyBorg ZumBot
archive.org_bot arquivo-web-crawler
arquivo.pt arquivo.pt
cah.io.community check1.exe
clark-crawler
cognitiveseo
crawl.sogou.com crawl.sogou.com
domainsproject.org evc-batch
facebookscraper
gopher
heritrix heritrix
instabid
isitwp.com
lwp-trivial lwp-trivial
meanpathbot mediawords
pcBrowser plumanalytics
probe-image-size polaris version
scalaj-http s1z.ru
scan.lol satoristudio.net
seocompany.store
seoscanners seoscanners
serpstatbot seostar
sitechecker.pro sogouspider
siteripz sp_auditbot
spyfu
sysscan sysscan
trendiction.com trendiction.com
ubermetrics-technologies.com webmeup-crawler
webgains-bot webpros.com
x22Mozilla webprosbot
xpymep1.exe zauba.io
zgrab zgrab

322
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider
404checker 404checker
ADmantX 404enemy
AIBOT
Aboundex Aboundex
Aboundexbot Acunetix
AiHitBot AiHitBot
Aipbot Alexibot
AllSubmitter
Alligator Alligator
Anthill AlphaBot
Apexoo Anarchie
Attach Ankit
Asterias
AwarioRssBot
BBBike
BDCbot BDCbot
BacklinkCrawler BDFetch
Bandit BackStreet
Badass
BatchFTP
Battleztar Bazinga Battleztar Bazinga
BetaBot BetaBot
Bitacle Blow
Bolt BotALot
Brandwatch Buck
Buddy BuiltBotTough
BuiltWith BuiltWith
Bullseye Bullseye
BunnySlippers
BuzzSumo BuzzSumo
CSHttp CSHttp
Calculon Cegbfeieh
CensysInspect CensysInspect
CheTeam ChinaClaw
CherryPicker Chlooe
Citoid
Claritybot Claritybot
Cosmos Cliqzbot
CrunchBot Copyscape
Curious
DBLBot DBLBot
DIIbot DIIbot
DSearch Demon
DTS Agent
Deusu Deusu
DigitalPebble Digincore
Dirbuster
Dispatch Dispatch
DnBCrawler-Analytics DittoSpyder
DomCopBot
DomainAppender DomainAppender
Download Wonder DomainCrawler
Dragonfly DomainSigmaCrawler
Domains Project
Drip
ECCP/1.0
EMail Wolf EMail Wolf
Ecxi EasyDL
Evil Ebingbong
EirGrabber
EroCrawler
Exabot
Extractor Extractor
Extreme Picture Finder
Ezooms Ezooms
FHscan FHscan
FemtosearchBot
FlashGet FlashGet
Foobot
Freeuploader Freeuploader
G-i-g-a-b-o-t GalaxyBot
GT::WWW Genieo
GermCrawler GermCrawler
Getintent GetRight
Go!Zilla GetWeb
Go-Ahead-Got-It
GoZilla GoZilla
Grabber Gotit
GrabNet
Grafula Grafula
GrapeFX GridBot
HEADMasterSEO HMView
HTMLparser Haansoft
HTTP::Lite HonoluluBot
HTTrack
Harvest
Humanlinks
HybridBot HybridBot
IDBTE4M Iblog
IDBot IlseBot
IRLbot Image Sucker
Image Fetch
IndeedBot IndeedBot
Indy Library InfoTekies
InfoNaviRobot Internet Ninja
Intelliseek
InternetSeer InternetSeer
Iria
Iskanie Iskanie
JennyBot JOC Web Spider
JamesBOT
Jetty Jetty
Jorgee JikeSpider
Keybot Translation-Search-Machine Joomla
LWP::Simple JustView
Lanshanbot Kinza
LibWeb Kozmosbot
Linkbot LNSpiderguy
Lipperhey LeechFTP
Litemage_walker Lftp
Lmspider Libwhisker
MFC_Tear_Sample Likse
MJ12bot LinkWalker
LinksManager
Lipperhey Spider
MIIxpc
Mag-Net Mag-Net
Magnet
Mail.RU_Bot
Majestic SEO Majestic SEO
Majestic-SEO
MarkMonitor MarkMonitor
Mass Downloader Mata Hari
Mb2345Browser MauiBot
MeanPath Bot
Meanpathbot
MegaIndex.ru
Metauri Metauri
Microsoft Data Access Microsoft Data Access
Microsoft URL Control
Minefield Minefield
Mister PiX
Moblie Safari Moblie Safari
Msrabot Mojeek
Musobot Mozlila
Mr.4x3
NICErsPRO NICErsPRO
NPbot Needle
Name Intelligence
NearSite
NetAnts NetAnts
NetLyzer NetLyzer
NetSpider NetZIP
Nettrack
Niki-bot Niki-bot
Nikto
NimbleCrawler
Nimbostratus Nimbostratus
Ninja Ninja
Nmap
Not
Nutch Nutch
Offline Navigator OrangeBot
OpenLinkProfiler PECL::HTTP
Openvas PageGrabber
OrangeSpider PageThing.com
POE-Component-Client-HTTP
Pandalytics
Papa Foto Papa Foto
PictureFinder Pavuk
Piepmatz PeoplePal
Pi-Monster
Picscout
Picsearch
Pimonster Pimonster
PleaseCrawl
Pump Pump
PyCurl PxBroker
Quick-Crawler QueryN Metasearch
RankingBot Rankivabot
ReGet
RealDownload
Reaper
RebelMouse RebelMouse
RocketCrawler RedesScrapy
Rogerbot RepoMonkey
SBIder SBIder
SEOkicks-Robot
SEOprofiler
SEOstats SEOstats
ScoutJet SalesIntelligent
ScanAlert
Screaming Screaming
ScreenerBot Searchestate
Seekport
SemanticJuice
Semrush Semrush
SemrushBot SeoSiteCheckup
SeobilityBot
Seomoz
Siphon Siphon
SiteCheckerBotCrawler
SiteSnagger SiteSnagger
Sitevigil Siteimprove
SlySearch
SmartDownload SmartDownload
Snake Snoopy
Snapbot SocialRankIOBot
Sociscraper Sosospider
Sogou web spider
Sottopop
Spammen Spammen
SpankBot
Spanner
SputnikBot
Sqlmap Sqlmap
Sqlworm Sqworm
Stripper Stripper
Sucuri SuperHTTP
SurveyBot Surfbot
Suzuran Swiftbot
Szukacz
T0PHackTeam T0PHackTeam
Teleport
Telesoft Telesoft
Telesphorep Telesphorep
Thumbor TheNomad
TightTwatBot Titan
Tracemyfile Tracemyfile
True_Robot Trendiction
Turingos Trendictionbot
Twice Twice
URLy Warning
Upflow Upflow
V-BOT V-BOT
VB Project
VCI VCI
VeriCiteCrawler Vagabondo
VidibleScraper VelenPublicWebCrawler
Virusdie VoidEYE
WEBDAV WEBDAV
WWWOFFLE WWW-Collector-E
WWW::Mechanize
Wallpapers Wallpapers
Wallpapers/3.0 Wallpapers/3.0
Web Enhancer WeSEE
Web Auto
Web Fetch
Web Fuck
Web Pix
Web Sauger Web Sauger
Web Sucker
WebBandit WebBandit
WebLeacher WebFetch
WebFuck
WebReaper
WebSauger WebSauger
WebStripper
WebSucker WebSucker
WebWhacker
WebZIP
Webshag Webshag
WebsiteExtractor Whacker
Webster Who.is Bot
Whack WinHTTrack
Whatweb
Widow
Wonderbot
Wprecon
Xaldon WebSpider Xaldon WebSpider
YoudaoBot Zitebot
Zade ZmEu
Zauba ZoomBot
Zermelo archive.org_bot
ZoominfoBot autoemailspider
adscanner cah.io.community
arquivo-web-crawler clark-crawler
backlink-check crawl.sogou.com
com.plumanalytics demandbase-bot
crawler.feedback domainsproject.org
crawler4j facebookscraper
dataforseo.com
gopher gopher
heritrix
instabid
ips-agent ips-agent
isitwp.com isitwp.com
iubenda-radar linkdexbot
lwp-request lwp-request
lwp-trivial lwp-trivial
meanpathbot
mediawords
netEstate NE Crawler netEstate NE Crawler
pcBrowser pcBrowser
polaris version plumanalytics
satoristudio.net s1z.ru
scalaj-http
scan.lol
seocompany.store seocompany.store
seostar serpstatbot
sysscan sexsearcher
siteripz
sogouspider
sp_auditbot
tAkeOut tAkeOut
ubermetrics-technologies.com trendiction.de
webgains-bot
webmeup-crawler webmeup-crawler
webpros.com webprosbot
x09Mozilla x09Mozilla
zauba.io zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
apper.de adrenalin-stk.ru
coslab.club alldezservice.kz
foto-telok.net calc-for-credit.ru
high-speed1.net kchaxton.com
hildinghr.se lolnex.us
pawli.eu mmog-play.ru
rednise.com picsforkeywordsuggestion.com
stanthonyscatholicchurch.org shillyourcoins.com
toys.erolove.in topappspro.com
wstroika.ru worldmusicfests.com

908
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
123kuma.com 1008.su
12bet.com 11235813.webzdarma.cz
12u.info 12masterov.com
1kdailyprofit.me 178evakuator178.ru
1stat.ru 24x7-server-support.site
2020iscoming.info 2728fb936f0.com
28n2gl3wfyb0.ru 2nt.ru
3th.co.in 2x2fan.ru
3w24.com 300richmond.co.nz
5forex.ru 3weekdiet.com
5kstore.com 5kstore.com
aaa38852e886ac4af1a3cff9b47cab6272.com 5ws.dating-app.ru
abctoppictures.net 883zy.com
abtasty.com a1security.com.ua
academia-nsk.org aaronabel.com
addray.pro absolutelyanalog.com
adloads.com academiacsmendoza.org
adprotect.net acads.net
adsloads.com activepr.ru
aimayangzhi.com adconscious.com
allbizne.co.ua adloads.net
allforminecraft.ru adman.se
allnews.md adnotbad.com
alt-servis.ru ads-cool.pro
alta-realestate.com adtiger.tk
ahrefs.com
ahrntal.verymes.xyz
aibolita.com
alert-fjg.xyz
alienwheel.es
allblogroll.com
allboard.xobor.de
alpharma.net
altamayoztourism.com altamayoztourism.com
amateurlivechat.org
ameblo.top ameblo.top
anaksma.info anaksma.info
anal-acrobats.hol.es
andpolice.com andpolice.com
angel1777.kz animal36.com
apispringsmartne-a.akamaihd.net anmysite.com
apartamentwroclaw.eu
apartment.ru
api.stathat.com
app5.letmacworkfaster.world
apper.de apper.de
apple.com-webbrowsing-security.science
arenanews.com.ua
arendakvartir.kz
arendas.net
articlesdirectoryme.info
as5000.com as5000.com
asdfz.pro asdfz.pro
asiengirls.net asseenontv.ru
atdedinotuho.tk asseenontvonline.ru
atley.eu.pn atlasvkusov.ru
autonew.biz audiobangout.com
autorn.ru audiofree.ru
avitocars.ru ausergrubhof.info
azartmsl.com avramstroy.ru
azbukadiets.ru azartmix.com
bagsonsale.online azazu.ru
baikaleminer.com azlex.uz
baixar-musicas-gratis.com babs.com.ua
baladur.ru backlink4u.net
balook.com baersaratov.ru
bambi.ck.ua balla-boo.se
best-deals-products.com banksrf.ru
bestadbid.com bannerspace.com
bestofferswalkmydogouteveryday.gq basedecor.ru
besttorrentknifta.weebly.com basisches-wasser.net
bet365.com battleforupdating.site
betterscooter.com beetpics.pw
bigcareer.info bestcalovebracelet.cn
bio.trade-jp.net betteroffers.review
bidbuy.co.kr
biglistofwebsites.com
bimatoprost-careprost.com.ua
bio-optomarket.ru
biocn.dx.am biocn.dx.am
bioinnovate.co blackplanet.com
bjetjt.com blagovest-med.ru
blackwitchcraft.ru blog.f00kclan.de
bongiornos.info bloggerads.net
bookmaker-bet.com blue-square.biz
breastaugmentation.co.za bmw.afora.ru
bombla.org
bonzybuddy.com
bookmark4you.com.biz
braintobe.win
breget74.ru breget74.ru
bristolhostel.com brewdom.ru
bristolhotel.com
buigas.00it.com
buyclomidonlaine.com buyclomidonlaine.com
buytizanidineonline.blogspot.com byme.se
buzzzg.men bzero1jewelry.net
bytimedance.ru
c-gainsbourg.com
callcafe.info callcafe.info
cam-kontakte.org canacopegdl.com
canrole.com candiceloves.us
canzoni.ru carstrends2015.com
carina-sy.de cartierlovebracelet.xyz
casinopinup-wins.com casinos4dummies.co.uk
cayado.snn.gr cementaresearch.se
censys.io charmstroy.info
cfcl.co.uk chatroulette.online
cheapjerseysfootballshop.com cheapestjerseysonwholesale.com
chocolateslim-u-srbiji.com checkm8.com
cialischmrx.com chocolateslim-en-france.com
claytransformations.info christianlouboutinshoes.xyz
clayvasedesigns.tk cialis-samples.com
cleanpcnow.com cialischsrx.com
clksite.com cleanersoft.com
club-samodelkin.ru clmforexeu.com
cmd.kz clodo.ru
cmhomestayagency.com cntravelre.com
codysbbq.com compliance-don.top
compliance-checker-7.info compliance-walter.top
compliance-checker.info comsysnet.com
compliance-george.top coprofam.org
concordexoticrentals.com copypaste.traderzplanet.in
construmac.com.mx covadhosting.biz
coolwallpapers4k.info cpajunkies.com
cornerstone-countertops.com crazyboost.pro
counciltally.com cubook.supernew.org
counter.bloke.com cubs-jerseys.us
creams.makeforum.eu customcatchcan.com
cryptoswap.biz customergrowthsystems.com
cutt.us davidovic.info
cwetochki.ru dear-diary.ru
dailystrength.org dekorkeramik.ru
darkages.info deshevo-nedorogo.ru
dating-app.ru deutschland123.de
dayibiao.com dev78.dioniqlabb.se
deadlinkchecker.com diminishedvalueoforegon.com
den-noch24.ru dnsrsearch.com
dickssportinggoods.com domashneeruporno.com
digest-project.ru doska-vsem.ru
digital-video-processing.com
digitalfaq.com
directivepub.com
discountbarbour.online
dividendo.ru
dneprsvet.com.ua
dojki-hd.com
domain-tracker.com
domination.ml
dorratex.tn
dostavimvdom.ru dostavimvdom.ru
download-of-the-warez.blogspot.com dwomlink.info
dvervmoskvu.ru eachdayisagift.review
easyukraine.com earncash.com.ua
ec-file.info ed-shop01.ru
edge.sharethis.com ege-essay.ru
ednorxmed.com eksprognoz.ru
elektrischezi.canalblog.com electro-prom.com
elkacentr.ru elektrischezigaretten1.blogse.nl
erotische-geschichten-xxl.com elektronischezigarettekaufen2.tumblr.com
etm-consult.de enpolis.ru
eurosamodelki.ru erectile.bid
excitacao.com ereko.ru
ezigarettenkaufen2.dreamwidth.org es5.com
fastcrawl.com eshop.md
fatmaelgarny.com eu-cookie-law.blogspot.com
fellowshipoftheminds.com eu-cookie-law.info
femdom.twiclub.in eupornstar.info
everflora.ru
everytalk.tv
expertblog.info
eyelike.com.ua
eyessurgery.ru
ezigaretteshop2.vefblog.net
f1nder.org
falopicm.pw
farmprofi.net
fashion.stellaconstance.co
fetishinside.com
fettefrauen.net fettefrauen.net
fickluder69.com fickluder69.com
figuringmoneyout.com filesvine.com
findthe.pet fillmewithhappiness.com
finuse.com financetip.eu
findclan.org
fitfloponline.store fitfloponline.store
fivedwld.cf fodelsedagspresenter.nu
foto-basa.com forex-procto.ru
franecki.net forms-mtm.ru
free-fbook-traffic.com formseo.com
free-share-buttons.com forpackningsutveckling.se
fun2cell.net fortevidyoze.net
fungamelands.com forum69.info
funnypica.com froggytube.com
funponsel.com gamblingnerd.com
gagrasector.ru gazobeton-p.com.ua
gallictures.com gcup.ru
gambarkatabaru.com ge0ip.org
gardene.ru gelezki.com
gaygalls.net getridofstretchmarks.org
generic-pills-online.com getyourimage.club
gidonline.one girlsfuckdick.com
girlspicsa.com gk-atlant.info
glastecfilms.com.my glall.ru
globetrotting-culture.ru glavtral.ru
gloverid.site go2album.com
gonextmedia.com gosarhivrt.ru
good-mummy.ru gosreg.amchs.ru
graphics8.info gq-catalog.gq
grupografico-pilar.com.ar greatzip.com
gtopstats.com gsou.cf
handicapvansarea.com gwebtools.com.br
havepussy.com hasshe.com
hazardky.net
hdhc.ru hdhc.ru
hdseriale.pl headpress.ru
herehloadibs.cf hentai-manga.porn
hetmanship.xyz hermesbirkinhandbagoutlets.com
heygidday.biz hidemyass.com
highspeed5.net highstairs-a.akamaihd.net
homelygarden.com hmywwogw.bloger.index.hr
homemade.gq honyaku.yahoofs.jp
hostcritique.com hopeonthestreet.co.uk
horoshieokna.com
hostingclub.lk hostingclub.lk
hotblognetwork.com hotblog.top
hoverboard360.at hotdl.in
hustoon.over-blog.com hotloans.ru
hyiphunter.org housediz.com
i-hobot.ru howtoclean.club
i4track.net howtowhitenteethfast.xyz
iamsport.org hscsscotland.com
iboss.com https-legalrc.biz
ideashome.id huimin764128.com
igrovyeavtomaty777.ru hystersister.com
ilbe.club ilmen.net
ilovevitaly.com
images.gyffu.com images.gyffu.com
importchinacoach-teach.com imitex-plus.ru
incanto.in.ua impotentik.com
inzn.ru incest-ru.com
iradiology.ru incolors.club
isitpaleo.info infokonkurs.ru
itservicesthatworkforyou.com infoupdate.org
javatex.co.id innodgfdriverhm.aircus.com
javitas.info intellektmedia.at
jeremyeaton.co internetartfair.com
jerseywholesalebizchina.com invitefashion.com
jyrxd.com istanbulit.com
kamagragelusa.net itis4you.com
kambasoft.com japan-bearings.ru
kareliatobacco.ru japfm.com
kartiniresto.com joyceblog.top
katjimej.blog.fc2.com jrpmakati.com
katushka.net jurajskie.info
kazan.zrus.org jyvopys.com
kinofak.net keywordsuggests.com
kinoplen.ru khadastoafarde.tk
klikbonus.com kino-filmi.com
klumba55.ru kopihijau.info
kol-energo.ru kovesszucs.atw.hu
konkursov.net krynica.info
kostenlos-sexvideos.com kutikomi.net
kraljeva-sutjeska.com ladov.ru
krasivye-devushki.net lankarns.com
krasula.pp.ua laptop-4-less.com
landoftracking.com laserpen.club
landscaping.center lechenie-gemorroya.com
lenpipet.ru lemon-ade.site
lenvred.org
lernur.net lernur.net
letmacworkfaster.site
letsrepair.in
letto.by
lexaprogeneric.link lexaprogeneric.link
libertybilisim.com lignofix.ua
limtu.ifmo.ru linerdrilling.com
linkbolic.com liver-chintai.org
liran-locks.com ljusihus.se
liveinternet.ro lkbennettstore.online
livesex-amateure.info lonely-mature.com
lodki-pvh.dp.ua
longadventure.com
lookingglassemb.com lookingglassemb.com
luchshie-topcasino.ru m0r0zk0-krava.ru
luciddiagnostics.in marketingvici.com
macdamaged.space marktforschung-stuttgart.com
mafcards.ru massage-info.nl
magicalfind-a.akamaihd.net matrixalchemy.com
maglid.ru mb140.ru
mailemedicinals.com
mainlinehobby.net
masserect.com
mebel-alait.ru mebel-alait.ru
mecash.ru meddesk.ru
medi-fitt.hu medicovi.com
megahdporno.net mega-polis.biz.ua
militarysale.pro michellblog.online
misterjtbarbers.com miniature.io
mjchamonix.org misandesign.se
mmoguider.ru mmog-play.ru
mojaocena.com monclercoats.xyz
monclerjacketsoutlet.pw money-every-day.com
monetizer.com-01.site mostantikor.ru
more-letom.ru msk.afora.ru
moscow-clining.ru muizre.ru
mrmoneymustache.com musflashtv.com
mtmtv.info musicstock.me
musicas.baixar-musicas-gratis.com musicvidz.ru
mydearest.co muycerdas.xxx
mylovelibrabry.com mysexpics.ru
mypets.by najaden.no
myphotopipe.com narco24.me
mystats.xyz netoil.no
nalogovyykodeks.ru netpics.org
narosty.com
neobux-bg.info
nerudlogistik.ru
net-radar.com
new-post.tk
newsperuse.com
newtechspb.ru
nextlnk12.com
nfljerseys.online
ngps1.ru ngps1.ru
noumeda.com nhl17coinsforps3.gratisblog.biz
ntic.fr nikitsyringedrivelg.pen.io
nudejapan.net no-rx.info
obuv-kupit.ru nosecret.com.ua
oda.as notfastfood.ru
odywpjtw.bloger.index.hr novgorod.xrus.org
ok.ru novosti-hi-tech.ru
onescreen.cc o-dachnik.ru
online-x.ru obiavo.ru
onlinefilmz.net oconto.ru
onlineporno.site oecnhs.info
orenburg-gsm.ru offf.info
oxotl.com omgtnc.com
paparazzistudios.com.au one.net.in
paradontozanet.ru onlinedomains.ru
parajumpersstore.online onlineslotmaschine.com
partner-stop.men openstat.com
pastaleads.com optikremont.ru
petsblogroll.com otbelivanie-zubov.com
photosaga.info ourville.info
piccdata.com partner-trustworthy.men
picsearch.com partybunny.ru
picsfair.com pechikamini.ru
picturesify.com pelfind.me
pisanieprac.info peugeot-club.org
play-movie.pl piski.top
pokerniydom.ru plastgran.com
polcin.de plastgranar.nu
poligon.com porno-transsexuals.ru
polytopesexempt.com pornoforadult.com
pony-business.com pornogad.com
popunder.net pornogig.com
popunder.ru
popup-fdm.xyz
porno-dojki.net
pornohub.me pornohub.me
pornotubexxx.name pornokajf.com
prchecker.info pornowarp.info
poshiv-chehol.ru
pospr.waw.pl
powc.r.ca.d.sendibm2.com
pozdravleniya-c.ru
predominant-invent.tk predominant-invent.tk
presleycollectibles.com pricheskaonline.ru
privacylocationforloc.com privatov-zapisi.ru
privatefx-in.ru privetsochi.ru
progonrumarket.ru prizefestival.mobi
proprostatit.com prizesbook.online
prostitutki-belgoroda.org profitfx.online
prostitutki-kharkova.org prointer.net.ua
prostitutki-yaroslavlya.org pron.pro
pukaporn.com prosperent.com
push-ad.com pussyfleet.com
pussyspace.net
pwwysydh.com pwwysydh.com
pyramidlitho.webs.com rank-checker.online
quick-offer.com ranksignals.com
ranking2017.ga rapidsites.pro
realizmobi.com realresultslist.com
realtytimes.com research.ifmo.ru
redirectingat.com
refads.pro
refudiatethissarah.info
reimageplus.com
remont-fridge-tv.ru
rennlist.com
replica-watch.ru
respublica-otel.ru
rewardpoll.com
ric.info
riders.ro
ringporno.com
rixpix.ru rixpix.ru
rock-cafe.info romhacking.ru
rp9.ru rukino.org
rubanners.com russkoe-zdorovie.ru
rutor.vip rvtv.ru
s-forum.biz sa-live.com
sakhboard.ru saltspray.ru
saletool.ru
samolet.fr
sanyuprojects.com sanyuprojects.com
saratov.xrus.org sawin.beth.webd.pl
sbt-aqua.ru
scanmarine.info
scanner-andrew.top scanner-andrew.top
scanner-george.top scanner-fred.top
scanner-john.top scanner-ivan.top
scanner-josh.top
schalke04fc.info
schlampen-treffen.com
scrapy.org
scrnet.biz.ua scrnet.biz.ua
sdelai-prosto.ru searchwebknow-a.akamaihd.net
searchimage.co selectads.men
searchimpression.com seoexperimenty.ru
searchkut.com
security60-e.com
sei80.com
seoanalyses.com
seokicks.de seokicks.de
service-core.ru service-core.ru
service.adtech.fr
service.adtech.us
sevendays.com.ua sevendays.com.ua
sexuria.net sex-tracker.de
sexy-screen-savers.com sex-watch.com
shillyourcoins.com sexblog.pw
shop-electron.ru sexpornotales.com
shopwme.ru sexvideo-sex.com
sientalyric.co sexvporno.ru
sigmund-freud.co.uk sexxdate.net
skatestick.bid sexyebonyteen.com
skincrate.net shivafurnishings.com
skypasss.com si-unique.com
sibtest.ru
similardeals.net
siteimprove.com
sledstvie-veli.net
smartadserver.com
smichovbike.cz smichovbike.cz
sms2x2.ru snow.nvr163.com
snsdeainavi.info
snworks.com snworks.com
socialseet.ru socas.pluto.ro
softlinesolutions.me socialbutton.xyz
soviet-portal.do.am sockshares.tv
spywarebegone.com soft-terminal.ru
srdrvp.com softtor.com
stair.registrydomainservices.com softwaretrend.net
stanthonyscatholicchurch.org songplanet.ru
startufa.ru soochi.co
soundfrost.org
spaceshipad.com
spb.ru
sports-supplements.us
ssconstruction.co
stal-rulon.ru
star61.de
stockspmb.info stockspmb.info
storehouse.ua strangeduckfilms.com
sundrugstore.com success-seo.com
sunflowerdrawingpaintings.blogspot.com superinterstitial.com
superkanpo.com supermesta.ru
support.nopeas.sk superoboi.com.ua
surgut.zrus.org svolze.com
svetodiodoff.ru tabletkinaodchudzanie.com.pl
szamponrevita.pl
tackletarts.co
teenforporn.com teenforporn.com
templates.franklinfire.co tehngr.ru
templates.radiodigital.co templates.radiodigital.co
teslathemes.com terrafootwear.us
tgtclick.com thedownloadfreeonlinegames.blogspot.com
thaismartloan.com thegameriders.com
theallgirlarcade.com thfox.com
thebitcoincode.com timdreby.com
titan-cloud.life timetorelax.biz
tizanidine4mgprice.blogspot.com
tizanidineduringpregnancy.blogspot.com tizanidineduringpregnancy.blogspot.com
tizanidinehcl4mgtablet.blogspot.com tizanidinesideeffects.blogspot.com
tizanidinehydrochloride4mgstreetvalue.blogspot.com tomck.com
tizanidinevssoma.blogspot.com tongkatmadura.info
top-deal.com.pl topclickguru.com
topmira.com torontoplumbinggroup.com
torrnada.ru torrentgamer.net
tour-line.net toursmaps.com
tourismvictoria.com tpu.ru
traxdom.ru tracksurf.daooda.com
trackzapper.com
trafficfactory.biz
tri-slona.org tri-slona.org
trkdf.com trustaffs.com
trucri.me try-rx.com
tupper-posuda.ru ttrraacckkrr.com
tvory.predmety.in.ua tubeline.biz
twitlinks.com tuckpointingmasonrysystems.com
uggsale.online twojebook.pl
uginekologa.com unrealcommander.org
uhdtv.website uroffer.link
uk-zheu20.ru ussearche.cf
unpredictable.ga uytmaster.ru
url-img.link vavilone.com
utrolive.ru vduplo.ru
vesnatehno.ru viagra.pp.ua
viddyoze.com
video-chat.cn video-chat.cn
video-production.com videotuber.ru
videochat.mx vip-dom.in
videochaty.ru vipsexfinders.com
videos-for-your-business.com viteonlusarezzo.it
viel.su vodkoved.ru
virtualbb.com volgograd.xrus.org
vita.com.hr vostoktrade.info
voluumtracker1.com vpnhowto.info
voprosotvet24.ru vsexkontakte.net
vozbujdenie.com warezaccess.com
wallpaperaccess.com watchinf.com
watchdogs-2.ru wcb.su
web-analytics.date
web-betting.ru
webcamdevochka.com webcamdevochka.com
weblo.com website-speed-checker.site
webshoppermac.com welck.octopis.com
website-speed-up.site wesharepics.site
wemedinc.com wfb.hatedriveapart.com
wesharepics.info wheelchairliftsarea.com
whatsupinfoley.com where-toget.com
wholesalejerseys-cheapest.com wholesalejerseychinashop.com
wildworld.site wholinkstome.com
wladimirpayen.com wildcattube.com
wnoz.de wiosenny-bon-1500.pl
works.if.ua worldinternetauthority.com
worldmusicfests.com worldmusicfests.com
x5market.ru worldtraveler.world
xcombear.ru www.kabbalah-red-bracelets.com
xjlottery.com x-diesel.biz
x-rates.ru
x-stars.ru
xcc24.pl
xchangetrak.com
xev.ru
xgftnlrt.bloger.index.hr
xlovecam.com xlovecam.com
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai xmronta.com
xn--24-glceagatoq7c2a6ioc.xn--p1ai xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
xn--80ab4aa2g.xn--p1ai xn--b1agm2d.net
xn--d1abj0abs9d.in.ua xn--oogle-wmc.com
xn--d1acah0c.xn--p1ai xnxxandxvideos.com
xtrafficplus.com xrus.org
xxxdatinglocal.us
yadro.ru yadro.ru
yebocasino.co.za youghbould.wordpress.com
yellowproxy.net yourtemplatefinder.com
yun56.co za-fun-offer.com
yurgorod.ru zaobao.com.sg
zacreditom.ru zeg-distribution.com
zeikopay.com zog.link
zhacker.net zone-kev717.info
zigarettenonlinekaufen1.bloog.pl zrus.org
znakom.sibtest.ru zscalertwo.net
zocaparj.kz zvuker.net
zreloeporno.tv zx6.ru

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
0c47f8422d3f.com 123kuma.com
12bet.com 12u.info
1hwy.com 2015god.org
2daytrendingnews.com aanapa.ru
40cg.com aaronabel.com
51.la alfabot.xyz
5kstore.com alldezservice.kz
70casino.online alvtank.se
7search.com amyfoxfitness.com
aad1f4acb0a373420d9b0c4202d38d94fa.com anlimebel.kiev.ua
akvamaster.dp.ua apiallgeniusinfo-a.akamaihd.net
allcredits.su apiportalmorecom-a.akamaihd.net
allwomen.info apisecretsaucebi-a.akamaihd.net
amanda-porn.ga ardimobileinfo.ml
apibatbrowsecom-a.akamaihd.net aucoinhomes.com
apilinkswiftco-a.akamaihd.net aviva-limoux.com
apimossnetinfo-a.akamaihd.net b2b-lounge.com
appearance-cool.com bang-hotties.com
arenanews.com.ua bear.gotcher.us
arenda-avtoprokat-krasnodar.ru best-games.today
arkkivoltti.net bestofupload.info
asseenontv.ru bibys.com
autobrennero.it blogsfunky672.weebly.com
autoplate.info bradleylive.xyz
ayakino.net brateg.xyz
bad-stars.net break-the-chains.com
best-mam.ru buyessayonline19.blogspot.ru
bestcalovebracelet.cn cartierbraceletsreplica.pw
bestcoin.cc cartierlovebraceletreplica.xyz
bestdooz.com castingbank.ru
bestkfiledxd.cf cayado.snn.gr
biketank.ga cbox.ws
binaryoptionscops.info centrumcoachingu.com
bingo8888.com cgi2.nintendo.co.jp
bio-optomarket.ru chcu.net
bioscorp.ru cheapjerseysbizwholesale.us
blackle.com cheapmoncler.pw
blavia.00author.com chinacheapelitejerseys.com
blog.yam.com chocolateslim-en-france.com
bluesman.nu classicakuhni.ru
boazpower.com cleansearch.net
bongiornos.info coin-hive.com
brimstonehillfortress.org compliance-barak.xyz
briomotor.co compliance-don.xyz
buckspar.gq compliance-donald.xyz
call-of-duty.info cookie-law-enforcement-ff.xyz
callstevens.com coolgamechannel.com
cardiosport.com.ua country-chic.ru
cctva.tv crystalslot.com
cdnnetwok.xyz dayibiao.com
cheap-pills-norx.com dear-diary.ru
cheatcode-lita12.rhcloud.com decoratinghomes.ga
chelnytruck.ru deplim.com
cialischmrx.com dimkino.ru
clickhype.com
clippingphotoindia.com
compliance-alexa.top
creditmoney.com.ua
damasarenai.info
decenttools.com
deda-moroza-zakaz.ru
design-ideas.info
digital-video-processing.com
digitalassetmanagement.site
discountliv.com
dividendo.ru
dktr.ru dktr.ru
doublepimp.com domainsatcost.com
dumuelave.xyz domoysshop.ru
dvr.biz.ua drpornogratisx.xxx
ecommerce-seo.org e-collantes.com
e-poker-2005.com
easycommerce.cf
elektir.ru elektir.ru
elektrozigaretten1.webs.com elektrischezigarette2.devhub.com
eloxal.ru elektronischezigarette2.mex.tl
enge-fotzen.info enhand.se
erotik0049.com eropho.com
escort-russian.com escort-russian.com
estibot.com essayassist.com
extremez.net everflora.ru
falllow.gq ezb.elvenmachine.com
floridamobilebillboards.com f07.de
forum20.smailik.org fanpagerobot.com
free-share-buttons.top farmprofi.net
freelifetimefuckbook.com ff30236ddef1465f88547e760973d70a.com
frivgame250.com filmidivx.com
gazporno.com fitnesspiks.com
glavtral.ru fordsonmajbor.cf
gomusix.com forex-indextop20.ru
googlemare.com forpostlock.ru
gosreg.amchs.ru forumprofi.de
graphics8.info fotzen-ficken.com
guildebzh.info freemags.cc
hasshe.com fuck-paid-share-buttons.xyz
hdimagegallery.net funcrushgames.com
headpharmacy.com fungamelands.com
heartofpayne.xyz gay-file.com
hesteel.pl geile-lelly.eu
home-task.com geileweiber.tk
iboss.com get-seo-domain.com
igtools.club getpopunder.com
imagerydatabase.com go2album.com
insomniagamingfestival.com goldandcard.ru
goodhumor24.com
goosefishpost.bid
graphid.com
gsasearchengineranker.pw
gwhwpxbw.bloger.index.hr
gz2.bbsoldes.fr
hamacapty.com
headpress.ru
homemature.net
hostcritique.com
images-free.net
incest-ru.com
indianmedicaltourismshop.com
iplusbit.blogspot.co.za iplusbit.blogspot.co.za
istmira.ru iqs.biz.ua
janettabridal.com itch.io
jikoman.info izhevsk.xrus.org
joy-penguin.com japan-bearings.ru
jurajskie.info javlibrary.cc
kerwinandcariza.com jobgirl24.ru
kiskinhouse.com journalhome.com
kiwi237au.tk kaidalibor.de
kladrus.ru kazrent.com
keywordbasket.com
kinocccp.net
krasnodar-avtolombards.ru
kunstaktien.info
lafriore.ru lafriore.ru
landscaping.center laxdrills.com
languagecode.com lineavabit.it
leadwayau.com linkbolic.com
limtu.ifmo.ru lol-smurfs.com
lincolntheatre.com longlifelomilomi.info
lion.misba.us magicplayer-s.acestream.net
lrac.info malwareremovalcenter.com
mafa-free.com marcoislandvacations.net
magiadinamibia.blogspot.com masturbate.co.uk
maridan.com.ua med-zdorovie.com.ua
martlinker.com metalonly.info
milblueprint.com mmgq.ru
minegam.com monitorwebsitespeed.com
miniature.io monsterdivx.tv
mixx.com mrsdalloways.com
modabutik.ru mylida.org
morocco-nomad-excursions.com narkologiya-orel.ru
mp3ritm.top naturtreenspicerx.pw
mydownlodablefiles.com neodownload.webcam
naphukete.ru nevansk.ru
narutonaruto.ru new7ob.com
nordstar.pro newyorkhotelsmotels.info
novosibirsk.xrus.org nrv.co.za
obnal.org nudejapan.net
okeinfo.online okayimage.com
onefilms.net onlinadverts.com
onlythegames.com origin-my.ru
orgasmus-virtual.com osnova3.ru
oyster-green.com oyster-green.com
partner-pop.men pagesense.com
peekyou.com par-fallen.ga
perederni.net pijoto.net
pewit.pw pipki.r.acdnpro.com
pk-services.ru pliks.pl
pochtovyi-index.ru popup-jdh.xyz
poiskzakona.ru porno-dojki.net
poptool.net pornoslive.net
pornohub.me poshiv-chehol.ru
potolokelekor.ru pospr.waw.pl
pro-okis.ru powenlite24.ru
prosmibank.ru pretty-mart.com
prostitutki-rostova.org prize44.com
prostitutki-tolyatti.org prodess.ru
rabot.host.sk profitsport.club
recordpage-a.akamaihd.net pussyfleet.com
redirectme.net putitin.me
remote-dba.de pyrodesigns.com.au
rentehno.ru quickloanbank.com
roma-kukareku.livejournal.com realtytimes.com
ruf777.com responsinator.com
rus-teh.narod.ru rightenergysolutions.com.au
ruspdd.com rixpix.ru
savefrom.com rvi.biz
scanner-brian.top sdrescher.net
scanner-george.top seobility.net
scanspyware.net seoholding.com
screentoolkit.com seorussian.ru
seansonline24.pl servisural.ru
search-goo.com sexytrend.ru
search.1and1.com shemalegalls.blogporn.in
see-your-website-here.com sibecoprom.ru
sendearnings.com siha.de
serdcenebolit.com sildenafil-tadalafil.info
sexad.net sispe.com.br
sextracker.com smart-balancewheel.com
shiksabd.com snjatie-geroinovoy-lomki.ru
shisha-swag.de social-s-iii.xyz
shitting.pro socialbuttons.xyz
simplesite.com soundfrost.org
simpoed.ufop.br space4updating.win
sitevalued.com specialfinanceoffers.com
sizeplus.work sportbetfair.com
slayerlife.com sribno.net
sloopyjoes.com stard.shop
smstraf.ru store-rx.com
socialseet.ru strigkaomsk.ru
spb.afora.ru
spy-app.info
stockspmb.info
studworks.org
subj.ukr-lit.com
superiends.org
superkanpo.com
superlist.biz superlist.biz
swiped.su takeflyte.com
t3chtonic.com td-l-market.ru
takeprofitsystem.com teguh.info
tam-gde-more.ru template-kid.com
taraz.xkaz.org terraclicks.com
tech4master.com tgtclick.com
thaismartloan.com theautoprofit.ml
thebluffs.com theendivechronicles.com
thetoiletpaper.com thetattoohut.com
thin.me.pn thin.me.pn
tizanidinemusclerelaxant.blogspot.com thruport.com
tizanidinestreetvalue.blogspot.com tiandeural.ru
tizanidinevscyclobenzaprine.blogspot.com tizanidinehcl4mgtab.blogspot.com
tksn.ru trahic.ru
tootoo.to triplepanda.xyz
toys.erolove.in tula.howotorg.ru
traffic2money.com
tsstcorpcddvdwshbbdriverfb.aircus.com
tuckpointingmasonrysystems.com
turbodsp.com
turist-strani.ru turist-strani.ru
turvgori.ru twodollarshows.com
tv-spoty.info tytoona.com
tvoystartup.ru uggbootsoutletsale.us
um-razum.ru urlopener.com
umaseh.com vgoloveboli.net
urengoy.pro vietimgy.pw
videochat.mx visa-china.ru
viveresaniesnelli.it warningwar.ru
vladimir.zrus.org webcamtalk.net
voditeltrezviy.ru webmasters.stackexchange.com
vodkoved.ru websites-reviews.com
vsesubwaysurfers.com white-truck.life
vvpg.ru wingsoffury2.com
warning.or.kr wladimirpayen.com
web-analytics.date www.event-tracking.com
webscouter.net x-lime.com
weburlopener.com x-lime.net
weddingdresses.xyz xblog.in
welck.octopis.com xgames-04.com
weprik.ru xn----9sbubg3ambdfl1j.xn--p1ai
x-diesel.org xxx-treker.ru
xblognetwork.com xxxguitars.com
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai xz618.com
y8games-free.com yottos.com
yellowstonesafaritours.com your-bearings.com
yoursearch.me youradexchange.com
zacreditom.ru youtuhe.com
zarajbuilders.com yur-p.ru
zarepta.com zakon-ob-obrazovanii.ru
zeroredirect7.com zaobao.com.sg
zigarettenonlinekaufen1.blox.pl zapatosenventa.info
zverokruh-shop.cz zarabotok--doma.ru
zetmaster.ru
zoogdinsney.com
zryydi.com
zvuker.net

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

14
dev-tools/test_units/ratelimittest-master.txt
View file

@ -1,21 +1,11 @@
<!DOCTYPE html> <html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html> ml><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.133 0; 10.1.0.166 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3533 ### Version: V4.2023.01.3534
### Updated: Sat Dec 31 22:01:23 UTC 2022 ### Updated: Mon Jan 2 10:03:35 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3533 ### Version: V4.2023.01.3534
### Updated: Sat Dec 31 22:01:23 UTC 2022 ### Updated: Mon Jan 2 10:03:35 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##