vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 06:15:06 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3456 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-07 10:17:50 +00:00
parent 52c9385a49
commit 9f49ecb2f6
47 changed files with 91573 additions and 91563 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3455 #### Version: V4.2022.11.3456
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3455 #### Version: V4.2022.11.3456
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3455 #### Version: V4.2022.11.3456
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11308
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3455 #### Version: V4.2022.11.3456
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11312
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11312
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11312
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11312
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 1; 10.1.0.102 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 0; 10.1.0.102 0;
127.0.0.1 0; 127.0.0.1 0;

11312
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 1; 10.1.0.102 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 0; 10.1.0.102 0;
127.0.0.1 0; 127.0.0.1 0;

11312
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 1; 10.1.0.102 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 0; 10.1.0.102 0;
127.0.0.1 0; 127.0.0.1 0;

11312
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3455 3456

11312
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11312
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.176.152.29 0; 20.169.168.61 0;

11312
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 1; 10.1.0.102 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 0; 10.1.0.102 0;
127.0.0.1 0; 127.0.0.1 0;

11312
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 1; 10.1.0.102 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 0; 10.1.0.102 0;
127.0.0.1 0; 127.0.0.1 0;

11312
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 1; 10.1.0.102 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 0; 10.1.0.102 0;
127.0.0.1 0; 127.0.0.1 0;

11312
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.176.152.29 0; 20.169.168.61 0;

11312
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.176.152.29 0; 20.169.168.61 0;

11312
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 1; 10.1.0.102 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Black Hole AllSubmitter
CazoodleBot Collector
CopyRightCheck DomainAppender
CyotekWebCopy RebelMouse
Evil SBIder
Haansoft Siteimprove
Mb2345Browser SpankBot
T0PHackTeam Suzuran
URLy.Warning adscanner
scalaj-http probe-image-size

308
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404enemy 01h4x.com
ADmantX ALittle Client
ASPSeek
Abonti Abonti
Aboundex Aboundex
Aboundexbot Aboundexbot
AhrefsBot Acunetix
AiHitBot
Aipbot
Alexibot
Alligator Alligator
AlphaBot
Anthill Anthill
AwarioSmartBot Apexoo
BBBike BackDoorBot
BLEXBot
Backlink-Ceck
Badass Badass
Bandit Bandit
Black Hole BatchFTP
Battleztar Bazinga
BetaBot
Bigfoot
Bitacle
BlackWidow BlackWidow
Blow Blow
Brandprotect Brandprotect
Buck Brandwatch
Buddy
BuiltBotTough BuiltBotTough
BuiltWith
BunnySlippers
BuzzSumo BuzzSumo
CODE87 CODE87
Calculon
CazoodleBot CazoodleBot
Cegbfeieh
CensysInspect CensysInspect
ChinaClaw Chlooe
Cloud mapping Citoid
Cogentbot Cogentbot
Copier
Copyscape
Crawling at Home Project
Crescent
CrunchBot
Curious
CyotekWebCopy CyotekWebCopy
DIIbot
DatabaseDriverMysqli DatabaseDriverMysqli
Demon Demon
Digincore Devil
DigitalPebble
Disco Disco
Discoverybot Discobot
DittoSpyder Dispatch
DnBCrawler-Analytics DnBCrawler-Analytics
DomainAppender DnyzBot
Domains Project Download Wonder
Dragonfly
Drip Drip
ECCP/1.0 EMail Siphon
EasyDL
Ebingbong
Ecxi Ecxi
EroCrawler Express WebPictures
ExtLinksBot
Extractor
Extreme Picture Finder Extreme Picture Finder
FHscan
FemtosearchBot FemtosearchBot
Foobot Flunky
FyberSpider FyberSpider
G-i-g-a-b-o-t Fyrebot
GT::WWW GT::WWW
Genieo GalaxyBot
GetRight GermCrawler
Getintent GrabNet
Go!Zilla Grabber
Go-Ahead-Got-It GridBot
GoZilla HMView
Grafula HTTP::Lite
GrapeFX Hloader
HEADMasterSEO HonoluluBot
HTTrack
Haansoft
Havij
HybridBot
IDBTE4M IDBTE4M
IRLbot IDBot
Id-search Id-search
Image Fetch
IndeedBot IndeedBot
Jorgee Indy Library
InfoTekies
Intelliseek
InternetSeer
IstellaBot
JetCar
Jetty
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Kinza Keyword Density
Kozmosbot Kozmosbot
LWP::Simple LNSpiderguy
Larbin Lanshanbot
LeechFTP Likse
LeechGet LinkScan
LexiBot Linkbot
Lftp LinksManager
LibWeb Lipperhey Spider
Libwhisker Lmspider
Lightspeedsystems Ltx71
LinkpadBot MFC_Tear_Sample
LinqiaMetadataDownloaderBot MIIxpc
MJ12bot MJ12bot
MSIECrawler MTRobot
Mag-Net
Magnet Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic12 Majestic12
MarkMonitor MarkMonitor
MarkWatch
Mass Downloader Mass Downloader
MauiBot Masscan
Meanpathbot
Mediatoolkitbot Mediatoolkitbot
Metauri Metauri
Minefield Minefield
Mister PiX Morfeus Fucking Scanner
Mojeek Mr.4x3
Mozlila Musobot
Msrabot NPbot
NICErsPRO Name Intelligence
Navroad Nameprotect
Nessus Needle
NetMechanic NetLyzer
NetZIP
Nettrack Nettrack
Ninja Netvibes
Not Nibbler
NimbleCrawler
Nimbostratus
Nuclei Nuclei
Octopus Nutch
Offline Explorer Offline Explorer
OnCrawl
OpenLinkProfiler
OpenVAS OpenVAS
OrangeBot Openfind
OrangeSpider OutclicksBot
OutfoxBot
PECL::HTTP PECL::HTTP
POE-Component-Client-HTTP
Page Analyzer
PageAnalyzer PageAnalyzer
PageGrabber PageScorer
Pandalytics PageThing.com
Papa Foto Papa Foto
Petalbot
Pi-Monster
Picscout Picscout
Picsearch Picsearch
PictureFinder Pixray
PleaseCrawl PleaseCrawl
Pu_iN ProPowerBot
ProWebWalker
PxBroker PxBroker
PyCurl
Quick-Crawler
RSSingBot RSSingBot
RankFlex RankActive
Rankivabot RankingBot2
RankurBot ReGet
RebelMouse
RedesScrapy
RepoMonkey
RocketCrawler RocketCrawler
SEOlyticsCrawler
SMTBot SMTBot
SalesIntelligent ScoutJet
ScanAlert
Scrapy Scrapy
ScrepyBot Searchestate
SearchmetricsBot SearchmetricsBot
Semrush Seekport
SemrushBot
SentiBot
SeobilityBot
Seomoz Seomoz
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteExplorer
SiteLockSpider
SiteSnagger
Siteimprove Siteimprove
Snake Sitevigil
SlySearch
Snoopy
SocialRankIOBot
Sociscraper
Sosospider Sosospider
Sottopop
SpaceBison
SpankBot SpankBot
Spanner
Spinn3r
SputnikBot SputnikBot
Sqworm Sqlmap
Steeler Sqlworm
Stripper Stripper
Sucker Sucker
Sucuri SuperBot
SuperHTTP
Surfbot Surfbot
Swiftbot T8Abot
Telesphorep Teleport
The Intraformant The Intraformant
TheNomad Thumbor
Titan Toata
Trendiction
Trendictionbot Trendictionbot
True_Robot
Turingos
Turnitin Turnitin
TurnitinBot TwengaBot
Twice
Typhoeus Typhoeus
UnisterBot UnisterBot
V-BOT Upflow
VCI Vagabondo
VeriCiteCrawler
VidibleScraper VidibleScraper
VoidEYE VoidEYE
Voltron Voil
WBSearchBot
WISENutbot WISENutbot
WPScan
Wallpapers
Wallpapers/3.0 Wallpapers/3.0
Web Pix WeSEE
Web Sauger Web Collage
WebCollage Web Fetch
WebEnhancer WebAuto
WebFuck WebCopier
WebGo IS WebImageCollector
WebLeacher
WebPix WebPix
WebSauger WebSauger
WebZIP WebWhacker
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Webshag
WebsiteExtractor WebsiteQuester
Whack WiseGuys Robot
Whacker
WinHTTrack
Woobot
Wotbox
Xaldon_WebSpider
Xenu Xenu
Zauba
Zeus Zeus
ZoomBot Zitebot
ZyBorg ZmEu
adscanner ZoominfoBot
archive.org_bot
arquivo-web-crawler
backlink-check
cah.io.community cah.io.community
clark-crawler clark-crawler
cognitiveseo crawler.feedback
demandbase-bot domainsproject.org
evc-batch
gopher
internetVista monitor
ips-agent ips-agent
isitwp.com
linkdexbot linkdexbot
lwp-trivial lwp-trivial
magpie-crawler
meanpathbot
mediawords mediawords
muhstik-scan
pcBrowser
plumanalytics plumanalytics
polaris version s1z.ru
probe-image-size
ripz
satoristudio.net satoristudio.net
serpstatbot scan.lol
siteripz seocompany.store
seoscanners
seostar
sitechecker.pro
sp_auditbot sp_auditbot
spyfu spyfu
sysscan
tAkeOut tAkeOut
trendiction.com trendiction.com
ubermetrics-technologies.com
voyagerx.com voyagerx.com
webgains-bot webgains-bot
webmeup-crawler webmeup-crawler
webprosbot webpros.com
x09Mozilla x22Mozilla
xpymep1.exe
zauba.io
zgrab

326
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 360Spider
ADmantX 404enemy
ALittle Client
ASPSeek
Abonti Abonti
Aboundex Aboundexbot
AfD-Verbotsverfahren AhrefsBot
AiHitBot
Aipbot
Alexibot Alexibot
Alligator AllSubmitter
AlphaBot AlphaBot
Anarchie
Anarchy
Anarchy99 Anarchy99
Ankit Ankit
Anthill
Apexoo Apexoo
Aspiegel AwarioRssBot
Atomseobot AwarioSmartBot
BDFetch BLEXBot
BackWeb BackDoorBot
Barkrowler Backlink-Ceck
Black Hole Badass
BlackWidow Battleztar Bazinga
Blackboard Blow
BlowFish BlowFish
Bolt Boardreader
BotALot Brandprotect
Brandwatch BuiltBotTough
BuiltWith Bullseye
BuzzSumo BunnySlippers
CODE87 CODE87
CSHttp
Calculon
Cegbfeieh Cegbfeieh
CherryPicker CheTeam
Chlooe Cliqzbot
Citoid
Claritybot
Cloud mapping Cloud mapping
Cogentbot Cogentbot
Copyscape Collector
Copier
Cosmos Cosmos
CrazyWebCrawler
Custo
CyotekWebCopy
DSearch DSearch
Digincore DTS Agent
Dirbuster Demon
Dispatch Deusu
DnyzBot Devil
DomCopBot DomCopBot
DomainAppender DomainStatsBot
Dotbot Dragonfly
Drip EMail Wolf
EasyDL Ecxi
EirGrabber
EroCrawler EroCrawler
Evil Evil
Express WebPictures Express WebPictures
Extreme Picture Finder
EyeNetIE EyeNetIE
Ezooms Fimap
FDM Firefox/7.0
FemtosearchBot FlashGet
Flunky
Fuzz
FyberSpider FyberSpider
Fyrebot G-i-g-a-b-o-t
GT::WWW GT::WWW
GalaxyBot Genieo
GetWeb GetRight
Gigabot GoZilla
Go!Zilla Gotit
Go-Ahead-Got-It Grafula
GrabNet GrapeshotCrawler
Grabber GridBot
HEADMasterSEO
HMView HMView
Heritrix Harvest
Image Fetch HonoluluBot
Image Sucker Humanlinks
Iria HybridBot
IRLbot
IlseBot
InfoTekies
InternetSeer
IstellaBot
JamesBOT JamesBOT
JennyBot
JetCar JetCar
Jetty Jetty
JustView
Jyxobot
Keyword Density
Kinza Kinza
Kozmosbot LWP::Simple
LNSpiderguy Larbin
Lanshanbot
Leap
LeechFTP
LeechGet LeechGet
LexiBot LexiBot
Lftp LibWeb
Libwhisker Libwhisker
Lightspeedsystems LieBaoFast
Likse Linkbot
LinkScan
LinkextractorPro LinkextractorPro
LinqiaMetadataDownloaderBot LinkpadBot
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Litemage_walker MIDown tool
Ltx71 MIIxpc
MSFrontPage MJ12bot
Magnet MQQBrowser
Mail.RU_Bot Mag-Net
MarkWatch Majestic SEO
Masscan Majestic-SEO
MeanPath Bot MarkMonitor
Mass Downloader
Mata Hari
MauiBot
Metauri Metauri
Minefield MicroMessenger
Microsoft Data Access
Mister PiX Mister PiX
Moblie Safari Mojolicious
Mojeek MolokaiBot
Msrabot Mozlila
NICErsPRO Mr.4x3
Name Intelligence Name Intelligence
Nameprotect
Navroad Navroad
NearSite
Needle Needle
NetMechanic NetLyzer
NetSpider NetZIP
NextGenSearchBot Nettrack
Nibbler
Ninja
Nmap Nmap
Nuclei Octopus
Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler Openfind
OpenVAS Openvas
OrangeBot
OrangeSpider OrangeSpider
OutfoxBot OutclicksBot
PageAnalyzer PECL::HTTP
PHPCrawl
PageGrabber PageGrabber
PageScorer
PageThing.com PageThing.com
Papa Foto
Pavuk
Petalbot Petalbot
Pi-Monster Picscout
PleaseCrawl Picsearch
Pockey ProWebWalker
PyCurl Probethenet
RankFlex Pu_iN
RankingBot QueryN Metasearch
Rankivabot Quick-Crawler
RealDownload RankActive
Recorder RankActiveLinkBot
ReGet
RedesScrapy RedesScrapy
RepoMonkey RepoMonkey
Ripper Ripper
RocketCrawler SBIder
SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOprofiler SEOprofiler
SISTRIX
SMTBot SMTBot
ScanAlert ScoutJet
ScreenerBot Scrapy
ScrepyBot
SearchmetricsBot SearchmetricsBot
Seekport Seekport
SemanticJuice SemanticJuice
SemrushBot SemrushBot
Seomoz Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteExplorer
SiteLockSpider SiteSnagger
Sitebeam Sitebeam
SlySearch SlySearch
SmartDownload
SocialRankIOBot SocialRankIOBot
Sottopop Sociscraper
Spammen Sogou web spider
Spbot
Spinn3r Spinn3r
Sqworm Sqlmap
SuperBot Sqlworm
Stripper
Sucuri
SuperHTTP SuperHTTP
Surfbot
Suzuran Suzuran
Swiftbot TeleportPro
Telesoft Telesphoreo
Twice The Intraformant
Titan
Tracemyfile
Turingos
Turnitin
TurnitinBot
TwengaBot
Typhoeus
URLy Warning
UnisterBot UnisterBot
Upflow V-BOT
VB Project VB Project
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
VoidEYE VoidEYE
Voltron WASALive-Bot
WEBDAV WBSearchBot
WWW-Collector-E WPScan
WWW::Mechanize WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers Wallpapers
WallpapersHD WallpapersHD
Web Auto
Web Collage
Web Enhancer
Web Fetch Web Fetch
Web Pix WebBandit
Web Sucker WebCopier
WebCollage
WebFetch
WebFuck WebFuck
WebImageCollector WebGo IS
WebLeacher WebLeacher
WebPix WebPix
WebSauger WebReaper
WebWhacker
Webshag Webshag
WebsiteExtractor Webster
WebsiteQuester
Whacker Whacker
Wotbox Whatweb
YoudaoBot Who.is Bot
Zade Widow
Zitebot WiseGuys Robot
ZmEu Woobot
ZoomBot Wprecon
ZumBot
adscanner adscanner
cah.io.community archive.org_bot
clark-crawler arquivo.pt
backlink-check
com.plumanalytics com.plumanalytics
domainsproject.org crawler.feedback
eCatch crawler4j
facebookscraper
gopher gopher
heritrix heritrix
iubenda-radar instabid
isitwp.com
lwp-trivial
magpie-crawler magpie-crawler
meanpathbot oBot
netEstate NE Crawler plumanalytics
page scorer probe-image-size
s1z.ru ripz
satoristudio.net scalaj-http
scan.lol
seocompany.store seocompany.store
seostar
serpstatbot serpstatbot
sexsearcher sexsearcher
sitechecker.pro sitechecker.pro
sp_auditbot siteripz
spyfu
sysscan sysscan
trendiction.com trendiction.com
trendiction.de voyagerx.com
webgains-bot
webmeup-crawler
webpros.com webpros.com
webprosbot webprosbot
x09Mozilla x22Mozilla
zauba.io zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
camdolls.com abcdefh.xyz
extremepornos.net balook.com
freelifetimefuckbook.com dnsrsearch.com
goodprotein.ru fba-mexico.com
greamimgo.pw fikasound.tk
likrot.com foto-weinberger.at
music7s.me naval.jislaaik.com
portside.cc partner-host.men
vita.com.hr vegascosmetics.ru
wyniki-lista.pl videochat.tv.br

936
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
03e.info 00webcams.com
1688.com 1001desert.com
100searchengines.com
11pikachu.ru
13tabs.com
18ps.ru
1hwy.com
1webmaster.ml
3dracergames.com
3weekdiet.com
4ureyesonly.com 4ureyesonly.com
76brighton.co.uk 5kstore.com
8gold.com 5tey7463.weebly.com
91abcw.com 7ooo.ru
aa8b68101d388c446389283820863176e7.com 8kisses.eu
aaronabel.com 8si.ru
accessoires-mode.in a.frcls.fr
addbin.men acmebtn.ml
ads-cool.pro add-add.men
ads-seo.men addblueoff.com.ua
adf.ly
adobereader-free.ru
adrenalin-stk.ru
ads.tremorhub.com
adserver-e7.com adserver-e7.com
affordablewebsitesandmobileapps.com adsref.men
agardomains.com adtech.de
ageofclones.com akvamaster.dp.ua
agysya.ru alert-fdm.xyz
akita.kz allornamenti.com
aksonural.ru amazingninja.com
aktivator-windows10.blogspot.com apiallgeniusinfo-a.akamaihd.net
aktobe.xkaz.org app-ready.xyz
alessandraleone.com
allcredits.su
allnews24.in
aloofly.com
alyeskaresort.com
am-se.com
am15.net
amateurmatch.com
amazingpic.net
analnoeporno.tv
anapa-inns.ru
anticrawler.org
apartmentratings.com
api.stathat.com
apimountainbikei-a.akamaihd.net
apiqualitinknet-a.akamaihd.net
apper.de apper.de
appfixing.space applyneedy.xyz
atmagroup.ru arate.ru
auto.rusvile.lt arclk.net
autolombard-krasnodar.ru arcteryxstore.online
automatic-seo.com arenanews.com.ua
aviav.co ausergrubhof.info
ayanaspa.com autobudpostach.club
ayeartoforget.com autoua.top
azadnegar.com
azartclub.org
azazaporn.com
azazu.ru
babyguns.ru babyguns.ru
backiee.com balook.com
baladur.ru bangla-chat-uk-paralud.ga
beachtoday.ru basisches-wasser.net
berdasovivan.ru battle.net
best-businessman.ru belinka.com.ua
best-seo-solution.com bestdooz.com
best-way.men bestimagecoollibrary.com
bestserials.com bin-brokers.com
bestssaker.com bio.trade-jp.net
bestwrinklecreamnow.com biographiya.com
bet-prognoz.com bioinnovate.co
big-cash.net blacktwhite.com
bizcheapjerseyswholesalechina.com board.f00d.de
bjgugu.net.ua boo-arts.com
blackwitchcraft.ru bosman.pluto.ro
blue-square.biz brainsandeggs.blogspot.com
bodybuilding-shop.biz bristolhotel.com.ua
bongiornos.info bupropion-sr-150-mg.us
boole.onlinewebshop.net businesn.men
brateg.xyz buy-cheap-online.info
break-the-chains.com buy-meds24.com
bscodecs.com buytizanidineonline.blogspot.com
buketeg.xyz camdenmemorials.com
bukleteg.xyz candelluxsklep.pl
buqayy0.livejournal.com cardsdumps.com
butstrap.space cartierlove2u.com
buyessay3.blogspot.ru cartujano-pre.de
buyparajumpers.online cbozhe.com
buytizanidineonlinenoprescription.blogspot.com cenokos.ru
buzz-porno.info
buzzonclick.com
buzzurl.jp
bycontext.com
c-gainsbourg.com
cablecar.us
candlewooddentalcentre.com.au
canvas.gloverid.site
carson.getenjoyment.net
carstrends2015.com
cartierreplica.xyz
castingbank.ru
cfacarrosserie74.com
cfcl.co.uk cfcl.co.uk
chatroulette.world changde.58.com
cheapbelstaff.online chatroulette.life
cheapjerseysbizwholesale.us chimiver.info
chocolateslim-u-srbiji.com
christianlouboutinreplica.pw christianlouboutinreplica.pw
cialischmrx.com cjs.com.ru
cialiswithoutadoctor.net clean-start.net
cibpenonptib.flu.cc clevernt.com
cindymatches.com co.lumb.co
clash-clans.ru cocyq.inwtrade.com
cleansearch.net cometorussia.net
clickso.com community.allhiphop.com
clippingphotoindia.com compliance-alex.xyz
compliance-don.top compliance-checker.info
compliance-nelson.top compliance-irvin.top
confib.ifmo.ru compliance-jack.top
contextualyield.com compliance-jess.top
cookie-law-enforcement-ff.xyz compliance-julia.top
cookielawblog.wordpress.com compliance-olga.top
coolyarddecorations.com concealthyself.com
courtshipgift.com constantaservice.net
creams.makeforum.eu contentlook.co
credit-online.ws coolingoods.com
culturevie.info counter.yadro.ru
customergrowthsystems.com country-chic.ru
damedingel.ya.ru credit-respect.ru
darmebel.com.ua daneshetabiat.com
data-ox.com dangphoto.trade
datsun-do.com darodar.com
dealighted.com deanmoore.ie
dekoration.us detsky-nabytek.info
dekorkeramik.ru dichvuvesinhhanoi.com
detoxmed24.ru dinosaurus.site
dev.citetick.com directivepub.com
dev78.dioniqlabb.se dnepr-avtospar.com.ua
dieta-personalna.pl dolohen.com
dimfour.com domain-tracker.com
div.as domination.ml
domashneeruporno.com dorratex.tn
dosug-lux.ru downloads-whatsapp.com
downloadkakaotalk.com drunkmoms.net
drugs-no-rx.info earncash.com.ua
dvd-famille.com easycommerce.cf
dvr.biz.ua ebonyporn.site
dynainbox.com
e-commerce-seo.com
e90post.com
ecookna.com.ua
eduserver.net
ee77ee.com ee77ee.com
efnor-ac.com
ege-russian.ru
egvar.net egvar.net
ekosmetyki.net.pl elektrozigaretten1.postbit.com
elektrischezigarette2.devhub.com enge-fotzen.info
elitedollars.com enginebay.ru
eloconcream.blogspot.com enskedesquashclub.se
eluxer.net envaseslotusama.com
en.altezza.travel erotikchat-24.com
encodable.com erotiktreff24.info
eng-lyrics.com eupornstar.info
epngo.bz excitacao.com
eropornosex.ru exdocsfiles.com
erotikstories.ru exlarseva.webblog.es
evening-dating-club.info extads.net
excaliburfilms.com extstat.com
exportshop.us eyelike.com.ua
eyes-on-you.ga ezigaretteshop2.vefblog.net
ezb.elvenmachine.com ezrvrentals.com
ezigarettenkaufen2.dreamwidth.org f012.de
ezigaretteshop.webs.com factorynightclub.com
fba-mexico.com falllow.gq
fermersovet.ru farmprofi.net
fickkontakte.org fatfasts-4tmz.com
fickluder69.com feargames.ru
filesmonster.porn feel-planet.com
filmetricsasia.com fettefrauen.net
filunika.com.ru filkhbr.com
firstdrugmall.ru financepoints.eu
fitfloponline.store fitfloponline.store
fiuxy.com flipper.top
folowsite.com flprog.com
forexgb.ru fm-upgrade.ru
formulaf1results.blogspot.com forex-procto.ru
forumprofi.de forminecrafters.ru
freakycheats.com forum.doctissimo.fr
freenom.link fr.netlog.com
fuckingawesome.com frameimage.org
fuel-gas.com freelifetimefuckbook.com
fun-mobi.pl fsakhalin.ru
funcrushgames.com fyl.com.ru
game-mmorpg.net gaming-journal.com
garciniaxt.us gamingspark.com
gay-file.com gaz-voshod.ru
gdebestkupit.ru ge0ip.org
gearsadspromo.club
geile-lelly.eu geile-lelly.eu
gepezz.info ggiaro.com
get-free-social-traffic.com glasof.es
getlaid-xxxhookupdirect.com globalsurfari.com
gg-arena.ru gloverid.site
gidonline.one go2album.com
gne8.com gok-kasten.net
go2jump.org goldenggames.com
golmau.host.sk
goodhousekeeping.com goodhousekeeping.com
googlepositions.com goodhumor24.com
googleseo.com.tr greendream.com.ua
googlsucks.com greenzaim.ru
gorodservis.ru groupmoney.ru
growshop.es gta-club.ru
gunsvicceadadebt.tk guitar-master.org
guruofcasino.com gymi.name
gwebtools.com.br haikuware.com
handsandlegs.ru hamptonoaks.ca
hcate.com handicapvantoday.com
heartofbeijing.blogspot.com hanwei.us
hebr.myddns-flir.com hermesbracelets.xyz
hermesreplica.win hesteel.pl
hkdiiohi.skyrock.com histock.info
hop.clickbank.net hollywoodactress.info
hoverboard360.es hostnow.men
hothot.ru
hotxnights.info
houdom.net
howlongdoestizanidinestayinyoursystem.blogspot.com
https-legalrc.biz
hulfingtonpost.com hulfingtonpost.com
hully.altervista.org
humanorightswatch.org humanorightswatch.org
hustoon.over-blog.com idegenvezeto.eu
hystersister.com ideibiznesa2015.ru
idc.com.ua ifirestarter.ru
ilo134uloh.com iframe-toloka.com
iideaidekonkatu.info
iklysha.ml
imagecoolpub.com imagecoolpub.com
inbabes.sexushost.com images-graphics-pics.com
incolors.club imagui.eu
indianmedicaltourismshop.com img.wallpaperstock.net
infospot.pt imitex-plus.ru
investpamm.ru immobiliaremassaro.com
invitefashion.com infodocsportal.com
inzn.ru intellego.info
iqoption.pro iplogger.org
itrevolution.cf iqbazar.ru
jagg.info iqs.biz.ua
javcoast.com isoveti.ru
javxxx18.com istizanidineanarcoticdrug.blogspot.com
jrpmakati.com ix20.ru
juliaworld.net ixora.pro
kabbalah-red-bracelets.com jacago.com
javrip.net
je7.us
job.icivil.ir
journeydownthescale.info
jyvopys.com
kakadu-interior.com.ua kakadu-interior.com.ua
keywordsking.com kam-dom.ru
khadastoafarde.tk kamagragelusa.net
kinoduh.ru kamalsinha.com
kinohit1.ru katjimej.blog.fc2.com
koronirealestate.gr kaz.kz
kosova.de kevblog.top
kountrylife.com kimcurlrvsms.com
krasnodar.ru kinobest.pl
kvartiry-remont.ucoz.ru kinoflux.net
klikbonus.com
klosetkitten.com
knogg.net
kol-energo.ru
konturkrasoty.ru
koral.se
kovesszucs.atw.hu
kozhniebolezni.com
krassh.ru
kung-fu-ru.com
labvis.host.sk labvis.host.sk
lacave.ntic.fr lacloop.info
landinez.co landoftracking.com
lazymanyoga.com larose.jb2c.me
le-clos-des-alouettes.com leadn.pl
libertybilisim.com lesbian.xyz
likesdesign.com lineavabit.it
limads.men littlesexdolls.com
linkbuddies.com livefixer.com
liquimondo.com locksmith.jp
listiseltemournlan.gq lollypopgaming.com
live-sexcam.tk lookingglassemb.com
lkbennettstore.online louboutinshoes.xyz
loadopia.com macnewtech.com
locatellicorretor.com.br mail.allnews24.in
lomza.info masthopehomes.com
love-baby.cz medicines-choice.com
lovi-moment.com.ua medispainstitute.com.au
lubetube.com medizinreisen.de
lumb.co mhtr.be
luxmagazine.cf minecraft-neo.ru
mackeeper-center.club mir-business-24.ru
macrotek.ru misswell.net
mamasuper.prom.ua mixx.com
manifestation.betteroffers.review mncrftpcs.com
martlinker.com mnogabukaff.net
matchpal-a.akamaihd.net moesexy.com
mathgym.com.au moetomnenie.com
mbiologi.ru moinozhki.com
mebel-ekb.com
mebelcomplekt.ru
med-zdorovie.com.ua
medi-fitt.hu
medosmotr-ufa.ru
metabar.ru
minyetki.ru
mirobuvi.com.ua
mjchamonix.org
mmostrike.ru
mojowhois.com
mojpregled.com mojpregled.com
mojpreskumanie.com monclerboots.xyz
mokrayakiska.com monetizationking.net
moneyzzz.ru monsterdivx.com
monthlywinners.com moscow-clining.ru
morf.snn.gr mp3downloadhq.com
moscow-region.ru mp3films.ru
mostantikor.ru mrmoneymustache.com
mowser.com mrvideospornogratis.xxx
mp3ritm.top msk-diplomat.com
mrbitsandbytes.com musezone.ru
mttwtrack.com muycerdas.xxx
mturkcontent.com mydoctorok.ru
my-floor.in.ua myhealthcare.com
mysex21.com myindospace.com
mywallpaper.top mylovelibrabry.com
narkologiya-belgorod.ru mystats.xyz
narkologiya-voronezh.ru naughtyconnect.com
naturtreenspicerx.pw naval.jislaaik.com
nero-us.com networkad.net
netcheckcdn.xyz nevansk.ru
networkcheck.xyz newstudio.tv
new-post.tk
newstraveller.ru
nfljerseysforsalewholesaler.com
nicefloor.co.uk
nicovideo.jp
northfacestore.online
nowtorrents.com
nrv.co.za nrv.co.za
nudo.ca nuker.com
nuit-artisanale.com o-o-11-o-o.com
nyfinance.ml office2web.com
o-dachnik.ru okuos.com
obiavo.su olympescort.com
oecnhs.info
ofanda.com
onblastblog.online
onclickpredictiv.com
oneclickfiles.com
onlyforemont.ru
onstrapon.purplesphere.in
openstat.com openstat.com
otbelivanie-zubov.com optitrade24.com
ownshop.cf optom-deshevo.ru
paintingplanet.ru origin-my.ru
pandroid.co osb.se11.ru
partybunny.ru outshop.ru
pekori.to oynat.info
pendelprognos.se paramountmarble.co.uk
partner-cdn.men
penisvergrotendepillennl.ovh
peugeot-club.org peugeot-club.org
pharmacyincity.com pictures-and-images.net
piccshare.com picturesfrom.com
picture-group.com piluli.info
pierrehardysale.online pintattoos.com
piter.xrus.org pinwallpaper.xyz
piulatte.cz pirateday.ru
pix-hd.com pizda.lol
popander.mobi playtap.us
porndl.org poisk-zakona.ru
pornoelita.info popserve.adscpm.net
pornogratisdiario.com poptool.net
pornoklad.ru porno-video-chati.ru
pr0fit-b0x.com pornohub.me
pornosok.ru
pornpost.in
pozvonim.com
presleycollectibles.com
pricheski-video.com pricheski-video.com
privatbank46.ru primedice.com
privatefx-in.ru printdirectforless.com
pro-okis.ru privetsochi.ru
prpops.com prodess.ru
ptr.ruvds.com proxyelite.biz
purplesphere.in proxyradar.com
psa48.ru
pssucai.info
pufip.com
putitin.me putitin.me
qwarckoine.com pyramidlitho.webs.com
rabotaetvse.ru ranking2017.ga
rankingchart.de realitykings.com
ranksays.com redhotfreebies.co.uk
rczhan.com registrysweeper.com
redirect.trafficreceiver.club reimageplus.com
registrydomainservices.com reklamuss.ru
ritlweb.com replicaclub.ru
rn-to-bsn.com restaurantlescampi.com
rucrypt.com riralmolamsaca.tk
ruscopybook.com roma-kukareku.livejournal.com
roofers.org.uk
rossmark.ru
royal-betting.net
runofilms.ru
russia-today-video.ru
saecsa.co saecsa.co
saltspray.ru safe-app.net
sanatorrii.ru sanidumps.com
saneyes.com sanyuprojects.com
santasgift.ml saratov.xrus.org
sarf3omlat.com sc-specialhost.com
savetubevideo.com scanner-alex.top
sbf441.com scanner-julia.top
sbtdesign.co.uk scanner-mark.top
scanner-margo.top seansonline24.pl
sdelai-prosto.ru segol.tv
searchmywindow-a.akamaihd.net semp.net
seasonvar.ru seo-2-0.com
seeingmeerkat.com service.adtech.us
selectads.men sfd-chess.ru
seminarygeorgia59.ga sharpchallenge.com
semxiu.com shijian.ac.cn
senger.atspace.co.uk sigmund-freud.co.uk
seocheckupx.com simple-share-buttons.com
seofied.com sinestesia.host.sk
seotoolsagency.com site-analyzer.com
serving.adbetclickin.pink
sexy-pings.com
sexy-screen-savers.com
sexytrend.ru
shlyahten.ru
sildenafil-tadalafil.info
site-auditor.online site-auditor.online
site-speed-check.site
site.ru site.ru
sixcooler.de siteheart.net
skanninge.se siteonomy.com
sladkoevideo.com sivs.ru
slutloadlive.com skutecznetabletkinaporostwlosow.pl
snomer1.ru slujbauborki.ru
soc-econom-problems.ru social-buttons.xyz
socialbutton.xyz social-s-ggg.xyz
sockshare.net socialmadesimple.com
soft-program.com softonicads.com
soft1.ru
sohoindia.net
solnplast.ru solnplast.ru
songplanet.ru spy-app.info
souvenir.cc stanthonyscatholicchurch.org
spb.afora.ru steebook.com
spl63.fr steelmaster.lv
sport7777.net
sps-shop.com
srdrvp.com
startufa.ru
stathat.com
stauga.altervista.org
sterva.cc sterva.cc
stevemonsen.com sticken.co
stickers-market.ru strangeduckfilms.com
stpicks.com stretchmate.net
supermodni.com.ua stroimajor.ru
svarbit.com stroiminsk.com
swiped.su stroiminsk.org
t3chtonic.com strongsignal-a.akamaihd.net
tacbelarus.ru strv.se
td-l-market.ru studentguide.ru
teesdaleflyballclub.co.uk stuffpride.com
template-kid.com surflinksmedical.com
testbotprocessor44.com swinginwithme.ru
thaisamkok.com swinon.site
thebestweightlosspills.ovh tarad.com
thebitcoincode.com taxi-v-eisk.ru
thenetinfo.com teastory.co
thepornsex.org telefonsex-ohne0900.net
thetattoohut.com tengohydar.tk
titan-ads.life testingads.pro
tizanidine4mgtablets.blogspot.com the-trader.net
tizanidineandgabapentin.blogspot.com thecoral.com.br
tizanidinepain.blogspot.com thedownloadfromwarez.blogspot.com
tizanidinerestlesslegsyndrome.blogspot.com thepokertimer.com
tizanidinevssoma.blogspot.com theporndude.com
thewomenlife.com
timdreby.com
tizanidineduringpregnancy.blogspot.com
tizanidineformigraines.blogspot.com
tizanidinehcl2mgsideeffects.blogspot.com
tizanidinehcl4mgtab.blogspot.com
tizanidinehcl4mgtablet.blogspot.com
tizanidinemusclerelaxant.blogspot.com
tizanidinesleepaid.blogspot.com
tkanorganizma.ru tkanorganizma.ru
tmearegion26.com tmm-kurs.ru
toondinsey.com tn811.us
top10registrycleaners.com tobeyouday.win
top250movies.ru top-study.work
topcasinoratings.ru top10-way.com
topsiteminecraft.com topsy.com
torrent-newgames.com torrent-to-magnet.com
track112.site traffic2money.com
trkur.com trafficshaper.com
truck-addzilla.life truck-land.life
tv-spoty.info tsyndicate.com
tvnewsclips.info twodollarshows.com
twitlinks.com txxx.com
tyumen.xrus.org tyumen.xrus.org
u17795.netangels.ru
ua.tc
ucsol.ru
uhdtv.website
uhod-za-sobakoj.ru
ukkelberg.no
ukrup.com ukrup.com
unimodemhalfduplefw.pen.io uprour.com
unrealcommander.org varikoz24.com
upstore.me
validdomain.xyz
vbabule.net vbabule.net
veselokloun.ru vduplo.ru
video-chat.love veerotech.com
videochat.world video-chat.in
voronezh.xrus.org vigrx-original.ru
vvon.co.uk vladimir.zrus.org
vzlomfb.com voltrknc1.com
vzlomsn.org vrnelectro.ru
warezkeeper.com watchinf.com
warning.or.kr weboptimizes.com
watchmyfb.pl weekes.biz.tc
waterpurifier.club whatsappdownload10.com
webjam.com whereiskentoday.com
website-audit.com.ua whiteproduct.com
websiteexplorer.info wholesalejerseychinaoutlet.com
welcomeauto.ru wholinkstome.com
wfb.hatedriveapart.com winx-play.ru
whatistizanidinehclusedfor.blogspot.com womensterritory.ru
wholesalejerseychinashop.com worldinternetauthority.com
wholesalejerseysgaa.com writersgroup580.web.fc2.com
whos.amung.us wufak.com
word-vorlagen.net www.888.com
worldmusicfests.com www.kabbalah-red-bracelets.com
xn--e1afanlbnfckd7c3d.xn--p1ai www.timer4web.com
xpresscare.ru x5market.ru
yaaknaa.info xmlinde.com
yachts-cruise.info xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
youraticles.pl xn----ctbbcjd3dbsehgi.xn--p1ai
yunque.pluto.ro xn----itbkqkfiq.xn--p1ai
yuweng.info yoluxuryevents.com
zakazfutbolki.com youradexchange.com
youtoner.it
z-master.ru
za-fun-offer.com
zahvat.ru
zaobao.com.sg
zapiszto.pl zapiszto.pl
zarenica.net zero1.it
zeroredirect8.com zeroredirect1.com
zhuravlev.info zigarettenonlinekaufen1.blox.pl
zmoda.hostreo.com zigzog.ru
znaniyapolza.ru znakomstvaonlain.ru
zolotoy-lis.ru zootravel.com
zoompegs.com zrus.org
zs2vm.top
zvetki.ru

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00author.com 0912701309f8ce.com
03p.info 1j7740kd.website
72-news.com 3-letter-domains.net
7ooo.ru 4replicawatch.net
8kisses.eu 9med.net
DomainStatsBot abusora.com
aa14ab57a3339c4064bd9ae6fad7495b5f.com adelly.bg
ad-words.ru ads.gold
adrunnr.com advocatemsk.ru
advancedcleaner.com afftrack001.com
advokat-grodno.by aibolita.com
alibestsale.com alf-img.com
alice.it algarveglobal.com
amateurgalls.com allblogroll.com
apartamentwroclaw.eu alphahoverboards.com
apple.com-webbrowsing-security.review am-se.com
arkkivoltti.net amazingpic.net
artdestshop.eu amyfoxfitness.com
asrv-a.akamoihd.net analytics-ads.xyz
asrvvv-a.akamaihd.net apidigidocketnet-a.akamaihd.net
azartmsl.com appmsr.org
backuperwebcam.weebly.com architecturebest.com
betune.onlinewebshop.net babosas.com
board.f00d.de badavit.com.ua
brothers-smaller.ru baditri.com
browsepulse-a.akamaihd.net bannerbank.ru
bugof.gq batanga.net
battle.net
becuo.com
beget.tech
belsetirehafi.tk
bestwebsitesawards.com
bidbuy.co.kr
billiard-classic.com.ua
bingoporno.com
blockety.co
blog.remote-computer.de
bloggedporn.com
boom-celebs.com
bryansk.zrus.org
buy-cheap-online.info
buypanicdisorderpill.com buypanicdisorderpill.com
cartierbracelet.xyz buzzurl.jp
cartierjusteunclou.xyz cakesplus.com.au
cerev.info cam-kontakte.org
changde.58.com candypeople.se
chatroulette.life casinorewards.info
chiblackhawks-jerseys.com cercacamion.it
chidporn.com chlooe.com
clickprotects.com chocolateslim-original.com
clmforexeu.com chocolateslim-u-srbiji.com
club-lukojl.ru compliance-checker-7.info
club-samodelkin.ru compliance-josh.top
coldfilm.ru connectingsingles.com
compliance-george.top construmac.com.mx
compliance-viktor.top cordstrap.cc
connectionstrenth.com cosmeticswomens-womensports.rhcloud.com
converse.ddsoldes.fr cottageofgrace.com
coolbar.pro date-withme.com
cpabegins.ru detalizaciya-tut.biz
cxpromote.com directivepub.com
dandiyabeats.in dominterior.org
daptravel.com eets.net
data-ox.com elidelcream.weebly.com
detsky-nabytek.info enbersoft.com
digital-video-processing.com ero-advertising.com
disruptingdinnerparties.com euromasterclass.ru
domcran.net everypony.ru
donna7753191.ru evrotekhservis.ru
dostavimvdom.ru exchanges-bet.com
duplicashapp.com ezigarettekaufen1.hpage.com
dvrlists.com f07.de
e2click.com
ekaterinburg.xrus.org
emerson-rus.ru
empirepoker.com
errorfixing.space
exlarseva.webblog.es
ezigarettenkaufen1.pagina.gr
f0815.de
fandlr.com
fastcrawl.com fastcrawl.com
fatmaelgarny.com femdom.twiclub.in
feorina.ru filmfanatic.com
fettefrauen.net firma-legion.ru
flash4fun.com firstsiteguide.com
forminecrafters.ru fitness-video.net
forpostlock.ru floridahuntingfishingadventures.com
forum-engineering.ru forex21.ru
foxweber.com forumprofi.de
fr-bearings.ru freza-sverlo.ru
frcls.fr fungamelands.com
freesitetest.com funnypica.com
frvo.alptandem.ru fyl.com.ru
funnel.co.za gameprimary.com
gambarkataku.co gavuer.ru
gearcraft.us ge0ip.org
generousdeal-a.akamaihd.net gerhardhealer.com
gepezz.info glastecfilms.com.my
girlspicsa.com gorabagrata.ru
gk170.ru
google-liar.ru
gotwebsite1.com
greamimgo.pw greamimgo.pw
greatzip.com
groupmoney.ru groupmoney.ru
gsasearchengineranker.pw hannasolution.ru
gsmtlf.ru hasshe.com
hahashka.ru hifidesign.ru
handicapvantoday.com home.myplaycity.com
hermesreplica.win
hildinghr.se
hopeonthestreet.co.uk
hoste.octopis.com hoste.octopis.com
hotel-mkad.ru hostingclub.lk
hotloans.ru hubbble.com
icaseclub.ru hvd-store.com
iccornacircri.cf ibb.com.ua
imagui.eu ido3.com
img.wallpaperstock.net igrovyeavtomaty777.ru
intellektmedia.at ilmexico.com
imagecoolpub.com
imagine-ex.co
int.search.tb.ask.com
intellego.info
invest-pamm.ru invest-pamm.ru
investpamm.ru investsuccess.org
itbc.kiev.ua johnnyhaley.top
japan-bearings.ru kalandranis.gr
jerseyswholesalechinalimited.com kashubadesign.ru
johannesburgsingles.co.za keywords-monitoring-success.com
junglenet-a.akamaihd.net kinosed.net
keywordspay.com komputers-best.ru
keywordsuggests.com kovrenok.ru
kino2018.cc kupiproday.com.ua
klejonka.info kvartir-remont.biz
kostenlos-sexvideos.com lapitec.eu
kosynka-games.ru law-check-nine.xyz
laexotic.com lider-zhaluzi.kiev.ua
lampokrat.ws lomb.co
law-check-eight.xyz lulea-auktionsverk.se
levaquin750mg.blogspot.com m4ever.net
likesdesign.com macfix.life
limtu.ifmo.ru magazin-pics.ru
linkarena.com magiadinamibia.blogspot.com
lodki-pvh.dp.ua marcoislandvacations.net
loginduepunti.it masturbate.co.uk
mafa-free.com mebel-vstroika.ru
magicplayer-s.acestream.net meteocast.net
makis.nu mhi-systems.ru
managerpak204.weebly.com michellblog.online
marmotstore.online milblueprint.com
martlinker.com militarysale.pro
matematikus.info missvietnam.org
medicine-4u.org mobilemedia.md
medicineseasybuy.com moncleroutletonline.win
memberty.com montredemarque.nl
menhealed.net moomi-daeri.com
mezaruk.info mosrif.ru
miniads.ca mrlmedia.net
monclercheap.xyz musicspire.online
monclervests.xyz mypets.by
mrpornogratis.xxx mysex21.com
mycaf.it nacap.ru
nakozhe.com narco24.me
newfilmsonline.ru netpics.org
nfljerseys.online next-dentists.tk
notfastfood.ru nikitabuch.com
novatech.vn nofreezingmac.work
o-o-11-o-o.com normalegal.ru
nowtorrents.com
nzfilecloud.weebly.com
oecnhs.info
offer.wpsecurity.website
old-rock.com old-rock.com
olympescort.com oralsexfilme.net
oren-cats.ru osagonline.ru
painting-planet.com page2rss.com
pearlisland.ru pavlodar.xkaz.org
percin.biz.ly pc4download.co
perm.xrus.org physfunc.ru
pinstake.com piski.top
plastweb.ru pizdopletka.club
pomoc-drogowa.cba.pl pogosh.com
popup.matchmaker.com poisk-zakona.ru
porndroids.com pontiacsolstice.info
pornoblood.com pony-business.com
pr0fit-b0x.com pornoinn.com
prizefestival.mobi pornotubs.com
prostitutki-rostova.org produkto.net
psvita.ru
pushdata.sendpulse.com pushdata.sendpulse.com
putevka24.ru pussyspace.net
quit-smoking.ga quickbuck.com
randalljhoward.com robot-forex.biz
randki-sex.com rollercoin.com
razyboard.com roma-kukareku.livejournal.com
rebrand.ly romhacking.ru
redirectingat.com
registry-cleaner.net
remont-mobile-phones.ru
remont-ustanovka-tehniki.ru
replicaclub.ru
reversing.cc
risparmiocasa.bz.it
royal-betting.net
rrutw.com
rusexy.xyz rusexy.xyz
s-forum.biz rustic-quiver.win
sad-torg.com.ua rutor.group
rutor.vip
saecsa.co saecsa.co
sdsjweb.com saneitconsulting.com
seoheap.com satoristudio.net
sex-tracker.de scat.porn
shijian.ac.cn securesmrt-dt.com
shoesonlinebuy.cn selectads.men
sierraapps.com seoexperimenty.ru
sofit-dmd.ru seofirmreviewsus.info
softomix.com sexsearch.com
sonata-arctica.wz.cz shoppingmiracles.co.uk
sortthemesitesby.com skyway24.ru
sotechco.co snegozaderzhatel.ru
spb-plitka.ru softxaker.ru
steelmaster.lv soietvousmaime.fr
streamin.to sovetogorod.ru
sygraem.com speedup-my.site
takeflyte.com star61.de
taqplayer.info starwars.wikia.com
taqywu51.soup.io stats-public.grammarly.io
thejournal.ru stmassage.ru
traffic100.com stop-gepatit.te.ua
trafficcentr.xyz strongsignal-a.akamaihd.net
traffpartners.com sukirgenk.dvrlists.com
trustl.life superiends.org
twiclub.in templates.radiodigital.co
unionmarkt.de thefarmergame.com
usswrite.com thenetinfo.com
valoresito.com thepokertimer.com
viagraneggrx.com tizanidineformigraines.blogspot.com
video-chat.cn tizanidinehcl2mgsideeffects.blogspot.com
video-chat.love tizanidinepain.blogspot.com
vipromoffers.com toondinsey.com
visitcambridge.org trahvid.com
vkak.ru turbodsp.com
turn-up-life.life
tx41tclega.ru
uchil.net
ucsol.ru
ukkelberg.no
ukrtvir.com.ua
vacuumscleaner.com
vapsy.com
verymes.xyz
video-hollywood.ru
videos-for-your-business.com
villakohlanta.nu
vksex.ru
voloo.ru
voluumtrk.com
vostoktrade.info
vzlomsn.org
wallpapers-best.com
warezkeeper.com warezkeeper.com
weboptimizes.com website-speed-check.site
weclipart.com wieseversa.no
wellcome2slovenia.ru womensplay.net
westen-z.life wpsecurity.website
whatistizanidine2mg.blogspot.com xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai
wineration.com xn--80aodinpgi.xn--p1ai
withstandingheartwarming.com
womens-journal.net
worst-sites.online
www.labves.ru
wzgyyq.com
x-diesel.org
xaijo.com
xmnb.net
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
xn----btbdvdh4aafrfciljm6k.xn--p1ai
xn----ctbigni3aj4h.xn--p1ai
xvideosporn.biz
yaaknaa.info yaaknaa.info
zaapplesales.blogspot.com yourtemplatefinder.com
zarepta.com youthreaders.com
zdesformula.ru zmoda.hostreo.com
zojirushi-products.ru
zoodrawings.com
zoolubimets.ru
zoominfo.com
zrelaya.pw
zverokruh-shop.cz
zvezdagedon.ru

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.165 0; 10.1.0.102 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3455 ### Version: V4.2022.11.3456
### Updated: Sun Nov 6 22:01:19 UTC 2022 ### Updated: Mon Nov 7 10:04:53 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3455 ### Version: V4.2022.11.3456
### Updated: Sun Nov 6 22:01:19 UTC 2022 ### Updated: Mon Nov 7 10:04:53 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##