vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.04.3063 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-04-23 22:07:21 +00:00
parent 479bb3bfc9
commit 11dde6e4b5
47 changed files with 95246 additions and 95236 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3062 #### Version: V4.2022.04.3063
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3062 #### Version: V4.2022.04.3063
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.04.3062 #### Version: V4.2022.04.3063
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11768
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.04.3062 #### Version: V4.2022.04.3063
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11772
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11772
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11772
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11772
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 1; 10.1.1.86 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 0; 10.1.1.86 0;
127.0.0.1 0; 127.0.0.1 0;

11772
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 1; 10.1.1.86 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 0; 10.1.1.86 0;
127.0.0.1 0; 127.0.0.1 0;

11772
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 1; 10.1.1.86 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 0; 10.1.1.86 0;
127.0.0.1 0; 127.0.0.1 0;

11772
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3062 3063

11772
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11772
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.64.94.131 0; 20.125.40.103 0;

11772
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 1; 10.1.1.86 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 0; 10.1.1.86 0;
127.0.0.1 0; 127.0.0.1 0;

11772
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 1; 10.1.1.86 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 0; 10.1.1.86 0;
127.0.0.1 0; 127.0.0.1 0;

11772
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 1; 10.1.1.86 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 0; 10.1.1.86 0;
127.0.0.1 0; 127.0.0.1 0;

11772
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.64.94.131 0; 20.125.40.103 0;

11772
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.64.94.131 0; 20.125.40.103 0;

11772
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 1; 10.1.1.86 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AiHitBot Chlooe
Harvest Claritybot
NetZIP InfoTekies
Octopus LinkextractorPro
Sqworm LinqiaScrapeBot
VidibleScraper Microsoft URL Control
Webster Mozlila
ips-agent Picsearch
ripz Sucker
scan.lol Xenu

296
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404enemy ALittle Client
80legs ASPSeek
AIBOT
Abonti
Aboundex Aboundex
Aboundexbot
AfD-Verbotsverfahren
AiHitBot
Aipbot
Alexibot Alexibot
AllSubmitter AlphaBot
Apexoo Anarchie
Atomseobot Anarchy
AwarioRssBot Ankit
AwarioSmartBot
BBBike BBBike
BDCbot BackDoorBot
BackStreet BackWeb
Badass Badass
Bigfoot BetaBot
Bitacle BlackWidow
Blow Blow
BlowFish
Bolt Bolt
Brandprotect BotALot
Buddy Brandwatch
Bullseye Buck
CazoodleBot CATExplorador
Cegbfeieh
CensysInspect
Citoid
Claritybot
Cliqzbot
Cocolyzebot Cocolyzebot
Collector Collector
Copier
CopyRightCheck
Cosmos Cosmos
Craftbot
Crawling at Home Project
CrunchBot CrunchBot
CyotekWebCopy CyotekWebCopy
Devil DBLBot
Digincore
DigitalPebble
Dirbuster
Discobot Discobot
Discoverybot Dispatch
DnBCrawler-Analytics DnBCrawler-Analytics
DomainCrawler DomainCrawler
Domains Project Download Wonder
ECCP/1.0 ECCP/1.0
Ecxi EasyDL
EroCrawler EroCrawler
Evil Exabot
Express WebPictures EyeNetIE
Ezooms FemtosearchBot
FDM Fimap
FHscan FlashGet
Flunky
Foobot Foobot
Freeuploader Freeuploader
FrontPage Fyrebot
FyberSpider GT::WWW
GetRight
GetWeb GetWeb
Go!Zilla Getintent
Go-Ahead-Got-It
GoZilla
Gotit Gotit
GrabNet
GrapeFX GrapeFX
HEADMasterSEO HTMLparser
HMView Haansoft
HaosouSpider Hloader
Heritrix
Humanlinks
HybridBot HybridBot
Id-search IDBot
IndeedBot IlseBot
Image Sucker
Indy Library Indy Library
InfoTekies
InterGET InterGET
IstellaBot Iria
JOC Web Spider
JamesBOT JamesBOT
JennyBot JennyBot
JetCar JetCar
Joomla Jetty
Jorgee Jorgee
JustView Keybot Translation-Search-Machine
Kinza Kinza
LNSpiderguy Lanshanbot
LWP::Simple Leap
LeechGet LeechFTP
LexiBot LexiBot
LieBaoFast LinkWalker
Lightspeedsystems
LinkextractorPro LinkextractorPro
LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
Lipperhey Spider LinqiaRSSBot
Litemage_walker LinqiaScrapeBot
Magnet MSIECrawler
MTRobot
Mag-Net
Mail.RU_Bot Mail.RU_Bot
Majestic-SEO Mata Hari
Majestic12
MarkWatch
Mb2345Browser Mb2345Browser
MeanPath Bot
Meanpathbot Meanpathbot
MegaIndex.ru MegaIndex.ru
Metauri MicroMessenger
Microsoft URL Control
Minefield
Moblie Safari Moblie Safari
Mojeek Mojeek
Morfeus Fucking Scanner MolokaiBot
Mozlila Mozlila
Mr.4x3 Msrabot
Musobot Musobot
NICErsPRO
NPbot
Nameprotect Nameprotect
Needle Nessus
NetLyzer NetAnts
NetMechanic NetMechanic
NetZIP NetZIP
Netcraft Netcraft
Not Nettrack
Netvibes
Nibbler
Niki-bot
Nuclei Nuclei
Offline Explorer
Offline Navigator
OnCrawl OnCrawl
OrangeSpider OpenLinkProfiler
OpenVAS
Openfind
OutclicksBot OutclicksBot
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageGrabber PageGrabber
PageThing.com PageThing.com
Pandalytics Pandalytics
Pi-Monster Pi-Monster
Picscout Picsearch
PictureFinder PictureFinder
Pimonster Pixray
PleaseCrawl Pockey
ProWebWalker PxBroker
Probethenet
Pu_iN
PyCurl PyCurl
QueryN Metasearch Quick-Crawler
RSSingBot RankActive
RankActiveLinkBot RankFlex
RankingBot RankingBot
Rankivabot
Re-re Re-re
Recorder ReGet
RedesScrapy RealDownload
Ripper
RocketCrawler RocketCrawler
Rogerbot Rogerbot
SEOkicks-Robot SEOkicks-Robot
SMTBot SEOlyticsCrawler
SalesIntelligent ScanAlert
Scanbot ScoutJet
ScrepyBot ScrepyBot
Seekport Semrush
SemanticJuice SentiBot
SeoSiteCheckup Seomoz
SeobilityBot Shodan
Site Sucker Siphon
SiteLockSpider SiteSnagger
SiteSucker SiteSucker
Sitebeam Sitebeam
SlySearch Siteimprove
SmartDownload Sitevigil
Snoopy Snake
SocialRankIOBot Snapbot
Sociscraper Sogou web spider
Sottopop SpankBot
Spanner Spanner
Spbot Spinn3r
SputnikBot
Sqlmap
Sqworm Sqworm
Steeler Steeler
Stripper Sucker
Surfbot SuperBot
SurveyBot SuperHTTP
Suzuran
Swiftbot Swiftbot
Teleport Teleport
Telesoft TeleportPro
Telesphorep
The Intraformant
TightTwatBot TightTwatBot
Trendictionbot Toata
Turnitin Tracemyfile
Trendiction
TwengaBot TwengaBot
Typhoeus URLy.Warning
URLy Warning UnisterBot
V-BOT Upflow
VB Project VB Project
VCI
Vacuum
VeriCiteCrawler VeriCiteCrawler
Virusdie
Voltron Voltron
WASALive-Bot WISENutbot
WBSearchBot
WPScan WPScan
WWW-Collector-E WWWOFFLE
Wallpapers/3.0
WallpapersHD
WeSEE
WebAuto WebAuto
Web Collage
Web Pix
WebBandit WebBandit
WebCollage
WebCopier WebCopier
WebPix WebFetch
WebSauger WebGo IS
WebImageCollector
WebLeacher
WebReaper
WebStripper WebStripper
WebWhacker WebWhacker
Webster WebZIP
Webalta
WebmasterWorldForumBot
WebsiteExtractor
Whacker Whacker
Whatweb Who.is Bot
WinHTTrack WinHTTrack
Wonderbot WiseGuys Robot
Wotbox
Wprecon Wprecon
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider
Xenu Xenu
YoudaoBot Zade
Zermelo Zauba
Zeus
Zitebot Zitebot
ZoomBot ZoomBot
ZumBot ZumBot
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt cah.io.community
backlink-check check1.exe
clark-crawler clark-crawler
coccocbot coccocbot
cognitiveseo
crawler.feedback crawler.feedback
eCatch
facebookscraper facebookscraper
gopher
instabid instabid
internetVista monitor isitwp.com
ips-agent
linkdexbot
lwp-request
lwp-trivial
mediawords mediawords
netEstate NE Crawler muhstik-scan
page scorer page scorer
pcBrowser s1z.ru
plumanalytics scan.lol
satoristudio.net seostar
seobility serpstatbot
seocompany.store sexsearcher
sitechecker.pro
sogouspider sogouspider
spyfu
sysscan
tAkeOut tAkeOut
trendiction.com
ubermetrics-technologies.com ubermetrics-technologies.com
webgains-bot
webpros.com webpros.com
webprosbot x09Mozilla
x22Mozilla x22Mozilla
zauba.io
zgrab zgrab

320
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 404checker
80legs ALittle Client
AIBOT
Abonti Abonti
AfD-Verbotsverfahren Aboundex
AhrefsBot Acunetix
Alexibot AiHitBot
AllSubmitter Alligator
AlphaBot
Anarchie
Anarchy99 Anarchy99
Anthill Anthill
Apexoo
Aspiegel Aspiegel
Asterias Asterias
Atomseobot
Attach Attach
BBBike BLEXBot
BDFetch BackStreet
BackWeb
Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass
Bandit Bandit
Bitacle Battleztar Bazinga
BlackWidow
Blackboard Blackboard
Blow
Brandprotect
Brandwatch Brandwatch
Buddy BuiltBotTough
BuiltWith CSHttp
BuzzSumo Calculon
CATExplorador Cegbfeieh
CCBot
CensysInspect CensysInspect
Chlooe CheTeam
Citoid CheeseBot
Claritybot CherryPicker
ChinaClaw
Cliqzbot Cliqzbot
Cloud mapping Cloud mapping
Cocolyzebot Collector
Copier Crawling at Home Project
Copyscape DBLBot
CyotekWebCopy
DIIbot DIIbot
DTS Agent DSearch
Demon
Devil
Digincore Digincore
Dirbuster
Discobot Discobot
Discoverybot
Dispatch Dispatch
DomainStatsBot DnyzBot
Domains Project DomainSigmaCrawler
Ecxi Dotbot
EMail Siphon
EasyDL
EirGrabber EirGrabber
Express WebPictures ExtLinksBot
FHscan Ezooms
FemtosearchBot FDM
Firefox/7.0 Firefox/7.0
FlashGet FlashGet
Flunky Flunky
FrontPage Foobot
GT::WWW FyberSpider
GetRight Fyrebot
GoZilla G-i-g-a-b-o-t
Gotit GalaxyBot
GrabNet Genieo
Grabber GermCrawler
GetWeb
Gigabot
GrapeFX GrapeFX
GridBot HEADMasterSEO
HTTP::Lite HTTrack
Haansoft
HaosouSpider
Harvest
Havij
Heritrix Heritrix
HonoluluBot HonoluluBot
HybridBot Humanlinks
IDBot IRLbot
Id-search Id-search
IlseBot IndeedBot
Image Sucker InfoNaviRobot
InfoTekies InfoTekies
Intelliseek Internet Ninja
InterGET Iskanie
InternetSeer
IstellaBot IstellaBot
JOC Web Spider JOC Web Spider
JetCar JamesBOT
JennyBot
Jetty
Joomla Joomla
JustView Keybot Translation-Search-Machine
Jyxobot
Kenjin Spider
Kozmosbot Kozmosbot
LWP::Simple LWP::Simple
Lanshanbot Lanshanbot
LeechGet Larbin
LexiBot Leap
LibWeb LeechFTP
Libwhisker Lftp
Lightspeedsystems
Likse Likse
Linkbot LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Litemage_walker Lipperhey
Ltx71 Lipperhey Spider
MFC_Tear_Sample MIIxpc
MQQBrowser
MSIECrawler MSIECrawler
MTRobot Mag-Net
Mail.RU_Bot Majestic SEO
Majestic12 Majestic12
MauiBot MarkWatch
MicroMessenger Mass Downloader
Mata Hari
Mb2345Browser
MeanPath Bot
MegaIndex.ru
Microsoft Data Access Microsoft Data Access
Minefield
Mister PiX Mister PiX
Mojolicious Morfeus Fucking Scanner
Mr.4x3 Msrabot
NPbot
Name Intelligence
Navroad
Net Vampire Net Vampire
NetAnts NetAnts
NetSpider NetSpider
Netcraft Nibbler
Netvibes Nmap
NextGenSearchBot Octopus
Niki-bot Offline Navigator
Nikto OnCrawl
Ninja
Not
Nuclei
Nutch
OpenLinkProfiler OpenLinkProfiler
OpenVAS
Openfind
Openvas
OrangeBot
OrangeSpider
PECL::HTTP
PHPCrawl
Page Analyzer Page Analyzer
PageGrabber PageGrabber
PageScorer PageScorer
Pavuk Pi-Monster
Petalbot Piepmatz
PictureFinder
Pimonster Pimonster
Pixray Pixray
PleaseCrawl Probethenet
Pockey Psbot
Pu_iN
PyCurl PyCurl
RankFlex RSSingBot
RankingBot
Re-re Re-re
RealDownload ReGet
Reaper Reaper
RebelMouse RebelMouse
Recorder Recorder
SMTBot Ripper
SalesIntelligent Rogerbot
Scanbot SBIder
SEOkicks
SEOstats
ScanAlert
ScoutJet
Scrapy Scrapy
ScreenerBot
ScrepyBot ScrepyBot
Searchestate Semrush
SemrushBot SeobilityBot
SentiBot
SeoSiteCheckup
Seomoz
Siphon Siphon
Site Sucker Site Sucker
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSucker Siteimprove
SmartDownload Snake
Snapbot Snapbot
Sociscraper
Sosospider Sosospider
Sottopop Sottopop
SpaceBison SpaceBison
Spammen
SpankBot SpankBot
Sqlworm Spanner
Stripper SputnikBot
SuperHTTP Sqlmap
Suzuran Sqworm
TeleportPro Sucker
The Intraformant Sucuri
Thumbor SuperBot
Toweyabot Surfbot
Tracemyfile SurveyBot
Swiftbot
Szukacz
Teleport
Telesphorep
Trendiction Trendiction
True_Robot Turingos
Turnitin TwengaBot
URLy Warning Twice
URLy.Warning Upflow
UnisterBot V-BOT
VB Project VB Project
VCI
Vacuum
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VidibleScraper
VoidEYE VoidEYE
Voil
Voltron Voltron
WASALive-Bot WASALive-Bot
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE Web Auto
WeSEE Web Fuck
Web Enhancer
Web Pix Web Pix
Web Sucker Web Sucker
WebAuto WebBandit
WebCollage WebFuck
WebEnhancer WebPix
WebFetch WebStripper
WebGo IS WebWhacker
WebReaper WebmasterWorldForumBot
WebZIP Website Quester
Webshag Webster
Whack Whack
Whatweb
Who.is Bot Who.is Bot
Widow WinHTTrack
WiseGuys Robot
Wonderbot
Wprecon
Xaldon WebSpider
Xenu
Zade
Zauba Zauba
Zeus
Zitebot
ZmEu
ZoomBot
ZumBot ZumBot
archive.org_bot cah.io.community
arquivo.pt clark-crawler
backlink-check
check1.exe
coccocbot coccocbot
cognitiveseo
crawler.feedback crawler.feedback
eCatch domainsproject.org
evc-batch
facebookscraper facebookscraper
gopher
heritrix heritrix
internetVista monitor isitwp.com
iubenda-radar
lwp-request
magpie-crawler
meanpathbot
mediawords mediawords
muhstik-scan
netEstate NE Crawler netEstate NE Crawler
oBot oBot
plumanalytics page scorer
pcBrowser
s1z.ru s1z.ru
seoscanners scan.lol
serpstatbot sogouspider
sexsearcher tAkeOut
siteripz trendiction.com
sp_auditbot
sysscan
ubermetrics-technologies.com ubermetrics-technologies.com
webgains-bot
webprosbot webprosbot
x09Mozilla x09Mozilla
x22Mozilla xpymep1.exe
zauba.io zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
aviav.co alienwheel.es
civilwartheater.com aucoinhomes.com
cypernhuset.se bigcities.org
enbersoft.com bodybuilding-shop.biz
fermersovet.ru clickso.com
ghazel.ru frankofficial.ru
nodding-passion.tk lashstudia.ru
ops.picscout.com nippon-bearings.ru
pcads.ru openmultipleurl.com
remont-comp-pomosh.ru wheelchairliftsarea.com

930
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com 1001watch.com.ua
1bet.com
1millionusd.xyz 1millionusd.xyz
4kpics.rocks 2kata.ru
4ur.click 3th.co.in
55wmz.ru 3weekdiet.com
8kisses.eu 4rent.ru
91abcw.com 7a2rnuey1tw9ar.ru
aa9d046aab36af4ff182f097f840430d51.com abcdeg.xyz
aac52006ec82a24e08b665f4db2b5013f7.com
activecampaign.dreamhosters.com
adcash.com adcash.com
adsloads.com adelly.bg
advancedsoftwaresupport.com adexprts.com
affrh2015.com adprotect.net
aimayangzhi.com adsref.men
all4invest.info advokateg.xyz
allcredits.su adzerg.com
alldezservice.kz affiliate-fr.com
allwomen.info afftrack001.com
amoi.tn akuhni.by
albamargroup.com
allbizne.co.ua
alpharma.net
alta-realestate.com
anabolics.shop anabolics.shop
androids-store.com animal36.com
anlimebel.kiev.ua apessay.com
apelsinnik.website apiallgeniusinfo-a.akamaihd.net
apibetweenlinesn-a.akamaihd.net apidigidocketnet-a.akamaihd.net
appfastplay.com apper.de
applicationg29.com arquivo.pt
arcadepages.com ascat.porn
arendas.net asrvvv-a.akamaihd.net
arius.tech atelielembrancaqueencanta.com.br
artefakct.com atout-energie-69.com
artel116.ru
asacopaco.tk
asia-forum.ru
australia-opening-times.com
autoblogger24.info
aviav.ru
avon-severozapad.ru
avtoarenda.by
azartniy-bonus.com azartniy-bonus.com
bad-stars.net azbukafree.com
bankcrediti.ru bamo.xsl.pt
bcmp.org basedecor.ru
beauty-clinic.ru beachtoday.ru
belinvestdom.by beatifullandscape.co
berrymall.ru bestofupload.info
bestmouthwash.club besttorrentknifta.weebly.com
bestwaystogetpaid.us bingoporno.com
bfz.biz
bio.trade-jp.net bio.trade-jp.net
biocn.dx.am blackle.com
bioscorp.ru blackplanet.com
biteg.xyz blogporn.in
bjorkbacken.nu bluerobot.info
blackcurranthumidifiers.site bmsco.co
bluejays-jerseys.us boleznikogi.com
bmw-ark.ru bongacams.com
bolegapakistan.com bookmark4you.com.biz
bookmark4you.com boostmyppc.com
bpro1.top brateg.xyz
bracketsmackdown.com brus.city
brandbucket.com buketeg.xyz
brandimensions.com bum.com.ru
browsepulse-a.akamaihd.net bupropion-sr-150-mg.us
buqyxa.rincian.info burkesales.com
buyk.host.sk buscarfoto.com
buypillsorderonline.com buttons-for-your-website.com
buyviagraa.blogspot.com buytizanidineonline.blogspot.com
buzz-porno.info c-english.ru
bytimedance.ru cactussoft.biz
calstaterealty.us cakemediahq.com.au
camsex.xxx-cam.webcam californianews.cf
candy-glam-hp.com
canzoni.ru
carabela.com.do
carina-sy.de carina-sy.de
cars-modification.net cars-modification.net
caveavins.fr carsplate.com
cbozhe.com cartierreplica.top
cete.ru cashonads.com
chatroulette.world cbcseward.com
cezartabac.ro
cgi2.nintendo.co.jp
chastnoeporno.com
chatroulette.video
cheapcigarettesc.info
cheatcode-lita12.rhcloud.com
check-host.net check-host.net
chelyabinsk.xrus.org checkhit.com
cherubinimobili.it chocolateslim-u-srbiji.com
chocolateslim-en-espana.com chuckguilford.com
chocolateslim-original.com cialgenisrx.com
ciekawinki.pl
civilwartheater.com
claytransformations.info claytransformations.info
clodo.ru cleanersoft.com
community.allhiphop.com clicksor.com
compliance-donald.xyz coindirect.io
compliance-irvin.top compliance-brian.top
compliance-viktor.top compliance-viktor.top
cookie-law-enforcement-ee.xyz compucelunlock.net
conciergegroup.org
contentsexpress.com
coolingoods.com
coprofam.org coprofam.org
copypaste.traderzplanet.in copypaste.traderzplanet.in
copyrightclaims.org copyrightclaims.org
corta.co counter.yadro.ru
cougfan.info countercrazy.com
cranly.net crackzplanet.com
credit-online.ws craftinsta.ru
csgo4.win cutt.us
cupday.com date-withme.com
customerpromos-a.akamaihd.net dealwifi.com
cyclobenzaprinevstizanidine.blogspot.com degerlund.net
dangphoto.trade delfin-aqua.com.ua
data.vtc.pw descargar-musica-gratis.net
davidovic.info
daydream-studio.ru
decenttools.com
decoratinghomes.ga
delitime.info
dfwu1013.info dfwu1013.info
diarioaconcagua.com dickssportinggoods.com
dichvuvesinhhanoi.com diegolopezcastan.com
dinosaurus.site diusyawiga.tk
dlya-android.org diy-handmade-ideas.com
dms-sw.ru dkvorota.ru
dnepropetrovsk.mistr-x.org dogbreedspicture.net
dogclothing.store dogoimage.com
dorratex.tn dokumentalkino.net
dotmass.top domainroam.win
downloadeer.net domashneeruporno.com
drugs-no-rx.info dominateforex.ml
dstroy.su domination.ml
earl-brown.info drillsaw.ru
ekto.ee druzhininevgeniy63.blogspot.com
elektrischezigarette1.blog.pl dscaas.website
elite-sex-finders.com ed-shop01.ru
educontest.net
election.interferencer.ru
elektrischeziga.livejournal.com
elidelcream.weebly.com
elitedollars.com elitedollars.com
elvenar.com energydiet-info.ru
empathica.com eng-lyrics.com
enge-fotzen.info enternet.ee
englishgamer.com epngo.bz
eropornosex.ru ereko.ru
erotik-kostenlos.net eropho.com
estibot.com erot.co
euromasterclass.ru escortplius.com
everflora.ru etur.ru
extractorandburner.com everytalk.tv
falllow.gq exci.ru
favorcosmetics.com exonline.info
fikasound.tk expresstoplivo.ru
findpik.com extrabot.com
fishingwholesale.us extremal-blog.com
fishtauto.ru extremepornos.net
flashahead.info ezigarettekaufen2.blox.pl
floating-share-buttons.com ezigarettekaufen2.mpbloggar.se
flowersbazar.com ezigarettenkaufen1.deviantart.com
fm-upgrade.ru ezigaretteshop.webs.com
foto-weinberger.at ezigaretteshop2.mywapblog.com
freegamesplay.online f012.de
freenode.info facialporntube.com
freevpn.space failingmarriege.blogspot.com
froggytube.com familienzahnaerzte.com
fanoboi.com
fantasticpixcool.com
fast-wordpress-start.com
fastcrawl.com
femmesdenudees.com
filmgo.ru
forums.toucharcade.com
fr-bearings.ru
franch.info
front.ru
fyxabomiw.ru fyxabomiw.ru
gael-s.ru
galeon.com galeon.com
galeria-zdjec.com gambarkatabaru.com
geile-lelly.eu gameprimary.com
geoads.com games.kolossale.ru
get-free-social-traffic.com gamewrath.com
getdot.ru ge0ip.net
gheus.altervista.org geckoandfly.com
gktt.ru geileweiber.tk
glasof.es generalporn.org
googglet.com getmyads24.com
gotorussia.com getridofstretchmarks.org
ghostvisitor.com
gidonline.one
glavtral.ru
gobongo.info
goforexvps.com
gonextmedia.com
goodnightjournal.com
gopixdatabase.com
gowreckdiving.com
gq-catalog.gq gq-catalog.gq
gratuitxblcodes.com greatzip.com
grtyi.com guge.io
hanink.biz.ly guildebzh.info
hausfrauensex18.com happy.new.yeartwit.com
hccoder.info healmytrauma.info
hdpixent.com helicalpile.us
headpharmacy.com
healgastro.com
helpmymacfaster.trade helpmymacfaster.trade
helvetia.com.ua
hermesbirkinhandbagoutlets.com hermesbirkinhandbagoutlets.com
hexpilot.com hifidesign.ru
hit-men.men hornymatches.com
hitcpm.com hostcritique.com
home.myplaycity.com hoterika.com
homedecorpicture.us hotsocialz.com
homelygarden.com htmlcorner.com
hostsshop.ru hully.altervista.org
hotenergy.ru hunterboots.online
hotshoppymac.com husky-shop.cz
hotxnights.info i4track.net
house.sieraddns.com ilikevitaly.com
hoverboard360.at images.gyffu.com
hoverboard360.se in-tandem.co
howopen.ru inbabes.sexushost.com
howtowhitenteethfast.xyz indetiske.ya.ru
iboss.com inet-traffic.com
id-forex.com infobanks.ru
igithab.com infospot.pt
imagecoolpub.com infostatsvc.com
immigrational.info investingclub.ru
indiakino.net investmac.com
infobabki.ru inzn.ru
infodocsportal.com isotoner.com
inspiring-desperate.tk j33x.com
intelhdgraphicsgtdrive6w.metroblog.com javidol.site
intimshop-fantasy.ru
iqoption-bin.com
iqoption.pro
iscblog.info
isitpaleo.info
itsdp3.com
jacago.com
jackwolfskinoutlet.online
javatex.co.id
javstock.com javstock.com
jjbabskoe.ru jestr.org
justbcause.com jobmarket.com.ua
kalb.ru jpcycles.com
kandidos.com kaac.ru
kanimage.com kazan.xrus.org
karpun-iris.ru kchaxton.com
kasino-money.pw kerch.site
ketrzyn.pl ketrzyn.pl
keywords-monitoring-success.com keywordsking.com
kimcurlrvsms.com kinocccp.net
kino-rating.ru kinostorm.net
kino-rf.ru kinozapas.org
kinopolet.net kiprinform.com
kirov.zrus.org kirov.zrus.org
kiskinhouse.com klin3952.ru
kuzinsp.ru konoplisemena.com
lafriore.ru koronirealestate.gr
laparfumotec.com krasnodar.xrus.org
laptoper.net lafourmiliaire.com
lenpipet.ru larose.jb2c.me
lider-zhaluzi.kiev.ua law-check-two.xyz
linkarena.com lazy-z.com
linkdebrideur.xyz lazymanyoga.com
linkredirect.biz leftofcentrist.com
lionking-1994.blogspot.com lego4x4.ru
listiseltemournlan.gq lernur.net
liveshoppersmac.com letolove.ru
livingcanarias.com lezbiyanki.net
lostaruban.com linerdrilling.com
lotto6888.com linkbuddies.com
louboutinreplica.pw linkpulse.com
luckyshop.net.ua lirunet.ru
macrotek.ru live-sexcam.tk
mafa-free.com live-sexchat.ru
magnetic-bracelets.ru loadopia.com
mansiondelrio.co long-beach-air-conditioning.com
lovelycraftyhome.com
lowephotos.info
lyngdalhudterapi.no
m5home.ru
mafcards.ru
magazintiande.ru
make-money-online.com
malwareremovalcenter.com
marktforschung-stuttgart.com
matb3aa.com matb3aa.com
meble-bogart.info mcadamssupplyco.com
mecash.ru medi-fitt.hu
medicines-choice.com megakino.net
mesbuta.info mesto-x.com
michaelkorsoutletstore.net meteostate.com
minecraft-rus.org miniads.ca
mmoguider.ru missvietnam.org
mojowhois.com mmostrike.ru
mokrayakiska.com mobicover.com.ua
montredemarque.nl mobilierland.com
morepoweronmac.trade modabutik.ru
moscow-region.ru moesen-ficken.com
moskva.nodup.ru moncleroutletonline.pw
mossmesi.com moncleroutletonline.xyz
mrbojikobi4.biz moneyviking-a.akamaihd.net
mrsdalloways.com more-letom.ru
mundoaberrante.com mosrif.ru
music7s.me moviezbonkerssk.cf
musicdaddy.net moxo.com
musicktab.com mp3films.ru
mxgetcode.com mp3ritm.top
mybackgroundlandscape.blogspot.com musflashtv.com
mybinaryoptionsrobot.com mustwineblog.com
myblogregistercm.tk muzaporn.com
mygameplus.com mydirtystuff.com
myshopmatemac.com myfreetutorials.com
myyour.eu mylovelibrabry.com
namecrumilchlet.tk namenectar.com
naperehresti.info natali-forex.com
narkologiya-belgorod.ru net-profits.xyz
narkologiya-orel.ru networkad.net
negociosdasha.com nextbackgroundcheck.gq
neks.info niroo.info
nelc.edu.eg nvformula.ru
netallergy.ru o-o-6-o-o.com
netanalytics.xyz odoratus.net
nextrent-crimea.ru ofanda.com
nosecret.com.ua offtopic.biz
notasprensa.info oklogistic.ru
novgorod.xrus.org one-gear.com
novosti-hi-tech.ru onemactrckr.com
nudo.ca online247.ml
o.light.d0t.ru ooo-gotovie.ru
oakleyglassesonline.us
oda.as
odesproperty.com
offer.camp
onko-24.com
onload.pw
openmultipleurl.com
opinionreelle.com
oralsexfilme.net oralsexfilme.net
orel-reshka.net osoznanie-narkotikam.net
orenburg-gsm.ru
p-business.ru p-business.ru
pandarastore.top painting-planet.com
park.above.com papasdelivery.ru
partner-high.men perm.xrus.org
pawli.eu pestomou.info
pdn-4.com petrovka-online.com
perm-profnastil.ru photorepair.ru
perosan.com
perso.wanadoo.es
pharmacyincity.com
photosaga.info photosaga.info
php-market.ru phrcialiled.com
pics-group.com pic-re.blogspot.com
picphotos.net
picturesmania.com picturesmania.com
pintattoos.com pills24h.com
pl.id-forex.com pinkduck.ga
playfortuna-play.ru pinwallpaper.top
poptool.net pisanieprac.info
pornhubforum.tk pixshark.com
porno-home365.com pizdeishn.com
pornohub.me pizza-imperia.com
pizza-tycoon.com
pk-services.ru
pl-top.pl
pogosh.com
poisk-zakona.ru
pooleroadmedicalcentre.co.uk
popup-hgd.xyz
porndroids.com
pornokorol.com
pornoslive.net
pornoted.com pornoted.com
pornotubexxx.name portal-eu.ru
portadd.men portside.xyz
posible.net pravoholding.ru
pozdrawleniya.com printie.com
pregnant.guru privhosting.com
pretty-mart.com prodess.ru
princeadvantagesales.com productarium.com
privatov-zapisi.ru promalp-universal.ru
prospekt-st.ru prostitutki-kharkova.org
prostitutki-tolyatti.org psa48.ru
purplestats.com
puteshestvennik.com puteshestvennik.com
py100.ru puzzleweb.ru
qld10000.net pwwysydh.com
raisedseo.com qwesa.ru
ranksonic.info rankia.com
rapidokbrain.com ranksays.com
rcb101.ru rapidsites.pro
realnye-otzyvy.info razvratnoe.org
reckonstat.info reeyanaturopathy.com
recordpage-a.akamaihd.net regionshop.biz
regdefense.com replicalouboutin.xyz
reimageplus.com restaurantlescampi.com
reklamuss.ru retailwith.com
renecaovillasale.online reversing.cc
responsive-test.net rocketchange.ru
robot-forex.biz rubbed.us
roofers.org.uk
ros-ctm.ru
rrutw.com
runtnc.net runtnc.net
rusoft-zone.ru ruscoininvest.company
russintv.fr rusexy.xyz
saddiechoua.com s-forum.biz
saneyes.com sakhboard.ru
sarahmilne.top sampleletters.net
satellite.maps.ilovevitaly.com saneitconsulting.com
sbwealthsolutions.ca sanjosestartups.com
scanner-ivan.top sarm.tk
scanner-olga.top sbornik-zakonov.ru
scanner-walt.top scanner-alexa.top
sdelatmebel.ru scanner-andrew.top
sdi-pme.com scanner-don.top
se-welding.ru scanner-julia.top
seansonline24.pl
search.1and1.com
searchaddis.com searchaddis.com
seinterface.com searchimpression.com
selectads.men seeingmeerkat.com
service-core.ru semprofile.com
sexkontakte-seite.com serptehnika.ru
sexkrasivo.net sexs-foto.com
sexspornotub.com sexuria.net
sfj-ror.no shama-rc.net
shop.acim.org shlyahten.ru
sispe.com.br shohanb.com
site3.free-share-buttons.com sibecoprom.ru
skladvaz.ru simple-image.com.ua
smart-balancewheel.com sinestesia.host.sk
socialbuttons.xyz site-analyzer.com
socialmonkee.com sitevalued.com
soheavyblog.com smallseotools.com
soserfis.com social-s-ggg.xyz
sovetogorod.ru social-s-hhh.xyz
soviet-portal.do.am socialseet.ru
spabali.org sofit-dmd.ru
space-worry.ml sohoindia.net
speechfoodie.com soundfrost.org
spl63.fr spacash.com
spyfu.com spasswelt.xyz
srgwebmail.nl spb.afora.ru
sstroy44.ru sport7777.net
stairliftstrue.com spymac.net
startwp.org srecorder.com
steamoff.net staticfs.host
stickers-market.ru strana-krasoty.ru
stockspmb.info streamin.to
store-rx.com styro.ru
strigkaomsk.ru suchenindeutschland.com
supermama.top sukirgenk.dvrlists.com
support.nopeas.sk surintech.ac.th
surcentro.com svetlotorg.ru
svetodiodoff.ru svolze.com
svtrd.com takeprofitsystem.com
t3chtonic.com tarad.com
tako3.com tasteidea.com
tattoo33.ru tedy.su
telesvoboda.ru telesvoboda.ru
texbaza.by thedownloadfreeonlinegames.blogspot.com
thin.me.pn thenews-today.info
ticketsys.inetwd.com timhost.ru
tiens2010.ru tizanidine4mgprice.blogspot.com
tizanidineandcipro.blogspot.com tizanidineforopiatewithdrawal.blogspot.com
tizanidineinfo.blogspot.com tizanidinehcl4mgtabinfo.blogspot.com
tizanidineonline.blogspot.com tizanidinehclsideeffects.blogspot.com
tizanidinepain.blogspot.com tizanidinepills.blogspot.com
tizanidinerecreationaluse.blogspot.com todohr.com
tizanidinestreetprice.blogspot.com toon-families.com
tizanidinezanaflex.blogspot.com top-deal.com.pl
tk-assortiment.ru top-karkas.ru
track-rankings.online topcasinoratings.ru
trafaret74.ru torrent-newgames.com
trafficmania.com torrent-to-magnet.com
tripper.de torture.ml
trubywriting.com toy-shop.top
tsyndicate.com toys.erolove.in
turkeyreport.tk track112.site
uchebavchehii.ru trafficshaper.com
udayavani.com trkur.com
udsgame.online truck-addzilla.life
uggbootsoutletsale.us truemfilelj.gq
uptime-gamma.net tuberkuleznik.ru
urldelivery.com tversvet.ru
urll.eu tvteleport.ru
urzedowski.eu typimga.pw
vegan-foods.us uhod-za-sobakoj.ru
viandpet.com unlimitdocs.net
videochat.life unrealcommander.org
vikistars.com urlopener.com
vip2ch.com vduplo.ru
visa-pasport.ru velpanex.ru
viagengrarx.com
video-camer.com
videochaty.ru
videosbox.ru
vilingstore.net
villacoloniale.com
vipromoffers.com
virus-schutzmasken.de
vkmusics.ru vkmusics.ru
vse-pesni.com vozbujdenie.com
vtcdns.com wardreapptokone.tk
vzlom-na-zakaz.com warning.or.kr
wallpapersist.com weburlopener.com
waterefficiency.co
webcamtalk.net
webmonetizer.net
webstatsdomain.org
wechatdownload10.com
weclipart.com weclipart.com
whengirlsgowild.com wemarketing.se
wholesalejerseychinashop.com westsextube.com
wholesalenfljerseys.us.com westvilletowingservices.co.za
withstandingheartwarming.com wfb.hatedriveapart.com
wladimirpayen.com wholesalejerseyscheapjerseys.us.com
woman-tampon.ru woman-orgasm.ru
wordpresscore.com worldtraveler.world
www.labves.ru wpthemedetector.co.uk
www.wohnkabinen-shop.de wtsindia.in
wwwadultcheck.com www.kabbalah-red-bracelets.com
wzgyyq.com x-true.info
x-lime.net xchat26.myfreecams.com
x-mix.info xn--d1abj0abs9d.in.ua
xaylapdiendanang.com xn--l1aengat.xn--p1ai
xerox-douglas.cf xnxxandxvideos.com
xgftnlrt.bloger.index.hr xtraffic.plus
xmladserver.com xxlargepop.com
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai xxxnatelefon.ru
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai yaderenergy.ru
xn----7sbifcamovvfggw9d.xn--p1ai yadro.ru
xn----itbkqkfiq.xn--p1ai yellowads.men
xn--80aaagvmjabrs1aoc9luc.xn--p1ai youdao.com
xn--b1adccaf1bzj.xn--p1ai yourporn.com
xn--lifehacer-1rb.com yur-p.ru
yachts-cruise.info zahvat.ru
yaminecraft.ru zajm-pod-zalog-nedvizhimosti.ru
yottos.com zeleznobeton.ru
youjizz.vc zeroredirect1.com
yunque.pluto.ro zigarettenonlinekaufen1.bloog.pl
zagadki.in.ua zixizop.net.ru
zatjmuzu.info zoompegs.com
zebramart.ru
zeroredirect10.com
zeroredirect12.com
zigarettenonl.canalblog.com
zigzog.ru
zoogdiesney.com
zoogdinsney.com
zreloeporno.tv

480
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
101billion.com 101raccoon.ru
1stat.ru 1flag.co.za
66cpwgln.space 1kdailyprofit.me
aab94f698f36684c5a852a2ef272e031bb.com 1pamm.ru
adf.ly 2015god.org
adman.se 2345.com
adminshop.com 2rich4bitches.com
admitad.com 3xforum.ro
adventureparkcostarica.com aaronabel.com
adzpower.com adhome.biz
affiliate-fr.com admeasures.com
afftrack001.com adssafeprotected.com
alarmobninsk.ru alkoravto.ru
albamargroup.com allknow.info
alfabot.xyz analytics-ads.xyz
allesohnegirls.net angigreene.com
analnoeporno.tv animal-drawings.com
apparel-offer.com animal36.com
approved.su apioutoboxnet-a.akamaihd.net
aramaicmedia.org apiwoodensealcom-a.akamaihd.net
artavenuegardenstudios.com arraty.altervista.org
artdeko.info asdfz.pro
asia-forum.ru asiengirls.net
autogrep.ru autochoixspinelli.com
autowebmarket.com.ua autovideobroadcast.com
avkzarabotok.com avtolombard-krasnodar.ru
azadnegar.com azartniy-bonus.com
azlex.uz backlink4u.net
babyguns.ru backlinkwatch.com
bestbrainprod.win bambi.ck.ua
bestdraws.com beauty-clinic.ru
beta.hotkeys.com besthoro.ru
bif-ru.info binaryoptionscops.info
binomo.com bio-optomarket.ru
bitcoinpile.com bmw-ark.ru
blackcurranthumidifiers.site bochemit.com.ua
boltalko.xyz brainzpod.win
bracketsmackdown.com break-the-chains.com
brandedleadgeneration.com buy-loft.ru
business-suggested.tk buyantiviralwp.com
cah.io.community cablecar.us
callawaygolfstore.online cacheimages.com
capsderfudd.tk cactussoft.biz
casablancamanor.co.za calc-for-credit.ru
celebsopics.com capture-room.com
cenokos.ru carsdined.org
cfjump.com carstrends2015.com
charmstroy.info cbozhe.com
cheatcode-lita12.rhcloud.com centrdebut.ru
check-this-out-now.online
chorus.terakeet.com
cienum.fr cienum.fr
citizenclsdriveri7.pen.io cityadspix.com
cleanallspyware.com civilwartheater.com
clksite.com clicksor.net
cntravelre.com clickzzs.nl
codysbbq.com club-lukojl.ru
compliance-barak.xyz compliance-alexa.xyz
computernetworksonline.com cool-wedding.net
cookielawblog.wordpress.com cowblog.fr
corta.co
crackguru.tk crackguru.tk
crafthubs.com cryptoswap.biz
cupday.com cutt.us
ddlmega.net cv.wallhade.co
detsky-nabytek.info czeshop.info
datsun-do.com
delitime.info
deutschland123.de
dev.citetick.com
dipstar.org dipstar.org
disruptingdinnerparties.com discountliv.com
divku.ru diy-handmade-ideas.com
domini.cat documentbase.net
donvito.unas.cz dojki-hd.com
dosug-lux.ru doktoronline.no
e-c.al dstroy.su
egvar.net dumpsmania24.com
elektrischezigaretten1.blogse.nl eaplay.ru
eloxal.ru ekosmetyki.net.pl
elstal.com.pl eldiariodeguadalajara.com
elvenmachine.com elektrischezigarette2.onsugar.com
erotikstories.ru eloconcream.blogspot.com
etenininrade.ga emailaccountlogin.co
eurosamodelki.ru englishgamer.com
fanrto.com estibot.com
fashionavenuegame.com ewebarticle.info
fertilitetsradgivningen.se failingmarriege.blogspot.com
fetishinside.com feargames.ru
fingerprintjs.com filkhbr.com
fix-website-errors.com filmfanatic.com
floridamhca.org fototravel.eu
flprog.com fres-news.com
foxweber.com gallictures.com
freefoto.ca gaming-journal.com
funnel.co.za getresponse.com
gamedayassist.com girlgamerdaily.com
gardene.ru gpms.org.my
gavuer.ru gsbs.com.ua
gdzkurokam.ru guardlink.com
get-free-social-traffic.com handicapvantoday.com
glcomputers.ru highspeed5.net
glogow.pl home.myplaycity.com
gombita.info hotel-mkad.ru
goo.ne.jp hully.altervista.org
goosefishpost.bid iboss.com
gratuitbaise.com ihc2015.info
greatdealshop.com images-free.net
guiadeserraazul.com infogame.name
hahashka.ru infoupdate.org
hitsbox.info insider.pro
hiwibyh.bugs3.com interesnie-faktu.ru
hopeonthestreet.co.uk intimshop-fantasy.ru
howopen.ru isitpaleo.info
iblogpress.xyz isotoner.com
iframe-toloka.com itsdp3.com
ikritikimou.gr izhevsk.zrus.org
ilmexico.com jav-fetish.com
indiakino.net jobgirl24.ru
investpamm.ru karachev-city.ru
iptool.xyz karusel-market.ru
iqs.biz.ua kochanelli.com
irkutsk.zrus.org labelwater.se
istizanidineanarcoticdrug.blogspot.com larose.jb2c.me
ivanovo.zrus.org legalrc.biz
iwantedmoney.com legionalpha.com
jmat.cn lenpipet.ru
kadashihotel.com lexiangwan.com
kaidalibor.de livetsomudvekslingstudent.bloggersdelight.dk
kenaba.su luckyday.world
kerei.ru lutherstable.org
keywordhut.com m.b00kmarks.com
kfon.eu masthopehomes.com
laserpen.club mathgym.com.au
lechtaczka.net mature.free-websites.com
lflash.ru medi-fitt.hu
liky.co.ua millionare.com
listiseltemournlan.gq
locksmith.jp
long-beach-air-conditioning.com
losangeles-ads.com
macfix.life
martlinker.com
masterseek.com
medkritika.ru
meduza-consult.ru
megakino.net
metaxalonevstizanidine.blogspot.com
misterjtbarbers.com misterjtbarbers.com
moetomnenie.com mixtapetorrent.com
mrvideospornogratis.xxx mncrftpcs.com
mtmtv.info mobifunapp.weebly.com
mygameplus.com mole.pluto.ro
nacap.ru moneyviking-a.akamaihd.net
nagdak.ru morepoweronmac.trade
narco24.me mountainstream.ms
narkologiya-orel.ru mulberryoutletonlineeu.com
noscrapleftbehind.co musicdaddy.net
nufaq.com muz-tracker.net
nvformula.ru mydirtystuff.com
o333o.com navalwiki.info
office-windows.ru net-profits.xyz
online-podarki.com news-readers.ru
openmediasoft.com nfljerseys.online
oren-cats.ru nonews.co
osoznanie-narkotikam.net novodigs.com
penisvergrotendepillennl.ovh panouri-solare-acoperis.com
pensplan.com pinstake.com
photokitchendesign.com pixell.club
phrcialiled.com pk-pomosch.ru
pic2fly.com pl-vouchers.com
pojdelo.weebly.com play.leadzupc.com
pooleroadmedicalcentre.co.uk popup-jdh.xyz
poppen-nw.net pornobrazzers.biz
porno-raskazy.ru pr0fit-b0x.com
pornofiljmi.com pro-tec.kz
pornolook.net profolan.pl
pornoreino.com prostitutki-tolyatti.org
princeadvantagesales.com purplestats.com
ptr.ruvds.com quick-seeker.com
ranking2017.ga relax.ru
regionshop.biz remmling.de
registry-cleaner.net remont-fridge-tv.ru
replica-watch.ru remorcicomerciale.ro
robotixix.com retreatia.com
rozalli.com sad-torg.com.ua
sampleletters.net santechnik.jimdo.com
sandhillsonline.com sanyuprojects.com
sarf3omlat.com savingsslider-a.akamaihd.net
scanmarine.info
scanner-elena.top
scanner-nelson.top
scenicmissouri.us scenicmissouri.us
sell-fb-group-here.com scmor.ilxc.cc
semp.net screentoolkit.com
shemale-sex.net search-goo.com
sherlock.se searchmywindow-a.akamaihd.net
shopvilleroyboch.com.ua seo-tools-optimizing.com
shtaketniki.ru shlyahten.ru
sientalyric.co sivs.ru
sildenafil-tadalafil.info skincrate.net
simple-image.com.ua small-games.biz
socialtrade.biz social-vestnik.ru
softwaretrend.net sockshare.net
softxaker.ru specialfinanceoffers.com
spaceshipad.com sptslmtrafms.com
supermama.top spy-sts.com
terraclicks.com spywareit.com
theheroes.ru spywarespy.com
time-japan.ru stairliftsarea.com
top1-seo-service.com starpages.net
staynplay.net
stop-zavisimost.com
store-rx.com
stromerrealty.com
tastyfoodideas.com
tedy.su
telefonsexi.com
tiens2010.ru
tizanidine6mg.blogspot.com
tnaionline.org
top10-online-games.com top10-online-games.com
torrent-newgames.com totu.us
tourcroatia.co.uk trafficcentr.xyz
toursmaps.com trafficjunky.net
tracksz.co tri-slona.org
trainoffend.ml tripper.de
trustl.life twiclub.in
tuberkulezanet.ru twodollarshows.com
turizm.bz undergroundcityphoto.com
tvnewsclips.info vacuumcleanerguru.com
ua-company.ru virtuagirl.com
ugogo.info
ukrtvory.in.ua
univerfiles.com
unrealcommander.biz
userequip.com
valsalud.com
veselokloun.ru
via-gra.webstarts.com
vids18.site
vilingstore.net
virus-respirators.com virus-respirators.com
vrotike.ru visa-china.ru
vtcdns.com viveresaniesnelli.it
vzubah.com vykupavto-krasnodar.ru
wallpaperaccess.com vysigy.su
vzlomtw.com
vzube.com
webmasterhome.cn webmasterhome.cn
website-speed-check.site westum.se
wjgony.com whatsappdownload10.com
worldoffiles.ru winx-play.ru
wpthemedetector.co.uk witherrom55.eklablog.fr
works.if.ua
wttavern.com
wufak.com
wyniki-lista.pl wyniki-lista.pl
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai x-diesel.biz
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai xjlottery.com
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
xn--80aanaardaperhcem4a6i.com xn----9sbebi2bvzr7h.xn--p1ai
xn--oogle-wmc.com xn--b1adccaf1bzj.xn--p1ai
xsfetish.org xxx-cam.webcam
xxxdatinglocal.us
yaoguangdj.com
yellowfootprints.com
yottos.com
youporn-ru.com youporn-ru.com
za-fun-offer.com youradexchange.com
zeikopay.com yourserverisdown.com
zeroredirect9.com zeleznobeton.ru
zoompegs.com zeroredirect2.com
zytpirwai.net zone-kev717.info
zscalertwo.net
zverokruh-shop.cz

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.54 0; 10.1.1.86 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3062 ### Version: V4.2022.04.3063
### Updated: Sat Apr 23 10:03:55 UTC 2022 ### Updated: Sat Apr 23 22:01:26 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3062 ### Version: V4.2022.04.3063
### Updated: Sat Apr 23 10:03:55 UTC 2022 ### Updated: Sat Apr 23 22:01:26 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##