vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3468 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-13 10:09:20 +00:00
parent 092f3e2db1
commit 501d3d487d
47 changed files with 90457 additions and 90447 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3467 #### Version: V4.2022.11.3468
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3467 #### Version: V4.2022.11.3468
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3467 #### Version: V4.2022.11.3468
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11170
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3467 #### Version: V4.2022.11.3468
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11174
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11174
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11174
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11174
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 1; 10.1.0.124 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 0; 10.1.0.124 0;
127.0.0.1 0; 127.0.0.1 0;

11174
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 1; 10.1.0.124 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 0; 10.1.0.124 0;
127.0.0.1 0; 127.0.0.1 0;

11174
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 1; 10.1.0.124 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 0; 10.1.0.124 0;
127.0.0.1 0; 127.0.0.1 0;

11174
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3467 3468

11174
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11174
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.65.107.190 0; 104.214.66.189 0;

11174
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 1; 10.1.0.124 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 0; 10.1.0.124 0;
127.0.0.1 0; 127.0.0.1 0;

11174
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 1; 10.1.0.124 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 0; 10.1.0.124 0;
127.0.0.1 0; 127.0.0.1 0;

11174
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 1; 10.1.0.124 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 0; 10.1.0.124 0;
127.0.0.1 0; 127.0.0.1 0;

11174
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.65.107.190 0; 104.214.66.189 0;

11174
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.65.107.190 0; 104.214.66.189 0;

11174
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 1; 10.1.0.124 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Cosmos BDCbot
Curious Cogentbot
SocialRankIOBot FDM
Sqworm IRLbot
Sucuri NetSpider
Wprecon ScreenerBot
Zitebot SemrushBot
magpie-crawler Shodan
satoristudio.net Wotbox
siteripz lwp-trivial

316
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404checker
360Spider 404enemy
Aboundex 80legs
Acunetix ADmantX
AIBOT
ALittle Client
AfD-Verbotsverfahren
AiHitBot
Aipbot Aipbot
Alexibot
AlphaBot
Anarchie Anarchie
Anarchy
Anarchy99 Anarchy99
Anthill Aspiegel
Asterias Attach
Atomseobot
AwarioSmartBot AwarioSmartBot
BBBike BDCbot
BackDoorBot BDFetch
BackStreet BLEXBot
BacklinkCrawler BacklinkCrawler
Bandit Badass
Barkrowler Barkrowler
BatchFTP
Battleztar Bazinga
Bigfoot
Bitacle Bitacle
Black Hole
BlackWidow
Boardreader Boardreader
Bolt Brandprotect
Brandwatch Buck
Buddy BuiltWith
BuiltBotTough Bullseye
BuzzSumo BunnySlippers
CATExplorador
CCBot
Calculon
CheTeam CheTeam
ChinaClaw CheeseBot
CherryPicker
Chlooe Chlooe
Claritybot Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot CrazyWebCrawler
Collector DIIbot
Copier DSearch
CopyRightCheck Demon
CrunchBot
Custo
DBLBot
DataCha0s
DatabaseDriverMysqli
Deusu Deusu
Disco DigitalPebble
DomainCrawler Dispatch
DomainSigmaCrawler DnyzBot
Domains Project Download Wonder
ECCP/1.0 Dragonfly
EMail Wolf Drip
Ebingbong
EirGrabber EirGrabber
Evil Exabot
Extractor
EyeNetIE
FHscan FHscan
FemtosearchBot FemtosearchBot
FrontPage Flunky
Fuzz Foobot
GT::WWW
GermCrawler
GetRight
Go!Zilla Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
Grabber Gotit
GrabNet
Grafula
GrapeFX
GridBot
HEADMasterSEO HEADMasterSEO
HTTP::Lite HMView
HTTrack HTTrack
Haansoft HaosouSpider
Havij Hloader
Heritrix Humanlinks
IDBTE4M IDBot
Iblog Image Fetch
IlseBot
Image Sucker
InfoNaviRobot InfoNaviRobot
Intelliseek InterGET
InternetSeer JOC Web Spider
JetCar JennyBot
JikeSpider Jorgee
Joomla
JustView
Keyword Density
LWP::Simple LWP::Simple
Lanshanbot Leap
Larbin
Lftp Lftp
LibWeb LibWeb
Lightspeedsystems Libwhisker
Likse LieBaoFast
LinkScan
LinkWalker LinkWalker
LinksManager LinkpadBot
LinqiaRSSBot LinqiaMetadataDownloaderBot
Lipperhey Spider Lipperhey
Ltx71 Ltx71
MFC_Tear_Sample
MIDown tool
MIIxpc MIIxpc
MJ12bot
MQQBrowser MQQBrowser
MTRobot MTRobot
Mag-Net
Magnet Magnet
Mail.RU_Bot MarkMonitor
Majestic SEO Mass Downloader
MarkWatch
Masscan Masscan
Mata Hari Mata Hari
MegaIndex.ru Mb2345Browser
Metauri MeanPath Bot
MicroMessenger Meanpathbot
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Mister PiX Mister PiX
Moblie Safari Morfeus Fucking Scanner
Mojolicious
MolokaiBot
Mozlila Mozlila
Mr.4x3
Musobot Musobot
NPbot
Name Intelligence Name Intelligence
NearSite Navroad
Needle Needle
NetAnts Net Vampire
NetLyzer NetMechanic
NetSpider
NetZIP NetZIP
Netcraft
Nettrack Nettrack
NextGenSearchBot NextGenSearchBot
Nibbler
Niki-bot Niki-bot
Nikto Nimbostratus
Ninja
Nmap
Not Not
Octopus Offline Explorer
Offline Navigator OnCrawl
OpenVAS OpenLinkProfiler
Openvas Openvas
OrangeSpider OrangeBot
Pi-Monster OutclicksBot
Piepmatz PECL::HTTP
PHPCrawl
PageScorer
Pandalytics
Papa Foto
Pimonster Pimonster
Psbot Psbot
PxBroker Pu_iN
Pump
QueryN Metasearch
Quick-Crawler
RSSingBot RSSingBot
RankFlex RankActive
RankActiveLinkBot
RankingBot RankingBot
Rankivabot Rankivabot
RankurBot RankurBot
ReGet
RebelMouse RebelMouse
RepoMonkey
Ripper
RocketCrawler
Rogerbot
SBIder SBIder
SEOprofiler SEOkicks
SISTRIX SISTRIX
SMTBot SMTBot
SalesIntelligent
ScoutJet ScoutJet
Scrapy Screaming
ScreenerBot
Searchestate
Seekport
SemanticJuice
Semrush
SemrushBot
SentiBot SentiBot
SeobilityBot SeobilityBot
Shodan Shodan
Site Sucker
SiteExplorer
SiteSnagger
SiteSucker
Sitebeam Sitebeam
Sitevigil Snapbot
SmartDownload
Snake
Snoopy Snoopy
SocialRankIOBot
Sociscraper Sociscraper
Sogou web spider
Sosospider Sosospider
SpaceBison
Spbot Spbot
Spinn3r Sucker
Sqlmap
Sqworm
Stripper
Sucuri Sucuri
SuperHTTP SurveyBot
Surfbot Suzuran
Swiftbot Swiftbot
Szukacz Szukacz
T8Abot
Telesoft
Telesphorep
The Intraformant
Thumbor Thumbor
Trendiction TightTwatBot
Trendictionbot Toata
Turingos
TwengaBot
Twice Twice
URLy Warning URLy.Warning
VCI VB Project
Vacuum Vagabondo
VeriCiteCrawler VeriCiteCrawler
VidibleScraper VidibleScraper
Voil
WASALive-Bot WASALive-Bot
WBSearchBot
WPScan WPScan
WWW-Mechanize WWW-Mechanize
WWWOFFLE WWW::Mechanize
Wallpapers
Wallpapers/3.0
WallpapersHD WallpapersHD
Web Collage Web Auto
Web Enhancer
Web Fuck Web Fuck
Web Sauger Web Sucker
WebAuto WebAuto
WebBandit
WebFuck
WebLeacher
WebPix WebPix
WebStripper WebReaper
WebWhacker WebSauger
WebZIP Webalta
WebmasterWorldForumBot
Website Quester Website Quester
WebsiteExtractor
WebsiteQuester WebsiteQuester
Who.is Bot Webster
Whatweb
Widow Widow
WinHTTrack
WiseGuys Robot WiseGuys Robot
Wonderbot
Wotbox
Xaldon WebSpider Xaldon WebSpider
YoudaoBot
Zade
Zeus Zeus
Zitebot Zitebot
ZmEu ZoominfoBot
ZumBot ZumBot
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt check1.exe
cah.io.community coccocbot
cognitiveseo
com.plumanalytics
crawl.sogou.com crawl.sogou.com
domainsproject.org crawler4j
eCatch dataforseo.com
gopher heritrix
internetVista monitor internetVista monitor
ips-agent isitwp.com
iubenda-radar linkdexbot
page scorer lwp-trivial
pcBrowser magpie-crawler
polaris version muhstik-scan
s1z.ru netEstate NE Crawler
scalaj-http probe-image-size
seocompany.store scan.lol
seobility
seostar
sexsearcher sexsearcher
siteripz
sp_auditbot sp_auditbot
spyfu
sysscan
trendiction.com trendiction.com
webgains-bot voyagerx.com
webmeup-crawler
webpros.com webpros.com
x09Mozilla
x22Mozilla
xpymep1.exe
zgrab zgrab

290
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404enemy
AIBOT AIBOT
Abonti Abonti
Aboundex
AhrefsBot AhrefsBot
AllSubmitter AllSubmitter
Anarchy99 Alligator
Ankit AlphaBot
Anarchie
Anarchy
Anthill
Aspiegel Aspiegel
AwarioSmartBot Atomseobot
BackStreet BLEXBot
Backlink-Ceck BackWeb
Badass
Bandit
Barkrowler
BatchFTP
Battleztar Bazinga
BetaBot BetaBot
Bigfoot Bigfoot
Bitacle
Black Hole Black Hole
Blackboard Blackboard
BlowFish
Bolt Bolt
Brandprotect Brandprotect
Brandwatch
Buck Buck
Buddy BuiltBotTough
BuiltWith
Bullseye Bullseye
BunnySlippers
BuzzSumo BuzzSumo
CATExplorador CazoodleBot
CensysInspect Cegbfeieh
CheTeam
CheeseBot CheeseBot
Citoid ChinaClaw
Cloud mapping Cliqzbot
Cogentbot
Collector Collector
Copier
Craftbot Craftbot
Custo Curious
CyotekWebCopy DTS Agent
DBLBot
DatabaseDriverMysqli DatabaseDriverMysqli
Demon
Deusu
Devil
Digincore Digincore
DigitalPebble Dirbuster
Discobot Disco
Discoverybot
Dispatch
DomainSigmaCrawler DomainSigmaCrawler
Download Wonder Drip
ECCP/1.0 ECCP/1.0
EMail Siphon Ecxi
Extractor EirGrabber
Evil
Exabot
ExtLinksBot
Extreme Picture Finder
FDM FDM
Firefox/7.0
FlashGet FlashGet
Flunky Freeuploader
Foobot
FrontPage FrontPage
G-i-g-a-b-o-t
GT::WWW GT::WWW
Genieo
GermCrawler GermCrawler
GetRight GetRight
GetWeb GetWeb
Getintent Gigabot
Go!Zilla
GoZilla GoZilla
GrabNet Grabber
Grafula
HMView HMView
Haansoft HTMLparser
HTTrack
HaosouSpider
Havij Havij
Heritrix Heritrix
HonoluluBot Hloader
Humanlinks
HybridBot
IDBTE4M
IDBot IDBot
IRLbot Id-search
Image Sucker Intelliseek
IndeedBot
InterGET InterGET
Internet Ninja Internet Ninja
InternetSeer
Iskanie Iskanie
IstellaBot Jbrofuzz
JamesBOT JetCar
Jetty JikeSpider
Kinza Joomla
Kozmosbot
LNSpiderguy LNSpiderguy
Lanshanbot LWP::Simple
Larbin LeechFTP
LexiBot LexiBot
Lftp
LibWeb LibWeb
Libwhisker
Lightspeedsystems Lightspeedsystems
Likse
LinkScan
LinkWalker
Linkbot Linkbot
LinkextractorPro
LinkpadBot
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey
Lipperhey Spider
Ltx71 Ltx71
MQQBrowser MIIxpc
MTRobot MTRobot
Mag-Net Mag-Net
Magnet Majestic SEO
Majestic12 Majestic-SEO
MarkWatch MarkMonitor
Mass Downloader
Mata Hari
Mb2345Browser
MeanPath Bot
Metauri Metauri
Microsoft URL Control MicroMessenger
Mojolicious Minefield
MolokaiBot
Mozlila Mozlila
Mr.4x3 Msrabot
Musobot Musobot
NearSite NearSite
Needle Needle
Nessus Net Vampire
NetAnts
NetLyzer NetLyzer
NetMechanic Netcraft
Nettrack Nettrack
Netvibes
Nibbler Nibbler
NimbleCrawler Nikto
Nimbostratus Nimbostratus
Nmap Nmap
Octopus
Offline Explorer Offline Explorer
OpenLinkProfiler Offline Navigator
OpenVAS
Openfind
Openvas
OrangeBot OrangeBot
OrangeSpider OrangeSpider
OutclicksBot OutclicksBot
PageThing.com PHPCrawl
Panscient Page Analyzer
PageAnalyzer
PageScorer
Papa Foto Papa Foto
Petalbot
Pi-Monster
Picscout Picscout
Picsearch
Pixray Pixray
PleaseCrawl
ProWebWalker ProWebWalker
Probethenet PyCurl
PxBroker
QueryN Metasearch QueryN Metasearch
RSSingBot
RankActive
RankFlex RankFlex
RankingBot RankingBot
RankingBot2 RepoMonkey
Rankivabot Ripper
RankurBot
ReGet
RebelMouse
Recorder
RocketCrawler RocketCrawler
Rogerbot SEOkicks
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SISTRIX
SEOstats ScoutJet
SMTBot
Scrapy Scrapy
ScreenerBot
SearchmetricsBot
Semrush Semrush
SemrushBot Seomoz
SeobilityBot Shodan
Siphon Site Sucker
SiteCheckerBotCrawler SiteExplorer
SiteSucker
Sitebeam
Siteimprove
SlySearch SlySearch
SmartDownload
Snake
Snoopy
SocialRankIOBot
Sociscraper Sociscraper
Sogou web spider Sogou web spider
Sottopop
Spammen
SpankBot
Spanner Spanner
Spbot Spbot
Spinn3r Spinn3r
Sqlmap SputnikBot
Sqlworm Sqworm
Stripper
Sucker
Sucuri
SuperBot SuperBot
SurveyBot Surfbot
Szukacz TeleportPro
T8Abot Thumbor
Teleport
Telesoft
Telesphoreo
TightTwatBot TightTwatBot
Toata Tracemyfile
Toweyabot Trendictionbot
Turingos
TurnitinBot
TwengaBot TwengaBot
Twice
Typhoeus Typhoeus
URLy.Warning UnisterBot
VB Project VB Project
VidibleScraper VCI
Voil VelenPublicWebCrawler
WASALive-Bot VeriCiteCrawler
WBSearchBot WBSearchBot
WEBDAV
WPScan WPScan
WWW::Mechanize
WWWOFFLE
Wallpapers/3.0
WeSEE
Web Collage
Web Enhancer Web Enhancer
Web Fetch Web Sauger
Web Pix Web Sucker
WebBandit WebAuto
WebCopier WebFuck
WebFetch
WebLeacher WebLeacher
WebPix
WebReaper WebReaper
WebSauger
WebStripper WebStripper
WebsiteQuester WebWhacker
Webster Webalta
WebmasterWorldForumBot
Webshag
Website Quester
Whack
WiseGuys Robot WiseGuys Robot
Wonderbot
Wprecon Wprecon
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Xenu Zitebot
YoudaoBot
Zade
Zauba
Zeus
ZmEu ZmEu
ZoomBot ZoomBot
ZoominfoBot ZumBot
cah.io.community adscanner
autoemailspider
check1.exe check1.exe
dataforseo.com clark-crawler
domainsproject.org crawler4j
eCatch
gopher gopher
heritrix
instabid instabid
lwp-request ips-agent
iubenda-radar
lwp-trivial lwp-trivial
netEstate NE Crawler magpie-crawler
mediawords
page scorer page scorer
pcBrowser
plumanalytics plumanalytics
polaris version
probe-image-size
s1z.ru
satoristudio.net
scalaj-http
seobility seobility
seoscanners seostar
serpstatbot sexsearcher
sitechecker.pro
sogouspider sogouspider
trendiction.de voyagerx.com
webmeup-crawler
webprosbot webprosbot
x09Mozilla
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
adloads.net addbin.men
cbcseward.com airmaxshoes-2016.com
changde.58.com applyneedy.xyz
cienum.fr hothot.ru
gomusix.com mxgetcode.com
hdseriale.pl ragecash.com
investzalog.ru remote-dba.de
jikoman.info sureone.pro
usacasino.com xtube.com
wladimirpayen.com zambini.ru

932
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com 03p.info
101flag.ru 123kuma.com
108shot.com 1314dh.com
123any.com 178evakuator178.ru
18ps.ru 1bet.com
21h2o.com 2015god.org
256bit.by 2728fb936f0.com
2kata.ru
2nt.ru
300richmond.co.nz 300richmond.co.nz
5000-cotydzien.com 4webmasters.org
5elementov.ru 5forex.ru
6128786.com 5kstore.com
7a2rnuey1tw9ar.ru 7wind.ru
7search.com
883zy.com 883zy.com
aa625d84f1587749c1ab011d6f269f7d64.com 8lufu.com
aa9bd78f328a6a41279d0fad0a88df1901.com a.pr-cy.ru
aa9d046aab36af4ff182f097f840430d51.com aadroid.net
adhome.biz above.com
adman.se abusora.com
admatic.com.tr acads.net
adminshop.com actionnooz.com
adpremium.org adcash.com
ads.gold addtoadd.men
adtech.fr ads-seo.men
adultactioncam.com adult-shop.com.ua
adultnet.in adultmeetonline.info
adzerg.com
adzpower.com
ageofclones.com ageofclones.com
ahmedabadwebs.com air-edem.ru
aihelen.net akuhni.by
alert-fdm.xyz aliexpresscashback.club
alfa9.com alive-ua.com
alienwheels.de allblogroll.com
alif-ba-ta.com allboard.xobor.de
alldezservice.kz allknow.info
allnews24.in
allwidewallpapers.com allwidewallpapers.com
alphacarolinas.com amazingpic.net
anal-acrobats.hol.es amt-k.ru
ananumous.ru anaseracresar.tk
apartamentwroclaw.eu andpolice.com
apibetweenlinesn-a.akamaihd.net android-systems.ru
apilinkswiftco-a.akamaihd.net animalrank.com
apparel-offer.com anmysite.com
appearance-cool.com apibatbrowsecom-a.akamaihd.net
applicationg29.com apidigidocketnet-a.akamaihd.net
arpelsreplica.xyz appfastplay.com
asscenihotosidea.blogspot.co.za arius.tech
atleticpharm.org asophoto.com
asrv-a.akamaihd.net
astana.xxxkaz.org
aucoinhomes.com aucoinhomes.com
autobudpostach.club auspolice.com
availit.weebly.com autowebmarket.com.ua
ayanaspa.com avramstroy.ru
azartniy-bonus.com bannerspace.com
azinoofficial777.ru bestimagecoollibrary.com
bablonow.ru bestsexyblog.com
bad-stars.net beta.hotkeys.com
bag77.ru bezlimitko.xyz
bannerswap.com bg6s0.com
battlecarnival.su bikini-image.com
best-coupon-offer.com bioca.org
bestkfiledxd.cf bitcoinremote.com
besttorrentknifta.weebly.com blackbabesporn.com
betshuckclean.com blog.axant.it
betterscooter.com bmw-ark.ru
bibys.com bnt-team.com
big-cash.net bonvillan.com
bitporno.sx braindaily.xyz
bizfly.info brothers-smaller.ru
bizru.info btvn.ru
bjorkbacken.nu buchananshardware.com
bloggers.nl buehne-fuer-menschenrechte.de
bobba.dzaba.com burger-imperia.com
boc.kz butstrap.space
bonanza-fortune.men bzero1jewelry.net
bonkers.name candypeople.se
bookmaker-bet.com canrioloadm.gq
boostmyppc.com cartierbraceletsreplica.pw
bracketsmackdown.com cartierlove2u.xyz
bradleylive.xyz cartiertrinity.xyz
braintobe.win cbb1smartlist12.click
brk-rti.ru cdncash.org
browserprotecter.com cementaresearch.se
buy-loft.ru cenoval.ru
buyantiviralwp.com censys.io
buyessaynow.biz cgi2.nintendo.co.jp
buzzzg.men chatroulette.si
bvlgariring.xyz chatroulette.video
cakemediahq.com.au cheapsergiorossi.online
calcularpagerank.com cheapwebsitehoster.com
callejondelpozo.es
cam-kontakte.org
camel-beach.com
candcstuccoandstone.com
canzoni.ru
capture-room.com
carrauterie.be
cartechnic.ru
catalogs-parts.com
cezartabac.ro
chatroulette.world
check-host.net check-host.net
checkpagerank.net checkm8.com
chorus.terakeet.com cherubinimobili.it
chiblackhawks-jerseys.com
christianlouboutinreplica.pw
chuckguilford.com
citetick.com
citysecurity.nu
clayvasedesigns.tk clayvasedesigns.tk
clicksotrk.com
clknsee.com clknsee.com
clksite.com clodo.ru
compliance-andrew.top coindirect.io
compliance-barak.xyz colehaanoutlet.store
compliance-willy.top colobit.biz
comsysnet.com compliance-don.xyz
contentsexpress.com compliance-irvin.xyz
cornomase.win compliance-ivan.top
countbertwistdisp26.soup.io compliance-margo.top
counter.bloke.com conocer-sanabria.com
cowlmash.com counter.yadro.ru
crackguru.tk cracksplanet.com
culturevie.info craftburg.ru
customboxes4less.com crafthubs.com
customcollegeessays.net crirussian.ru
customerguru.in cwetochki.ru
customsua.com.ua dalavia.ru
cutt.us danceuniverse.ru
dangphoto.trade daretodonate.co
decenttools.com darodar.com
dengi-pod-zalog-nedvizhimosti.ru data-ox.com
deniven.1bb.ru daydream-studio.ru
diegolopezcastan.com db.speedup.gdn
digitalassetmanagement.site decoratinghomes.ga
dinkolove.ya.ru decorazilla.com
dneprsvet.com.ua dekoration.us
dogsrun.net delayreferat.ru
dominateforex.ml delitime.info
downloads-whatsapp.com delta-line.men
draniki.org deplim.com
dscaas.website dfiles.me
dumuelave.xyz digital-sale.su
e-avon.ru directrev.com
edwinkonijn.com.au discovertreasurenow.com
electronicadirect.com divatraffic.com
elkacentr.ru djihispano.com
emediate.eu dmmspy.com
enternet.ee documentsite.net
erectile.bid doktoronline.no
ero-advertising.com domain-tracker.com
eropho.net domain.webkeyit.com
erotiktreff24.info domainsphoto.com
eshop.md domik-derevne.ru
eupornstar.info dorratex.tn
evening-dating-club.info dosug-lux.ru
everypony.ru download-of-the-warez.blogspot.com
drivotracker.com
dwomlink.info
e-collantes.com
e-kwiaciarz.pl
e-poker-2005.com
eas-seo.com
ednorxmed.com
eduserver.net
egovaleo.it
electricwheelchairsarea.com
elektronischezi.livejournal.com
elitepcgames.com
embedle.com
empathica.com
enthuse.computernetworksonline.com
eonpal.com
erotik0049.com
es5.com
evidencecleanergold.com
ewebarticle.info ewebarticle.info
excitacion.info
expertblog.info expertblog.info
extrabot.com ezigarettekaufen2.mpbloggar.se
ezigarettekaufen2.yolasite.com ezigarettenshop2.postbit.com
f012.de facebook-mobile.xyz
farm26.ru fashion-mk.net
fashionindeed.ml fashionindeed.ml
fast-wordpress-start.com fba-mexico.com
fellowshipoftheminds.com fil.ru
femdom.twiclub.in filefilter.weebly.com
fetishinside.com filmi-onlain.info
ff30236ddef1465f88547e760973d70a.com firesub.pl
fickenprivat.info firstdrugmall.ru
fifa-coins.online flash4fun.com
filesmonster.porn
filmetricsasia.com
filmfanatic.com
findanysex.com
finteks.ru
finuse.com
flashbannernow.com flashbannernow.com
formseo.com flauntyoursite.com
foto-sisek.porngalleries.top florida-tourism.net
foto-weinberger.at focalink.com
free-deals.faith forensicpsychiatry.ru
freelotto.com forex.osobye.ru
freeseedsonline.com forpostlock.ru
freewebs.com forum69.info
frighteningremain.cf forumprofi.de
front.ru free-stock-illustration.com
g33.org free-today.com
gavuer.ru friendflnder.com
gaygalls.net frivgame250.com
genericlowlatencyasiodriverhq.aircus.com fx-brokers-review.com
germes-trans.com gallery.rennlist.com
girlgamerdaily.com gate5.co.za
gktt.ru gay-site.store
glopages.ru gcup.ru
gok-kasten.net geilehausfrauen.net
goldpanningtools.com getresponse.com
golmau.host.sk ghazel.ru
goodwinmetals.co gigixo.com
googlsucks.com go2album.com
gratuitbaise.com gojiberriess.apishops.ru
green-tea.tv googlefeud.com
gsasearchengineranker.top gosreg.amchs.ru
guidefs.ru greenshop.su
guitar-master.org growboxbl.ru
gyffu.com guruofcasino.com
handicapvansarea.com hd-filmy.net
hasfun.com hdseriale.pl
havepussy.com hem.passagen.se
hermesbracelets.xyz himazin.info
hermesreplica.pw home-task.com
highspeed5.net homeinns.com
homelygarden.com hostingclub.lk
honyaku.yahoofs.jp howtowhitenteethfast.xyz
hosting-tracker.com hplaserjetpdriver8y.pen.io
hotchatdate.com hustoon.over-blog.com
hotel-mkad.ru ibmdatamanagement.co
htmlcorner.com id-forex.com
humbmosquina.tk ido3.com
husky-shop.cz igrovyeavtomaty777.ru
iccornacircri.cf impresagaia.it
ideoworld.org incest-ru.com
iloveitaly.ru incomekey.net
ilovevitaly.com interesnie-faktu.ru
imabase.com
imperiafilm.ru
impisr.edunsk.ru
incanto.in.ua
infowarcraft.ru
innoslicon.com
intellekt21.ru
interfucks.net interfucks.net
iqoption.com intermesh.net
intim-uslugi.info
ip-guide.com
iqoption.pro iqoption.pro
irunfar.com irkutsk.zrus.org
japfm.com ivearchenceinflu.cf
javidol.site jackpotchances.com
javxxx18.com
jerseyssportsshop.com jerseyssportsshop.com
jerseywholesalebizchina.com jose.mulinohouse.co
jobgirl24.ru junglenet-a.akamaihd.net
justdating.online jurajskie.info
justprofit.xyz justucalling32211123456789.tk
kartiniresto.com kakadu-interior.com.ua
katushka.net katushka.net
kazan.xrus.org kazan.zrus.org
keki.info keywordsuggests.com
kerei.ru khadastoafarde.tk
keywordteam.net kinofak.net
kino-rf.ru klikbonus.com
kinosed.net kol-energo.ru
kirov.zrus.org komputers-best.ru
klitimg.pw
knowsitall.info
kostenloser-sex.com kostenloser-sex.com
kvartir-remont.biz kvartir-remont.biz
kvartiry-remont.ucoz.ru
labvis.host.sk labvis.host.sk
laexotic.com ladsup.com
larger.io lafourmiliaire.com
law-check-seven.xyz lapitec.eu
lecbter-relationships.ga larchik.net
letmacworkfaster.world legalrc.biz
lflash.ru lemon-ade.site
lietaer.com letto.by
linkwithin.com lion.misba.us
liveinternet.ro loadopia.com
localmatchbook.com lob.com.ru
loginduepunti.it lockerz.com
lrac.info lombia.co
m0r0zk0-krava.ru lostaruban.com
mafa-free.com louisvuittonoutletstore.net
mainhp.net lumb.co
makedo.ru m.b00kmarks.com
manifestation.betteroffers.review mackeeper-center.club
manve.info macrotek.ru
mapquestz.us magazintiande.ru
marsgatan.com magicalfind-a.akamaihd.net
master-muznachas.ru magnetic-bracelets.ru
manuscript.su
marktforschung-stuttgart.com
max-eclat.men max-eclat.men
maxxtor.eu max-p.men
mebel-vstroika.ru medi-fitt.hu
medispainstitute.com.au meet-flirt-dating.com
megahdporno.net metafilter.com
megavolt.net.ua metaxalonevstizanidine.blogspot.com
metallosajding.ru mex-annushka.ru
michaelkorssaleoutletonline.net mhi-systems.ru
mielec.pl milblueprint.com
minegam.com miniads.ca
missvietnam.org misslike.ru
monarhs.info misterjtbarbers.com
money-every-day.com mobifunapp.weebly.com
morocco-nomad-excursions.com mojaocena.com
mototsikl.org monclerjacketsoutlet.pw
mouselink.co moncleroutletonline.pw
mp3films.ru moneymaster.ru
mp3s.club moroccosurfadventures.com
mrbitsandbytes.com moscow.online-podarki.com
mrcsa.com.au mosdverka.ru
mrmoneymustache.com musezone.ru
muschisexbilder.com music.utrolive.ru
mycouponizemac.com myshopmatemac.com
mydownloadengine.com mystats.xyz
myfreemp3.eu
myhitmp3.club
myhydros.org
myperiod.club
na-telefon.biz na-telefon.biz
nakozhe.com
nakrutka.cc nakrutka.cc
naturalbreakthroughsresearch.com napalm51.nut.cc
naturalpharm.com.ua naturtreenspicerx.pw
naval.jislaaik.com netvouz.com
netallergy.ru news-readers.ru
new-post.tk nlfjjunb5.ru
newfilmsonline.ru oakridgemo.com
newtechspb.ru obiavo.in
nordvpn.com offer.wpsecurity.website
novatech.vn offtopic.biz
novgorod.xrus.org okmedia.sk
nuker.com omoikiri-japan.ru
nvformula.ru onclickpredictiv.com
nyfinance.ml onlinemegax.com
oconto.ru openlibrary.org
odoratus.net openmultipleurl.com
of-ireland.info ordernorxx.com
offf.info orenburg-gsm.ru
ok-ua.info ororodnik.goodbb.ru
onlineku.com ostrovtaxi.ru
ostroike.org outshop.ru
oynat.info
ozoz.it
panchro.xyz panchro.xyz
paparazzistudios.com.au partner-pop.men
partner-trustworthy.men pcboa.se
pdn-4.com perfection-pleasure.ru
pearlisland.ru perm.xrus.org
perederni.net petitions.whitehouse.gov
phelissota.xyz phuketscreen.com
picurams.pw placid-rounded-coast.glitch.me
piter.xrus.org plaff-go.ru
piuminiita.com playlott.com
pk-services.ru
play-mp3.com
pochemychka.net
pochtovyi-index.ru pochtovyi-index.ru
pokerniydom.ru pokerniydom.ru
polimga.pw popugaychiki.com
pontiacsolstice.info popup-hgd.xyz
popads.net pornogig.com
porno-transsexuals.ru pornohub.me
pornofiljmi.com pornokajf.com
pornoforadult.com pornosmola.info
pornokorol.com princeadvantagesales.com
pornolook.net progonrumarket.ru
pornpost.in promalp-universal.ru
potolokelekor.ru psvita.ru
preparevideosafesystem4unow.site purchasepillsnorx.com
priora-2.com qwertty.net
prizeestates.cricket randalljhoward.com
pronekut.com randki-sex.com
prostitutki-novgoroda.org rapevideosmovies.com
prostitutki-odessa.org realting-moscow.ru
qcstrtvt.bloger.index.hr rebuildermedical.com
qpypcx.com recipedays.com
radiogambling.com refererx.com
rapidgator-porn.ga renecaovillasale.online
razleton.com restaurantlescampi.com
rebelmouse.com retailwith.com
redirect.trafficreceiver.club rethinkwasteni.info
redirectme.net riders.ro
redirlock.com rimedia.org
rednise.com rixpix.ru
remont-comp-pomosh.ru rogervivierforsale.com
remontvsamare.su rp9.ru
resant.ru runtnc.net
rezeptiblud.ru rus-teh.narod.ru
rfid-locker.co rvi.biz
ringtonepartner.com sad-torg.com.ua
rocis.site sadaholding.com
roofers.org.uk salmonfishingsacramentoriver.com
rootandroid.org savememoney.co.za
rospromtest.ru sbdl.no
ruex.org.ua scanner-jack.top
rybalka-opt.ru scanner-jessica.top
saitevpatorii.com scanner-john.top
savefrom.com scanner-viktor.top
sbf441.com scanner-walt.top
scanner-mark.top searchinquire.com
scottbywater.com security60-e.com
sdsjweb.com seofied.com
se-welding.ru seoholding.com
seansonline24.pl serptehnika.ru
searchengineranker.email service.adtech.us
securesmrt-dt.com sexstream.pl
seinterface.com sexyebonyteen.com
seksotur.ru sharpchallenge.com
senger.atspace.co.uk ship-marvel.co.ua
seo-2-0.com
seo18.su
seonetwizard.com
seoprofiler.com
sexfreepornoxxx.com
sexiporno.net
sexuria.net
sharebutton.to
sheerseo.com
shisha-swag.de
shoesonlinebuy.cn shoesonlinebuy.cn
shopvilleroyboch.com.ua shopvilleroyboch.com.ua
sicfor.bcu.cc shtaketniki.kz
sigmund-freud.co.uk signal03.ru
siha.de sisi-go.ru
simple-image.com.ua site-speed-check.site
simplepooltips.com site5.com
sitevaluation.org siteonomy.com
sitopreferito.it skuteczna-dieta.co.pl
skanninge.se slim.sellany.ru
skylta.com slonechka.ru
smailik.org smailik.org
smart-scripts.com sneakyboy.com
softwaretrend.net sofit-dmd.ru
space-worry.ml sosdepotdebilan.com
spl63.fr sovinsteel.ru
spy-sts.com spammen.de
srecorder.com spb.afora.ru
stairliftsarea.com spravka130.ru
stariy-baku.com stackthatbucks.com
staynplay.net strangeduckfilms.com
stop-gepatit.te.ua stretchmate.net
storehouse.ua strfls.com
studworks.org styro.ru
subj.ukr-lit.com support.nopeas.sk
sukarame.net
supermesta.ru
surintech.ac.th surintech.ac.th
swagbucks.com svbur.ru
syvertsen-da.no sweepstakes.rewardit.com
szamponrevita.pl swinginwithme.ru
tackletarts.co swinon.site
takethatad.com sygraem.com
teastory.co talant-factory.ru
teenbbw.yopoint.in tecnoteakviareggio.it
teesdaleflyballclub.co.uk
telefonsex-ohne0900.net telefonsex-ohne0900.net
telefonsexkostenlos.tk telefonsexkostenlos.tk
thecoral.com.br thaisamkok.com
theplacetoupdating.pw thecoolimages.net
thfox.com theguardlan.com
ticketsys.inetwd.com thepornsex.org
titangel-vietnam.com tizanidine6mg.blogspot.com
tizanidinedosage.blogspot.com tizanidinedosageforsleep.blogspot.com
tizanidinehcl2mg.blogspot.com tizanidineformigraines.blogspot.com
tizanidineinteractions.blogspot.com tizanidinenarcotic.blogspot.com
tizanidinesideeffectsweightloss.blogspot.com tizanidinepills.blogspot.com
tnaionline.org tk-assortiment.ru
toondinsey.com topads.men
topappspro.com topvidos.ru
torrent-to-magnet.com torrent-to-magnet.com
tracksz.co trafaret74.ru
trkdf.com trafficjunky.com
turbo-suslik.org trustl.life
twodollarshows.com twitlinks.com
typer.one twu.com.ua
udayavani.com uchebavchehii.ru
ukrtvir.com.ua ukkala.xyz
uogonline.com umityangin.net
uroki.net underthesite.com
us-america.ru uni.me
veselokloun.ru unisexjewelry.org
vesnatehno.ru uptime-alpha.net
via-gra.webstarts.com uptime-gamma.net
urdoot.win
urldelivery.com
utiblog.fr
valkiria-tk.ru
viandpet.com
viberdownload10.com viberdownload10.com
vids18.site videochat.tv.br
videositename.com
viel.su viel.su
visa-china.ru viven.host.sk
vizag.kharkov.ua vriel.batcave.net
vozbujdenie.com vseigru.one
vsdshnik.com wardreapptokone.tk
vuclip.com wareseeker.com
vvon.co.uk waterpurifier.club
vzlomsn.org watsonrealtycorp.com
vzlomtw.com wcb.su
wallpaperstock.net wdfdocando.com
watracker.net webix.biz
webjam.com webradiology.ru
websitevaluebot.com webs.com
wesharepics.site weburlopener.com
whatistizanidine2mg.blogspot.com weddingdresses.xyz
winwotgold.pl whengirlsgowild.com
womama.ru wholesalenfljerseys.us.com
wordkeyhelper.com wiosenny-bon-1500.pl
wrona.it works.if.ua
www.pinnacle-bets.com worldis.me
www.timer4web.com xblognetwork.com
x-diesel.info xcombear.ru
xclicks.net xdoza.com
xxx-treker.ru xgames-04.com
yaoguangdj.com xjrul.com
youbloodyripper.com xmnb.net
youjizz.vc xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
zastroyka.org xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
zazagames.org xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
zeroredirect9.com xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
zynax.ua xn--80abgj3a5acid6ghs.top
xn--80ahdheogk5l.xn--p1ai
xn--b1addnj3cah.xn--p1ai
xn--b1agm2d.net
xn--d1abj0abs9d.in.ua
xwatt.ru
xxxmania.top
youradulthosting.com
yourmovies.pl
yoursite.com
ypmuseum.ru
zeroredirect7.com
znaniyapolza.ru
znaturaloriginal.com
zoogdisany.com
zryydi.com

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
178evakuator178.ru
1bet.com 1bet.com
2345.com 83net.jp
3th.co.in DomainStatsBot
8lufu.com abtasty.com
999webdesign.com actulite.com
adminshop.com adimmix.com
alphacarolinas.com ads.tremorhub.com
alpinism.ru adtech.us
aludecor.info adverster.com
amazingninja.com advertisingtag.net
anniemation.com aero2.ru
apiqualitinknet-a.akamaihd.net aimayangzhi.com
approved.su ald2014.org
apxeo.info alert-fdm.xyz
arabgirls.adultgalls.com allornamenti.com
arclk.net alyeskaresort.com
arraty.altervista.org amazingpic.net
articlesdirectoryme.info angry-fermi-7633.arukascloud.io
as5000.com anonymizeme.pro
aviav.co api.stathat.com
aviav.org arenda-yeisk.ru
avtoarenda.by arendovalka.xyz
awency.com asseenontvonline.ru
babespcs.com azbukafree.com
bangla-chat-uk-paralud.ga badavit.com.ua
bazaronline24.ru baladur.ru
beauby.ru balans.shahterworld.org
bedcapdealers.com balla-boo.se
bestadbid.com bambi.ck.ua
bestbrainprod.win bestcalovebracelet.cn
bestpornuha.com bestserials.com
bestsexyblog.com bin-brokers.com
betshuckclean.com bitcoinpile.com
bget.ru blobar.org
bigcities.org blog.koreadaily.com
bjetjt.com board.f00d.de
brateg.xyz boo-arts.com
burn-fat.ga bookmark4you.com.biz
buttons-for-website.com brokergid.ru
buyviagraa.blogspot.com buckspar.gq
buzz-porno.info buyantiviralwp.com
buzzurl.jp
c-gainsbourg.com c-gainsbourg.com
callejondelpozo.es cakesplus.com.au
canrioloadm.gq callcafe.info
capture-room.com candytiner.com
cartierjusteunclou.xyz carapics.com
cartierlovebraceletreplica.xyz cartierlovestore.com
chelnytruck.ru cheapmoncler.pw
christianlouboutinsaleonline.us cheapmoncler.win
ci.ua cienum.fr
citysecurity.nu cinemaenergy-hd.ru
cleanersoft.com circlesl.com
compliance-fred.top cleanpcnow.com
compucelunlock.net com-secure.download
comsysnet.com com-supportcenter.website
coolgramgoods.com compliance-alex.top
counterbot.com compliance-george.xyz
data-ox.com cookingmeat.ru
dekorkeramik.ru cvety24.by
dildofotzen.net cwetochki.ru
dkvorota.ru cypernhuset.se
e-commerce-seo1.com delayreferat.ru
edwinkonijn.com.au demenageur.com
eropornosex.ru dimkino.ru
esfchat.tk domznaniy.ru
eyelike.com.ua dostavka-v-ukrainu.ru
ezofest.sk download-of-the-warez.blogspot.com
fapgon.com downloads-whatsapp.com
fashionindeed.ml efkt.jp
fickblock18.com ekn-art.se
filmetricsasia.com ekto.ee
filmfanatic.com electricwheelchairsarea.com
elektrischezigarettekaufen2.cowblog.fr
energy-ua.com
es5.com
escortslet.net
essayassist.com
exchangeit.gq
extlinks.com
fanrto.com
farm26.ru
fetroshok.ru
fifa-coins.online
find1friend.com
firstdrugmall.ru
flash4fun.com
flytourisme.org flytourisme.org
focalink.com forum-engineering.ru
free-share-buttons.blogspot.com foto-telok.net
free-today.com front.ru
freeuploader.ml
freevpn.space
fsalas.com
ftns.ru ftns.ru
funnel.co.za fz139.ttk.ru
funtoonez.com game-mmorpg.net
fyl.com.ru gaming-journal.com
gallictures.com gk170.ru
genericviagrasildenafiled.net golaya.pw
global-ics.co.za
glopages.ru
goblacked.com
goodhousekeeping.com goodhousekeeping.com
googleseo.com.tr googlemare.com
groupmoney.ru gov.yanao.ru
gsmlab.pl greatidea.marketing
gsmtlf.ru gstatey.net
hawaiisurf.com hamacapty.com
hdimagelib.com hostcritique.com
helicalpile.us houtings.xyz
iblogpress.xyz hoverboard360.nl
ibmdatamanagement.co howmuchdoestizanidinecost.blogspot.com
ideawheel.com i-midias.net.br
igithab.com idc.com.ua
imgata.com idealtits.net
inet-traffic.com ilo134uloh.com
inspiring-desperate.tk in-tandem.co
intellektmedia.at inboxdollars.com
iomoio.net incomekey.net
ipostroika.ru insurple.com
iqoption.pro iqs.biz.ua
iscblog.info istizanidinelikexanax.blogspot.com
isistaylorporn.info javidol.site
itrevolution.cf javlibrary.cc
itronics.ca jerseyswholesalechinalimited.com
ivanstroi.ru journalhome.com
jacago.com jumptap.com
javcoast.com justprofit.xyz
job.icivil.ir justucalling32211123456789.tk
johnrobertsoninc.com karachev-city.ru
kam-dom.ru kazan.zrus.org
katjimej.blog.fc2.com ketoanhanoi.info
keywordblocks.com kimcurlrvsms.com
kinohall.ru kinozapas.com
klin3952.ru kirov.zrus.org
kmd-pto.ru konoplisemena.com
krynica.info koptims.tiu.ru
landmania.ru krasnodar-avtolombards.ru
languagecode.com kumuk.info
lego4x4.ru kvartir-remont.biz
letslowbefast.site ladov.ru
littlecity.ch letsart.ru
littlesexdolls.com li-er.ru
liveinternet.ru linksharingt.com
lyrster.com loadopia.com
macdamaged.tech m3gadownload.pl
madot.onlinewebshop.net macnewtech.com
martlinker.com marketingvici.com
miniads.ca matb3aa.com
mitrasound.ru matrixalchemy.com
monarhs.info mega-bony-2017.pl
msk.afora.ru menetie.ru
multgo.ru mere.host.sk
musezone.ru meteostate.com
myghillie.info moneyteam24.com
nardulan.com monitorwebsitespeed.com
narkologiya-peterburg.ru moskva.nodup.ru
naturalpharm.com.ua mundoaberrante.com
nbsproject.ru musicvidz.ru
nina.az mwtpludn.review
northfacestore.online myshopmatemac.com
obiavo.uz naturalshair.site
oda.as o00.in
onlinemeetingnow.com odoratus.net
optitrade24.com of-ireland.info
osnova3.ru oneshotdate.com
ostrovtaxi.ru ostrovtaxi.ru
papasdelivery.ru ovirus.ru
paradontozanet.ru partner-stop.men
parlament.biz pdamods.ru
percin.biz.ly perederni.net
piluli.info perfectpracticeweb.com
pipki.r.acdnpro.com petedrummond.com
piratecams.com picmoonco.pw
pochemychka.net pladform.ru
polimga.pw plastgran.com
popserve.adscpm.net plugingeorgia.com
pornoted.com poem-paying.gq
prefersurvey.net pornohd1080.online
preparevideosafesystem4unow.site pornotubexxx.name
prizesbook.online powitania.pl
promover.org pravoholding.ru
prospekt-st.ru proekt-mos.ru
proxyradar.com prostitutki-kharkova.org
rainbowice.ru pulse33.ru
rating-bestcasino.com raavidesigns.com
registry-clean-up.net razleton.com
rootandroid.org reimageplus.com
saveriopiazza.it rethinkwasteni.info
search.1and1.com reversing.cc
searchlock.com rialp.getenjoyment.net
seonetwizard.com rogervivierforsale.com
serw.clicksor.com rossmark.ru
seryeznie-znakomstva.ru running-line.ru
sex-pr.net russia-today-video.ru
sex-tracker.com russintv.fr
sex.hotblog.top rvi.biz
shitmovs.com samolet.fr
sibvitr.ru sarf3omlat.com
sitevalued.com satellite.maps.ilovevitaly.com
slayerlife.com sbornik-zakonov.ru
smailik.org scalerite.co.za
snworks.com scanner-jessica.top
soblaznu.net se-welding.ru
social-button.xyz sexystrippe.info
soda.media sharebutton.net
staff.prairiesouth.ca si-unique.com
starwars.wikia.com silvermature.net
stopnarco.ru slavkokacunko.de
studiokamyk.com.pl small-game.com
suhanpacktech.com snjack.info
suture.co srdrvp.com
swinginwithme.ru srecorder.com
tandvardshuset.net stanthonyscatholicchurch.org
teenporn18.net storehouse.ua
tehngr.ru sukarame.net
telegramdownload10.com supermodni.com.ua
the-trader.net superstarfloraluk.com
thegioixekhach.com superstats.com
thegolfclub.info tds-advert002.info
ticketsys.inetwd.com telemetryverification.net
tizanidine4mguses.blogspot.com thecoral.com.br
tizanidinedruginteractions.blogspot.com theguardlan.com
tizanidinepills.blogspot.com theheroes.ru
tizanidinevsvalium.blogspot.com thewebsitetemplate.info
tobeyouday.win tizanidinedosageforsleep.blogspot.com
tizanidinehcl4mgtablet.blogspot.com
tongkatmadura.info tongkatmadura.info
top-karkas.ru tooplay.com
topappspro.com topvidos.ru
topmira.com tourismvictoria.com
torture.ml trafficfactory.biz
tracksurf.daooda.com trafficmp.com
tsan.net traffictrade.life
turbo-suslik.org try-rx.com
turvgori.ru tsstcorpcddvdwshbbdriverfb.aircus.com
twinderbella.com usiad.net
unimodemhalfduplefw.pen.io venerologiya.com
uptimechecker.com ventelnos.com
ural-buldozer.ru via-energy-cumpara.com
vabasa.inwtrade.com vip.51.la
vancleefreplica.pw volume-pills.biz
vertaform.com webix.me
videositename.com wholesalejerseysgaa.com
virtualbb.com wholesalenfljerseys.us.com
viteonlusarezzo.it wingsoffury2.com
vseigru.one x-mix.info
warningwar.ru xcombear.ru
wttavern.com xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
www.kabbalah-red-bracelets.com yellowstonesafaritours.com
x-diesel.org yogamatsexpert.com
xn--80aikhbrhr.net youbrainboost.asia
xtremeeagles.net
xwatt.ru
xxxasianporn.net
yoopsie.com
yourads.website yourads.website
yoursearch.me yourothersite.com
yugo-star.ru zog.link
zajm-zalog-krasnodar.ru
zetgie.com.pl
zhacker.net
zigarettenonlinekaufen2.drupalgardens.com
zzlgxh.com

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.253 0; 10.1.0.124 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3467 ### Version: V4.2022.11.3468
### Updated: Sat Nov 12 22:01:24 UTC 2022 ### Updated: Sun Nov 13 10:03:32 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3467 ### Version: V4.2022.11.3468
### Updated: Sat Nov 12 22:01:24 UTC 2022 ### Updated: Sun Nov 13 10:03:32 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##