vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.10.3423 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-10-21 10:10:39 +00:00
parent 099516837b
commit 0273a7a6b6
47 changed files with 92982 additions and 92972 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3422 #### Version: V4.2022.10.3423
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3422 #### Version: V4.2022.10.3423
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.10.3422 #### Version: V4.2022.10.3423
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11486
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.10.3422 #### Version: V4.2022.10.3423
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11490
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11490
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11490
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11490
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 1; 10.1.0.203 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 0; 10.1.0.203 0;
127.0.0.1 0; 127.0.0.1 0;

11490
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 1; 10.1.0.203 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 0; 10.1.0.203 0;
127.0.0.1 0; 127.0.0.1 0;

11490
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 1; 10.1.0.203 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 0; 10.1.0.203 0;
127.0.0.1 0; 127.0.0.1 0;

11490
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3422 3423

11490
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11490
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
168.61.6.51 0; 20.230.119.199 0;

11490
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 1; 10.1.0.203 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 0; 10.1.0.203 0;
127.0.0.1 0; 127.0.0.1 0;

11490
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 1; 10.1.0.203 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 0; 10.1.0.203 0;
127.0.0.1 0; 127.0.0.1 0;

11490
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 1; 10.1.0.203 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 0; 10.1.0.203 0;
127.0.0.1 0; 127.0.0.1 0;

11490
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
168.61.6.51 0; 20.230.119.199 0;

11490
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
168.61.6.51 0; 20.230.119.199 0;

11490
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 1; 10.1.0.203 1;
127.0.0.1 1; 127.0.0.1 1;

14
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
CSHttp BetaBot
CheTeam CheTeam
LinqiaMetadataDownloaderBot DomainAppender
FlashGet
GoZilla
Lipperhey Lipperhey
NextGenSearchBot NetAnts
Nutch SuperHTTP
Panscient
RepoMonkey
TeleportPro TeleportPro
webmeup-crawler TheNomad

294
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404checker 404checker
404enemy
80legs
ADmantX
AIBOT
ASPSeek ASPSeek
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot
AiHitBot AiHitBot
Alexibot AlphaBot
Anarchie Anarchie
Aspiegel Aspiegel
Asterias
Atomseobot Atomseobot
Attach
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BackStreet BDCbot
Backlink-Ceck BDFetch
BackDoorBot
BacklinkCrawler
Badass Badass
Bandit Bandit
Barkrowler Barkrowler
Battleztar Bazinga Battleztar Bazinga
BetaBot Black Hole
Bitacle BlowFish
BlackWidow BotALot
Bolt Brandprotect
Brandwatch
Buck
BuiltBotTough BuiltBotTough
BuzzSumo Bullseye
CATExplorador
CODE87 CODE87
CSHttp Cegbfeieh
Calculon
CensysInspect CensysInspect
CheTeam ChinaClaw
CheeseBot Claritybot
Citoid Cloud mapping
Collector Cocolyzebot
Copier
CopyRightCheck CopyRightCheck
Crawling at Home Project
Crescent Crescent
DBLBot Curious
DIIbot DSearch
DTS Agent Demon
Devil Devil
Digincore Digincore
DigitalPebble DittoSpyder
Dirbuster DomainSigmaCrawler
Discobot Dotbot
DnBCrawler-Analytics
DomainAppender
Drip Drip
ECCP/1.0
EMail Siphon EMail Siphon
EMail Wolf
Ebingbong
Ecxi Ecxi
ExtLinksBot EirGrabber
Extreme Picture Finder
FDM
FHscan
FemtosearchBot FemtosearchBot
Fimap Fimap
Freeuploader
Fuzz Fuzz
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW GalaxyBot
GermCrawler
GetWeb GetWeb
Getintent Getintent
Gigabot
Go!Zilla Go!Zilla
GrabNet Grafula
Grabber
GrapeFX GrapeFX
HEADMasterSEO GrapeshotCrawler
HTMLparser HTTP::Lite
HTTrack
Haansoft Haansoft
Havij Havij
Heritrix Heritrix
Hloader Hloader
Humanlinks Humanlinks
IDBot IDBot
IndeedBot Iblog
InfoNaviRobot Image Fetch
Intelliseek InfoTekies
Iskanie InterGET
JOC Web Spider JOC Web Spider
JetCar Jbrofuzz
Jetty Joomla
Jorgee
JustView JustView
Jyxobot Kenjin Spider
LWP::Simple Keybot Translation-Search-Machine
Keyword Density
Lanshanbot
Larbin Larbin
Leap Leap
LeechGet LexiBot
Lftp Lightspeedsystems
LibWeb Linkbot
LieBaoFast LinksManager
LinkpadBot LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey LinqiaScrapeBot
Litemage_walker Lipperhey Spider
Lmspider Lmspider
MSIECrawler Ltx71
MIDown tool
MSFrontPage
Mag-Net Mag-Net
Magnet
Mail.RU_Bot
Majestic SEO
Majestic12 Majestic12
MarkWatch MarkWatch
Mass Downloader
Masscan Masscan
Mata Hari MauiBot
Mb2345Browser Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot Mister PiX
MegaIndex.ru
Metauri
Microsoft URL Control
Minefield
Moblie Safari
Mojeek Mojeek
Mojolicious Mojolicious
Morfeus Fucking Scanner
Mr.4x3
Msrabot
Musobot Musobot
Name Intelligence NPbot
NearSite NetLyzer
NetSpider NetMechanic
NetZIP Netcraft
Nettrack Netvibes
NextGenSearchBot
Niki-bot Niki-bot
Nmap Nikto
Nuclei Ninja
Nutch Octopus
Offline Navigator Offline Explorer
OpenLinkProfiler OnCrawl
OpenVAS OpenVAS
Openvas OrangeBot
OutfoxBot
PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer
PageAnalyzer
PageGrabber PageGrabber
PageThing.com
Pandalytics Pandalytics
Pi-Monster Pavuk
Picscout
Pixray Pixray
Pockey ProWebWalker
Psbot
Pump
PxBroker PxBroker
QueryN Metasearch RankFlex
Quick-Crawler
RSSingBot
RankingBot
Rankivabot Rankivabot
RankurBot
Reaper Reaper
RebelMouse Recorder
RepoMonkey RepoMonkey
Rogerbot Ripper
SEOkicks
SEOstats SEOstats
SMTBot SMTBot
SalesIntelligent Scanbot
ScoutJet ScoutJet
ScreenerBot ScrepyBot
SearchmetricsBot Searchestate
Seekport
Semrush
SemrushBot SemrushBot
SentiBot
Shodan Shodan
SiteExplorer SiteCheckerBotCrawler
SiteSucker SiteSucker
Sitebeam Siteimprove
SmartDownload
Snake Snake
Snoopy Snoopy
Sosospider Sociscraper
Sottopop Sottopop
SpaceBison
Spammen
SpankBot SpankBot
Spinn3r Spbot
Sqlmap Sqlmap
Sqlworm Sucker
Steeler Sucuri
SuperHTTP
SurveyBot SurveyBot
Suzuran
Szukacz
T0PHackTeam
T8Abot T8Abot
Teleport
Telesoft
Telesphorep Telesphorep
The Intraformant TheNomad
Thumbor
TightTwatBot TightTwatBot
Toweyabot Titan
Tracemyfile Trendictionbot
Turingos
Turnitin Turnitin
TwengaBot TurnitinBot
URLy Warning
URLy.Warning
UnisterBot UnisterBot
Upflow Vagabondo
VB Project Virusdie
VCI VoidEYE
Vacuum
VeriCiteCrawler
Voil Voil
WBSearchBot
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWWOFFLE
Wallpapers
Wallpapers/3.0 Wallpapers/3.0
Web Auto WallpapersHD
WeSEE
Web Collage Web Collage
Web Enhancer Web Pix
WebEnhancer Web Sauger
WebReaper WebAuto
WebSauger WebCollage
WebZIP WebFuck
WebmasterWorldForumBot WebImageCollector
WebsiteExtractor WebPix
WebsiteQuester Webalta
Whacker Webshag
WinHTTrack Webster
Who.is Bot
Widow
WiseGuys Robot WiseGuys Robot
Wonderbot
Woobot Woobot
Wotbox Wprecon
Xaldon_WebSpider Xaldon WebSpider
Xenu
YoudaoBot
Zauba Zauba
Zermelo ZoomBot
Zitebot
ZoominfoBot ZoominfoBot
adscanner
archive.org_bot archive.org_bot
backlink-check backlink-check
cah.io.community
check1.exe
clark-crawler
cognitiveseo cognitiveseo
com.plumanalytics crawler.feedback
crawler4j
dataforseo.com dataforseo.com
domainsproject.org evc-batch
facebookscraper facebookscraper
heritrix internetVista monitor
isitwp.com isitwp.com
linkdexbot lwp-trivial
magpie-crawler magpie-crawler
plumanalytics netEstate NE Crawler
pcBrowser
polaris version polaris version
ripz
s1z.ru
seobility seobility
seoscanners seoscanners
serpstatbot
siteripz siteripz
sogouspider
sp_auditbot sp_auditbot
spyfu tAkeOut
trendiction.com trendiction.com
trendiction.de trendiction.de
webgains-bot voyagerx.com
webmeup-crawler webpros.com
webprosbot webprosbot
x09Mozilla
x22Mozilla x22Mozilla
zauba.io zauba.io

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 01h4x.com
Aboundex 360Spider
Acunetix 80legs
ADmantX
Abonti
Aboundexbot
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot
Aipbot
Alexibot Alexibot
Alligator AllSubmitter
AlphaBot
Anarchie Anarchie
Ankit Anarchy99
Anthill Anthill
Aspiegel Aspiegel
Asterias
Atomseobot Atomseobot
Attach
AwarioRssBot
AwarioSmartBot
BDCbot
BLEXBot BLEXBot
BackDoorBot BackDoorBot
BacklinkCrawler BackStreet
Badass
Bandit Bandit
Barkrowler Barkrowler
BatchFTP BatchFTP
Black Hole
BlackWidow
Boardreader
BotALot
Brandprotect
Brandwatch
Buck Buck
Buddy Buddy
BuiltWith Bullseye
BuzzSumo
CATExplorador CATExplorador
CCBot CCBot
Calculon CODE87
Cegbfeieh CSHttp
CensysInspect CensysInspect
Cliqzbot CheTeam
Cloud mapping ChinaClaw
Copier Claritybot
Cocolyzebot
CopyRightCheck CopyRightCheck
Copyscape Copyscape
Cosmos
Crawling at Home Project
Crescent Crescent
CrunchBot Custo
Curious DSearch
DBLBot DTS Agent
DIIbot DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Demon
Deusu
Devil
Dirbuster
Discobot Discobot
DomCopBot Discoverybot
DittoSpyder
DnyzBot
DomainCrawler DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
Download Wonder
ECCP/1.0 ECCP/1.0
EMail Siphon EMail Siphon
Ezooms EMail Wolf
FHscan EroCrawler
Extractor
Fimap Fimap
FlashGet FlashGet
Flunky Fuzz
Freeuploader G-i-g-a-b-o-t
GermCrawler GT::WWW
GalaxyBot
Genieo
GetRight GetRight
GetWeb
Getintent
Gigabot Gigabot
Go-Ahead-Got-It Go!Zilla
Gotit
Grafula
GrapeshotCrawler GrapeshotCrawler
HMView GridBot
HTTP::Lite HTTP::Lite
HaosouSpider Humanlinks
Havij HybridBot
IlseBot IDBot
Internet Ninja IRLbot
InternetSeer Id-search
Iria InfoNaviRobot
IstellaBot InfoTekies
JOC Web Spider Intelliseek
InterGET
Iskanie
JamesBOT JamesBOT
Jbrofuzz
JennyBot JennyBot
JetCar JetCar
Keyword Density Jetty
Kozmosbot Kenjin Spider
Lanshanbot Kinza
Leap LNSpiderguy
LWP::Simple
Larbin
LeechGet LeechGet
Lftp
Libwhisker Libwhisker
LinkScan LinqiaMetadataDownloaderBot
LinkextractorPro LinqiaScrapeBot
LinkpadBot Lipperhey
LinqiaRSSBot
Lipperhey Spider Lipperhey Spider
MIIxpc MJ12bot
MSFrontPage MTRobot
MSIECrawler Mail.RU_Bot
Mag-Net Majestic SEO
Magnet
Majestic-SEO
Majestic12 Majestic12
MarkMonitor
MarkWatch
Mass Downloader Mass Downloader
Masscan Meanpathbot
Mb2345Browser MicroMessenger
Mediatoolkitbot Microsoft URL Control
Metauri Mojolicious
Microsoft Data Access
Minefield
Moblie Safari
MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila
Msrabot
Musobot Musobot
NICErsPRO Name Intelligence
Nameprotect
Navroad
NearSite NearSite
Needle Needle
Nessus
NetLyzer NetLyzer
NetMechanic
NetSpider NetSpider
Netvibes Netvibes
Nibbler Nikto
NimbleCrawler
Nimbostratus
Nmap Nmap
Nuclei Not
Offline Explorer Offline Navigator
Openvas OnCrawl
OutclicksBot
OutfoxBot
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageAnalyzer
PageGrabber PageGrabber
PageScorer
PageThing.com PageThing.com
Pandalytics Pi-Monster
Picscout Piepmatz
Pimonster ProWebWalker
ProPowerBot Probethenet
Psbot Psbot
Pump Pu_iN
RankActiveLinkBot RankFlex
Rankivabot RankingBot2
RankurBot RankurBot
Re-re Re-re
ReGet Recorder
Reaper RepoMonkey
RocketCrawler Ripper
Rogerbot
SBIder SBIder
SEOlyticsCrawler SEOstats
SMTBot SISTRIX
Scanbot Screaming
Scrapy
ScreenerBot ScreenerBot
ScrepyBot ScrepyBot
Searchestate Searchestate
SearchmetricsBot
SemrushBot SemrushBot
SentiBot
SeoSiteCheckup SeoSiteCheckup
Seomoz
Siphon
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteLockSpider SiteLockSpider
SiteSnagger
SmartDownload SmartDownload
Snapbot
Snoopy Snoopy
Sociscraper
Sogou web spider Sogou web spider
Sosospider SpaceBison
Sottopop Spammen
Spanner
SputnikBot SputnikBot
Sqlmap Sqworm
Sucuri
SuperBot
Suzuran
Szukacz
T0PHackTeam
T8Abot
Teleport Teleport
TeleportPro Telesphorep
Telesphoreo The Intraformant
TheNomad
Thumbor
Titan
Toata Toata
Tracemyfile Trendiction
Trendictionbot
True_Robot
Turingos Turingos
Turnitin Turnitin
Twice Twice
Typhoeus
URLy Warning URLy Warning
URLy.Warning
UnisterBot UnisterBot
VCI VB Project
Vacuum
Vagabondo Vagabondo
VeriCiteCrawler Virusdie
VidibleScraper
VoidEYE
Voil Voil
Voltron Voltron
WASALive-Bot
WBSearchBot
WEBDAV
WISENutbot
WWW-Collector-E
WWW-Mechanize WWW-Mechanize
WWWOFFLE WWWOFFLE
Wallpapers WeSEE
Web Collage
Web Enhancer
Web Fetch Web Fetch
Web Fuck
WebEnhancer WebEnhancer
WebFuck WebFuck
WebGo IS WebReaper
WebPix
WebSauger
WebStripper
WebSucker
WebWhacker WebWhacker
WebZIP WebZIP
Webalta WebmasterWorldForumBot
Webshag Website Quester
Webster WebsiteExtractor
Whack Whack
Whatweb Whacker
Who.is Bot Who.is Bot
WiseGuys Robot WiseGuys Robot
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Zermelo Zauba
Zeus Zeus
Zitebot
ZoomBot ZoomBot
arquivo-web-crawler ZoominfoBot
arquivo.pt
cah.io.community cah.io.community
check1.exe check1.exe
com.plumanalytics clark-crawler
crawler4j demandbase-bot
facebookscraper eCatch
heritrix
instabid
internetVista monitor
ips-agent
isitwp.com isitwp.com
linkdexbot iubenda-radar
meanpathbot meanpathbot
probe-image-size probe-image-size
s1z.ru s1z.ru
satoristudio.net satoristudio.net
scan.lol
seobility seobility
seocompany.store
seoscanners seoscanners
seostar
serpstatbot serpstatbot
sexsearcher sysscan
sitechecker.pro trendiction.de
siteripz ubermetrics-technologies.com
trendiction.com
webmeup-crawler
webpros.com webpros.com
webprosbot webprosbot
x09Mozilla x09Mozilla
x22Mozilla zauba.io
xpymep1.exe zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
ascat.porn 123any.com
bavsac.com amung.us
cdncash.org hazardky.net
ez8motelseaworldsandiego.com hybrid.ru
layola.biz.tc obiavo.com
mixtapetorrent.com psa48.ru
onefilms.net salutmontreal.com
purplestats.com vremya.eu
sabaapress.com xfluro.com
tomatis.gospartner.com zoominfo.com

942
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1-99seo.com 01apple.com
24videos.tv 03p.info
28n2gl3wfyb0.ru 0daymusic.org
2kata.ru 123cha.com
45en.ru 12masterov.com
7fon.ru 18ps.ru
9med.net 1webmaster.ml
DomainStatsBot 256bit.by
4pp13.com
7ooo.ru
7zap.com
a.pr-cy.ru a.pr-cy.ru
a14download.com aa8b68101d388c446389283820863176e7.com
aa9d046aab36af4ff182f097f840430d51.com activepr.ru
aad1f4acb0a373420d9b0c4202d38d94fa.com addbin.men
actices.com adtech.fr
ad-words.ru advokat-grodno.by
adimmix.com
adultactioncam.com
advertex.info
aero2.ru aero2.ru
afftrack001.com agardomains.com
ahrntal.verymes.xyz agysya.ru
aihelen.net albamargroup.com
akita.kz alert-hgd.xyz
akuhni.by alfa-img.com
albuteroli.com all-number.com
alfabot.xyz aludecor.info
alif-ba-ta.com andpolice.com
alive-ua.com androids-store.com
all-streaming-media.com anlimebel.kiev.ua
amanda-porn.ga anti-virus-removal.info
amazon-seo-service.com apartment.ru
ameblo.jp apifasterlightin-a.akamaihd.net
android-systems.ru apilinkswiftco-a.akamaihd.net
ap.senai.br appearance-cool.com
apartamentwroclaw.eu arabgirls.adultgalls.com
apiallgeniusinfo-a.akamaihd.net arpelsreplica.xyz
apidiamondatanet-a.akamaihd.net artpicso.com
apiqualitinknet-a.akamaihd.net as5000.com
apisecretsaucebi-a.akamaihd.net
apispringsmartne-a.akamaihd.net
app5.letmacworkfaster.world
apple.com-cleaner.systems
arabsexxxtube.com
arate.ru
artavenuegardenstudios.com
artdestshop.eu
arvut.org
asiengirls.net
atlant-auto.info
aucoinhomes.com aucoinhomes.com
autoplate.info autobrennero.it
avtochehli.by autowebmarket.com.ua
ayanaspa.com avirasecureserver.com
ayerbo.xhost.ro ayeartoforget.com
ayodhya.co bad-stars.net
azte.ch bankmib.ru
babosas.com batanga.net
bankiem.pl
bankofthewext.com
bausparen.bz.it bausparen.bz.it
bedandbreakfast.com
belinvestdom.by
berdasovivan.ru berdasovivan.ru
bestcoin.cc beremenyashka.com
bestdooz.com berrymall.ru
besthoro.ru bestofferswalkmydogouteveryday.gq
bestimagecoollibrary.com betshuckclean.com
bestssaker.com betwinservice.com
besttorrentknifta.weebly.com bimatoprost-careprost.com
betonka.pro binomo.kz
bird1.ru bio.trade-jp.net
biteg.xyz
bizlist.com.de
bjetjt.com bjetjt.com
blavia.00author.com blackcurranthumidifiers.site
blogig.org boeuklad.com
bmw.afora.ru bonzbuddy.com
bodybuilding-shop.biz boobsimge.com
brandehk.dk bosman.pluto.ro
brg8.com brainzpod.win
bristolhostel.com brendbutik.ru
brus.city brillianty.info
buigas.00it.com brimstonehillfortress.org
buqayy0.livejournal.com burger-imperia.com
business-made-fun.com buscarfoto.com
bytimedance.ru buzzzg.men
c-english.ru callawaygolfstore.online
call-of-duty.info canvas.gloverid.site
candy-glam-hp.com capture-room.com
cdncash.net carabela.com.do
chatroulette.life carsplate.com
cheatcode-lita12.rhcloud.com carstrends2015.com
checkm8.com cashkitten-a.akamaihd.net
checkpagerank.net celejihad.info
chinajerseyswholesalecoupons.com ceotrk.com
christianlouboutinreplica.pw cfjump.com
cialgenisrx.com cheapmoncler.pw
cialisndbrx.com cialisovercounteratwalmartusa.com
cienum.fr cienum.fr
clash-clans.ru co.lumb.co
cleansearch.net coindirect.io
clipartnew.com communisave.co.za
clknsee.com companies-catalog.com
compliance-checker.info
compliance-donald.xyz compliance-donald.xyz
compliance-fred.xyz compliance-jessica.top
constantaservice.net compliance-john.top
continent-e.tv compliance-margo.top
coolgramgoods.com concordexoticrentals.com
coolyarddecorations.com copro.pw
copypaste.traderzplanet.in copypaste.traderzplanet.in
counciltally.com cowblog.fr
courtshipgift.com crackguru.tk
cracksplanet.com dafatiri.com
crd.clan.su dailystrength.org
cscwtalkto.site date-withme.com
custodieva.ru day-news.info
custom-product-labels.com dealwifi.com
czeshop.info design-ideas.info
d2jsp.org diminishedvalueoforegon.com
data-mining.tk dirtpics.pw
de.zapmeta.com distver.ru
decenttools.com dktr.ru
delitime.info docstoc.com
dev78.dioniqlabb.se domaincdn.xyz
devochki.top domcran.net
directivepub.com download-wallpaper.net
doeco.ru drugs-no-rx.info
doktoronline.no duplicashapp.com
dollartree.info dynainbox.com
domainroam.win elektrozigaretten1.tumblr.com
dominateforex.ml elitesportsadvisor.com
download-of-the-warez.blogspot.com elmacho.xyz
downloadeer.net elvel.com.ua
downtuptv.gq embedle.com
doxyporno.com emergencyneeds.org
dscaas.website
dvervmoskvu.ru
eas-seo.com
efkt.jp
egovaleo.it
elektrischezigarette2.onsugar.com
elektrischezigarettekaufen2.cowblog.fr
elektronischezigarettekaufen1.myblog.de
elektrozigarette2.wordpressy.pl
elstal.com.pl
empis.magix.net empis.magix.net
escort-russian.com encodable.com
eshop.md englishdictionaryfree.com
eupornstar.info englishgamer.com
ezigaretteshop2.vefblog.net eorogo.top
failingmarriege.blogspot.com eralph.tk
faithe.top eu-cookie-law.blogspot.com
fandlr.com euromasterclass.ru
fashionavenuegame.com evehemming.blogspot.com.au
ewebarticle.info
extractorandburner.com
ezigarettenshop1.yolasite.com
facebook-mobile.xyz
facecup.top
favornews.com favornews.com
femdom.twiclub.in fickblock18.com
fermersovet.ru fickkontaktehobbyhuren.com
fickenbumsen.net filesclub.net
fix-website-errors.com filesmonster.porn
find1friend.com
findpik.com
findsexguide.com
flash4fun.com
flashahead.info
flauntyoursite.com
flipper.top flipper.top
flooringinstallation-edmonton.com floridahuntingfishingadventures.com
foodgid.net
forexgb.ru
formseo.com formseo.com
foto-sisek.porngalleries.top formulaantiuban.com
fototravel.eu forum.tvmir.org
free-social-buttons.com forum69.info
free411games.com freakycheats.com
freewareseek.com free-laptop-reward.com
frustrated-favorable.gq free-share-buttons.com
fuckmill.com freecamdollars.com
fz139.ttk.ru freelotto.com
galeria-zdjec.com freewlan.info
gambarkataku.co freshdz.com
gaytube.com gael-s.ru
geilehausfrauen.net galaxyflowers.ru
gelezki.com galaxys6manual.info
getmyads24.com gamebackyard.com
girlgamerdaily.com gaygalls.net
given2.com gaypornmovie.net
glass-msk.ru gayxperience.com
goldenggames.com gazobeton-p.com.ua
golfresa.lucania.se geoads.com
gonextmedia.com getdot.ru
gotorussia.com gidonline.one
gov.yanao.ru girls-ufa.ru
gratuitxblcodes.com gktt.ru
gsou.cf global-ics.co.za
healmytrauma.info go2mike.ru
help.tpu.ru gojiberriess.apishops.ru
hesteel.pl gok-kasten.net
hetmanship.xyz goldadpremium.com
high-speed1.net goodnightjournal.com
histats.com gosreg.amchs.ru
gotcher.us
gratis-sexkontakte.com
gta-club.ru
gta-top.ru
guitar-master.org
gungamesz.com
hawaiisurf.com
hayate.biz
healing-dysplasia.ru
hermesreplica.pw
historichometeam.com
hit-kino.ru hit-kino.ru
hmywwogw.bloger.index.hr homemature.net
hollywoodactress.info howmuchdoestizanidinecost.blogspot.com
homedecoguide.info hreade.com
hoporno.com huimin764128.com
hotchatdate.com hulfingtonpost.com
hotdl.in hustoon.over-blog.com
hotelcrocenzi.sm i-midias.net.br
hoverboard360.se i-service.kz
howtoclean.club ifirestarter.ru
hoztorg-opt.ru igadgetsworld.com
husky-shop.cz igithab.com
hydropump.su ilovevitaly.ru
ie.57883.net imadedinner.net
igrovyeavtomaty777.ru imagerydatabase.com
ikritikimou.gr imagui.eu
images-graphics-pics.com impresagaia.it
images.gyffu.com increasewwwtraffic.info
imgarcade.com intermesh.net
impisr.ru internetartfair.com
in-tandem.co iscblog.info
informatiecentro.be istizanidineanopiate.blogspot.com
infostatsvc.com itronics.ca
inmate-locator.us ivanovo.zrus.org
int.search.mywebsearch.com jennyfire.ru
intelhdgraphicsgtdrive6w.metroblog.com
investsuccess.org
iphantom.com
ivoiretechnocom.ci
jerseychinabizwholesale.us
jimmychoosale.online
jobgirl24.ru jobgirl24.ru
jolic2.com joessmogtestonly.com
journeydownthescale.info katadhin.co
kaac.ru
kalandranis.gr
kaz.kz
keki.info keki.info
kenaba.su keywordhut.com
keywordblocks.com kladrus.ru
keywordglobal.co.za konturkrasoty.ru
keywords-monitoring-success.com kumuk.info
kichenaid.ru labplus.ru
kiprinform.com landscapebackgrounds.blogspot.com
klitimg.pw landscaping.center
kol-energo.ru lasvegaslockandsafe.com
koronirealestate.gr laxob.com
kosova.de light.ifmo.ru
kwzf.net lightinghomes.net
ladov.ru linkredirect.biz
lancheck.net lionking-1994.blogspot.com
leadwayau.com lisque.batcave.net
leboard.ru listiseltemournlan.gq
lechenie-gemorroya.com liveinternet.ru
letslowbefast.today livejasmin.com
lexaprogeneric.link liveu.infoteka.hu
lezbiyanki.net loadopia.com
lifehacĸer.com locationdesaison.com
lineavabit.it lonely-mature.com
linkwithin.com long-beach-air-conditioning.com
lion.misba.us longlifelomilomi.info
live-sexcam.tk
lizace.com
loadingpages.me
lollypopgaming.com
lolnex.us
lotto6888.com lotto6888.com
lovelycraftyhome.com louisvuittonoutletstore.net
m5home.ru lsp-awak-perikanan.com
mackeeper-land-672695126.us-east-1.elb.amazonaws.com lyngdalhudterapi.no
mail.allnews24.in lyrics.home-task.com
makis.nu magicplayer-s.acestream.net
mall.uk
mapquestz.us mapquestz.us
medanestesia.ru marmitaco.cat
medictube.ru matematikus.info
matrixalchemy.com
meddesk.ru
mediafresh.online
medtherapy.ru
meduza-consult.ru meduza-consult.ru
megavolt.net.ua michaelkorssaleoutletonline.net
menetie.ru mielec.pl
mhi-systems.ru millionare.com
michellblog.online minet.club
minegam.com minharevisao.com
missvietnam.org mini-modus.ru
mlvc4zzw.space mixed-wrestling.ru
mmgq.ru moje-recenze.cz
mobilemedia.md movpod.in
mobot.site moz.com
mokrayakiska.com mp3s.club
moneytop.ru msk-diplomat.com
moneyviking-a.akamaihd.net msk.afora.ru
mrmoneymustache.com musflashtv.com
mtmtv.info musirc.com
mwtpludn.review
my-big-family.com
my-floor.in.ua my-floor.in.ua
mydearest.co myghillie.info
mylovelibrabry.com myiptest.com
mymobilemoneypages.com naj-filmy24.pl
myonigroup.com najaden.no
namenectar.com namecrumilchlet.tk
nardulan.com narkologiya-penza.ru
netvouz.com nastroyke.net
neks.info
nelc.edu.eg
new7ob.com new7ob.com
nmrk.ru newstudio.tv
noclegonline.info nextconseil.com
nofreezingmac.click nfljerseyscheapbiz.us
nomuos.it nhl17coins.exblog.jp
novatech.vn nhl17coinsforps3.gratisblog.biz
novosibirsk.xrus.org nina.az
o-o-6-o-o.ru njkmznnb.ru
obiavo.in nonameread45.live
obiavo.su nosecret.com.ua
oda.as notaria-desalas.com
oksrv.com novodigs.com
omsk.xrus.org nucia.biz.ly
onlineporno.site o-dachnik.ru
openstat.com omoikiri-japan.ru
oren-cats.ru onclickpredictiv.com
ostroike.org onlinadverts.com
palvira.com.ua online-hit.info
paphoselectricianandplumber.com optom-deshevo.ru
parajumpersoutlet.online ordernorxx.com
partner-stop.men orgasmus-virtual.com
paydayloanslocal.com ourville.info
pdns.download outshop.ru
perso.wanadoo.es pc-test.net
pestomou.info pdamods.ru
photo-clip.ru pdn-4.com
pdns.cz
pensplan4u.com
perm-profnastil.ru
perosan.com
pewit.pw
picsearch.com picsearch.com
picsforkeywordsuggestion.com pictures-and-images.net
piuminiita.com picturesify.com
platesauto.com pinkduck.ga
pinup-casino1.ru
pirateday.ru
pix-hd.com
platezhka.net
ploenjitmedia.azurewebsites.net
plusnetwork.com plusnetwork.com
porndairy.in pomoc-drogowa.cba.pl
porno-raskazy.ru popup-fdm.xyz
pornogratisdiario.com popup-jdh.xyz
pornosemki.info pornowarp.info
pornpost.in pr0fit-b0x.com
pospr.waw.pl pretty-mart.com
potolokelekor.ru preventheadacheguide.info
powc.r.ca.d.sendibm2.com privat-girl.net
printie.com prizefestival.mobi
priora-2.com prom23.ru
profitsport.club proxyelite.biz
profitwithalex.info ptr.ruvds.com
promodj.com qor360.com
propranolol40mg.blogspot.com quit-smoking.ga
prostitutki-almata.org razamicroelectronics.com
prostitutki-kharkova.org rcb101.ru
prostitutki-rostova.org
quizzitch.net
ranksignals.com
reckonstat.info reckonstat.info
rednise.com restaurantlescampi.com
reklama1.ru rfserial.net
remont-komputerov-notebook.ru rockprogblog.com
remontbiz.ru rollercoin.com
replica-watch.ru rome2rio.com
research.ifmo.ru rootandroid.org
restorator-msk.ru rustic-quiver.win
riralmolamsaca.tk rusvideos.su
rocis.site rvtv.ru
rock-cafe.info rybalka-opt.ru
rp9.ru sack.net
runofilms.ru sape.top
russia-today-video.ru sarm.tk
sa-rewards.co.za sbetodiodnye-lampy.ru
saitevpatorii.com scalerite.co.za
satellite.maps.ilovevitaly.com scanner-julia.top
sayyoethe.blogspot.co.za score-ads.men
sbprabooks.com
scanner-walt.top
schlampen-treffen.com
scmor.ilxc.cc
sdelai-prosto.ru sdelai-prosto.ru
search-error.com seadragonherbery.com
search.1and1.com searchimage.co
seoanalyses.com seccioncontrabajo.com
sergiorossistore.online seofirmreviewsus.info
sexkontakte-seite.com serpstat.com
sexxdate.net service.adtech.fr
sexy-pings.com sex-foto.pw
ship-marvel.co.ua sexreliz.com
sexy-screen-savers.com
sexytrend.ru
sharebutton.net
sharebutton.to
shoplvlv.us shoplvlv.us
shopperifymac.com shymkent.xkaz.org
shoppingmiracles.co.uk sibtest.ru
simon3.ru sigmund-freud.co.uk
sisiynas.ru signal03.ru
sitebeam.net sildenafilcitratemed.com
skutecznetabletkinaporostwlosow.pl silverage.ru
skylta.com silvercash.com
slavkokacunko.de silvermature.net
socialbookmarksubmission.org sisi-go.ru
sofit-dmd.ru siteexpress.co.il
sosdepotdebilan.com slowmacfaster.trade
sovinsteel.ru smart-balancewheel.com
spacash.com social-s-hhh.xyz
sports-supplements.us sonnikforme.ru
stairliftstrue.com spabali.org
steebook.com sport7777.net
stjamesschool.info squidoo.com
stopnarco.ru sstroy44.ru
streamin.to staff.prairiesouth.ca
stroy-portal22.ru stal-rulon.ru
suggest-keywords.com star61.de
sukarame.net startufa.ru
supermama.top startwp.org
supervesti.ru stathat.com
sureone.pro staticfs.host
surfbuyermac.com steelmaster.lv
sygraem.com strag-invest.ru
taboola.com strongholdsb.ru
tagil.zrus.org subj.ukr-lit.com
taihouse.ru sucsesofinspiration.com
talant-factory.ru summerlinhomes411.info
tandvardshuset.net superlist.biz
tattooreligion.ru surflinksmedical.com
telefonsex-ohne0900.net svbur.ru
telegramdownload10.com svnuppsalaorebro.se
thefarmergame.com symbaloo.com
theporndude.com tabletkinaodchudzanie.com.pl
timhost.ru tarad.com
tizanidine6mg.blogspot.com testbotprocessor44.com
tizanidinecapsules.blogspot.com tfxiq.com
tizanidinehclsideeffects.blogspot.com thegamerznetwork.com
tizanidinehydrochloride2mg.blogspot.com thegolfclub.info
tizanidineonline.blogspot.com thepantonpractice.co.uk
tizanidinepain.blogspot.com titan-cloud.life
tizanidinerecreationaluse.blogspot.com tizanidineforopiatewithdrawal.blogspot.com
tkanorganizma.ru tizanidinehcl2mg.blogspot.com
token-lab.org tizanidinestreetprice.blogspot.com
top250movies.ru tn811.us
topcar-krasnodar.ru tongkatmadura.info
topflownews.com top10registrycleaners.com
torrent-to-magnet.com toproadrunner5.info
torrentdownloadhub.com torrentgamer.net
torture.ml traffic-club.info
tour-line.net traffic2cash.org
toy-shop.top trustl.life
trafficmp.com tsan.net
trafficnetzwerk.de
traxdom.ru
trkur.com
trubywriting.com
tsyndicate.com tsyndicate.com
ttrraacckkrr.com turbo-suslik.org
tube8.com typimga.pw
uasb.ru
ublaze.ru ublaze.ru
ucsol.ru uggbootsoutletsale.us
uggsale.online ukkelberg.no
ukrtvir.com.ua unrealcommander.org
ukrtvory.in.ua uprour.com
undergroundcityphoto.com uptime-alpha.net
usadacha.net userequip.com
uvozdeckych.info vashsvet.com
vchulkax.com vasileostrovsky-rayon.ru
vduplo.ru vekzdorov.ru
videochat.world versaut.xxx-cam.webcam
videotuber.ru videochat.cafe
vintontech.info videojam.tv
vip-file.com videokrik.net
vipsexfinders.com videonsk.com
vipsiterip.org vipps.com.my
vitanail.ru vizitki.net
vkak.ru vkontarkte.com
vucms.com vksaver-all.ru
w-journal.ru vladhistory.com
wait3sec.org vseuznaem.com
walkme.com vzglyadriv.kg
wanker.us waycash.net
watch-movies.ru webads.co.nz
waterefficiency.co webscouter.net
webcamtalk.net website-datenbank.de
webmasterhome.cn wedding0venues.tk
webpromotion.ae
website-speed-check.site
webtherapy.ru
wedding-salon.net
wellcome2slovenia.ru wellcome2slovenia.ru
wfb.hatedriveapart.com westvilletowingservices.co.za
wingsofrefuge.net white-truck.life
witclub.info wholinkstome.com
wjgony.com wildnatureimages.com
wnoz.de word-vorlagen.net
wordpress-crew.net workle.website
worst-sites.online works.if.ua
wurr.voila.net worldoffiles.ru
www.souvenirua.com woweb.com.ua
x69ty.ru wrona.it
xchat26.myfreecams.com ww1943.ru
xitjw.info www.wohnkabinen-shop.de
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai wygraj-skiny.win
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai x-true.info
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai xep.info
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai
xn--80aikhbrhr.net xn--90acenikpebbdd4f6d.xn--p1ai
xn--oogle-wmc.com xn--lifehacer-1rb.com
xvideospornoru.com xnxx699.com
xwatt.ru xrus.org
yatrk.xyz xtrafficplus.com
yougetsignal.com xvideosporn.biz
zakazvzloma.com xxxdatinglocal.us
zakon-ob-obrazovanii.ru yaaknaa.info
zakonobosago.ru yopoint.in
zambini.ru yougame.biz
zaobao.com.sg yourporn.com
zarajbuilders.com youtubologia.it
zerocash.msk.ru zaidia.xhost.ro
zeroredirect.com zajm-pod-zalog-nedvizhimosti.ru
zeroredirect10.com zaloro.com
zigarettenonlinekaufen2.drupalgardens.com zero1.it
znakom.sibtest.ru zixizop.net.ru
zoompegs.com znakomstvaonlain.ru
znaniyapolza.ru
zoogdisany.com
zoolubimets.ru
zscalertwo.net
zytpirwai.net

490
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
0912701309f8ce.com 000free.us
11pikachu.ru 2000k.ru
123locker.com 55wmz.ru
4ureyesonly.com 5ws.dating-app.ru
absolute-s.ru 66cpwgln.space
adult-shop.com.ua 8lufu.com
aibolita.com abbanreddy.com
aliexpresscashback.club acads.net
allwomen.info adanih.com
animaltoplist.com adrunnr.com
apccargo.com allboard.xobor.de
apifasterlightin-a.akamaihd.net am15.net
arkartex.ru amt-k.ru
autochoixspinelli.com androids-store.com
aviav.eu aosexkontakte.net
avramstroy.ru apimegabrowsebiz-a.akamaihd.net
bannertesting.com apimossnetinfo-a.akamaihd.net
baoxaydung.com.vn asrv-a.akamoihd.net
bard-real.com.ua atdedinotuho.tk
barnfurnituremart.com ayerbo.xhost.ro
azartmsl.com
azbaseimages.net
azbukafree.com
babosas.com
backuperwebcam.weebly.com
bagsonsale.online
balans.shahterworld.org
bamo.xsl.pt
bannerbank.ru
beclean-nn.ru beclean-nn.ru
bigames.online bestpriceninja.com
bizru.info betshuckclean.com
blackhatworth.com bibys.com
blog.axant.it blackwitchcraft.ru
blog.yam.com bongiornos.info
bodybuilding-shop.biz bookmaker-bet.com
bronzeaid-a.akamaihd.net brokergid.ru
browserprotecter.com bryansk.zrus.org
buyviagraa.blogspot.com businescoop.men
carfax.com.ua businesxxl.com
cartierlovestore.com buttons-for-website.com
charmstroy.info camel-beach.com
chinaelitecheapjerseys.com canrole.com
chocoslim.pro carina-sy.de
clarithromycin500mg.com carloans.com
clips.ua.ac.be cartierjusteunclou.xyz
collegeessay19.blogspot.ru cerev.info
compliance-barak.xyz cheapjerseysbizwholesale.us
compliance-jack.top christianlouboutinreplica.win
compliance-julia.top club-lukojl.ru
connectionstrenth.com compliance-john.top
cookie-law-enforcement-aa.xyz conboy.us
cougfan.info connectingsingles.com
counter.yadro.ru coslab.club
crafthubs.com counciltally.com
cranly.net cowlmash.com
cubook.supernew.org craftburg.ru
dcj-nn.ru crd.clan.su
ddsoldes.fr customchocolate.business-for-home.com
dieswaene.com cv.wallhade.co
dieta-personalna.pl d2jsp.org
discountbarbour.online data-mining.tk
divatraffic.com day-news.info
dkvorota.ru den-noch24.ru
dogsrun.net dev.citetick.com
dojki-hd.com digital-sale.su
dosug-lux.ru digital-video-processing.com
draniki.org dimkino.ru
drupa.com discountliv.com
edudocs.net divci-hry.info
eets.net documentbase.net
ekobata.ru domaincheck.io
eloxal.ru doxysexy.com
eropho.com drunkmoms.net
erotikstories.ru dyt.net
eshop.md e-poker-2005.com
evening-dating-club.info elektronischezi.livejournal.com
excitacao.com elektronischezigarettekaufen2.tumblr.com
exportshop.us enbersoft.com
extener.org ero-advertising.com
fellowshipoftheminds.com erotische-geschichten-xxl.com
ferieboligkbh.dk faithe.top
find1friend.com figuringmoneyout.com
flowersbazar.com filerockstar298.weebly.com
flprog.com firma-legion.ru
forpackningsutveckling.se firstdrugmall.ru
forums.toucharcade.com fototravel.eu
foto-sisek.porngalleries.top free-share-buttons.com
gambarkatabaru.com free-share-buttons.top
gamezblox.com fullgirl.ru
gaypornmovie.net gambarkataku.co
gaz-voshod.ru games.kolossale.ru
geckoandfly.com gayxperience.com
gentamicineyedrops.blogspot.com genetworx.com
glcomputers.ru gettpromos.com
grand-chlen.ru glass-msk.ru
greatgrace.ru go2album.com
growmyfunds.ca goldadpremium.com
hanink.biz.ly goosefishpost.bid
hatdc.org groupmoney.ru
hobbyhuren-datenbank.com handsandlegs.ru
house.sieraddns.com hanwei.us
ie.57883.net hausfrauensex18.com
iinstalll-fii1leis.jus0wil.pp.ua hoholikik.club
inet-traffic.com hotblog.top
investzalog.ru howmuchdoestizanidinecost.blogspot.com
irkutsk.online-podarki.com ibmdatamanagement.co
iweblist.info icaseclub.ru
jackpotchances.com insomniagamingfestival.com
janavibekken.no integritylandscapeservices.com
jaxcube.info ispac.org
jerseyssportsshop.com ivearchenceinflu.cf
jewelryandfiligree.com iyasimasennka.com
jyvopys.com jerseywholesalechinabiz.com
karpun-iris.ru kareliatobacco.ru
keywordspay.com katadhin.co
kfon.eu kazinogames.lv
kemerovo.zrus.org
keywordhouse.com
kino-ecran.ru
kinogolos.ru
kladrus.ru
kovrenok.ru kovrenok.ru
leftofcentrist.com krasnodar.ru
legalrc.biz krasnodar.zrus.org
lida-ru.com landscaping.center
lightinghomes.net laparfumotec.com
linkwithin.com laserpen.club
liran-locks.com life.biz.ua
liverpool.gsofootball.com lignofix.ua
losangeles-ads.com limtu.ifmo.ru
lutherstable.org listiseltemournlan.gq
malls.com loadopia.com
manimpotence.com lob.com.ru
medicines-choice.com m-google.xyz
medmajor.ru m4ever.net
megavolt.net.ua macfix.life
methodsmarketing.com matsdale.com
mex-annushka.ru mcar.in.ua
microstatic.pl migente.com
mindeyegames.com miniads.ca
minegam.com mir-limuzinov.ru
missclub.info mistr-x.org
mitrasound.ru mmoguider.ru
moneytop.ru mncrftpcs.com
morlat.altervista.org morf.snn.gr
moviezbonkerssk.cf muz-shoes.ru
musicktab.com myhydros.org
mxgetcode.com mysexpics.ru
mydownloadengine.com newsperuse.com
myhealthcare.com njpalletremoval.com
nalogovyykodeks.ru nmrk.ru
northfacestore.online nordvpn.com
novodigs.com novatech.vn
ntic.fr offer.wpsecurity.website
nullrefer.com office2web.com
o-o-8-o-o.ru ohmyrings.com
oakleyglassesonline.us oksrv.com
offside2.5v.pl online-sbank.ru
onlinadverts.com paydayloanslocal.com
paperwritingservice17.blogspot.ru pensplan.com
partner-pop.men picsearch.com
pdns.download pistonclasico.com
picphotos.net ploenjitmedia.azurewebsites.net
pics-group.com plugingeorgia.com
pierrehardysale.online porno-raskazy.ru
pio.polytopesexempt.com porno-transsexuals.ru
piulatte.cz pornosee.info
pixelrz.com pornosemki.info
podshipniki-nsk.ru pornoslive.net
porn-w.org postclass.com
pornoforadult.com powitania.pl
pozdravleniya-c.ru prodess.ru
predominant-invent.tk puserving.com
privatefx.all4invest.info rankingchart.de
pronekut.com recipedays.ru
proxyradar.com remont-komputerov-notebook.ru
pwwysydh.com royalcar-ufa.ru
quebec-bin.com sady-urala.ru
rangoman.date saecsa.co
reeyanaturopathy.com saneitconsulting.com
referencemoi.com scanmarine.info
reklama1.ru scanner-jess.top
remybutler.fr scanspyware.net
ricorsogiustizia.org seadragonherbery.com
robotixix.com seocheckupx.com
rocis.site seoholding.com
royal-investments.net seoprofiler.com
roznica.com.ua seorankinglinks.us
runstocks.com serialsway.ucoz.ru
rusenvironmental.net sexs-foto.top
rvi.biz shopfishing.com.ua
saitevpatorii.com shoppingjequiti.com.br
scanner-don.top silvercash.com
searchwebknow-a.akamaihd.net sindragosa.comxa.com
sexpornotales.com siteaero.com
sexyali.com slkrm.ru
shopcheermakeup.info
shoppingmiracles.co.uk
similardeals.net
sivs.ru
skladvaz.ru
slowmacfaster.trade slowmacfaster.trade
socialmonkee.com space4updating.win
spb-plitka.ru sptslmtrafms.com
spylog.com spywarespy.com
starpages.net steamoff.net
staticfs.host store-rx.com
stefanbakosab.se storehouse.ua
stroilka.info superkanpo.com
strongholdsb.ru
sugvant.ru
surflinksmedical.com surflinksmedical.com
surgut.zrus.org swinon.site
talant-factory.ru teletype.in
thebestweightlosspills.ovh thebluffs.com
tizanidineandcipro.blogspot.com thepokertimer.com
tizanidineandgabapentin.blogspot.com tizanidinevscyclobenzaprine.blogspot.com
torrent-newgames.com tizanidinewithdrawal.blogspot.com
torrent-to-magnet.com tracfone.com
torrents.cd
tour-line.net
trackzapper.com
tradedeals.biz tradedeals.biz
trafficjunky.net trafficcentr.xyz
tula.howotorg.ru truck-addzilla.life
underthesite.com tube8.com
unlimitdocs.net typer.one
urldelivery.com umnovocaminho.com
usswrite.com upupa.net
uzporno.mobi vadimkravtcov.ru
validccseller.com validdomain.xyz
valmetrundan.se verymes.xyz
vesnatehno.com viddyoze.com
video-hollywood.ru videochat.bz
videonsk.com videochaty.ru
vintontech.info vinsit.ru
vrnelectro.ru visitcambridge.org
w7s.ru vkgaleria.com
waterefficiency.co vkontaktemusic.ru
webix.biz vladimir.zrus.org
webjam.com vozbujdenie.com
webradiology.ru waterpurifier.club
websitevaluebot.com wejdz-tu.pl
win-spy.com wemarketing.se
witherrom55.eklablog.fr winwotgold.pl
world-mmo.com wttavern.com
worst-sites.online ww1943.ru
wyniki-lista.pl wygraj-skiny.win
x-diesel.org x-musics.com
x-mix.info xedserver.com
xgames-04.com xingzi-vision.com
xn----itbkqkfiq.xn--p1ai xn--24-glceagatoq7c2a6ioc.xn--p1ai
xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai xn--d1abj0abs9d.in.ua
xvideosporn.biz xtube.com
xxxmania.top xxx-treker.ru
yebocasino.co.za yougetsignal.com
youradexchange.com zaim-pod-zalog-krasnodar.ru
youtoner.it zaloadi.ru
yun56.co zeroredirect5.com
zarenica.net
zetmaster.ru
zigzog.ru
zytpirwai.net

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.93 0; 10.1.0.203 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3422 ### Version: V4.2022.10.3423
### Updated: Thu Oct 20 22:01:27 UTC 2022 ### Updated: Fri Oct 21 10:04:54 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3422 ### Version: V4.2022.10.3423
### Updated: Thu Oct 20 22:01:27 UTC 2022 ### Updated: Fri Oct 21 10:04:54 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##