vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.02.3566 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-02-03 10:09:23 +00:00
parent cfdddb04ad
commit baf2563be7
48 changed files with 105296 additions and 105293 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.02.3565 #### Version: V4.2023.02.3566
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.02.3565 #### Version: V4.2023.02.3566
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2023.02.3565 #### Version: V4.2023.02.3566
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

13014
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2023.02.3565 #### Version: V4.2023.02.3566
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

13018
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

13018
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

13018
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

13018
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 1; 10.1.0.219 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 0; 10.1.0.219 0;
127.0.0.1 0; 127.0.0.1 0;

13018
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 1; 10.1.0.219 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 0; 10.1.0.219 0;
127.0.0.1 0; 127.0.0.1 0;

13018
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 1; 10.1.0.219 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 0; 10.1.0.219 0;
127.0.0.1 0; 127.0.0.1 0;

13018
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3565 3566

13018
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

13018
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.227.50 0; 13.88.61.211 0;

13018
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 1; 10.1.0.219 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 0; 10.1.0.219 0;
127.0.0.1 0; 127.0.0.1 0;

13018
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 1; 10.1.0.219 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 0; 10.1.0.219 0;
127.0.0.1 0; 127.0.0.1 0;

13018
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 1; 10.1.0.219 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 0; 10.1.0.219 0;
127.0.0.1 0; 127.0.0.1 0;

13018
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.227.50 0; 13.88.61.211 0;

13018
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
138.91.227.50 0; 13.88.61.211 0;

13018
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 1; 10.1.0.219 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Buddy AiHitBot
BuiltWith Apexoo
Grafula EasyDL
HTTrack GetRight
JetCar Picscout
OpenVAS Sottopop
Openvas VCI
Trendiction WebSucker
arquivo.pt plumanalytics
zgrab scalaj-http

312
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker
ADmantX
AIBOT AIBOT
ASPSeek
Abonti Abonti
Aboundex Acunetix
Aboundexbot
AfD-Verbotsverfahren
AiHitBot AiHitBot
AllSubmitter Alexibot
AlphaBot
Anarchie Anarchie
Anarchy
Anarchy99 Anarchy99
Apexoo Ankit
Anthill
Asterias Asterias
Atomseobot Attach
AwarioSmartBot AwarioSmartBot
BBBike BDCbot
BLEXBot BackDoorBot
BackWeb BackStreet
BatchFTP BacklinkCrawler
Badass
Bandit
Barkrowler
Battleztar Bazinga
Bitacle
BlackWidow
Blow Blow
Boardreader Boardreader
BotALot BotALot
Brandprotect Buddy
BuiltWith
Bullseye Bullseye
CSHttp CATExplorador
Calculon Calculon
Claritybot Cegbfeieh
Cliqzbot Cliqzbot
Cloud mapping Cocolyzebot
Cogentbot
Copier Copier
Copyscape Craftbot
Crawling at Home Project CrunchBot
Custo DIIbot
DataCha0s DataCha0s
DatabaseDriverMysqli Deusu
Demon Devil
Digincore
Dirbuster Dirbuster
Discoverybot Disco
Dispatch
DnBCrawler-Analytics
DnyzBot DnyzBot
DomainCrawler DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
Domains Project Domains Project
Drip Download Wonder
ECCP/1.0 Dragonfly
EMail Siphon
EMail Wolf EMail Wolf
EasyDL Ecxi
Ebingbong Exabot
EirGrabber
EroCrawler
Evil
Express WebPictures Express WebPictures
ExtractorPro ExtLinksBot
Extreme Picture Finder Extreme Picture Finder
FDM Ezooms
FHscan FHscan
FemtosearchBot Fimap
Foobot
Freeuploader
FrontPage FrontPage
Fuzz
FyberSpider FyberSpider
G-i-g-a-b-o-t Genieo
GT::WWW
GermCrawler
GetWeb
Go!Zilla
Gotit Gotit
GrabNet Grabber
GridBot GrapeFX
HMView HEADMasterSEO
HTTP::Lite
HTTrack HTTrack
HaosouSpider
Harvest Harvest
Havij
Hloader Hloader
HonoluluBot HonoluluBot
IlseBot Humanlinks
Internet Ninja IDBTE4M
IDBot
IRLbot
Iblog
Indy Library
InfoNaviRobot
Intelliseek
InterGET
InternetSeer InternetSeer
IstellaBot IstellaBot
Jbrofuzz Jbrofuzz
JetCar
JikeSpider JikeSpider
JustView JustView
Keyword Density
Kinza Kinza
Leap Kozmosbot
LeechFTP LWP::Simple
LeechGet LexiBot
LibWeb LinkextractorPro
Libwhisker LinkpadBot
LieBaoFast Lipperhey
Lipperhey Spider
Litemage_walker Litemage_walker
Ltx71
MFC_Tear_Sample
MIDown tool MIDown tool
MIIxpc MJ12bot
Mag-Net Mail.RU_Bot
Majestic12
MarkMonitor
MarkWatch MarkWatch
Masscan
Mata Hari Mata Hari
Meanpathbot MauiBot
MegaIndex.ru Mb2345Browser
MeanPath Bot
MicroMessenger MicroMessenger
Microsoft Data Access
Minefield
Mister PiX Mister PiX
Moblie Safari Moblie Safari
Mojeek
Morfeus Fucking Scanner
Mozlila
Mr.4x3 Mr.4x3
Musobot
NICErsPRO NICErsPRO
NPbot NPbot
Name Intelligence
Nameprotect Nameprotect
Navroad Navroad
NearSite Needle
Nessus NetAnts
Nettrack NetLyzer
Netvibes NetSpider
NextGenSearchBot Netcraft
Niki-bot
Nikto Nikto
NimbleCrawler
Nimbostratus Nimbostratus
Ninja
Nuclei Nuclei
Nutch
Octopus Octopus
OnCrawl Openfind
OpenLinkProfiler
OpenVAS
OrangeBot
OrangeSpider OrangeSpider
OutclicksBot POE-Component-Client-HTTP
OutfoxBot
PHPCrawl
PageAnalyzer PageAnalyzer
Panscient Pandalytics
Petalbot Papa Foto
PeoplePal
Pi-Monster
Picscout Picscout
Picsearch
Piepmatz
Pimonster
Pixray Pixray
ProPowerBot Probethenet
Psbot
PyCurl PyCurl
Quick-Crawler
RSSingBot
RankActive
RankActiveLinkBot RankActiveLinkBot
RankingBot
RankingBot2 RankingBot2
Rankivabot
RankurBot RankurBot
Re-re Re-re
Reaper
RedesScrapy
Ripper
RocketCrawler RocketCrawler
SBIder SEOkicks
ScoutJet SEOstats
SISTRIX
ScanAlert
Scrapy Scrapy
ScreenerBot ScreenerBot
ScrepyBot ScrepyBot
Searchestate
Seekport
SentiBot SentiBot
SeobilityBot SeoSiteCheckup
Seomoz Seomoz
Shodan
Siphon
SiteCheckerBotCrawler
SiteSnagger SiteSnagger
SiteSucker Siteimprove
Sitevigil
SlySearch SlySearch
Snake
Snapbot Snapbot
Snoopy SocialRankIOBot
Sociscraper
Sosospider Sosospider
Sottopop
SpaceBison SpaceBison
Spammen
Spanner
Spinn3r
SputnikBot
Sqlworm
Sqworm
Steeler
Sucker Sucker
Swiftbot SuperBot
T0PHackTeam SuperHTTP
TeleportPro Szukacz
Telesoft TightTwatBot
Telesphorep Trendictionbot
TheNomad True_Robot
Thumbor TwengaBot
Toweyabot Typhoeus
Trendiction
Turingos
Turnitin
URLy Warning URLy Warning
UnisterBot URLy.Warning
Vacuum Upflow
V-BOT
VB Project
Vagabondo
VeriCiteCrawler
Voil Voil
WEBDAV Voltron
WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers/3.0 Wallpapers
WallpapersHD WallpapersHD
WeSEE Web Collage
Web Auto Web Fetch
Web Fuck Web Fuck
Web Pix Web Pix
WebAuto
WebBandit
WebCopier
WebGo IS
WebImageCollector
WebLeacher
WebPix
Web Sauger Web Sauger
WebZIP Web Sucker
WebmasterWorldForumBot WebCollage
WebCopier
WebFetch
WebPix
WebStripper
WebWhacker
Webalta
Webshag Webshag
Website Quester Website Quester
Whack WebsiteExtractor
Whacker Webster
Widow Widow
Wonderbot WiseGuys Robot
Zauba Woobot
ZoomBot YoudaoBot
Zade
Zitebot
ZoominfoBot
ZyBorg ZyBorg
arquivo-web-crawler archive.org_bot
check1.exe backlink-check
clark-crawler coccocbot
cognitiveseo cognitiveseo
crawler.feedback com.plumanalytics
crawler4j domainsproject.org
demandbase-bot eCatch
evc-batch evc-batch
heritrix heritrix
instabid
ips-agent ips-agent
iubenda-radar linkdexbot
lwp-request
magpie-crawler magpie-crawler
mediawords mediawords
muhstik-scan oBot
probe-image-size page scorer
s1z.ru polaris version
satoristudio.net satoristudio.net
sitechecker.pro scalaj-http
sp_auditbot seoscanners
sexsearcher
sogouspider
spyfu spyfu
sysscan
tAkeOut
trendiction.de
ubermetrics-technologies.com
voyagerx.com
webgains-bot
webpros.com
x09Mozilla
x22Mozilla x22Mozilla
xpymep1.exe

308
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404checker
80legs 80legs
ADmantX
ALittle Client ALittle Client
ASPSeek
Abonti Abonti
Aboundexbot Aboundexbot
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
Aipbot
Alexibot Alexibot
Alligator AllSubmitter
AlphaBot Anarchie
Ankit Ankit
Anthill Anthill
Aspiegel Aspiegel
Atomseobot Atomseobot
AwarioSmartBot Attach
BackDoorBot AwarioRssBot
BDFetch
BackStreet
BackWeb BackWeb
Backlink-Ceck Backlink-Ceck
Badass Badass
Battleztar Bazinga Bandit
Bitacle Bitacle
Blow Blackboard
BlowFish
Boardreader Boardreader
BotALot Bolt
Brandwatch Brandwatch
BuiltBotTough BuiltWith
BunnySlippers Bullseye
CATExplorador CSHttp
CazoodleBot CazoodleBot
Cegbfeieh
CensysInspect CensysInspect
CheTeam
CherryPicker CherryPicker
ChinaClaw
Chlooe Chlooe
Claritybot Craftbot
Cocolyzebot CyotekWebCopy
Cogentbot
CopyRightCheck
Copyscape
Crawling at Home Project
CrunchBot
DSearch DSearch
DTS Agent DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli
Demon Demon
Discobot Deusu
Discoverybot Digincore
Dirbuster
DittoSpyder DittoSpyder
DnBCrawler-Analytics DnyzBot
DomainAppender
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
Download Wonder Dragonfly
ECCP/1.0 Drip
EMail Siphon EMail Siphon
Ecxi Ecxi
EirGrabber
Evil
Exabot
Express WebPictures Express WebPictures
ExtLinksBot
Extractor
Extreme Picture Finder Extreme Picture Finder
EyeNetIE FemtosearchBot
FHscan Fimap
Firefox/7.0 Firefox/7.0
FlashGet FlashGet
Flunky Flunky
Foobot
Freeuploader Freeuploader
FrontPage Fyrebot
FyberSpider G-i-g-a-b-o-t
GT::WWW
GalaxyBot GalaxyBot
Genieo
GetRight GetRight
Getintent
Gigabot Gigabot
Grafula GoZilla
HMView GrabNet
HTMLparser
HTTP::Lite
Haansoft
HaosouSpider HaosouSpider
IDBot Havij
Iblog Heritrix
Id-search Hloader
Image Fetch Humanlinks
Image Sucker HybridBot
Indy Library IDBTE4M
IRLbot
IndeedBot
InfoNaviRobot InfoNaviRobot
InfoTekies
Intelliseek
Internet Ninja
InternetSeer InternetSeer
Iria JetCar
IstellaBot Jorgee
JOC Web Spider JustView
JamesBOT Jyxobot
Jetty
Kenjin Spider Kenjin Spider
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
LNSpiderguy Kinza
Larbin LWP::Simple
Leap Lanshanbot
LeechFTP
LeechGet LeechGet
Libwhisker
Lightspeedsystems Lightspeedsystems
LinkWalker Likse
LinkextractorPro
LinkpadBot
LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaScrapeBot
Lipperhey
Ltx71 Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc
MQQBrowser MQQBrowser
MSFrontPage MSFrontPage
MTRobot MarkWatch
Magnet Mass Downloader
MarkMonitor MeanPath Bot
Mata Hari Mediatoolkitbot
Metauri Metauri
Mister PiX Mojolicious
MolokaiBot MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Musobot
Mr.4x3
Msrabot
NICErsPRO
Nameprotect Nameprotect
NearSite Navroad
Nessus Nessus
Net Vampire NetLyzer
Nimbostratus NetMechanic
NetSpider
Netvibes
Nibbler
NimbleCrawler
Ninja
Nmap
Not
Nutch
Offline Explorer Offline Explorer
OpenLinkProfiler Offline Navigator
OnCrawl
Openfind
Openvas
OrangeBot OrangeBot
OrangeSpider
OutfoxBot OutfoxBot
Page Analyzer Panscient
Pandalytics Picscout
PeoplePal PictureFinder
Petalbot
Pi-Monster
Piepmatz
Pimonster Pimonster
ProPowerBot PleaseCrawl
ProWebWalker ProWebWalker
Probethenet Psbot
PxBroker Pu_iN
PyCurl Pump
RSSingBot RSSingBot
Re-re RankActiveLinkBot
Recorder RankFlex
RepoMonkey RankingBot
RankurBot
ReGet
Reaper
RebelMouse
RedesScrapy
Ripper Ripper
RocketCrawler Rogerbot
SEOlyticsCrawler
SEOprofiler
SEOstats SEOstats
SMTBot SISTRIX
Screaming SalesIntelligent
SearchmetricsBot ScanAlert
Scrapy
Seekport Seekport
SemrushBot Semrush
SentiBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot
Seomoz
Shodan Shodan
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSucker SiteSnagger
Siteimprove Siteimprove
Sitevigil SlySearch
SmartDownload
Snake
Snapbot Snapbot
SocialRankIOBot
Sociscraper
SpankBot
Spbot
Spinn3r Spinn3r
Sqlworm Sqlmap
Steeler Steeler
Sucker
Sucuri Sucuri
SuperBot SurveyBot
SuperHTTP Suzuran
Surfbot Swiftbot
Szukacz Szukacz
T0PHackTeam
Teleport
Telesphoreo
Telesphorep Telesphorep
The Intraformant
TheNomad
Titan Titan
True_Robot Tracemyfile
Trendiction
Turingos Turingos
Turnitin TurnitinBot
Twice TwengaBot
URLy Warning URLy.Warning
UnisterBot Upflow
VB Project V-BOT
Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler VidibleScraper
Virusdie Virusdie
Voltron Voltron
WBSearchBot WASALive-Bot
WWW-Mechanize WPScan
WeSEE WWW::Mechanize
Wallpapers
WallpapersHD
Web Auto Web Auto
Web Collage
Web Pix Web Pix
Web Sucker Web Sucker
WebCollage WebEnhancer
WebFuck
WebSauger
WebStripper WebStripper
WebWhacker
WebZIP
Webalta
Webshag
Website Quester Website Quester
Webster Webster
Whack
Whacker
Who.is Bot Who.is Bot
Widow WiseGuys Robot
WinHTTrack Wotbox
Wonderbot
Wprecon Wprecon
Xenu Xaldon WebSpider
Zade Zade
Zitebot ZoomBot
ZmEu
ZoominfoBot
ZyBorg
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
autoemailspider check1.exe
backlink-check clark-crawler
crawler.feedback cognitiveseo
crawler4j
dataforseo.com
evc-batch
facebookscraper
gopher gopher
heritrix
internetVista monitor internetVista monitor
ips-agent linkdexbot
isitwp.com lwp-request
lwp-trivial
mediawords mediawords
netEstate NE Crawler netEstate NE Crawler
page scorer
pcBrowser pcBrowser
plumanalytics polaris version
s1z.ru satoristudio.net
seobility scan.lol
seocompany.store
seoscanners
seostar
sitechecker.pro
spyfu spyfu
trendiction.com webmeup-crawler
trendiction.de webpros.com
x22Mozilla webprosbot
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
alert-fdm.xyz getresponse.com
apispringsmartne-a.akamaihd.net ghostvisitor.com
clickcash.com larose.jb2c.me
fiuxy.com medtherapy.ru
front.to mistr-x.org
googleseo.com.tr moscow-region.ru
io9.com owohho.com
sbricur.com play.leadzupc.com
snjatie-geroinovoy-lomki.ru virtualbb.com
xn--hxazdsfy.blogspot.com woweb.com.ua

934
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
101billion.com 178evakuator178.ru
11235813.webzdarma.cz 2728fb936f0.com
12u.info 273-fz.ru
14b.info 45en.ru
1webmaster.ml 4k-player.pl
2x2fan.ru 4ur.click
4istoshop.com 5-steps-to-start-business.com
4pp13.com 5000-cotydzien.com
5kstore.com 5forex.ru
7yue.org 76brighton.co.uk
999webdesign.com 7fon.ru
7minuteworkout.com
888.com
8si.ru
a.pr-cy.ru a.pr-cy.ru
a14download.com aasoldes.fr
aa9bd78f328a6a41279d0fad0a88df1901.com abcdeg.xyz
aa9d046aab36af4ff182f097f840430d51.com abctoppictures.net
aac500b7a15b2646968f6bd8c6305869d7.com acortarurl.es
abiente.ru actulite.com
actionnooz.com adbetclickin.pink
adcash.com addray.pro
adinterax.com adviceforum.info
adman.gr afora.ru
adrunnr.com aihelen.net
adult3dgames.com all4wap.ru
adultnet.in alltheviews.com
affrh2015.com alphaforum.ru
afftrack001.com an-donut.com
afslankpillen2017nl.eu animalrank.com
agahinameh.com anmysite.com
alert-fjg.xyz apiitsthirteende-a.akamaihd.net
allesohnegirls.net apimossnetinfo-a.akamaihd.net
allwomen.info apiportalmorecom-a.akamaihd.net
alot.com app-ready.xyz
altamayoztourism.com apper.de
aludecor.info apple.com-webbrowsing-security.science
alyeskaresort.com approved.su
amateurmatch.com arcadepages.com
ameblo.jp arenda-avtoprokat-krasnodar.ru
ample-awards-today.us
anaksma.info
ananumous.ru
android-vsem.org
animenime.ru
apibrowsesmartne-a.akamaihd.net
apilinkswiftco-a.akamaihd.net
apiwebwebgetcom-a.akamaihd.net
arclk.net
arewater.com arewater.com
asscenihotosidea.blogspot.co.za atley.eu.pn
auto4style.ru atmovs.com
avon-ukraine.com automobile-spec.com
awency.com aviav.eu
ayodhya.co avtolombard-krasnodar.com
backlinkwatch.com azazaporn.com
balkanfarma.org azte.ch
bang-hotties.com b00kmarks.com
bausparen.bz.it bablonow.ru
beauty-b0x.pl babyguns.ru
berdasovivan.ru bag77.ru
bestchoice.cf bala.getenjoyment.net
besthomemadepornsites.com bayareaaeroclub.org
bestmouthwash.club beachpics.com
bimatoprost-careprost.com.ua beauby.ru
binomo.com becuo.com
bio-optomarket.ru berrymall.ru
bizfly.info best-games.today
biznesluxe.ru bestdooz.com
blockworld.ru bestdraws.com
blog.remote-computer.de bestfortraders.com
bluejays-jerseys.us bestwebsitesawards.com
bmw.afora.ru bidbuy.co.kr
bmwhighperformers.com bigpictures.club
brainsandeggs.blogspot.com bikini-image.com
brateg.xyz bitcoinremote.com
browsepulse-a.akamaihd.net bkgr.se
bumskontakte.org blog.koreadaily.com
buy-meds24.com blogqpot.com
buytizanidineonline.blogspot.com blogs.rediff.com
buytizanidineonlinenoprescription.blogspot.com bluesman.nu
buzzsumo.com bobba.dzaba.com
buzzzg.men book-bets.com
calcularpagerank.com bookmaker-bet.com
brewdom.ru
brus-vsem.ru
bum.com.ru
business-suggested.tk
bwlx.prepedu.cn
callawaygolfstore.online callawaygolfstore.online
callstevens.com canzoni.ru
candiceloves.us capture-room.com
canrioloadm.gq carmuffler.net
cartierbracelet.xyz carrauterie.be
cartierjusteunclou.xyz carsdined.org
cartierlove2u.xyz carstrends2015.com
cartierreplica.pw cdn.walkme.com
cashkitten-a.akamaihd.net celebsopics.com
caulderwoodkennels.com cellfun.mobi
cbb1smartlist12.click cercacamion.it
cbcseward.com cfacarrosserie74.com
ccbill.com cheap-trusted-backlinks.com
cejewelry.xyz chocolateslim-original.com
changde.58.com christianlouboutinshoes.xyz
cheapestjerseys-wholesale.com cienum.fr
cheapjerseysa.com cinemaflix.website
chorus.terakeet.com clarithromycin500mg.com
ciproandtizanidine.blogspot.com clothesforcash.com
claytransformations.info clubfashionista.com
cleanallspyware.com collegeessay19.blogspot.ru
clicksor.com compliance-barak.xyz
clicksor.net compliance-brian.top
codq.info compliance-nelson.top
coin-hive.com consorzioilmosaico.org
coldfilm.ru cornomase.win
collegerentals.com cpajunkies.com
colobit.biz csgo4.win
compiko.info customergrowthsystems.com
compliance-elena.xyz czeshop.info
conciergegroup.org dalavia.ru
conocer-sanabria.com danschawbel.com
cookie-law-enforcement-ff.xyz
crafthubs.com
cryptoswap.biz
cs-passion.pl
cubook.supernew.org
culad.com
custom-product-labels.com
cyclobenzaprinevstizanidine.blogspot.com
dafatiri.com
dangphoto.trade
daretodonate.co daretodonate.co
davebestdeals.com data1.scopich.com
ddsoldes.fr
deluxedumps.com
deniven.1bb.ru
deshevo-nedorogo.ru deshevo-nedorogo.ru
detskie-zabavi.ru digilander.libero.it
dieswaene.com dolohen.com
digital-sale.su
documentsite.net
domaincdn.xyz
domaineaneblanc.com
domainroam.win
domik-derevne.ru
dorratex.tn
dosug-lux.ru
dotmass.top
download-wallpaper.net
downloadkakaotalk.com downloadkakaotalk.com
drpornogratisx.xxx doxysexy.com
dstroy.su dscaas.website
dtm-spain.com dyshagi.ru
duawitchrarato.tk easycommerce.cf
eachdayisagift.review
edge.sharethis.com
eduinfosite.com eduinfosite.com
eduserver.net ekosmetyki.net.pl
ekto.ee elektrozigaretten1.webs.com
elektrischezigarette1.blog.pl elektrozigaretten2.yn.lt
elektrischezigarette2.onsugar.com elmifarhangi.com
elektronischezigarettekaufen1.myblog.de elvel.com.ua
elitepcgames.com en.home-task.com
elvenar.com escortslet.net
es5.com essayservicewriting.org
essay-writing.work ezigarettenkaufen1.pagina.gr
essaypro.com ezofest.sk
etenininrade.ga f00kclan.de
evaashop.ru factorynightclub.com
excaliburfilms.com famix.xyz
expertblog.info fanoboi.com
fast-wordpress-start.com fanrto.com
ferretsoft.com fashion.stellaconstance.co
fettefrauen.net fba-mexico.com
fickkontakte.org fbdownloader.com
filesclub.net ferrotodo.com
filmci.pro filesdatabase.net
filmfanatic.com findthe.pet
flowersbazar.com footbalive.org
fodelsedagspresenter.nu forum.tvmir.org
folowsite.com frcls.fr
forex.osobye.ru free-share-buttons.com
formulaf1results.blogspot.com freelotto.com
foxjuegos.com frequiry.com
foxweber.com freshmac.space
foxydeal.com fuckingawesome.com
free-gluten.ru funnymama.com
front.ru fx-brokers-review.com
ftns.ru fyl.com.ru
fuel-gas.com fym.com.ru
futbolkisales.ru gameonasia.com
fz139.ttk.ru gardene.ru
gamebackyard.com gaygalls.net
gamerscorps.com gemara.com
gay.adultgalls.com ghostvisitor.com
gazoblok.net.ua gigapeta.com
get-your-social-buttons.info gktt.ru
getridofstretchmarks.org
glavprofit.ru glavprofit.ru
gobongo.info goldandcard.ru
goo.ne.jp golfresa.lucania.se
googst2.ru gosmeb.ru
gowreckdiving.com gox.com.ua
gq-catalog.gq greendream.com.ua
gratis-sexkontakte.com growshop.es
gsmtlf.ru gta-top.ru
gstatey.net gwebtools.com.br
guardlink.com
hamacapty.com
hatedriveapart.com hatedriveapart.com
hd-filmy.net hdimagelib.com
hdhc.ru hdseriale.pl
headpress.ru hdxnxxtube.mobi
heatpower.ru hit-kino.ru
helvetia.com.ua hitcpm.com
herehloadibs.cf homedecoguide.info
himazin.info howmuchdoestizanidinecost.blogspot.com
hitsbox.info howopen.ru
holidaypics.org howtowhitenteethfast.xyz
homik.pw hunterboots.online
hoste.octopis.com hut1.ru
hotloans.ru hydropump.su
huimin764128.com idegenvezeto.eu
id-forex.com igadgetsworld.com
idc.com.ua imediadesk.com
ihtec2019.org infektsii.com
ikearugs.xyz infobanks.ru
ilmen.net intellekt21.ru
inbabes.sexushost.com interferencer.ru
instakink.com invivo.hu
iomoio.net iphantom.com
iqoption.com iqs.biz.ua
ivanstroi.ru itbc.kiev.ua
iyasimasennka.com itrevolution.cf
j33x.com ixora.pro
javlibrary.cc izhevsk.xrus.org
jerseywholesalebizchina.com izhevsk.zrus.org
joingames.org jav-fetish.site
jaxcube.info
jikoman.info
johnrobertsoninc.com
journalhome.com journalhome.com
juliadiets.com justucalling32211123456789.tk
jumptap.com jwcialislrt.com
justkillingti.me jwss.cc
kamagragelusa.net kaac.ru
kamen-e.ru
kashubadesign.ru kashubadesign.ru
kellyonline.xyz kenaba.su
kerei.ru kerch.site
keywesthideaways.co kerwinandcariza.com
kfon.eu ketrzyn.pl
kiskinhouse.com keywordspay.com
kleine-titten.biz khadastoafarde.tk
klikbonus.com kihi.gdn
klitimg.pw kiinomaniak.pl
kmgamex.cf koleso24.com.ua
knowsitall.info kometars.xyz
knowyournextmove.com kupit-adenu.ru
kol-energo.ru
kosmetyki.tm.pl
kostenlos-sexvideos.com
kostenloser-sex.com
kredit-blog.ru
kunstaktien.info
kursy-ege.ru kursy-ege.ru
laapp.com kvartiry-remont.ucoz.ru
ladsup.com kwzf.net
lampokrat.ws labvis.host.sk
lapolis.it lawyersinfo.org
larger.io laxdrills.com
laulini.soclog.se layola.biz.tc
law-check-seven.xyz lenpipet.ru
law-check-two.xyz lernur.net
lazy-z.com lflash.ru
lechtaczka.net lifehacĸer.com
legalrc.biz lightinghomes.net
lifeinsurancekb.com likesdesign.com
linkdebrideur.xyz lion.misba.us
livejasmin.com littlesexdolls.com
lombia.co liupis.com
luciddiagnostics.in liveinternet.ro
m5home.ru liverpool.gsofootball.com
macrotek.ru livetsomudvekslingstudent.bloggersdelight.dk
magento-crew.net loftdigital.eu
magiadinamibia.blogspot.com lonfon.xyz
make-money-online.com lovelycraftyhome.com
marcowebonyodziezowe.pl lutherstable.org
matpre.top lyrster.com
m1media.net
m4ever.net
mafa-free.com
manifestyourmillion.com
maranbrinfo.com.br
marblestyle.ru
marcoislandvacations.net
mebeldekor.com.ua mebeldekor.com.ua
meble-bogart.info mediafresh.online
medicovi.com meltwater.com
member-quiz.com member-quiz.com
metascephe.com mielec.pl
metaxalonevstizanidine.blogspot.com mikrobiologies.ru
mezaruk.info milblueprint.com
mhtr.be millionare.com
michaelkorssaleoutletonline.net minecraft-rus.org
microsearch.ru miniads.ca
missclub.info mmstat.com
mojowhois.com mncrftpcs.com
mojpregled.com mobifunapp.weebly.com
mobilemedia.md
mobilierland.com
mojaocena.com
moncleroutletonline.pw moncleroutletonline.pw
monsterdivx.com montredemarque.nl
moz.com morepoweronmac.trade
moroccosurfadventures.com
mp3s.club
mrbitsandbytes.com
mtmtv.info
musezone.su
musflashtv.com
music.utrolive.ru music.utrolive.ru
mybackgroundlandscape.blogspot.com muz-tracker.net
mydirtyhobby.com myblogregistercm.tk
namenectar.com myhitmp3.club
naphukete.ru myra.top
nardulan.com nacap.ru
nastroyke.net nakozhe.com
neks.info naturalpharm.com.ua
netoil.no neobux-bg.info
netpics.org nerudlogistik.ru
news-readers.ru nina.az
no-rx.info no-rx.info
november-lax.com nonameread45.live
novosti-hi-tech.ru nottyu.xyz
o-o-6-o-o.com nzfilecloud.weebly.com
o333o.com o-o-11-o-o.com
obiavo.in obiavo.com
obuv-kupit.ru ofanda.com
oddamzadarmo.eu offers.bycontext.com
offer.wpsecurity.website offf.info
omoikiri-japan.ru olvanto.ru
onclkads.com onescreen.cc
onemantrip.com
oneminutesite.it
onetravelguides.com onetravelguides.com
onko-24.com onlinadverts.com
onlinefilmz.net online.ktc45.ru
onstrapon.purplesphere.in online7777.com
opinionreelle.com onlyforemont.ru
ooo-gotovie.ru
openfrost.com
ops.picscout.com
orion-code-access.net
osoznanie-narkotikam.net osoznanie-narkotikam.net
owathemes.com
paclitor.com
painting-planet.com
parlament.biz parlament.biz
partnerworkroom.men pendelprognos.se
partybunny.ru perm-profnastil.ru
php-market.ru perosan.com
picmoonco.pw
picsforkeywordsuggestion.com picsforkeywordsuggestion.com
pihl.se picturesboss.com
pinsdaddy.com piluli.info
porno.simple-image.com.ua pinstake.com
pornoklad.net pinup-casino1.ru
pornosok.ru piski.top
powc.r.ca.d.sendibm2.com piter.xrus.org
preg.marketingvici.com pizdeishn.com
privatamateure.com pk-pomosch.ru
privetsochi.ru playlott.com
productarium.com pod-muzyku.club
proposal-engine.com poptool.net
puzzleweb.ru poquoson.org
quelle.ru pornobrazzers.biz
rank-checker.online pornoklad.ru
realitykings.com posible.net
reimageplus.com pretty-mart.com
remmling.de printie.com
renewablewealth.com privacyassistant.net
rentalcarnavi.info produkto.net
rentehno.ru proekt-gaz.ru
rep-am.com prom23.ru
responsive-test.net promodj.com
respublica-otel.ru prostitutki-tyumeni.org
retailwith.com putevka24.ru
retreatia.com quickloanbank.com
reversing.cc rada.ru
richinvestmonitor.com razyboard.com
riralmolamsaca.tk replica-watch.ru
rockprogblog.com resultshub-a.akamaihd.net
romhacking.ru rixpix.ru
rumamba.com rock-cafe.info
ru-dety.ru
rubanners.com
rusenvironmental.net rusenvironmental.net
russia-tao.ru rustic-quiver.win
russintv.fr rvi.biz
sady-urala.ru s-forum.biz
sammyweaver.com sale-japan.com
sbtdesign.co.uk samolet.fr
scanner-alex.top sampleletters.net
scanner-donald.top sbf441.com
scanner-jessica.top sbprabooks.com
scanner-marwin.top scanner-jack.top
scanner-olga.top scanner-olga.top
scrnet.biz.ua scanner-walter.top
se.bnt-team.com scanspyware.net
searchimage.co schlampen-treffen.com
seksotur.ru scopich.com
seo-2-0.com sdelai-prosto.ru
servingnotice.com search.xtconnect.com
sexreliz.com searchimpression.com
silvercash.com semaltmedia.com
simpoed.ufop.br seoexperimenty.ru
sitebeam.net seorankinglinks.us
sledstvie-veli.net seorankinglinks.xyz
small-games.biz sergiorossistore.online
snjack.info servicecenter.co.ua
social-fun.ru sex-sex-sex5.com
socialtrade.biz sex-tracker.com
sockshares.tv sexblog.pw
softxaker.ru sexiporno.net
soheavyblog.com sexkontakte-seite.com
spacash.com sexsaoy.com
spidtest.org sexy-screen-savers.com
sport7777.net sexystrippe.info
stardevine.com sfj-ror.no
stariy-baku.com shemale-sex.net
sterva.cc shiksabd.com
store-rx.com shitmovs.com
stretchmate.net shitting.pro
stroicol.net si-unique.com
stroyplus.ru sibecoprom.ru
signoredom.com
sildenafil-tadalafil.info
simon3.ru
site-speed-check.site
site.ru
sizeplus.work
sklad-24.ru
skuteczna-dieta.co.pl
skyway24.ru
slavia.info
slomm.ru
slowmac.tech
social-search.me
socialbookmarksubmission.org
softomix.com
songplanet.ru
sortthemesitesby.com
sotechco.co
spasswelt.xyz
splendorsearch-a.akamaihd.net
spywareit.com
stats-public.grammarly.io
steame.ru
strana-krasoty.ru
studentguide.ru
studiofaca.com
studiokamyk.com.pl studiokamyk.com.pl
stuff-about-money.com sucsesofinspiration.com
sundrugstore.com suggest-keywords.com
superkanpo.com surffoundation.nl
supers.com.ua tacbelarus.ru
support.nopeas.sk tam-gde-more.ru
surgut.zrus.org
swingerseiten.com
swinginwithme.ru
tamada69.com
taqplayer.info taqplayer.info
tastyfoodideas.com tattooha.com
tech4master.com tds-advert005.info
testingads.pro the-usa-games.blogspot.com
thaismartloan.com thegamerznetwork.com
thenetinfo.com thin.me.pn
tiens2010.ru threecolumnblogger.com
titangel-vietnam.com tizanidine4mg.blogspot.com
tizanidinedrugtest.blogspot.com tizanidine4mgstreetvalue.blogspot.com
tizanidineduringpregnancy.blogspot.com tizanidine4mgtablets.blogspot.com
tizanidinehcl2mgtablet.blogspot.com tizanidinehcl4mgtab.blogspot.com
tizanidinestreetprice.blogspot.com tizanidinehydrochloride2mg.blogspot.com
tomatis.gospartner.com tizanidinesideeffectsweightloss.blogspot.com
top-study.work tobeyouday.win
top10-online-games.com toys.erolove.in
topanasex.com traffic100.com
topcar-krasnodar.ru traffmonster.info
torontoplumbinggroup.com trkur.com
torrentgamer.net
totu.info
toursmaps.com
toy-shop.top
trafficshaper.com
traffictrade.life
trk-4.net
trubywriting.com
tubeoffline.com
tula.mdverey.ru tula.mdverey.ru
univerfiles.com twittruth.com
uprour.com tytoona.com
uptime-gamma.net u-cheats.ru
urdoot.win ua-company.ru
uasb.ru
ublaze.ru
um-razum.ru
urll.eu urll.eu
urlopener.com urlopener.com
usdx.us uroffer.link
uzungil.com v-doc.co
validccseller.com
valmetrundan.se valmetrundan.se
veopornogratis.xxx vandrie-ict.nl
viandpet.com vbabule.net
virus-respirators.com vbtracker.net
visitcambridge.org vduplo.ru
via-energy-order.com
viberdownload10.com
videos-for-your-business.com
vipms.ru
virtuagirl.com
viteonlusarezzo.it viteonlusarezzo.it
vltai.com voronezh.xrus.org
vozbujdenie.com vroze.com
vpnhowto.info watchinf.com
vse-pesni.com watchmyfb.pl
wdrake.com wdfdocando.com
webinstantservice.com webads.co.nz
webpromotion.ae webadvance.club
website-speed-up.site westvilletowingservices.co.za
wejdz-tu.pl whengirlsgowild.com
wemedinc.com whereverdesperate.gq
wetgames.ru white-truck.life
winwotgold.pl wildworld.site
wiosenny-bon-1500.pl wiosenny-bon-1500.pl
wladimirpayen.com witclub.info
word-vorlagen.net wormix-cheats.ru
wordpresscore.com www.ehscloud.cn
www.kabbalah-red-bracelets.com
www.pinnacle-bets.com
x-diesel.biz x-diesel.biz
x-lime.net xblog.in
xcc24.pl xmlinde.com
xcombear.ru xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xerox-douglas.cf xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai
xmnb.net xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xn--80ahdheogk5l.xn--p1ai xn--lifehacer-1rb.com
xn--l1aengat.xn--p1ai xnxx699.com
xxxdatinglocal.us xtremeeagles.net
yellowstonevisitortours.com xxx-treker.ru
yhit.press xxxmania.top
yougetsignal.com xxxtube69.com
your-bearings.com yaoguangdj.com
yourporngay.com youbrainboost.asia
yourtemplatefinder.com yougotanewdomain.com
yurgorod.ru youporn-forum.ga
zajm-zalog-krasnodar.ru youporn-ru.com
zapatosenventa.info yur-p.ru
zapiszto.pl zaimite.ru
zeroredirect.com zaobao.com.sg
zeroredirect12.com
zeroredirect9.com zeroredirect9.com
zhuravlev.info zhorapankratov7.blogspot.com
zolotoy-lis.ru zigarettenonlinekaufen1.bloog.pl
zoompegs.com zone-kev717.info
zscaler.net
zygophyceous.womanstars.site

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
12masterov.com 1-free-share-buttons.com
3rbseyes.com 101lesbian.xyz
7minuteworkout.com 11pikachu.ru
8jn.dating-app.ru 123any.com
aad1f4acb0a373420d9b0c4202d38d94fa.com 4senses.co
abclauncher.com 7xc4n.com
abtasty.com adultfriendfinder.com
adexprts.com advertisingtag.net
adimmix.com afftrack001.com
ads-seo.men aghanyna.com
agysya.ru
ahrefs.com
alborzan.com alborzan.com
alekseevec.ru allpdfmags.net
alfa9.com allstatesugarbowl.org
alfabot.xyz alot.com
allfinweb.com alphahoverboards.com
altamayoztourism.com amateurlivechat.org
am15.net ampower.me
anjalika.co.in anal-acrobats.hol.es
apelsinnik.website android-systems.ru
apps-analytics.net apiwebwebgetcom-a.akamaihd.net
apptester.tk appartement-stumm.at
arcteryxsale.online atmagroup.ru
artparquet.ru audiofree.ru
astana.xxxkaz.org azte.ch
autotrends.today back.dog
avkzarabotok.com bannerswap.com
azbuka-mo.ru battlecarnival.su
b00kmarks.com battleforupdating.site
bad-stars.net bavsac.com
bang-hotties.com bear.gotcher.us
basisches-wasser.net bfz.biz
beachdriveblog.com bg6s0.com
best-coupon-offer.com bgviagrachrx.com
best-mam.ru bio-market.kz
biteg.xyz bizru.info
bjgugu.net.ua blavia.00author.com
blackwitchcraft.ru bloggerads.net
bmw.afora.ru blogos.kz
bnt-team.com
bodybuilding-shop.biz
bookmaker-bet.com bookmaker-bet.com
boom-celebs.com bradleylive.xyz
bpro1.top breastaugmentation.co.za
brendbutik.ru bristolhotel.com.ua
burger-tycoon.com btc4u.ru
buynorxx.com buron.pw
buzzzg.men c-gainsbourg.com
bvlgariring.xyz cakemediahq.com.au
cardiosport.com.ua cdnanalytics.xyz
cartierlove2u.com cementaresearch.se
cheapjerseysbizwholesale.us cheap-pills-norx.com
choosecuisine.com cheapestjerseys-wholesale.com
cityadspix.com checkm8.com
classicakuhni.ru chiblackhawks-jerseys.com
clickbank.net christianlouboutinoutlet.win
clickbanksites.info cialgenisrx.com
coccoc.com cialisovercounteratwalmartusa.com
cookie-law-enforcement-ee.xyz club-musics.ru
coolyarddecorations.com coinsspb.com
cowblog.fr compliance-alexa.top
compliance-jessica.top
comsysnet.com
coop-gamers.ru
cpajunkies.com cpajunkies.com
craftinsta.ru cyclobenzaprinevstizanidine.blogspot.com
credit.co.ua daooda.com
danashop.ru daretodonate.co
ddlmega.net daydream-studio.ru
dealitright.click ddospanel.com
devochki.top ddsoldes.fr
doreenblog.online deadlinkchecker.com
droidlook.net deutschland123.de
drunkmoms.net digital-video-processing.com
electrik-avenue.com discovertreasure-a.akamaihd.net
elektrozigarette1.dreamwidth.org disruptingdinnerparties.com
elite-sex-finders.com diusyawiga.tk
energydiet-info.ru divisioncore.com
enternet.ee dms-sw.ru
ereko.ru dollartree.info
europages.com.ru domaincdn.xyz
exchangeit.gq dumuelave.xyz
femmesdenudees.com eandsgallery.com
fettefrauen.net erectile.bid
fickkontakte.org espaceinventoristes.com
fighrofacciufreesig.ga etm-consult.de
filesvine.com evaashop.ru
filunika.com.ru exlarseva.webblog.es
freenode.info fickenbumsen.net
froggytube.com filmfanatic.com
fsalas.com financepoints.eu
fullgirl.ru findpics.pw
fun-mobi.pl fingerprintjs.com
gamerextra.com fireads.men
gateway.zscalertwo.net flytourisme.org
gaytube.com forms-mtm.ru
gdebestkupit.ru foxjuegos.com
geckoandfly.com frcls.fr
goforexvps.com free-share-buttons.blogspot.com
good-mummy.ru free-social-buttons.com
googglet.com galeria-zdjec.com
googlefeud.com gearcraft.us
googst2.ru genetworx.com
hanink.biz.ly girlsfuckdick.com
harmonyglen.us gkvector.ru
hausfrauensex18.com gloverid.site
hawaiisurf.com guarrasdelporno.xxx
hd720kino.ru healgastro.com
healmytrauma.info homik.pw
highspeed5.net htmlcorner.com
historichometeam.com ideoworld.org
hobbyhuren24.net igru-xbox.net
hostcritique.com imgata.com
howlongdoestizanidinestayinyoursystem.blogspot.com inclk.com
hundejo.com indianmedicaltourismshop.com
iamsport.org instasexyblog.com
idealtits.net invest-pamm.ru
incanto.in.ua ispac.org
intellektmedia.at itservicesthatworkforyou.com
intimshop-fantasy.ru ivanovo.zrus.org
isitpaleo.info
itrevolution.cf
jav-idol.com
joessmogtestonly.com joessmogtestonly.com
katjimej.blog.fc2.com jose.mulinohouse.co
kellyonline.xyz jurajskie.info
keywordblocks.com justdating.online
kino2018.club ketoanhanoi.info
konkursowo-24.pl keywordsdoctor.com
keywordsuggests.com
keywordteam.net
kinoduh.ru
kinofak.net
lafourmiliaire.com
lakomka.com.ua lakomka.com.ua
lancheck.net laptoper.net
lasvegaslockandsafe.com
lemon-ade.site
lightinghomes.net lightinghomes.net
linkbolic.com linkarena.com
linkredirect.biz lomb.co
lion.misba.us lombardfinder.ru
longgreen.info luxmagazine.cf
losangeles-ads.com mabdoola.blogspot.com
lostfilm-online.ru macdamaged.space
macrotek.ru malwareremovalcenter.com
markjaybeefractal.com mebelcomplekt.ru
mb140.ru
mebeldekor.com.ua
meble-bogart.info
meetlocalchicks.com
mjchamonix.org mjchamonix.org
mncrftpcs.com mmofreegames.online
monarhs.info mobile.ok.ru
moneyteam24.com mundoaberrante.com
morefastermac.trade music7s.me
mototsikl.org musicktab.com
mulberryoutletonlineeu.com my-floor.in.ua
myghillie.info myperiod.club
net-radar.com myplaycity.com
new-post.tk myshopmatemac.com
nalogovyykodeks.ru
nextconseil.com nextconseil.com
nikitsyringedrivelg.pen.io noclegonline.info
o-dachnik.ru okeinfo.online
obiavo.uz olvanto.ru
obsessionphrases.com omoikiri-japan.ru
of-ireland.info omsk.xrus.org
ownshop.cf oren-cats.ru
paclitor.com ostroike.org
palocco.it ourtherapy.ru
pdfprof.com par-fallen.ga
photorepair.ru pc-virus-d0l92j2.pw
picphotos.net pc4download.co
picsfair.com pcboa.se
plastgranar.nu penisvergrotendepillennl.ovh
poems.com.ua perfectplanned.com
pogruztehnik.ru petedrummond.com
poisk-zakona.ru pinup-casino1.ru
pio.polytopesexempt.com
pl.id-forex.com
polybuild.ru
polytopesexempt.com
pons-presse.com
porndairy.in porndairy.in
porno-play.net pornogig.com
privacylocationforloc.com postclass.com
proekt-mos.ru prizeestates.cricket
prointer.net.ua pukaporn.com
projefrio.com.br q-moto.ru
prostitutki-astana.org qld10000.net
prostitutki-kiev.org relatodelpresente.com.ar
pussyfleet.com remybutler.fr
quick-offer.com replicalouboutin.xyz
radiodigital.co retailwith.com
realresultslist.com retreatia.com
rennlist.com rialp.getenjoyment.net
rewardpoll.com russintv.fr
rezeptiblud.ru samo-soznanie.ru
ruex.org.ua satoristudio.net
rusvideos.su savefrom.com
sardinie.us scanhub.ru
scanner-margo.top scanner-barak.top
search.xtconnect.com searchimage.co
seoexperimenty.ru searchinterneat-a.akamaihd.net
seonetwizard.com semaltmedia.com
serw.clicksor.com seomarketings.online
sexblog.pw sexpornotales.com
site5.com shitmovs.com
slujbauborki.ru sledstvie-veli.net
social-s-hhh.xyz social-buttons.com
social-s-iii.xyz stroicol.net
spb.afora.ru surfbuyermac.com
spinazdrav.ru tasteidea.com
stop-zavisimost.com the-usa-games.blogspot.com
stpolice.com ticketsys.inetwd.com
superiends.org timhost.ru
supervesti.ru tizanidinesideeffects.blogspot.com
surintech.ac.th topflownews.com
tattoo33.ru torrent-newgames.com
teletype.in traffictrade.life
tengohydar.tk tubeline.biz
testbotprocessor44.com tvand.ru
thefarmergame.com typer.one
thegioixekhach.com upproar.com
thepantonpractice.co.uk uvozdeckych.info
tizanidine4mguses.blogspot.com valuado.com
tizanidinehcl2mgtablet.blogspot.com via-energy-order.com
tizanidinehcl4mgtablet.blogspot.com video-chat.cn
top10-online-games.com vielporno.net
topdownloads.ru vmnmvzsmn.over-blog.com
toproadrunner5.info vriel.batcave.net
topvidos.ru wallpapersist.com
tpu.ru warezaccess.com
tracksz.co waterefficiency.co
traffic100.com waycash.net
traffique.net web-analytics.date
treasuretrack-a.akamaihd.net webjam.com
twiclub.in website-speed-checker.site
uchil.net whatzmyip.net
uggsale.online winwotgold.pl
unrealcommander.org wladimirpayen.com
uptime.com xaylapdiendanang.com
urccvfmc.bloger.index.hr xchangetrak.com
viberdownload10.com xep.info
videochat.cafe xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai
videositename.com xn--80aeahghtf8ac5i.xn--p1ai
vipps.com.my xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai
vvpg.ru xnxx699.com
w-journal.ru yellowstonesafaritours.com
watch-movies.ru yougotanewdomain.com
webalan.ru yourserverisdown.com
webkeyit.com yourtemplatefinder.com
weburlopener.com zaim-pod-zalog-krasnodar.ru
whengirlsgowild.com zarabotok--doma.ru
whipme.yopoint.in zeroredirect12.com
wovis.site zigarettenonlinekaufen2.bloog.pl
wrz0iuebwhp5fg.freeddns.com zoompegs.com
wygraj-teraz.com
youtubedownload.org
zaimhelp.ru
zapiszto.pl
zerocash.msk.ru
zeroredirect11.com
zeroredirect9.com
zhacker.net
zoomovies.org

167
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -8,6 +8,86 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -414,83 +494,6 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
@ -581,14 +584,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.135 0; 10.1.0.219 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.02.3565 ### Version: V4.2023.02.3566
### Updated: Thu Feb 2 10:03:46 UTC 2023 ### Updated: Fri Feb 3 10:03:26 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.02.3565 ### Version: V4.2023.02.3566
### Updated: Thu Feb 2 10:03:46 UTC 2023 ### Updated: Fri Feb 3 10:03:26 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##