vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3463 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-10 22:07:06 +00:00
parent 893150d83e
commit 63bdd24177
46 changed files with 98798 additions and 98798 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3462 #### Version: V4.2022.11.3463
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3462 #### Version: V4.2022.11.3463
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3462 #### Version: V4.2022.11.3463
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12214
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3462 #### Version: V4.2022.11.3463
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12218
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12218
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12218
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12218
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 1; 10.1.3.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 0; 10.1.3.48 0;
127.0.0.1 0; 127.0.0.1 0;

12218
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 1; 10.1.3.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 0; 10.1.3.48 0;
127.0.0.1 0; 127.0.0.1 0;

12218
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 1; 10.1.3.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 0; 10.1.3.48 0;
127.0.0.1 0; 127.0.0.1 0;

12218
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3462 3463

12218
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12218
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.97.116.40 0; 20.10.30.231 0;

12218
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 1; 10.1.3.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 0; 10.1.3.48 0;
127.0.0.1 0; 127.0.0.1 0;

12218
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 1; 10.1.3.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 0; 10.1.3.48 0;
127.0.0.1 0; 127.0.0.1 0;

12218
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 1; 10.1.3.48 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 0; 10.1.3.48 0;
127.0.0.1 0; 127.0.0.1 0;

12218
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.97.116.40 0; 20.10.30.231 0;

12218
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.97.116.40 0; 20.10.30.231 0;

12218
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 1; 10.1.3.48 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BacklinkCrawler EyeNetIE
Barkrowler FDM
DIIbot Keyword Density
MQQBrowser LinqiaMetadataDownloaderBot
Mister PiX MIIxpc
Re-re MSIECrawler
Web Enhancer Papa Foto
WebLeacher Szukacz
ZmEu Vagabondo
dataforseo.com WebSucker

322
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
ASPSeek 360Spider
Abonti 404checker
Aboundex ADmantX
Aboundexbot
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
Aipbot Aipbot
Alligator
AlphaBot
Anarchie
Anarchy Anarchy
Ankit Anarchy99
Anthill Anthill
Aspiegel
Asterias
Atomseobot Atomseobot
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BLEXBot BBBike
BDFetch
BackDoorBot
BackStreet BackStreet
BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler
Badass
Barkrowler Barkrowler
BatchFTP BatchFTP
Bigfoot Battleztar Bazinga
Bitacle BetaBot
Blow Blow
BlowFish Bolt
Buddy Brandprotect
Buck
BuiltBotTough
Bullseye Bullseye
BunnySlippers
CCBot CCBot
CSHttp CheeseBot
CazoodleBot
Cegbfeieh
CherryPicker CherryPicker
ChinaClaw Chlooe
Claritybot Claritybot
Cocolyzebot Cliqzbot
Copier Cogentbot
Collector
CopyRightCheck
Copyscape
Cosmos Cosmos
Craftbot
Crawling at Home Project Crawling at Home Project
Curious CrunchBot
Custo DIIbot
DSearch
DataCha0s DataCha0s
Devil DatabaseDriverMysqli
Disco Demon
Discoverybot DigitalPebble
Dispatch Discobot
DnBCrawler-Analytics DittoSpyder
DnyzBot DnyzBot
DomCopBot DomainStatsBot
DomainAppender Domains Project
Download Wonder Dragonfly
Drip Ecxi
EMail Siphon EirGrabber
Ebingbong Express WebPictures
Exabot
ExtLinksBot ExtLinksBot
ExtractorPro ExtractorPro
FDM
FHscan FHscan
Fimap FlashGet
Fyrebot Flunky
GalaxyBot Foobot
GetRight Freeuploader
GetWeb GT::WWW
Gigabot Genieo
Go!Zilla GermCrawler
Getintent
Go-Ahead-Got-It Go-Ahead-Got-It
Grafula GoZilla
GrapeshotCrawler GrabNet
Grabber
GrapeFX
GridBot
HMView HMView
HTMLparser
HTTrack
Haansoft
HaosouSpider HaosouSpider
Harvest HybridBot
Havij IDBot
Hloader
Humanlinks
Id-search Id-search
IlseBot IlseBot
Image Fetch Image Fetch
IndeedBot InfoTekies
InterGET Iria
Internet Ninja Iskanie
JamesBOT IstellaBot
JetCar Joomla
JustView Jyxobot
Keybot Translation-Search-Machine
Kinza
Kozmosbot Kozmosbot
Leap Larbin
LeechGet LeechGet
Likse Libwhisker
LieBaoFast
LinkScan LinkScan
Linkbot
LinkextractorPro
LinksManager LinksManager
LinqiaRSSBot LinqiaMetadataDownloaderBot
Lipperhey Spider MJ12bot
MQQBrowser
MSFrontPage MSFrontPage
MSIECrawler
MTRobot MTRobot
Mag-Net Mag-Net
Mail.RU_Bot MarkWatch
Majestic SEO
Majestic-SEO
Mass Downloader Mass Downloader
Masscan Mata Hari
MauiBot MauiBot
Mb2345Browser
MeanPath Bot
Mediatoolkitbot
MegaIndex.ru
Metauri Metauri
Microsoft Data Access MicroMessenger
Minefield Microsoft URL Control
Moblie Safari MolokaiBot
Mojolicious Morfeus Fucking Scanner
Mozlila
Mr.4x3
Msrabot
Musobot
NICErsPRO NICErsPRO
NPbot
Nameprotect
Navroad Navroad
NearSite NearSite
Net Vampire Needle
NetLyzer
NetSpider
Nettrack
Netvibes Netvibes
NextGenSearchBot
Nikto
Nimbostratus
Ninja
Nmap
Not
Offline Explorer
Offline Navigator Offline Navigator
OnCrawl
Openfind
OrangeBot
OutclicksBot
OutfoxBot
PECL::HTTP PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageAnalyzer
PageGrabber
PageThing.com
Pandalytics
Panscient
Papa Foto
PeoplePal
Picscout
PictureFinder PictureFinder
Pimonster
Pixray Pixray
PleaseCrawl PleaseCrawl
Pockey Pockey
Psbot ProPowerBot
Pump Pu_iN
Quick-Crawler Quick-Crawler
RSSingBot
RankActive RankActive
RankActiveLinkBot RankurBot
Rankivabot ReGet
RealDownload
Reaper Reaper
RepoMonkey RebelMouse
Ripper Recorder
RocketCrawler SEOkicks
Rogerbot
SEOkicks-Robot SEOkicks-Robot
SEOprofiler
SISTRIX
ScoutJet ScoutJet
ScrepyBot Searchestate
Semrush
SemrushBot SemrushBot
Seomoz
Shodan Shodan
SiteLockSpider Siphon
Sitevigil Site Sucker
SiteSnagger
Sitebeam
SlySearch SlySearch
Snake
Snapbot Snapbot
Spammen Snoopy
Spinn3r Sosospider
Sottopop
SpaceBison
SputnikBot SputnikBot
Sucker Sucuri
SuperHTTP SuperBot
Surfbot Suzuran
SurveyBot
Swiftbot Swiftbot
T0PHackTeam T0PHackTeam
TeleportPro Teleport
Telesphoreo
Telesphorep Telesphorep
TheNomad TheNomad
Titan Thumbor
Trendiction Toweyabot
TurnitinBot Tracemyfile
TwengaBot Turingos
Twice Turnitin
Typhoeus
URLy Warning URLy Warning
UnisterBot UnisterBot
Upflow
V-BOT V-BOT
Vacuum VB Project
VCI
Vagabondo Vagabondo
VelenPublicWebCrawler VoidEYE
VidibleScraper
Voil Voil
WISENutbot Voltron
Wallpapers WASALive-Bot
WWW-Collector-E
WWW-Mechanize
Wallpapers/3.0 Wallpapers/3.0
WeSEE Web Fuck
Web Auto Web Pix
Web Enhancer WebCollage
WebEnhancer WebCopier
WebFuck
WebPix
WebSauger WebSauger
WebZIP WebStripper
WebWhacker
Webalta Webalta
WebmasterWorldForumBot
WebsiteQuester
Whack Whack
Whatweb Whacker
Widow WinHTTrack
WiseGuys Robot
Wonderbot
Woobot
Wotbox Wotbox
Wprecon Xaldon WebSpider
Xenu Xenu
Zermelo Zermelo
Zitebot Zeus
ZoomBot
ZoominfoBot ZoominfoBot
ZumBot arquivo-web-crawler
ZyBorg
autoemailspider autoemailspider
backlink-check com.plumanalytics
cah.io.community crawl.sogou.com
check1.exe crawler4j
clark-crawler dataforseo.com
cognitiveseo
crawler.feedback
demandbase-bot demandbase-bot
facebookscraper domainsproject.org
evc-batch
heritrix
internetVista monitor internetVista monitor
ips-agent ips-agent
iubenda-radar linkdexbot
muhstik-scan lwp-request
meanpathbot
mediawords
netEstate NE Crawler netEstate NE Crawler
pcBrowser page scorer
probe-image-size scalaj-http
ripz
s1z.ru
scan.lol scan.lol
seobility seobility
seocompany.store
seoscanners seoscanners
seostar sexsearcher
sitechecker.pro sitechecker.pro
sogouspider sogouspider
sysscan sysscan
ubermetrics-technologies.com tAkeOut
trendiction.com
trendiction.de
voyagerx.com voyagerx.com
x22Mozilla webpros.com
zauba.io x09Mozilla
zgrab zgrab

298
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider
404enemy 404enemy
80legs
AIBOT
ALittle Client
ASPSeek ASPSeek
Abonti Aboundexbot
AiHitBot Acunetix
AhrefsBot
Aipbot Aipbot
Alexibot Alexibot
AllSubmitter
Alligator Alligator
Anarchie
Anarchy Anarchy
Anarchy99
Anthill Anthill
Asterias Apexoo
Attach Aspiegel
AwarioRssBot Atomseobot
BackStreet AwarioSmartBot
BBBike
BLEXBot
BackDoorBot
BackWeb BackWeb
Badass BacklinkCrawler
Barkrowler Bandit
Battleztar Bazinga Battleztar Bazinga
BetaBot
Bitacle
Black Hole
BlackWidow BlackWidow
Blackboard Blackboard
Bolt BlowFish
BotALot BuiltWith
Buddy CATExplorador
BuiltBotTough CODE87
CSHttp
CazoodleBot CazoodleBot
CheeseBot Chlooe
Claritybot
Cloud mapping Cloud mapping
Cogentbot Cogentbot
CopyRightCheck Copier
Curious Copyscape
Cosmos
Crawling at Home Project
CrazyWebCrawler
CrunchBot
Custo
DBLBot DBLBot
DTS Agent DIIbot
DatabaseDriverMysqli DataCha0s
Devil
Dirbuster Dirbuster
Disco Discobot
DittoSpyder DittoSpyder
DnyzBot
DomCopBot DomCopBot
DomainAppender DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
Ebingbong Ebingbong
Ecxi Ecxi
EirGrabber
Evil Evil
Exabot
Express WebPictures
ExtLinksBot
ExtractorPro
Extreme Picture Finder
EyeNetIE
Ezooms Ezooms
FHscan Firefox/7.0
FlashGet Foobot
Freeuploader
FrontPage
Fuzz Fuzz
FyberSpider G-i-g-a-b-o-t
Genieo Genieo
GetRight Getintent
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GrabNet
GrapeFX GrapeFX
GrapeshotCrawler GrapeshotCrawler
HMView HEADMasterSEO
HTMLparser
HTTP::Lite HTTP::Lite
HTTrack Haansoft
Harvest HaosouSpider
Hloader Hloader
HonoluluBot
Humanlinks Humanlinks
HybridBot
IDBTE4M IDBTE4M
IDBot Iblog
Image Fetch Id-search
Image Sucker Image Sucker
Indy Library
InfoNaviRobot
InfoTekies InfoTekies
Intelliseek Intelliseek
InterGET
InternetSeer InternetSeer
Iria Iria
JOC Web Spider Kenjin Spider
JamesBOT Keybot Translation-Search-Machine
JennyBot LWP::Simple
JetCar
Joomla
Jorgee
Jyxobot
Keyword Density
LNSpiderguy
Larbin Larbin
Leap
LeechGet LeechGet
Libwhisker Lftp
LieBaoFast LieBaoFast
Likse Lightspeedsystems
Linkbot LinkScan
LinkextractorPro LinkextractorPro
LinksManager LinqiaRSSBot
LinqiaMetadataDownloaderBot Lipperhey
Lipperhey Spider
Lmspider Lmspider
Ltx71 Ltx71
MJ12bot
MSIECrawler
Majestic12
MarkWatch MarkWatch
MeanPath Bot Mass Downloader
MegaIndex.ru Masscan
Mozlila MauiBot
Mr.4x3 Mb2345Browser
NPbot Meanpathbot
Name Intelligence Mediatoolkitbot
Microsoft Data Access
Minefield
Mister PiX
Moblie Safari
Musobot
NearSite
NetAnts NetAnts
NetLyzer NetSpider
NetMechanic
NextGenSearchBot
Nibbler
Niki-bot
NimbleCrawler NimbleCrawler
Nimbostratus Nimbostratus
Ninja
Not
Octopus
OnCrawl
OpenVAS
Openfind Openfind
Openvas
OrangeBot OrangeBot
OutclicksBot OutclicksBot
OutfoxBot OutfoxBot
PHPCrawl
POE-Component-Client-HTTP
Page Analyzer
PageGrabber PageGrabber
Pandalytics PageThing.com
Panscient Picscout
Pavuk Probethenet
Pi-Monster Psbot
Piepmatz
Pimonster
Pixray
PleaseCrawl
ProPowerBot
Pu_iN Pu_iN
PxBroker RankingBot
PyCurl RankingBot2
Quick-Crawler Re-re
ReGet ReGet
RealDownload RealDownload
Reaper Reaper
RebelMouse
RedesScrapy
RepoMonkey RepoMonkey
SEOlyticsCrawler Ripper
SEOstats RocketCrawler
SMTBot Rogerbot
ScanAlert Scanbot
Searchestate Scrapy
Screaming
ScrepyBot
SearchmetricsBot
Seekport
Semrush Semrush
SemrushBot SemrushBot
SeobilityBot SentiBot
Seomoz
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteSnagger
SiteSucker
Siteimprove Siteimprove
Snake Sitevigil
Snapbot
Snoopy Snoopy
SocialRankIOBot Sosospider
Sogou web spider Spbot
SpaceBison Sqlworm
Spammen
SpankBot
Sqworm Sqworm
Stripper Stripper
Sucker
Sucuri Sucuri
SurveyBot SuperHTTP
Szukacz Suzuran
Swiftbot
T8Abot
Teleport Teleport
Telesoft
Telesphoreo Telesphoreo
TheNomad The Intraformant
Thumbor Thumbor
Trendiction Tracemyfile
True_Robot True_Robot
URLy Warning Turingos
Turnitin
Twice
URLy.Warning URLy.Warning
V-BOT UnisterBot
Vacuum VeriCiteCrawler
Vagabondo VidibleScraper
VelenPublicWebCrawler Voil
Virusdie Voltron
WASALive-Bot
WBSearchBot WBSearchBot
WISENutbot
WWW-Collector-E WWW-Collector-E
Wallpapers/3.0 WWW::Mechanize
Wallpapers
WeSEE
Web Collage Web Collage
Web Fetch Web Fetch
Web Fuck
Web Pix Web Pix
WebAuto Web Sauger
Web Sucker
WebBandit
WebCollage
WebCopier WebCopier
WebEnhancer
WebFetch WebFetch
WebImageCollector
WebLeacher WebLeacher
WebPix WebSauger
WebReaper WebStripper
WebSucker WebWhacker
Webalta Webalta
Webshag
Website Quester Website Quester
WebsiteExtractor
WebsiteQuester
Webster Webster
Whack
Widow
Wonderbot Wonderbot
Woobot Woobot
Wotbox
Xaldon WebSpider
Xaldon_WebSpider
Zeus Zeus
Zitebot ZoominfoBot
ZoomBot archive.org_bot
adscanner arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider
backlink-check backlink-check
coccocbot coccocbot
cognitiveseo com.plumanalytics
crawl.sogou.com crawl.sogou.com
domainsproject.org domainsproject.org
internetVista monitor evc-batch
ips-agent isitwp.com
magpie-crawler iubenda-radar
linkdexbot
meanpathbot meanpathbot
page scorer mediawords
polaris version muhstik-scan
netEstate NE Crawler
probe-image-size
ripz ripz
satoristudio.net
seobility seobility
seostar seoscanners
sexsearcher sitechecker.pro
sogouspider
sp_auditbot sp_auditbot
spyfu spyfu
sysscan
trendiction.de
ubermetrics-technologies.com
webgains-bot
webmeup-crawler
webprosbot webprosbot
x09Mozilla
x22Mozilla x22Mozilla
xpymep1.exe
zauba.io zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
duplicashapp.com bitcoinpile.com
ertelecom.ru buqayy0.livejournal.com
fxund.us healmytrauma.info
locustdesign.co ku6.com
mailemedicinals.com nlfjjunb5.ru
mega-bony-2017.pl roofers.org.uk
sk.golden-praga.ru titan-ads.life
testingads.pro worldtraveler.world
vipsiterip.org xn--c1acygb.xn--p1ai
www.888.com zone-kev717.info

916
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
101billion.com 1pamm.ru
101lesbian.xyz 23kw.ru
123cha.com 4senses.co
123movies.love 5i2.net
1j7740kd.website 7zap.com
28n2gl3wfyb0.ru DomainStatsBot
3rbseyes.com aa625d84f1587749c1ab011d6f269f7d64.com
4pp13.com abtasty.com
76brighton.co.uk adcash.com
7a2rnuey1tw9ar.ru adssafeprotected.com
7search.com
7yue.org
98oi.ru
aa9d046aab36af4ff182f097f840430d51.com
abclauncher.com
abwa.tk
actionnooz.com
actulite.com
adf.ly
adktrailmap.com
admeasures.com
adultactioncam.com
adultfullhd.com adultfullhd.com
adventureparkcostarica.com adverster.com
akama.com affordablewebsitesandmobileapps.com
agadelha.com.br
agreda.pluto.ro
agroeconom.kz
aibolita.com
ald2014.org ald2014.org
alevinefesleri.com alevinefesleri.com
alif-ba-ta.com aliexpresscashback.club
allbizne.co.ua allbizne.co.ua
allnews.md allfinweb.com
alpharma.net
alta-realestate.com alta-realestate.com
ample-awards-today.us animalrank.com
ampower.me
anal-acrobats.hol.es
anapa-inns.ru
animaltoplist.com
antons-transporte.de
apartmentbay.ru
apiappenableinfo-a.akamaihd.net apiappenableinfo-a.akamaihd.net
apimegabrowsebiz-a.akamaihd.net apiholdingmypage-a.akamaihd.net
appfixing.space apps-analytics.net
arabgirls.adultgalls.com arendatora.ru
aramaicmedia.org asia-forum.ru
arate.ru avkzarabotok.info
architecturebest.com azte.ch
arendas.net babyguns.ru
arkkivoltti.net baikaleminer.com
articlesdirectoryme.info balkanfarma.org
asdfz.pro baltgem.com
asrvvv-a.akamaihd.net barnfurnituremart.com
asscenihotosidea.blogspot.co.za
audiofree.ru
autobudpostach.club
aviav.org
avirasecureserver.com
avkzarabotok.com
avtocredit-legko.ru
awaybird.ru
azartmsl.com
azbukafree.com
bagcionderlab.com
banki76.ru
basedecor.ru basedecor.ru
bbsoldes.fr bdsmgalls.net
beachfix.co beatifullandscape.co
beclean-nn.ru beauty-b0x.pl
beget.tech becuo.com
belsetirehafi.tk best-games.today
benchmarkcommunications.co.uk bfz.biz
bestbuy.ca bgviagrachrx.com
bestcalovebracelet.cn bharatdefencekavach.com
bestcoin.cc bidr.trellian.com
bestfortraders.com biglistofwebsites.com
bestwaystogetpaid.us bloggedporn.com
bibys.com blogqpot.com
bigcities.org blogsfunky672.weebly.com
bigpictures.club brains2.biz
billiard-classic.com.ua brandedleadgeneration.com
bird1.ru break-the-chains.com
bitcoinremote.com
bitcoins-live.ru
blog.remote-computer.de
bloggerads.net
blogig.org
boazpower.com
bocoarchives.org
brendbutik.ru brendbutik.ru
bristolhostel.com brus.city
btvn.ru buchananshardware.com
bupropion-sr-150-mg.us bukleteg.xyz
buron.pw bum.com.ru
buyessaynow.biz burger-imperia.com
bvlgariring.xyz buy-cheap-online.info
bwlx.prepedu.cn carloans.com
cartierlove2u.com cartierlove.xyz
cartierloveringreplica.xyz casablancamanor.co.za
celebsopics.com cdncash.net
centre-indigo.org.ua cfacarrosserie74.com
cfcl.co.uk cheapestjerseysonwholesale.com
cgi2.nintendo.co.jp cheapjerseysfootballshop.com
changde.58.com cheapmoncler.xyz
chcu.net check-host.net
cheapmarmot.online chinese-amezon.com
chimiver.info christianlouboutinoutlet.win
cineacademy.ru cialischmrx.com
cinemaenergy-hd.ru cialisovercounteratwalmartusa.com
clothesforcash.com citetick.com
coccoc.com civilwartheater.com
cognitiveseo.com cleansearch.net
compliance-barak.top cmhomestayagency.com
compliance-don.top coinsspb.com
compliance-mark.top compliance-jessica.top
compliance-olga.top compliance-mary.top
compliance-viktor.top coolgramgoods.com
compliance-walt.top coolyarddecorations.com
computernetworksonline.com copro.pw
consorzioilmosaico.org countercrazy.com
converse.ddsoldes.fr cowlmash.com
cosmetic.donna7753191.ru credit.co.ua
countbertwistdisp26.soup.io customcollegeessays.net
cs-passion.pl cypernhuset.se
cwetochki.ru dailystrength.org
dairyindia.in dame-ns.kz
dandiyabeats.in daptravel.com
darodar.com datsun-do.com
data1.scopich.com deanmoore.ie
daydream-studio.ru
ddsoldes.fr
de.zapmeta.com
dekoration.us
dengi-pod-zalog-nedvizhimosti.ru
derevesendeco.com
destinationrealestate.com
detective01.ru
detoxmed24.ru detoxmed24.ru
deutschland123.de dickssportinggoods.com
dev.citetick.com dildofotzen.net
digital-video-processing.com div.as
disruptingdinnerparties.com divatraffic.com
divku.ru dmmspy.com
dokumentalkino.net dojki-hd.com
dolohen.com dominterior.org
domainsatcost.com donna7753191.ru
download-walpaperhd.blogspot.com downloadkakaotalk.com
downloadmefiranaratb1972.xpg.com.br druzhbany.ru
doxysexy.com dvrlists.com
e-poker-2005.com dyt.net
earthmagic.info e-avon.ru
ebooknovel.club e-c.al
ec-file.info e90post.com
ekspertmed.com eets.net
elektronischezigarettekaufen1.myblog.de efkt.jp
elektrozigarettekaufen2.blogse.nl efnor-ac.com
elmifarhangi.com electronicadirect.com
elvel.com.ua elektrischezigarette1.blog.pl
enge-fotzen.info elektrischezigaretten2.beeplog.com
enpolis.ru elektrozigarette2.wordpressy.pl
essay-writing.work elexies.info
essaypro.com elmacho.xyz
empirepoker.com
empis.magix.net
encodable.com
erotik-kostenlos.net
essayservicewriting.org
este-line.com.ua este-line.com.ua
euronis-free.com etur.ru
eurosamodelki.ru
evaashop.ru evaashop.ru
evrotekhservis.ru executehosting.com
exct.net ez8motelseaworldsandiego.com
exportshop.us f5mtrack.com
extremal-blog.com fanpagerobot.com
ezigarettekaufen2.yolasite.com faracontrol.ir
failingmarriege.blogspot.com farm26.ru
fakehandbags.xyz fatfasts-4tmz.com
fantasticpixcool.com fatmaelgarny.com
fearcrow.com fetishinside.com
fickluder69.com filesmonster.porn
filerockstar298.weebly.com filunika.com.ru
film-one.ru findsexguide.com
filmania-x.ru fivedwld.cf
financepoints.eu flowersbazar.com
firstsiteguide.com forex.osobye.ru
fodelsedagspresenter.nu forexunion.net
forex-indextop20.ru forzeronly.com
formulaf1results.blogspot.com fr-bearings.ru
forsex.info free-share-buttons.com
forum.tvmir.org free-share-buttons.top
foto-basa.com free411games.com
freakycheats.com freelinkbuilding.website.tk
free-fbook-traffic.com freesitetest.com
freefoto.ca freewebs.com
freegamesplay.online freewlan.info
freshwallpapers.info frivgame250.com
funnymama.com fuckingawesome.com
fxund.us fun2cell.net
galeon.com funnel.co.za
game-top.su gallery.rennlist.com
gamebackyard.com gay-file.com
gamerscorps.com gaz-voshod.ru
gateway.zscalerone.net gcup.ru
gemara.com generic-pills-online.com
gemgrab-a.akamaihd.net getaclueamerica.com
getmyads24.com getmyads24.com
girlspicsa.com gfaq.ru
gkvector.ru gheus.altervista.org
global-ics.co.za gk-atlant.info
globetrotting-culture.ru glastecfilms.com.my
golden-praga.ru glavprofit.ru
gorodservis.ru globalscam.ga
gotorussia.com goodprotein.ru
gourcy.altervista.org graphid.com
grtyi.com groupmoney.ru
gsasearchengineranker.xyz guidefs.ru
guod.me
guruofcasino.com guruofcasino.com
h2monline.com handicapvantoday.com
habermetre.com
happy.new.yeartwit.com happy.new.yeartwit.com
hauleddes.com hdimagelib.com
hazardky.net hdwalls.xyz
hdimagegallery.net headpress.ru
helicalpile.us healmytrauma.info
herehloadibs.cf hildinghr.se
hit-kino.ru histats.com
hmywwogw.bloger.index.hr hitsbox.info
howtoclean.club homelygarden.com
huhn.altervista.org homik.pw
humanelydrew.com horoshieokna.com
humbmosquina.tk hoterika.com
hut1.ru hotgirlhdwallpaper.com
hystersister.com ilovevitaly.xyz
ideibiznesa2015.ru images-graphics-pics.com
igithab.com intellekt21.ru
iklysha.ml
ilovevitaly.ru
imabase.com
imagerydatabase.com
imediadesk.com
impotentik.com
inclk.com
infazavr.ru
infobabki.ru
int.search.mywebsearch.com
intelhdgraphicsgtdrive6w.metroblog.com
iplogger.org
iscblog.info iscblog.info
iskalko.ru istizanidineacontrolledsubstance.blogspot.com
istripper.com izhevsk.xrus.org
j-times.ru jackpotchances.com
j33x.com jackwolfskinoutlet.online
jaxcube.info janerikholst.se
jeremyeaton.co javidol.site
jerseyswholesalechinalimited.com jetsli.de
kam-dom.ru keywordsdoctor.com
kamagragelusa.net
kazan.zrus.org
kerch.site
kiinomaniak.pl
kino-ecran.ru
kino-key.info kino-key.info
kino-rating.ru kino-rf.ru
kinoflux.net kinotorka.ru
kinoplen.ru kiwe-analytics.com
kirov.zrus.org komputernaya-pomosh-moscow.ru
kladrus.ru konica.kz
kmgamex.cf konpax.com
kozhniebolezni.com krasnodar.zrus.org
lacasamorett.com ks1234.com
lampokrat.ws kunstaktien.info
landreferat.ru kurbappeal.info
law-check-eight.xyz ladov.ru
lalalove.ru
laptop-4-less.com
lasvegaslockandsafe.com
law-check-seven.xyz law-check-seven.xyz
lemon-ade.site layola.biz.tc
lider-zhaluzi.kiev.ua lazymanyoga.com
lifeinsurancekb.com ldrtrack.com
likesdesign.com liky.co.ua
linkpulse.com lincolntheatre.com
liumimgx.pw locustdesign.co
localflirtbuddies.com lulea-auktionsverk.se
lockerz.com lyrics.home-task.com
lomb.co macresource.co.uk
longadventure.com mafa-free.com
lordzfilmz.me makis.nu
louboutinreplica.xyz manifestyourmillion.com
louboutinshoes.xyz map028.com
mackeeper-center.club marinetraffic.com
macotool.com marsgatan.com
mailemedicinals.com marwer.info
mainlinehobby.net masterseek.com
makedo.ru
makemoneyonline.com
makenahartlin.com
mall.uk
malls.com
marblestyle.ru
matchpal-a.akamaihd.net matchpal-a.akamaihd.net
medicines-choice.com maxthon.com
medicovi.com mb140.ru
medispainstitute.com.au meds-online24.com
meetingrainstorm.bid missvietnam.org
micasainvest.com mmostrike.ru
michaelkorssaleoutletonline.net mobplayer.net
mobplayer.ru moesen-ficken.com
mockupui.com moesonce.com
modnie-futbolki.net monarchfind-a.akamaihd.net
moesexy.com moncleroutletonline.pw
monarhs.info morf.snn.gr
monetizer.com-01.site
money-every-day.com
moscow-region.ru moscow-region.ru
mostantikor.ru most.gov.iq
mototsikl.org mp3downloadhq.com
mountainstream.ms mp3ringtone.info
musicktab.com mp3s.club
mxgetcode.com mrinsidesales.com
my-big-family.com mrmoneymustache.com
mygameplus.ru mulberryoutletonlineeu.com
myhitmp3.club musezone.su
namenectar.com muzaporn.com
naperehresti.info muztops.ru
new-post.tk myfreecams.com
nextrent-crimea.ru myfreetutorials.com
nicovideo.jp myghillie.info
nmrk.ru myindospace.com
nofreezingmac.work nagdak.ru
noumeda.com narkologiya-belgorod.ru
nyfinance.ml narkologiya-peterburg.ru
offergroup.info naturalbreakthroughsresearch.com
naturalshair.site
naval.jislaaik.com
needtosellmyhousefast.com
newstraveller.ru
newstudio.tv
nfljerseysforsalewholesaler.com
nhl17coins.exblog.jp
nightvision746.weebly.com
nomuos.it
nootrino.com
nottyu.xyz
nucia.biz.ly
nudejapan.net
nvformula.ru
nw-servis.ru
oakleyglassesonline.us
obesidadealgarve.com
obsessionphrases.com
oconto.ru
of-ireland.info
office-windows.ru
offtime.ru offtime.ru
onclkads.com offtopic.biz
ooo-olni.ru onemactrckr.com
openmediasoft.com onlinetvseries.me
optikremont.ru onthemarch.co
orsonet.ru orion-code-access.net
osnova3.ru outclicks.net
otbelivanie-zubov.com outpersonals.com
oxotl.com outrageousdeal-a.akamaihd.net
ozas.net ownshop.win
page2rss.com oyster-green.com
paighambot.com panchro.co.uk
pammik.ru partner-high.men
pattersonsweb.com partners-ship.pro
petrovka-online.com
peugeot-club.org peugeot-club.org
physfunc.ru physfunc.ru
picanalyzer.data-ox.com
piccdata.com
pintattoos.com pintattoos.com
pinwallpaper.xyz piulatte.cz
poppen-nw.net pixshark.com
popup-hgd.xyz plastgranar.nu
pornotubs.com play-movie.pl
porto.abuilder.net polybuild.ru
princevc.com popugaychiki.com
privhosting.com pornhubforum.tk
professionalwritingservices15.blogspot.ru pornmania.pl
profitsport.club pornodojd.ru
profitwithalex.info pornoklad.net
promoheads.com pornophoto.xyz
prostitutki-kharkova.org postclass.com
prostitutki-yaroslavlya.org powc.r.ca.d.sendibm2.com
pulse33.ru priora-2.com
rankexperience.com privat-girl.net
rarbg.to privatbank46.ru
psvita.ru
qexyfu.bugs3.com
qualitymarketzone.com
quickbuck.com
raavidesigns.com
rabotaetvse.ru
raisedseo.com
rating-casino2021.ru rating-casino2021.ru
razorweb-a.akamaihd.net razamicroelectronics.com
reklama1.ru razvratnoe.org
repeatlogo.co.uk realmonte.net
rff-cfal.info redirectingat.com
ribieiendom.no regionshop.biz
ric.info registrationdomainsite.com
risparmiocasa.bz.it renecaovilla.online
rubbed.us renhacklids.tk
ruf777.com responsinator.com
runofilms.ru respublica-otel.ru
ruscoininvest.company rfserial.net
rvi.biz royal-investments.net
s1z.ru russkoe-zdorovie.ru
salutmontreal.com sa-live.com
santechnik.jimdo.com saecsa.co
sbwealthsolutions.ca sammlungfotos.online
scanner-andrew.top saveindex.xyz
scanner-barak.top scanner-brian.top
scanner-don.top
scat.porn scat.porn
schlampen-treffen.com scenicmissouri.us
se.bnt-team.com se.bnt-team.com
searchimpression.com searchlock.com
searchwebknow-a.akamaihd.net seminarygeorgia59.ga
seo-tools-optimizing.com senger.atspace.co.uk
seo-traffic-ranking.info seo-traffic-ranking.info
seorankinglinks.us seoanalyses.com
serdcenebolit.com seofied.com
sexblog.pw seotoolsagency.com
sexxdate.net sextracker.be
shisha-swag.de sexyebonyteen.com
sisiynas.ru sfj-ror.no
site3.free-share-buttons.com sheerseo.com
shlyahten.ru
shmetall.com.ua
shoppingjequiti.com.br
simpoed.ufop.br
site-auditor.online
site-speed-checker.site
site.ru
siteexpress.co.il siteexpress.co.il
skylta.com sitiz.club
smartshoppymac.com sivs.ru
social-buttons.xyz slayerlife.com
social-s-ggg.xyz smallseotools.com
social-s-iii.xyz snworks.com
sockshares.tv sodexo.com
soft-terminal.ru soheavyblog.com
spasswelt.xyz spasswelt.net
sport-video-obzor.ru spidtest.org
squidoo.com spywarespy.com
sstroy44.ru stair.registrydomainservices.com
steamoff.net startufa.ru
stjamesschool.info stickers-market.ru
store-rx.com stmassage.ru
studiofaca.com strag-invest.ru
suchenindeutschland.com strangeduckfilms.com
streha-metalko.si
sudexpert66.ru
sukirgenk.dvrlists.com
superiends.org
supermodni.com.ua supermodni.com.ua
supernew.org superstarfloraluk.com
surfbuyermac.com suralink.com
surffoundation.nl surflinksmedical.com
surgut.zrus.org
svbur.ru
svolze.com svolze.com
tabakur77.com swimpool.ca
tandvardshuset.net takeprofitsystem.com
tasteidea.com tampabaywatch.org
taxi-v-eisk.ru
taximytishi.ru
tech4master.com
teenporn18.net teenporn18.net
telefonsex-ohne0900.net textads.men
thaisamkok.com thegolfclub.info
the-usa-games.blogspot.com theheroes.ru
thedownloadfromwarez.blogspot.com tizanidinehcl4mgtablet.blogspot.com
thenews-today.info
tizanidinemusclerelaxant.blogspot.com tizanidinemusclerelaxant.blogspot.com
tizanidinepain.blogspot.com tjkckpytpnje.com
tizanidinesideeffects.blogspot.com tomck.com
tizanidinevsvalium.blogspot.com tonerbox.kz
tomatis.gospartner.com tooplay.com
toon-families.com topbestgames.com
topcasinoratings.ru topmira.com
touchmods.fr
toyota.7zap.com
tradedeals.biz
traffic-club.info traffic-club.info
tryrating.com trafficfactory.biz
tuberkuleznik.ru traffmonster.info
turvgori.ru trahvid.com
twittrading.com treasuretrack-a.akamaihd.net
tx41tclega.ru trubywriting.com
tzritel.tk truck-addzilla.life
tubeoffline.com
turbabitload.weebly.com
tversvet.ru
u-cheats.ru
ugogo.info ugogo.info
ukrtextbook.com ultramart.biz
umaseh.com
umnovocaminho.com umnovocaminho.com
unece.org undergroundcityphoto.com
unlimitdocs.net unlimitdocs.net
unrealcommander.org unpredictable.ga
uptime.com uptime.com
uroki.net urccvfmc.bloger.index.hr
urzedowski.eu usiad.net
usbggettwku.ga vacuumscleaner.com
veselokloun.ru valmetrundan.se
via-energy-cumpara.com valuado.com
video-woman.com varikoz24.com
vkontarkte.com velpanex.ru
vsdshnik.com viagroid.ru
vseigru.one video-chat.in
videospornogratisx.net
vipcallsgirls.com
virus-schutzmasken.de
vizitki.net
vkgaleria.com
vkmusics.ru
vonradio.com
vremya.eu
vsesubwaysurfers.com vsesubwaysurfers.com
vykup-avto-krasnodar.ru w-journal.ru
wallpapers-best.com wareseeker.com
watchmygf.net watchmygf.net
waysbetter.cn wcb.su
webix.me webs.com
webjam.com websiteaccountant.de
weblo.com websiteexplorer.info
webstatsdomain.org welck.octopis.com
weburlopener.com wemedinc.com
wedding0venues.tk westen-v.life
westermarkanjou.se whatzmyip.net
whatistizanidinehclusedfor.blogspot.com whereverdesperate.gq
wholesalejerseychinaoutlet.com wholesalejerseys-cheapest.com
wholesalejerseysgaa.com wholesalejerseysgaa.com
wildnatureimages.com wineitudes.wordpress.com
worldtraveler.world withstandingheartwarming.com
wowas31.ucoz.ru writingservices17.blogspot.ru
www.kabbalah-red-bracelets.com www.souvenirua.com
x-true.info wzgyyq.com
xedserver.com x-diesel.org
xmlinde.com x-mix.info
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai x-stars.ru
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai xbaboon.com
xn--80aikhbrhr.net xkaz.org
xmladserver.com
xn--lifehacer-1rb.com xn--lifehacer-1rb.com
xportvusbdriver8i.snack.ws yeartwit.com
xtubeporno.net yebocasino.com
xwatt.ru yes-com.com
xxlargepop.com ynymnwbm.bloger.index.hr
xxxsiterips.xyz yottos.com
yellocloud.be yougetsignal.com
yellowfootprints.com youresponsive.com
yougame.biz yoursearch.me
ypmuseum.ru yugk.net
yurgorod.ru zagadki.in.ua
zaimite.ru
zambini.ru
zarabiaj-dzis.pl zarabiaj-dzis.pl
zero1.it zarabotok--doma.ru
zhuravlev.info zeg-distribution.com
zoogdinsney.com zetgie.com.pl
zscalertwo.net zigarettenonlinekaufen2.drupalgardens.com
zygophyceous.womanstars.site zigzog.ru
zlatnajesen.com
zog.link
zoolubimets.ru
zootoplist.com
zrizvtrnpale.tk
zscalerone.net
zynax.ua
zzbroya.com.ua

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
101billion.com 1j7740kd.website
12u.info 2itech.ru
178evakuator178.ru 2rich4bitches.com
1stat.ru 45en.ru
4inn.ru 51unlim.ru
4senses.co 6128786.com
4ureyesonly.com 83net.jp
888.com aa625d84f1587749c1ab011d6f269f7d64.com
8gold.com aasoldes.fr
adsfresh.men adidas.frwebs.fr
adsloads.com adloads.net
adviceforum.info affiliate-fr.com
advocatemsk.ru alive-ua.com
adzerg.com alldubai.biz
agecheckadult.com alltheviews.com
alfa9.com aloofly.com
allcredits.su aludecor.info
alphaforum.ru amyfoxfitness.com
amoi.tn anal-acrobats.hol.es
ananumous.ru apioasisspacenet-a.akamaihd.net
anapa-inns.ru arkkivoltti.net
animalrank.com avirasecureserver.com
animebox.com.ua aviva-limoux.com
anniemation.com avon-severozapad.ru
apiitsthirteende-a.akamaihd.net awency.com
arabseyes.com backgroundpictures.net
arendas.net
arius.tech
arpelsreplica.xyz
asrvrep-a.akamaihd.net
autotrends.today
awaybird.ru
azartniy-bonus.com
azbuka-mo.ru
babespcs.com
balkanfarma.org
balook.com
bbsoldes.fr
besthomemadepornsites.com besthomemadepornsites.com
bigcareer.info bestmouthwash.club
bingo8888.com bestofferswalkmydogouteveryday.gq
billiard-classic.com.ua
biznesrealnost.ru
bjorkbacken.nu
boc.kz boc.kz
bosman.pluto.ro bvlgariring.xyz
brianhenry.co centre-indigo.org.ua
businescoop.men cialis-samples.com
buyparajumpers.online cibpenonptib.flu.cc
cardul.ru cjmc.info
cartierlovestore.xyz coaching-netz.info
cartiertrinity.xyz colobit.biz
cayado.snn.gr com-onlinesupport.site
cienum.fr concealthyself.com
claytransformations.info coolgramgoods.com
cleanallspyware.com countercrazy.com
clevernt.com crackguru.tk
compliance-ivan.xyz crafthubs.com
consorzioilmosaico.org
converse.ddsoldes.fr
cscwtalkto.site
customchocolate.business-for-home.com
cutt.us cutt.us
dcj-nn.ru dame-ns.kz
detskie-konstruktory.ru date-withme.com
deadlinkchecker.com
descargar-musica-gratis.net
dfwu1019.info dfwu1019.info
dikqlyremy.info dispo.de
discountbarbour.online dnsrsearch.com
doctissimo.fr dogsrun.net
domainsatcost.com draniki.org
downloads-whatsapp.com drev.biz
dvr.biz.ua droidlook.net
dynainbox.com earl-brown.info
e-avon.ru
ecommerce-seo.org ecommerce-seo.org
eduserver.net eldiariodeguadalajara.com
elektronischezigarettekaufen2.tumblr.com electrik-avenue.com
elvenar.com elexies.info
enginebay.ru elkacentr.ru
extrabot.com elvel.com.ua
englishgamer.com
eonpal.com
erotikstories.ru
excitacao.com
extremal-blog.com
ezb.elvenmachine.com ezb.elvenmachine.com
ezigarettekaufen2.yolasite.com fba-mexico.com
f07.de fetlifeblog.com
femdom.twiclub.in filesmonster.porn
forexunion.net flagcounter.me
forodvd.com fodelsedagspresenter.nu
freza-sverlo.ru forums.toucharcade.com
frustrated-favorable.gq foxjuegos.com
ftns.ru freetangodownload.com
gaming-journal.com fym.com.ru
gazporno.com gallery.rennlist.com
gok-kasten.net gambarkatabaru.com
goodwinmetals.co games.kolossale.ru
guge.io gardene.ru
guiadeserraazul.com girlspicsa.com
gungamesz.com gkvector.ru
guod.me global-ics.co.za
hamacapty.com globatur.ru
hd-film.pl glogow.pl
healmytrauma.info gotomontenegro.net
hem.passagen.se gsasearchengineranker.top
highspeed5.net habermetre.com
hobild.net hitsbox.info
hostsshop.ru homedo.fabpage.com
hotshoppymac.com hop.clickbank.net
humbmosquina.tk hostcritique.com
hydropump.su houseofrose.com
iloveitaly.ru hulfingtonpost.com
incanto.in.ua i4track.net
internet-apteka.ru iinstalll-fii1leis.jus0wil.pp.ua
investyb.com imagefinder.site
istanbulit.com increasewwwtraffic.info
ivanstroi.ru insider.pro
jbl-charge.info instakink.com
jeremyeaton.co invitefashion.com
kakadu-interior.com.ua invivo.hu
kambasoft.com itservicesthatworkforyou.com
keki.info izismile.com
keywords-monitoring-success.com jennyfire.ru
kidskunst.info juliaworld.net
kinofak.net jumptap.com
kinopolet.net kino-key.info
kndxbkdx.bloger.index.hr kirov.zrus.org
knogg.net klosetkitten.com
konpax.com
koptims.tiu.ru koptims.tiu.ru
kuzinsp.ru lemon-ade.site
la-fa.ru locationdesaison.com
larutti.ru long-beach-air-conditioning.com
lasvegaslockandsafe.com lunchrock.co
lidoradio.com mail.allnews24.in
lol-smurfs.com marcoislandvacations.net
lostaruban.com mebelcomplekt.ru
louboutinreplica.xyz mebeldekor.com.ua
love-baby.cz med-zdorovie.com.ua
lumb.co megaapteka.ru
lunamedia.co menetie.ru
manifestation.betteroffers.review mezaruk.info
maridan.com.ua microsearch.ru
mcar.in.ua minyetki.ru
medicineseasybuy.com mirzonru.net
medmajor.ru mmofreegames.online
menstennisforums.com moinozhki.com
mikrobiologies.ru mojaocena.com
mmog-play.ru mojpreskumanie.com
moncleroutletonline.pw monclerjacketsoutlet.pw
moncleroutletonline.win money-for-placing-articles.com
more-letom.ru moneyteam24.com
mp3downloadhq.com montredemarque.nl
multgo.ru moomi-daeri.com
mybackgroundlandscape.blogspot.com musicas.baixar-musicas-gratis.com
myplaycity.com muznachas-service.ru
november-lax.com mydoctorok.ru
okel.co myonlinepayday.co
ooomeru.ru myshopmatemac.com
oranga.host.sk myxdate.info
parlament.biz myyour.eu
petsblogroll.com napalm51.nut.cc
picscout.com naperehresti.info
piratecams.com newyorkhotelsmotels.info
piulatte.cz nhl17coins.exblog.jp
pix-hd.com nightvision746.weebly.com
pixelrz.com obiavo.net
play-movie.pl openfrost.net
pokemongooo.ml osnova3.ru
pooleroadmedicalcentre.co.uk osoznanie-narkotikam.net
popunder.net ourville.info
pornoinn.com outrageousdeal-a.akamaihd.net
priceg.com ozoz.it
propranolol40mg.blogspot.com paydayonlinecom.com
prostitutki-novgoroda.org percin.biz.ly
qwertty.net photostudiolightings.com
rank-checker.online pictures-and-images.com
reelheroes.net picturesify.com
referencemoi.com pinwallpaper.top
replica-watch.ru pixshark.com
reyel1985.webnode.fr plastjulgranar.se
rock-cafe.info pogruztehnik.ru
ruex.org.ua popander.mobi
rustic-quiver.win porno-raskazy.ru
sanjosestartups.com prodvigator.ua
pronorm.fr
quangcaons.com
raschtextil.com.ua
realting-moscow.ru
research.ifmo.ru
rfserial.net
ruspdd.com
rusvideos.su
s-forum.biz
samolet.fr
sankt-peterburg.nodup.ru sankt-peterburg.nodup.ru
sarm.tk sardinie.us
savingsslider-a.akamaihd.net securityallianceservices.com
sbdl.no sendearnings.com
scanner-mark.top servethis.com
seeingmeerkat.com sextracker.be
seryeznie-znakomstva.ru shisha-swag.de
sethrollins.net shoesonlinebuy.xyz
sexgalleries.top silvercash.com
shemalegalls.blogporn.in siteexpress.co.il
site5.com skypasss.com
social-s-hhh.xyz sladkoevideo.com
solinf.co soda.media
soserfis.com space-worry.ml
souvenir.cc statistici.ro
spammen.de stockspmb.info
spanking.to storehouse.ua
spy-sts.com styro.ru
srdrvp.com taboola.com
steame.ru taqplayer.info
sukarame.net tecspb.ru
supers.com.ua telefonsex-ohne0900.net
superstats.com theendivechronicles.com
surcentro.com thefds.net
tackletarts.co ticketsys.inetwd.com
takeflyte.com timetorelax.biz
telefonsexsofort.tk timhost.ru
thaisamkok.com tizanidinehcl4mgtablet.blogspot.com
thebluffs.com tizanidineonline.blogspot.com
thenetinfo.com torrents.life
thexart.club toyota.7zap.com
tiens2010.ru trafaret74.ru
tizanidinewithdrawal.blogspot.com trafficjunky.com
tracklead.net traffmonster.info
traffic2cash.org treasuretrack-a.akamaihd.net
trailer.cinemaflix.website turbodsp.com
twodollarshows.com tytoona.com
tx41tclega.ru ukrtextbook.com
u555u.info uptime.com
uamtrk.com urlopener.com
uhdtv.website usacasino.com
univerfiles.com usdx.us
unpredictable.ga vacances-voyages.info
uptime-delta.net vandrie-ict.nl
urldelivery.com vapsy.com
ventelnos.com viktoria-center.ru
veopornogratis.xxx w7s.ru
vykup-avto-krasnodar.ru weburlopener.com
webpromotion.ae wesharepics.com
website-datenbank.de westen-z.life
wetgames.ru whatsupinfoley.com
whatistizanidinehclusedfor.blogspot.com whatzmyip.net
whereiskentoday.com wildcattube.com
wleuaprpxuvr.ga ws.ampower.me
ww2awards.info www.wohnkabinen-shop.de
x-lime.net x-diesel.com
x-true.info xgftnlrt.bloger.index.hr
xedserver.com xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xn--80aodinpgi.xn--p1ai xn----btbdvdh4aafrfciljm6k.xn--p1ai
xnxx-n.com xn--1-8sbcpb0bdm8k6a.xn--p1ai
xtremeeagles.net xolodremont.ru
xuki.us xxxnatelefon.ru
xxxguitars.com zeroredirect12.com
xzlive.com zeroredirect7.com
yaminecraft.ru zoogdinsney.com
yebocasino.com zoogdisany.com
yoursite.com zreloeporno.tv
ytmnd.com zvooq.eu
zakazvzloma.com zx6.ru
zakonobosago.ru zzbroya.com.ua
zawyna.ua
zdesformula.ru

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.91 0; 10.1.3.48 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3462 ### Version: V4.2022.11.3463
### Updated: Thu Nov 10 10:03:23 UTC 2022 ### Updated: Thu Nov 10 22:01:19 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3462 ### Version: V4.2022.11.3463
### Updated: Thu Nov 10 10:03:23 UTC 2022 ### Updated: Thu Nov 10 22:01:19 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##