vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.12.2771 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-12-10 22:07:23 +00:00
parent a201a687f6
commit 074523e31f
46 changed files with 71305 additions and 71305 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2770 #### Version: V4.2021.12.2771
#### Bad Referrers Blocked: [7080](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7080](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2770 #### Version: V4.2021.12.2771
#### Bad Referrers Blocked: [7080](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7080](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.12.2770 #### Version: V4.2021.12.2771
#### Bad Referrers Blocked: [7080](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7080](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8776
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.12.2770 #### Version: V4.2021.12.2771
#### Bad Referrers Blocked: [7080](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7080](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8780
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8780
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8780
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8780
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.1.142 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.1.142 0;
127.0.0.1 0; 127.0.0.1 0;

8780
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.1.142 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.1.142 0;
127.0.0.1 0; 127.0.0.1 0;

8780
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.1.142 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.1.142 0;
127.0.0.1 0; 127.0.0.1 0;

8780
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2770 2771

8780
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8780
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.109.169.78 0; 20.115.130.181 0;

8780
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.1.142 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.1.142 0;
127.0.0.1 0; 127.0.0.1 0;

8780
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.1.142 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.1.142 0;
127.0.0.1 0; 127.0.0.1 0;

8780
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.1.142 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.1.142 0;
127.0.0.1 0; 127.0.0.1 0;

8780
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.109.169.78 0; 20.115.130.181 0;

8780
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.109.169.78 0; 20.115.130.181 0;

8780
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.1.142 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Kinza Anarchy99
MeanPath Bot Cosmos
Meanpathbot EasyDL
Musobot InterGET
NearSite LinqiaMetadataDownloaderBot
RocketCrawler MFC_Tear_Sample
Searchestate MicroMessenger
Spinn3r NetSpider
scalaj-http Re-re
voyagerx.com Sosospider

306
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404enemy 404enemy
80legs
ADmantX
AIBOT AIBOT
ALittle Client ASPSeek
Abonti
Aboundex
Aboundexbot Aboundexbot
AhrefsBot AhrefsBot
Alexibot AiHitBot
AlphaBot AllSubmitter
Anarchy99 Anarchie
Anarchy
Ankit Ankit
Anthill Apexoo
Aspiegel
Asterias
Attach Attach
AwarioSmartBot BDCbot
BLEXBot BLEXBot
BackDoorBot BackDoorBot
Badass
Barkrowler Barkrowler
Battleztar Bazinga Battleztar Bazinga
BetaBot BetaBot
Bitacle
Black Hole
Blow Blow
Bolt BlowFish
Brandwatch Boardreader
Brandprotect
Buddy Buddy
BuiltBotTough
BuiltWith BuiltWith
BuzzSumo Bullseye
CATExplorador BunnySlippers
CODE87 CODE87
Calculon Calculon
CazoodleBot
CheeseBot
CherryPicker CherryPicker
ChinaClaw
Citoid Citoid
Cloud mapping Cocolyzebot
Collector Cogentbot
Copier Crawling at Home Project
Cosmos CrunchBot
Craftbot Custo
CrazyWebCrawler
DBLBot DBLBot
DIIbot DIIbot
DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Demon
Devil Devil
Digincore Digincore
DigitalPebble
Dirbuster
Dispatch Dispatch
DnyzBot
DomCopBot DomCopBot
DomainAppender DomainAppender
DomainCrawler
DomainStatsBot
Domains Project Domains Project
Download Wonder Drip
ECCP/1.0 EMail Siphon
EMail Wolf Ebingbong
EasyDL Ecxi
Express WebPictures EirGrabber
ExtLinksBot Evil
Extractor
Extreme Picture Finder
EyeNetIE
Ezooms Ezooms
Fimap FDM
FlashGet FemtosearchBot
Flunky Flunky
Foobot
FrontPage FrontPage
FyberSpider
Fyrebot
G-i-g-a-b-o-t
GT::WWW GT::WWW
Gigabot Genieo
GetRight
Go!Zilla
GoZilla
Gotit Gotit
Grafula Grafula
GrapeFX
GridBot GridBot
HEADMasterSEO HEADMasterSEO
HMView
HTMLparser
HTTP::Lite HTTP::Lite
HTTrack HTTrack
Heritrix Haansoft
Hloader Havij
Humanlinks
HybridBot HybridBot
IDBTE4M
IDBot
IRLbot
Id-search Id-search
Image Sucker
IndeedBot IndeedBot
InterGET Intelliseek
IstellaBot Internet Ninja
JamesBOT JetCar
Jbrofuzz
JennyBot
Jetty Jetty
JustView Joomla
Jorgee
Jyxobot Jyxobot
Keyword Density Kinza
LWP::Simple Kozmosbot
Larbin
Leap Leap
LexiBot LexiBot
Lftp
LibWeb LibWeb
LinkScan LieBaoFast
Lightspeedsystems
Likse
LinkWalker LinkWalker
LinksManager Linkbot
LinqiaRSSBot Linkdexbot
LinkextractorPro
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Litemage_walker
Lipperhey Spider
Ltx71 Ltx71
MJ12bot MQQBrowser
MSIECrawler MSFrontPage
Mag-Net MTRobot
Magnet Mail.RU_Bot
Majestic-SEO Majestic-SEO
Masscan Masscan
Mb2345Browser Mediatoolkitbot
Meanpathbot
MegaIndex.ru
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Mojeek Minefield
Mojolicious Mister PiX
Morfeus Fucking Scanner
Msrabot
Musobot
NICErsPRO NICErsPRO
Navroad NetAnts
NearSite NetLyzer
Net Vampire NetZIP
NetMechanic Netcraft
Niki-bot Nettrack
Nikto
Ninja
Not
Nutch Nutch
Octopus
Offline Navigator Offline Navigator
OnCrawl
OpenVAS
Openfind Openfind
OrangeBot
OrangeSpider OrangeSpider
PECL::HTTP PHPCrawl
POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageGrabber
Pandalytics Pandalytics
Pavuk
PeoplePal
Pi-Monster Pi-Monster
Picscout
Picsearch Picsearch
PictureFinder Piepmatz
Pimonster Pimonster
Pixray Pixray
PleaseCrawl Probethenet
Pockey
ProPowerBot
Pu_iN Pu_iN
PxBroker
PyCurl PyCurl
Quick-Crawler QueryN Metasearch
RankActive Re-re
Rankivabot ReGet
RebelMouse RealDownload
RedesScrapy
RepoMonkey
Ripper Ripper
RocketCrawler RocketCrawler
SBIder SEOkicks-Robot
SEOprofiler SEOprofiler
SISTRIX ScanAlert
SMTBot Scanbot
Scrapy ScoutJet
Searchestate Screaming
SearchmetricsBot ScreenerBot
Seomoz SemanticJuice
SiteCheckerBotCrawler SentiBot
SeoSiteCheckup
SeobilityBot
Shodan
SiteSnagger
Sitebeam
Sitevigil
SlySearch
SmartDownload SmartDownload
SocialRankIOBot
Sociscraper
Sosospider
Sottopop Sottopop
SpaceBison SpaceBison
SputnikBot SpankBot
Stripper Spanner
SurveyBot Spinn3r
Sqlmap
SuperBot
Suzuran Suzuran
Szukacz Szukacz
T0PHackTeam T0PHackTeam
TheNomad TeleportPro
Thumbor Telesoft
Titan Telesphoreo
Tracemyfile Telesphorep
The Intraformant
Trendictionbot Trendictionbot
Turingos Turnitin
Twice TwengaBot
URLy.Warning URLy.Warning
UnisterBot
Upflow
V-BOT V-BOT
VCI VB Project
VeriCiteCrawler Vacuum
Virusdie Vagabondo
WASALive-Bot VidibleScraper
Voil
WEBDAV
WWW-Mechanize
Wallpapers Wallpapers
Web Auto Wallpapers/3.0
Web Collage Web Enhancer
Web Fetch Web Fetch
Web Pix WebBandit
WebCollage WebFetch
WebGo IS
WebImageCollector
WebLeacher
WebWhacker
WebZIP
Webalta Webalta
WebmasterWorldForumBot WebsiteQuester
Webshag Whack
WebsiteExtractor Wonderbot
Whacker Wotbox
Whatweb Xaldon WebSpider
Who.is Bot
Widow
WinHTTrack
Xaldon_WebSpider Xaldon_WebSpider
YoudaoBot
Zade Zade
Zauba Zermelo
Zeus Zeus
ZmEu
ZoomBot
ZumBot ZumBot
ZyBorg ZyBorg
arquivo-web-crawler adscanner
archive.org_bot
arquivo.pt
autoemailspider
backlink-check backlink-check
com.plumanalytics clark-crawler
crawl.sogou.com crawl.sogou.com
crawler.feedback crawler.feedback
domainsproject.org crawler4j
evc-batch demandbase-bot
heritrix eCatch
gopher
ips-agent ips-agent
isitwp.com isitwp.com
meanpathbot
mediawords
muhstik-scan muhstik-scan
polaris version netEstate NE Crawler
probe-image-size page scorer
pcBrowser
ripz ripz
s1z.ru s1z.ru
scan.lol satoristudio.net
seostar sexsearcher
serpstatbot
siteripz siteripz
sogouspider trendiction.com
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webmeup-crawler webmeup-crawler
webpros.com xpymep1.exe
zauba.io
zgrab

324
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
80legs 80legs
AIBOT ASPSeek
ALittle Client Abonti
Aboundex Aboundexbot
Acunetix AhrefsBot
AiHitBot AiHitBot
Alexibot Alligator
AlphaBot
Anarchie Anarchie
Anthill Ankit
Aspiegel Aspiegel
Asterias
Attach Attach
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BDCbot BDCbot
BDFetch BLEXBot
Backlink-Ceck Bandit
BlackWidow BatchFTP
Blow Black Hole
Boardreader Boardreader
BotALot BotALot
Brandprotect Brandwatch
Buck Buddy
BuzzSumo BuiltWith
Bullseye
CCBot CCBot
CODE87 CODE87
CSHttp
Calculon
CazoodleBot
Cegbfeieh
CheTeam
CheeseBot CheeseBot
ChinaClaw ChinaClaw
Chlooe
Citoid Citoid
Cliqzbot Claritybot
Cocolyzebot Cocolyzebot
Copier Copier
CopyRightCheck
Copyscape
Cosmos
Crescent
CrunchBot CrunchBot
DIIbot CyotekWebCopy
DSearch
DTS Agent DTS Agent
DataCha0s DatabaseDriverMysqli
Demon Demon
Deusu Deusu
Devil
Digincore
DigitalPebble
Dirbuster Dirbuster
Discobot DomainAppender
Dispatch DomainCrawler
DnBCrawler-Analytics DomainSigmaCrawler
Domains Project Dragonfly
Dotbot Drip
Download Wonder EMail Siphon
ECCP/1.0
EMail Wolf EMail Wolf
Ebingbong EasyDL
Exabot EroCrawler
Express WebPictures Evil
ExtLinksBot
Extractor Extractor
Extreme Picture Finder
Ezooms
FHscan FHscan
FemtosearchBot FemtosearchBot
Fimap
Flunky Flunky
FrontPage FrontPage
Fuzz Fuzz
FyberSpider FyberSpider
G-i-g-a-b-o-t GT::WWW
GalaxyBot GalaxyBot
GermCrawler GermCrawler
GetRight Getintent
Go-Ahead-Got-It Gigabot
GoZilla Grafula
GrabNet GridBot
GrapeshotCrawler HTMLparser
HEADMasterSEO HTTrack
HTTP::Lite
Haansoft
Harvest Harvest
Heritrix Heritrix
Hloader
Humanlinks Humanlinks
IDBot IDBot
Iblog IRLbot
Id-search IlseBot
Indy Library Image Fetch
InfoNaviRobot Image Sucker
InterGET IndeedBot
Internet Ninja Intelliseek
Iria IstellaBot
Iskanie
JOC Web Spider
JamesBOT
Jbrofuzz Jbrofuzz
JetCar Jetty
Jyxobot Jorgee
Keyword Density Kinza
Larbin Kozmosbot
LexiBot Lanshanbot
LeechFTP
Libwhisker
LinkScan LinkScan
LinkWalker
Linkbot Linkbot
Linkdexbot Linkdexbot
LinkpadBot LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey Spider Lipperhey Spider
Ltx71 Ltx71
MFC_Tear_Sample
MIIxpc
MJ12bot
MSFrontPage MSFrontPage
Mag-Net Mail.RU_Bot
Majestic12 Majestic SEO
Majestic-SEO
MarkMonitor MarkMonitor
MarkWatch
Mass Downloader Mass Downloader
Masscan Meanpathbot
Mata Hari
Mb2345Browser
MeanPath Bot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Moblie Safari Minefield
Mister PiX
Mojolicious Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mr.4x3 Mr.4x3
NICErsPRO Msrabot
Navroad Musobot
Net Vampire Name Intelligence
NetMechanic Nameprotect
Netcraft NearSite
Niki-bot Needle
Nessus
NetZIP
Netvibes
NextGenSearchBot
NimbleCrawler NimbleCrawler
Nimbostratus
Nmap Nmap
Nuclei Not
Octopus Nutch
OnCrawl
OpenLinkProfiler
OpenVAS OpenVAS
Openfind Openfind
OrangeSpider Openvas
POE-Component-Client-HTTP OutclicksBot
Page Analyzer OutfoxBot
PageGrabber PageAnalyzer
PageThing.com PageScorer
Pandalytics Panscient
PeoplePal PeoplePal
Picscout Picsearch
PictureFinder ProPowerBot
Piepmatz
Pockey
ProWebWalker ProWebWalker
PxBroker Pu_iN
PyCurl
QueryN Metasearch QueryN Metasearch
RankActiveLinkBot Quick-Crawler
RankFlex RankFlex
RankingBot RankingBot
Rankivabot Rankivabot
RankurBot
Reaper Reaper
RebelMouse Recorder
RedesScrapy RepoMonkey
Ripper RocketCrawler
Rogerbot
SEOkicks SEOkicks
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOstats SEOprofiler
SISTRIX SISTRIX
ScanAlert SMTBot
ScoutJet Scanbot
Scrapy ScrepyBot
ScreenerBot SemrushBot
Searchestate SentiBot
SearchmetricsBot SeoSiteCheckup
Semrush
SeobilityBot
Shodan
Siphon Siphon
Site Sucker Site Sucker
SiteExplorer SiteLockSpider
SiteSucker
Sitebeam Sitebeam
Siteimprove
Sitevigil Sitevigil
SocialRankIOBot SlySearch
Sociscraper Sogou web spider
Sottopop Sottopop
SpaceBison Spammen
Spanner SpankBot
Spinn3r Spinn3r
Sqworm
Steeler Steeler
SurveyBot Sucker
Suzuran SuperBot
Swiftbot Swiftbot
The Intraformant Telesphorep
Titan TheNomad
TwengaBot Thumbor
Typhoeus Trendictionbot
Upflow Turingos
URLy Warning
URLy.Warning
V-BOT
VB Project VB Project
VidibleScraper VCI
Vacuum
Vagabondo
VeriCiteCrawler
Virusdie Virusdie
Voil Voil
WASALive-Bot Voltron
WEBDAV WEBDAV
WallpapersHD WWWOFFLE
Wallpapers/3.0
WeSEE WeSEE
Web Enhancer
Web Fuck Web Fuck
WebEnhancer Web Pix
WebLeacher Web Sucker
WebGo IS
WebReaper WebReaper
WebSauger WebSauger
WebStripper WebSucker
WebWhacker
WebZIP
Webalta
Webshag Webshag
Website Quester
WebsiteQuester WebsiteQuester
Webster
Whatweb Whatweb
WiseGuys Robot Who.is Bot
WinHTTrack
Wotbox Wotbox
YoudaoBot YoudaoBot
Zauba ZoomBot
Zermelo ZumBot
Zeus adscanner
Zitebot autoemailspider
ZyBorg backlink-check
arquivo.pt cah.io.community
clark-crawler coccocbot-web
com.plumanalytics cognitiveseo
dataforseo.com crawl.sogou.com
eCatch crawler.feedback
gopher domainsproject.org
heritrix evc-batch
ips-agent facebookscraper
instabid
lwp-request
magpie-crawler
meanpathbot meanpathbot
muhstik-scan oBot
pcBrowser page scorer
plumanalytics plumanalytics
polaris version s1z.ru
satoristudio.net seobility
serpstatbot seoscanners
trendiction.com seostar
trendiction.de sexsearcher
ubermetrics-technologies.com sogouspider
voyagerx.com sp_auditbot
webmeup-crawler tAkeOut
webprosbot webgains-bot
x09Mozilla
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
brakehawk.com cakesplus.com.au
cubook.supernew.org dstroy.su
drillsaw.ru gazoblok.net.ua
hahashka.ru ix20.ru
hyipmanager.in letslowbefast.today
onlinemeetingnow.com monetizer.com-01.site
pinwallpaper.top montredemarque.nl
tizanidinehcl2mg.blogspot.com pons-presse.com
webnode.me shama-rc.net
yourothersite.com templates.franklinfire.co

928
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1008.su 00author.com
1flag.co.za 18ps.ru
24videos.tv 2345.com
2drittel.de 28n2gl3wfyb0.ru
3dracergames.com 2x2fan.ru
4pp13.com 4replicawatch.net
4ureyesonly.com
4webmasters.org
5-steps-to-start-business.com 5-steps-to-start-business.com
7ooo.ru 5elementov.ru
8gold.com 5u.com
9med.net 66cpwgln.space
aadroid.net 7minuteworkout.com
adamoads.com 883zy.com
adf.ly 8si.ru
adobereader-free.ru 8xv8.com
a1security.com.ua
aac500b7a15b2646968f6bd8c6305869d7.com
aanapa.ru
acheterviagrafr24.com
activecampaign.dreamhosters.com
acunetix-referrer.com
ad-words.ru
adbetclickin.pink
addblueoff.com.ua
addray.pro
adrenalin-stk.ru adrenalin-stk.ru
adserver-e7.com adtech.us
adsloads.com adultwebhosting.info
adtech.de afslankpillen2017nl.eu
adultgalls.com agapovdg.ru
adultnet.in alevinefesleri.com
advancedcleaner.com alphacarolinas.com
advocatemsk.ru
ageofclones.com
aimayangzhi.com
airmaxshoes-2016.com
aksonural.ru
alert.scansafe.net
alphaforum.ru
alyeskaresort.com
anal-acrobats.com
android4fun.org android4fun.org
apiadanaknet-a.akamaihd.net animal36.com
apiappenableinfo-a.akamaihd.net apartmentratings.com
apiholdingmypage-a.akamaihd.net apispringsmartne-a.akamaihd.net
apioutoboxnet-a.akamaihd.net apparel-offer.com
apiwoodensealcom-a.akamaihd.net approved.su
appfixing.space apps-analytics.net
appsecurityr.com aquarium-pro.ru
arcadeplayhouse.com
ardimobileinfo.ml
arenda-avtoprokat-krasnodar.ru arenda-avtoprokat-krasnodar.ru
arius.tech arendovalka.xyz
arkkivoltti.net asacopaco.tk
artdestshop.eu asrv-a.akamaihd.net
asdfg.pro autodo.info
asiavirtualsolutions.com autoua.top
asscenihotosidea.blogspot.co.za autovideobroadcast.com
atovh.local-finders.com aviapanda.ru
autobrennero.it avkzarabotok.com
autobudpostach.club avon-ukraine.com
autorn.ru azazu.ru
autowebmarket.com.ua babyguns.ru
aviav.eu backuperwebcam.weebly.com
aviav.org baixar-musicas-gratis.com
ayanaspa.com banan.tv
ayeartoforget.com bankofthewext.com
azlex.uz bannerads.de
baksman.com basisches-wasser.net
balook.com bausparen.bz.it
bannerspace.com bazaronline24.ru
baoxaydung.com.vn beauty-b0x.pl
beclean-nn.ru best-deals-products.com
benchmarkcommunications.co.uk besthoro.ru
best-mam.ru bestmobilityscooterstoday.com
best-seo-software.xyz bestmouthwash.club
bestadbid.com
bestchoice.cf
bestcoin.cc
bestdraws.com
bestofupload.info bestofupload.info
bestserials.com bestprofits-there.com
bestwebsiteawards.com beyan.host.sk
beta.hotkeys.com big-cash.net
bidbuy.co.kr bigpictures.club
bingo8888.com binaryoptionscops.info
binomo.kz bingoporno.com
bitcoins-live.ru bioscorp.ru
blog.f00kclan.de bizlist.com.de
bluesalt.co blackbabesporn.com
boltalko.xyz blacktwhite.com
boo-arts.com blagovest-med.ru
boostmyppc.com bolezniorganov.ru
brandedleadgeneration.com bouda.kvalitne.cz
brandehk.dk brandehk.dk
brewdom.ru bravegirlsclub.com
burger-imperia.com break-the-chains.com
brokergid.ru
brus.city
buckspar.gq
businesxxl.com
buydissertation.net buydissertation.net
bvlgaribracelet.xyz buypanicdisorderpill.com
byme.se c1.onedmp.com
cactussoft.biz callawaygolfoutlet.online
calc-for-credit.ru camsex.xxx-cam.webcam
calstaterealty.us candiceloves.us
cartierlovebraceletreplica.xyz candytiner.com
catalogs-parts.com canrole.com
chiptuninger.com canzoni.ru
choosecuisine.com carfax.com.ua
christianlouboutinshoes.xyz carstrends2015.com
ciceros.co cartierlove2u.xyz
cineacademy.ru cb.iphantom.com
citizenclsdriveri7.pen.io cctva.tv
classiquebijoux.ru cenokos.ru
clickbanksites.info cheapestjerseys-wholesale.com
chidporn.com
ciekawinki.pl
cigarpass.com
cleanpcnow.com
clevernt.com
clickaider.com
clicksor.com
clips.ua.ac.be clips.ua.ac.be
clknsee.com cocyq.inwtrade.com
colehaanoutlet.store com-cleaner.systems
compliance-andrew.top compliance-george.xyz
compliance-brian.top compliance-julianna.top
compliance-mary.top
compliance-walt.top compliance-walt.top
cornerstone-countertops.com compliance-walter.top
coslab.club conboy.us
costablue.xyz conciergegroup.org
counciltally.com confib.ifmo.ru
crynet.cc cookie-law-enforcement-ff.xyz
cupday.com cottageofgrace.com
cwetochki.ru covid-schutzmasken.de
dairyindia.in crazyprotocol.com
daneshetabiat.com creditmoney.com.ua
daooda.com czat.wp.pl
data1.scopich.com daretodonate.co
davidovic.info dataloading.net
date-withme.com
dawlenie.com dawlenie.com
demian.kz detoxmed24.ru
dengi-pod-zalog-nedvizhimosti.ru
detective01.ru
detki-opt.ru
diamond-necklace.info diamond-necklace.info
dieta-personalna.pl dms-sw.ru
divisioncore.com dnepr-avtospar.com.ua
documentbase.net dnepropetrovsk.mistr-x.org
domain.webkeyit.com doeco.ru
domashneeruporno.com domaincrawler.com
dosugrostov.site dominterior.org
downloadmefiranaratb1972.xpg.com.br dorratex.tn
droidlook.net download-of-the-warez.blogspot.com
druzhbany.ru dprtb.com
drugstoreforyou.com
dumpsmania24.com dumpsmania24.com
dumuelave.xyz dumuelave.xyz
duplicashapp.com dvd-famille.com
dyt.net e-commerce-seo1.com
easytuningshop.ru eets.net
ebooknovel.club ekspresihati.info
educatemetv.com el-nation.com
elektronischezigarettekaufen1.myblog.de electronicadirect.com
elektrozigaretten2.yn.lt elektrozigarettekaufen2.blogse.nl
energydiet24.ru elektrozigaretten1.postbit.com
englishdictionaryfree.com elidelcream.weebly.com
enskedesquashclub.se elstal.com.pl
erotische-geschichten-xxl.com emerson-rus.ru
europeanwatches.ru eshop4u.jp
event-tracking.com esnm.ru
excitacao.com essaypro.com
exonline.info european-torches.ru
expertnaya-ocenka.ru executehosting.com
extlinks.com ezigarettenkaufen1.pagina.gr
extractorandburner.com ezigaretteshop2.vefblog.net
extstat.com f-loaded.de
eyessurgery.ru f0815.de
ezigarettenshop1.yolasite.com falcoware.com
femdom.twiclub.in fanpagerobot.com
fettefrauen.net fatmaelgarny.com
filmania-x.ru fefo.gdn
filmci.pro ferieboligkbh.dk
filmfanatic.com filmetricsasia.com
financehint.eu firma-legion.ru
financetip.eu
finansov.info
flashbannernow.com
floridamobilebillboards.com
flytourisme.org
fok.nl
footballfarrago.com
forex-procto.ru
fortevidyoze.net fortevidyoze.net
forzeronly.com foto-telok.net
free-share-buttons.com foto-weinberger.at
free-social-buttons.xyz fototravel.eu
free-today.com foxweber.com
free-traffic.xyz freelinkbuilding.website.tk
freesitetest.com freeseedsonline.com
frequiry.com frequiry.com
freshsuperbloop.com friendflnder.com
frustrated-favorable.gq fsalas.com
fullgirl.ru fuckmill.com
funcrushgames.com funnel.co.za
funnymama.com
funponsel.com funponsel.com
gael-s.ru fyxabomiw.ru
gallerily.com gallery.rennlist.com
gazobeton-p.com.ua gamingspark.com
geilehausfrauen.net gay-site.store
genericviagrasildenafiled.net ge0ip.net
get-your-social-buttons.info ghostvisitor.com
getdot.ru gilbertbanda.net
getresponse.com glall.ru
girlgamerdaily.com gloverid.site
gk-atlant.info goforexvps.com
gosmeb.ru googlefeud.com
greatdealshop.com gq-catalog.gq
gribkovye-zabolevaniya.com greatgrace.ru
guod.me gsmtlf.ru
handsandlegs.ru gstatey.net
healbio.ru hard-porn.mobi
heartofpayne.xyz hasfun.com
hermesbelts.xyz hdimagegallery.net
highstairs-a.akamaihd.net heartofbeijing.blogspot.com
hermesbracelets.xyz
highland-homes.com
histock.info histock.info
hop.clickbank.net hjaoopoa.top
hotchatdate.com host-tracker.com
humbmosquina.tk hothot.ru
hundejo.com hotshoppymac.com
ibmdatamanagement.co hoverboard360.de
hoverboardforsaledirect.com
iboss.com
ictizanidinehcl4mg.blogspot.com
ido3.com
ie.57883.net ie.57883.net
iflycapetown.co.za ikritikimou.gr
images-free.net imagine-ex.co
images.gyffu.com imguramx.pw
imitex-plus.ru immobiliaremassaro.com
importchinacoach-teach.com impotentik.com
inboxdollars.com ip-guide.com
incep.imagine-ex.co iptool.xyz
interferencer.ru isistaylorporn.info
internetartfair.com islamtoday.co.za
investpamm.ru itrevolution.cf
investyb.com iusstf.org
iqoption.com iyasimasennka.com
iscblog.info izhevsk.zrus.org
isitpaleo.info jav-fetish.site
isoveti.ru javitas.info
istanbulit.com jeremyeaton.co
istizanidinelikexanax.blogspot.com job.icivil.ir
istock-mebel.ru journeydownthescale.info
itronics.ca jpcycles.com
ivoiretechnocom.ci justbcause.com
iwantedmoney.com kanimage.com
japfm.com karachev-city.ru
javxxx18.com kaz.kz
jofucipiku.tk kellyonline.xyz
jrcigars.com kidskunst.info
juliaworld.net kinomaniatv.pl
kartiniresto.com kiprinform.com
kemerovo.zrus.org kiwe-analytics.com
kinoflux.net
kinohall.ru
kinohit1.ru
kit-opt.ru
klin3952.ru klin3952.ru
knowyournextmove.com koleso24.com.ua
kostenlos-sexvideos.com kollekcioner.ru
kovrenok.ru kollesa.ru
kumuk.info kongoultry.net
kustanay.kz krasivye-devushki.net
kutikomi.net kretpicf.pw
lacloop.info kuzinsp.ru
law-check-eight.xyz lezbiyanki.net
lecbter-relationships.ga lincolntheatre.com
lescinq.com link.ac
letmacworkfaster.site linkarena.com
letolove.ru linkredirect.biz
lexaprogeneric.link liran-locks.com
liver-chintai.org liumimgx.pw
ljusihus.se livefixer.com
locatellicorretor.com.br loadingpages.me
lodki-pvh.dp.ua loadopia.com
lol-smurfs.com locationdesaison.com
longadventure.com lomb.co
lostaruban.com lombia.com
mackeeper-center.club lsp-awak-perikanan.com
manyresultshub-a.akamaihd.net lumb.co
marcogrup.com luxup.ru
marinetraffic.com lyrics.home-task.com
mature.free-websites.com m4ever.net
mcadamssupplyco.com macnewtech.com
managerpak204.weebly.com
mantramusic.ru
mearns-tractors.co.uk mearns-tractors.co.uk
mebel-vstroika.ru mediaoffers.click
medicines-choice.com meetlocalchicks.com
meds-online24.com
meduza-consult.ru
meetingrainstorm.bid
megaindex.ru megaindex.ru
methodsmarketing.com menhealed.net
michellblog.online menstennisforums.com
metaxalonevstizanidine.blogspot.com
microsearch.ru
missclub.info missclub.info
mmofreegames.online
mnimmigrantrights.net
mobicover.com.ua mobicover.com.ua
mobplayer.ru mobile.ok.ru
moncleroutletonline.pw mobsfun.net
moscow-region.ru mojpreskumanie.com
mowser.com monitorwebsitespeed.com
morpicert.pw
moyakuhnia.ru
moyaterapiya.ru
moz.com moz.com
mozello.ru musicas.baixar-musicas-gratis.com
muizre.ru musicstock.me
mydeathspace.com myonigroup.com
myonlinepayday.co myonlinepayday.co
myshopmatemac.com myperiod.club
najaden.no mywallpaper.top
naturalbreakthroughsresearch.com nac-bearings.ru
networkcheck.xyz nalogovyy-kodeks.ru
new-post.tk narkologiya-belgorod.ru
newsperuse.com narkologiya-orel.ru
newyorkhotelsmotels.info narosty.com
next-dentists.tk narutonaruto.ru
nextconseil.com national-today-winning-winner.club
nikkiewart.ru nero-us.com
nordvpn.com netvouz.com
northfacestore.online newtechspb.ru
notaria-desalas.com nfljerseys.online
nufaq.com nfljerseyscheapchinabiz.com
nuup.info nisuturnetdgu.tk
olympescort.com npoet.ru
onlinedomains.ru nrv.co.za
openmultipleurl.com nucia.biz.ly
openstat.com obnal.org
ops.picscout.com offers.bycontext.com
optibuymac.com online-templatestore.com
ostroike.org onlineslotmaschine.com
openmediasoft.com
oren-cats.ru
orhonit.com
origin-my.ru
outclicks.net
p-business.ru
papasdelivery.ru
partner-stop.men
partnerworkroom.men
pastaleads.com
patol01.pw patol01.pw
perl.dp.ua pdns.download
pictures-and-images.com perederni.net
piratecams.com perfectpracticeweb.com
pisanieprac.info phpdevops.com
pix-hd.com picphotos.net
pixshark.com picquery.com
pl.aasoldes.fr pills24h.com
plastweb.ru pio.polytopesexempt.com
platinumdeals.gr pivka.xyz
pokemon-go-play.online pizza-tycoon.com
polytopesexempt.com popugaychiki.com
pornhub-ru.com pornoklad.net
porno-play.net porto.abuilder.net
pornokajf.com pozdrawleniya.ru
pornoplen.com
pornosok.ru
pornozhara.com
powenlite24.ru
preconnubial.usuby.site preconnubial.usuby.site
professionaldieselcare.com primedice.com
profolan.pl privatbank46.ru
privatov-zapisi.ru
professionalwritingservices15.blogspot.ru
prohoster.info prohoster.info
promalp-universal.ru promover.org
prostitutki-odessa.org pronorm.fr
prostitutki-yaroslavlya.org prosmibank.ru
pushdata.sendpulse.com purplesphere.in
qwarckoine.com qcstrtvt.bloger.index.hr
ranksonic.org radiogambling.com
razorweb-a.akamaihd.net ragecash.com
realnye-otzyvy.info rapidokbrain.com
recipedays.ru razvratnoe.org
redirlock.com real-time-analytics.com
registry-cleaner.net rebuildermedical.com
remont-fridge-tv.ru remote-dba.de
repeatlogo.co.uk replica-watch.ru
research.ifmo.ru replicaclub.ru
respublica-otel.ru rightenergysolutions.com.au
restorator-msk.ru rock-cafe.info
rezeptiblud.ru rockma.se
ricorsogiustizia.org royal-betting.net
riders.ro rrutw.com
robertefuller.com runtnc.net
ru-dety.ru russia-today-video.ru
s-forum.biz rvi.biz
sack.net sa-live.com
saletool.ru sa-rewards.co.za
saneitconsulting.com sad-torg.com.ua
satellite.maps.ilovevitaly.com sakhboard.ru
saveriopiazza.it sammlungfotos.online
scanner-mary.top samoiedo.it
scanner-olga.top sanyuprojects.com
schlampen-treffen.com savetubevideo.com
secret.xn--oogle-wmc.com scanner-john.top
security60-e.com sdrescher.net
securityallianceservices.com searchkut.com
seeingmeerkat.com sell-fb-group-here.com
seminarygeorgia59.ga sendearnings.com
seo-tools-optimizing.com seryeznie-znakomstva.ru
seoexperimenty.ru sex-foto.pw
seokicks.de
seonetwizard.com
sexkontakte-seite.com
sexkontakteao.info sexkontakteao.info
sexreliz.com shikiso.info
sexy-pings.com shlyahten.ru
sexyebonyteen.com shoesonlinebuy.cn
shiksabd.com shopperifymac.com
shop.xz618.com shoppytoolmac.com
shopsellcardsdumps.com sierraapps.com
silvercash.com silverage.ru
sirpornogratis.xxx simple-image.com.ua
site5.com simplesite.com
siteimprove.com site-analyzer.com
sitiz.club siteheart.net
skutecznetabletkinaporostwlosow.pl smartadserver.com
slayerlife.com
smadihome.com
smartbalanceworld.com smartbalanceworld.com
soheavyblog.com smokewithrabbits.com
space2019.top soc-econom-problems.ru
spidtest.space social-buttons.com
stardevine.com social-fun.ru
starpages.net social-s-ggg.xyz
startufa.ru socialbutton.xyz
stmassage.ru sodexo.com
stop-zavisimost.com softtor.com
studiofaca.com soietvousmaime.fr
svnuppsalaorebro.se songplanet.ru
swingerseiten.com spinnerco.ca
tabakur77.com sribno.net
tako3.com strigkaomsk.ru
tandvardshuset.net stroiminsk.org
tanieaukcje.com.pl stroy-portal22.ru
td-l-market.ru summerlinhomes411.info
teesdaleflyballclub.co.uk susanholtphotography.com
tengohydar.tk svarkagid.com
terraclicks.com sweepstakes.rewardit.com
terrafootwear.us symphonyintegratedhealthcare.com
theprofitsmaker.net tagil.zrus.org
thesmartsearch.net tamada69.com
time-japan.ru tedxrj.com
titelhelden.eu templates.franklinfire.co
tizanidine4mgstreetprice.blogspot.com the-torrent-tracker.blogspot.com
tizanidinehcl2mgtablet.blogspot.com the-trader.net
tizanidinevscyclobenzaprine.blogspot.com thecoral.com.br
tizanidinezanaflex.blogspot.com themestotal.com
toolsky.com tizanidinehclsideeffects.blogspot.com
top-study.work tizanidinemusclerelaxant.blogspot.com
toproadrunner5.info tizanidinesleepaid.blogspot.com
torrent-newgames.com tkanorganizma.ru
torrentdownloadhub.com tobeyouday.win
trackmedia101.com toondinsey.com
trafficfactory.biz tracklead.net
trafficmania.com traffique.net
trafficnetzwerk.de trainoffend.ml
tripper.de tripper.de
trustl.life triumf-realty.ru
try-rx.com trubywriting.com
tt-ipd.info truck-rece.life
turn-up-life.life tsyndicate.com
uac.net.au tubeline.biz
undergroundcityphoto.com tuckpointingmasonrysystems.com
unpredictable.ga tula.howotorg.ru
txxx.com
typer.one
tyumen.xrus.org
uni.me
unrealcommander.org
upstore.me
uptime-gamma.net uptime-gamma.net
upupa.net urldelivery.com
ussearche.cf urlopener.com
v720hd.ru us-america.ru
vacuumcleanerguru.com
vadimkravtcov.ru vadimkravtcov.ru
vchulkax.com video--production.com
velen.io vinylvault.co.uk
vgoloveboli.net volume-pills.biz
via-energy-cumpara.com
video-production.com
vinsit.ru
vip-parfumeria.ru
vksaver-all.ru
voditeltrezviy.ru
vsesubwaysurfers.com vsesubwaysurfers.com
vtc.pw w3javascript.com
wallpapersist.com wallpapers-best.com
watchmyfb.pl warezkeeper.com
webadvance.club watchmygf.net
website-datenbank.de wdfdocando.com
websiteaccountant.de web-analytics.date
wesharepics.com weblibrary.win
wesharepics.site website-audit.com.ua
westum.se weburok.com
whatzmyip.net wedding-salon.net
wheelchairliftsarea.com
white-truck.life
wholesalejerseychinaoutlet.com wholesalejerseychinaoutlet.com
wholinkstome.com wholesalejerseys-cheapest.com
wikes.20fr.com www.888.com
wineitudes.wordpress.com www.bookmaker-bets.com
wnhjavlhezp.gq x-musics.com
womensterritory.ru x-stars.ru
wordkeyhelper.com
wowcasinoonline.ooo
ws.ampower.me
wurr.voila.net
www.kabbalah-red-bracelets.com
x-lime.com
x-mix.info
xboxster.ru xboxster.ru
xcombear.ru xedserver.com
xitjw.info xerox-douglas.cf
xn--d1abj0abs9d.in.ua xn----dtbndd4ae7eub.top
xn--d1aifoe0a9a.top xn--h1ahbi.com.ua
xn--e1aggki3c.xn--80adxhks xnxx-n.com
xolodremont.ru xportvusbdriver8i.snack.ws
xtrafficplus.com xxlargepop.com
xwatt.ru xxxtube69.com
xxx-cam.webcam yaaknaa.info
ynymnwbm.bloger.index.hr yourporn.com
youjizz.vc zaim-pod-zalog-krasnodar.ru
youresponsive.com zarabiaj-dzis.pl
youtoner.it zhacker.net
yur-p.ru zhongwenlink.com
yuweng.info zhorapankratov7.blogspot.com
zakon-ob-obrazovanii.ru zigarettenonlinekaufen1.bloog.pl
zarepta.com zolotoy-lis.ru
zeroredirect6.com zx6.ru
zetmaster.ru
znaniyapolza.ru
zoominfo.com
zrus.org
zskdla.site
zytpirwai.net

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
34.gs 5i2.net
5tey7463.weebly.com 7fon.ru
actionnooz.com 999webdesign.com
adloads.com acortarurl.es
adrenalin-stk.ru adtech.us
akuhni.by advertisingtag.net
alfapro.ru alfabot.xyz
amateurlivechat.org allornamenti.com
amazon-adsystem.com animali.deagostinipassion.it
anabolics.shop arewater.com
angkortours.vn artdeko.info
aosheng-tech.com asacopaco.tk
apibatbrowsecom-a.akamaihd.net asiengirls.net
appsecurityr.com asrvrep-a.akamaihd.net
arate.ru atleticpharm.org
arclk.net atofilms.com
articlesdirectoryme.info avirasecureserver.com
aviav.org ayanaspa.com
avkzarabotok.com balans.shahterworld.org
bablonow.ru banan.tv
balla-boo.se beauty-b0x.pl
bestadbid.com bestofupload.info
biketank.ga bestsexyblog.com
bizzliving.com bif-ru.info
blue-square.biz bio-market.kz
bombla.org bonvillan.com
brandov.ru bookmaker-bet.com
bvlgaribracelet.xyz boostmyppc.com
callawaygolfoutlet.online bouda.kvalitne.cz
cartierlovestore.com buehne-fuer-menschenrechte.de
cartiertrinity.xyz buigas.00it.com
cascadelink.org builtwith.com
castingbank.ru bwlx.prepedu.cn
caveavins.fr cablecar.us
cbb1smartlist12.click canoncdriverq3.pen.io
citizenclsdriveri7.pen.io carsdined.org
clickprotects.com carstrends2015.com
compliance-checker.info cartierbraceletsreplica.pw
compliance-don.top cartierlovestore.xyz
compliance-irvin.top cayado.snn.gr
compliance-margo.top censys.io
construmac.com.mx charmstroy.info
coolwallpapers-hd.com cheapmoncler.win
coolwallpapers4k.info check-this-out-now.online
chiptuninger.com
clmforexeu.com
commoncrawl.org
compliance-john.top
conocer-sanabria.com
cookie-law-enforcement-ee.xyz
cougfan.info
counter.bloke.com
counterbot.com counterbot.com
cpajunkies.com customchocolate.business-for-home.com
cyclobenzaprinevstizanidine.blogspot.com
dailystrength.org dailystrength.org
danschawbel.com daretodonate.co
dekoration.us dealighted.com
detalizaciya-tut.biz deplim.com
dnepropetrovsk.mistr-x.org domznaniy.ru
domainsphoto.com easycommerce.cf
doxyporno.com elexies.info
doxysexy.com eng-lyrics.com
drillsaw.ru ezigarettekaufen2.blox.pl
elektrischezigaretten1.blogse.nl f00kclan.de
elitesportsadvisor.com familyholiday.ml
eloconcream.blogspot.com fikasound.tk
enginebay.ru footballfarrago.com
eshop4u.jp foxinsocks.ru
espaceinventoristes.com freenode.info
eu-cookie-law.info freeseedsonline.com
ewebarticle.info freeuploader.ml
ezigarettenshop2.myblog.de freewareseek.com
fakehandbags.xyz fun-mobi.pl
family1st.ca fun2cell.net
fast-wordpress-start.com geilehausfrauen.net
favornews.com germetiki.com.ua
fenoyl.batcave.net ghostvisitor.com
fitnesspiks.com gidonline.one
forsex.info gigapeta.com
foxjuegos.com goforexvps.com
free411games.com golaya.pw
freegamesplay.online goodly.pro
freeuploader.com graphid.com
freevpn.space gsmlab.pl
fsakhalin.ru gsmtlf.ru
gabeshop.ru hccoder.info
gamingspark.com hidemyass.com
genetworx.com homik.pw
gg.zzyjxs.com house.sieraddns.com
glass-msk.ru hptwaakw.blog.fc2.com
goo.ne.jp husky-shop.cz
googglet.com hustoon.over-blog.com
gopro-online.info i-service.kz
gpirate.com imgata.com
gpms.org.my instakink.com
greatidea.marketing intellego.info
groupmoney.ru jmat.cn
gsasearchengineranker.site jobgirl24.ru
gwebtools.com jongose.ninja
hao123.com kerei.ru
hdseriale.pl keywesthideaways.co
heartofpayne.xyz kicknights.gq
hildinghr.se kinomaniatv.pl
homemade.gq kinopolet.net
hotdl.in kletkimehan.ru
houdom.net krafte.ru
insta-add.pro krassh.ru
insurple.com lancheck.net
intelhdgraphicsgtdrive6w.metroblog.com landscapebackgrounds.blogspot.com
interfucks.net life.biz.ua
ispaniya-costa-blanca.ru linkdebrideur.xyz
j33x.com lion.conboy.us
japan-bearings.ru littlesunraiser.com
kaliningrad.zrus.org live-sexcam.tk
keywords-monitoring-success.com lovasszovetseg.hu
komputernaya-pomosh-moscow.ru low-format.ru
kosmetyki.tm.pl macresource.co.uk
kumuk.info magiadinamibia.blogspot.com
laexotic.com manyresultshub-a.akamaihd.net
liran-locks.com marketingvici.com
lob.com.ru mebeldekor.com.ua
marcogrup.com medicinacom.ru
mebel-vstroika.ru medtherapy.ru
mecash.ru meduza-consult.ru
mediawhirl.net minegam.com
memberty.com mmostrike.ru
michellblog.online moviezbonkerssk.cf
minharevisao.com mttwtrack.com
mir-business-24.ru my-bc.ru
mixtapetorrent.com myfreetutorials.com
mountainstream.ms
movies-in-theaters.net
movpod.in
mustwineblog.com
myhydros.org
mykings.pw mykings.pw
naturalshair.site myra.top
newsperuse.com mywallpaper.top
newstaffadsshop.club narkologiya-belgorod.ru
nomuos.it nbsproject.ru
netallergy.ru
netfacet.net
next-dentists.tk
nlfjjunb5.ru
nofreezingmac.work
notaria-desalas.com notaria-desalas.com
novgorod.xrus.org o-o-6-o-o.ru
nzfilecloud.weebly.com ok-ua.info
okroshki.ru one.net.in
onemantrip.com onlinadverts.com
onlyporno.ru ozas.net
optibuymac.com
oren-cats.ru
osoznanie-narkotikam.net
oyster-green.com
page2rss.com
pagesense.com pagesense.com
paleohub.info partnerworkroom.men
palvira.com.ua perfectpracticeweb.com
petedrummond.com pinup-casino1.ru
phobia.us pivka.xyz
plastweb.ru plastgran.com
porngalleries.top ploenjitmedia.azurewebsites.net
pornodojd.ru poisk-zakona.ru
portal-eu.ru pornogratisdiario.com
portnoff.od.ua pornohub.me
porto.abuilder.net pornosok.ru
pospr.waw.pl pro-poly.ru
pricheskaonline.ru psoriasis-file.trade
princeadvantagesales.com pyrodesigns.com.au
prizeestates.cricket
profit-opportunity.com
prostitutki-odessa.org
pulseonclick.com
qwertty.net qwertty.net
razamicroelectronics.com radiodigital.co
reining.lovasszovetseg.hu raschtextil.com.ua
rockprogblog.com remybutler.fr
rosbalt.com.ua respublica-otel.ru
rusenvironmental.net satellite.maps.ilovevitaly.com
rvtv.ru savetubevideo.com
s8-nowy-wygraj.comli.com scanner-jessica.top
sady-urala.ru scanner-mary.top
santechnik.jimdo.com scopich.com
savefrom.com sdsjweb.com
sayyoethe.blogspot.co.za se-welding.ru
sbt-aqua.ru searchlock.com
scanner-jess.top seomarketings.online
seeingmeerkat.com service.adtech.us
semalt.com sex-tracker.com
seobility.net shemale-sex.net
seorankinglinks.com shoppingmiracles.co.uk
sexpornotales.com shtora66.ru
share-buttons-for-free.com sitechecker.pro
shariki-zuma-lines.ru snegozaderzhatel.ru
shopvilleroyboch.com.ua softtor.com
siteonomy.com sogimlecal.tk
smailik.org solinf.co
softlinesolutions.me sovetogorod.ru
softxaker.ru spb-plitka.ru
soviet-portal.do.am sstroy44.ru
stairliftsarea.com star61.de
stpolice.com startwp.org
strongsignal-a.akamaihd.net steamoff.net
studiofmp.com sterva.cc
superkanpo.com streetfire.net
stromerrealty.com
stroyhelp-dv.ru
stuffpride.com
supermama.top
superstarfloraluk.com superstarfloraluk.com
symphonyintegratedhealthcare.com supervesti.ru
szamponrevita.pl susanholtphotography.com
tampabaywatch.org symbaloo.com
thegameriders.com taqplayer.info
thelottosecrets.com template-kid.com
tilido.com templates.radiodigital.co
tizanidineandhydrocodone.blogspot.com thefotosgratis.eu
tonivedu.it thepornsex.org
truemfilelj.gq threecolumnblogger.com
tubo360.com timdreby.com
tuckpointingmasonrysystems.com tizanidinehcl4mgisitanarcotic.blogspot.com
twittrading.com tizanidinerecreationaluse.blogspot.com
tzritel.tk tizanidinezanaflex.blogspot.com
udayavani.com top10-way.com
uptime.com topclickguru.com
uptimechecker.com toproadrunner5.info
uroki.net toursmaps.com
vezuviy.su truck-addzilla.life
viagra-soft.ru ucsol.ru
videochat.cafe unmaroll.ya.ru
videokrik.net v24s.net
villacoloniale.com video-woman.com
vip-parfumeria.ru videochaty.ru
visionwell.com.cn videospornogratisx.net
vpnmouse.com virtuagirl.com
web-analytics.date voloomoney.com
webnode.me wallpapersdesk.info
wellcome2slovenia.ru warningwar.ru
whatzmyip.net webstatsdomain.org
where-toget.com white-truck.life
wholinkstome.com wonderfulflowers.biz
woodyguthrie.se wordpresscore.com
writingservices17.blogspot.ru
wrona.it
ww1943.ru ww1943.ru
x69ty.ru xboxster.ru
xerox-douglas.cf xev.ru
xjlottery.com xingzi-vision.com
xlovecam.com xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
xmlinde.com xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xn--b1ag5cfn.xn--p1ai
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai xn--oogle-wmc.com
xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks xxx-cam.webcam
xn----9sbebi2bvzr7h.xn--p1ai yellowads.men
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai you-shall-not-pass.is74.ru
xn--80ab4aa2g.xn--p1ai yougame.biz
xnxx-n.com youjizz.com
youraticles.pl yourmovies.pl
yourserverisdown.com yourtemplatefinder.com
zdesoboi.com yur-p.ru
zigzog.ru zaobao.com.sg
zionstar.net zhacker.net
zone-kev717.info znakomstvaonlain.ru
zryydi.com zrelaya.pw
zs2vm.top zvezdagedon.ru

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.1.142 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2770 ### Version: V4.2021.12.2771
### Updated: Fri Dec 10 10:03:35 UTC 2021 ### Updated: Fri Dec 10 22:01:30 UTC 2021
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2770 ### Version: V4.2021.12.2771
### Updated: Fri Dec 10 10:03:35 UTC 2021 ### Updated: Fri Dec 10 22:01:30 UTC 2021
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##