vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.01.2875 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-01-24 22:07:39 +00:00
parent 216a0c45bb
commit d6051172a8
46 changed files with 93048 additions and 93048 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2874 #### Version: V4.2022.01.2875
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2874 #### Version: V4.2022.01.2875
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.01.2874 #### Version: V4.2022.01.2875
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11496
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.01.2874 #### Version: V4.2022.01.2875
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11500
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11500
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11500
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11500
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 1; 10.1.0.11 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 0; 10.1.0.11 0;
127.0.0.1 0; 127.0.0.1 0;

11500
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 1; 10.1.0.11 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 0; 10.1.0.11 0;
127.0.0.1 0; 127.0.0.1 0;

11500
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 1; 10.1.0.11 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 0; 10.1.0.11 0;
127.0.0.1 0; 127.0.0.1 0;

11500
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2874 2875

11500
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11500
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.52.38 0; 168.61.166.128 0;

11500
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 1; 10.1.0.11 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 0; 10.1.0.11 0;
127.0.0.1 0; 127.0.0.1 0;

11500
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 1; 10.1.0.11 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 0; 10.1.0.11 0;
127.0.0.1 0; 127.0.0.1 0;

11500
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 1; 10.1.0.11 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 0; 10.1.0.11 0;
127.0.0.1 0; 127.0.0.1 0;

11500
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.52.38 0; 168.61.166.128 0;

11500
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.52.38 0; 168.61.166.128 0;

11500
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 1; 10.1.0.11 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Citoid Chlooe
EroCrawler Craftbot
Fyrebot Discobot
IndeedBot GalaxyBot
InfoNaviRobot WebGo IS
JOC Web Spider WebReaper
JetCar Who.is Bot
VidibleScraper YoudaoBot
Zeus dataforseo.com
satoristudio.net voyagerx.com

290
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider
404checker 404checker
80legs
Abonti
Aboundex
Aboundexbot Aboundexbot
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot Alexibot
AiHitBot AllSubmitter
Aipbot AlphaBot
Anthill
Aspiegel
Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot
BBBike
BDFetch
BLEXBot
BackDoorBot BackDoorBot
BackWeb
Backlink-Ceck
BacklinkCrawler
Bandit
Barkrowler Barkrowler
BatchFTP BatchFTP
Battleztar Bazinga
BetaBot BetaBot
Bigfoot Bitacle
BlackWidow Black Hole
BlowFish BlowFish
CATExplorador Boardreader
CCBot Bolt
CODE87 Brandwatch
Buck
Bullseye
CSHttp
Calculon Calculon
Cegbfeieh Cegbfeieh
CensysInspect CheeseBot
Citoid CherryPicker
Claritybot
Cloud mapping
Cocolyzebot
Copyscape Copyscape
CrazyWebCrawler Cosmos
CrunchBot Craftbot
Curious Custo
CyotekWebCopy CyotekWebCopy
DIIbot DIIbot
Dirbuster Dirbuster
DittoSpyder Discobot
Discoverybot
Dispatch
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot DomCopBot
DomainSigmaCrawler DomainAppender
Dotbot
Download Wonder Download Wonder
Drip
ECCP/1.0
EasyDL EasyDL
Ebingbong Ebingbong
Express WebPictures Ecxi
ExtractorPro FHscan
Ezooms
FDM
FemtosearchBot FemtosearchBot
Fimap
Firefox/7.0 Firefox/7.0
Foobot
Fuzz Fuzz
G-i-g-a-b-o-t G-i-g-a-b-o-t
GalaxyBot GetWeb
GetRight
Gigabot
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla Gotit
Grafula Grabber
GrapeshotCrawler HEADMasterSEO
HTTP::Lite HaosouSpider
HTTrack
Haansoft
Heritrix Heritrix
Hloader HonoluluBot
Humanlinks
Iblog
IndeedBot IndeedBot
InterGET InterGET
Internet Ninja Internet Ninja
Iria InternetSeer
Iskanie
IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT JamesBOT
JennyBot JennyBot
JetCar
Jetty Jetty
JikeSpider
Joomla
Jorgee
JustView JustView
Kinza Kenjin Spider
Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple
Lanshanbot
Larbin Larbin
LexiBot
Lftp Lftp
Likse LibWeb
Libwhisker
LinkScan LinkScan
Linkbot
LinkextractorPro LinkextractorPro
LinkpadBot LinkpadBot
LinqiaRSSBot LinqiaScrapeBot
Litemage_walker Lipperhey
Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc
MQQBrowser MQQBrowser
MSIECrawler
Mag-Net Mag-Net
Magnet Mass Downloader
Mail.RU_Bot
Majestic12
Masscan Masscan
Mata Hari Mata Hari
Microsoft Data Access Mb2345Browser
Mojolicious Meanpathbot
Metauri
MicroMessenger
Minefield
MolokaiBot
Mozlila Mozlila
Msrabot NICErsPRO
Name Intelligence NPbot
Navroad Nameprotect
Nessus NetLyzer
Net Vampire NetMechanic
NetZIP NetZIP
Netcraft
Nettrack
Netvibes
NextGenSearchBot NextGenSearchBot
Not Nikto
Nuclei NimbleCrawler
Nimbostratus
Nmap
Nutch
Octopus Octopus
Offline Explorer Offline Explorer
Offline Navigator
OpenLinkProfiler OpenLinkProfiler
Openvas Openfind
OrangeBot OrangeBot
OutclicksBot OrangeSpider
OutfoxBot OutfoxBot
PECL::HTTP PECL::HTTP
PHPCrawl
PageAnalyzer PageAnalyzer
PageScorer Pandalytics
Papa Foto Pavuk
PeoplePal Pimonster
Petalbot ProWebWalker
Picscout
PictureFinder
PleaseCrawl
ProPowerBot
Psbot Psbot
Pump PxBroker
PyCurl
Quick-Crawler Quick-Crawler
RSSingBot RankActiveLinkBot
RankFlex
RankingBot
RankurBot RankurBot
Recorder Ripper
RedesScrapy
RepoMonkey
SBIder SBIder
SISTRIX SEOkicks
ScanAlert SEOkicks-Robot
SEOlyticsCrawler
SalesIntelligent
ScoutJet
Scrapy
ScreenerBot ScreenerBot
SearchmetricsBot SearchmetricsBot
SemrushBot SemrushBot
SentiBot SentiBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz
Shodan Shodan
Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteSnagger
SiteSucker SiteSucker
SlySearch
Snapbot Snapbot
SocialRankIOBot Snoopy
Sogou web spider Sogou web spider
Sosospider
Sottopop Sottopop
SpaceBison SpaceBison
Spbot Spbot
SputnikBot SputnikBot
Sqlmap Sqlmap
Stripper Sqlworm
Sucker Sqworm
SuperBot
SuperHTTP
Surfbot Surfbot
Suzuran Suzuran
Szukacz Szukacz
T0PHackTeam T0PHackTeam
T8Abot Teleport
Telesphorep Telesphoreo
The Intraformant
Thumbor Thumbor
TightTwatBot
Titan Titan
Toata Toata
Toweyabot
Trendiction Trendiction
Trendictionbot Turnitin
True_Robot
TurnitinBot
TwengaBot TwengaBot
Twice
Typhoeus Typhoeus
URLy Warning URLy.Warning
UnisterBot
Upflow Upflow
VB Project V-BOT
VelenPublicWebCrawler Vacuum
Virusdie Virusdie
WBSearchBot Voil
Voltron
WASALive-Bot
WISENutbot
WPScan WPScan
Wallpapers WWWOFFLE
WallpapersHD
WeSEE WeSEE
Web Fuck Web Collage
WebAuto Web Enhancer
WebCollage Web Sauger
WebCopier WebEnhancer
WebFetch WebFetch
WebLeacher WebGo IS
WebImageCollector
WebPix WebPix
WebStripper WebStripper
WebWhacker WebSucker
WebmasterWorldForumBot WebZIP
Website Quester Website Quester
Whack WebsiteExtractor
Whatweb Whacker
Widow Widow
WinHTTrack
WiseGuys Robot WiseGuys Robot
Woobot Xenu
Wotbox
Wprecon
Zade Zade
Zermelo Zermelo
ZoomBot ZmEu
ZoominfoBot ZyBorg
arquivo.pt adscanner
autoemailspider autoemailspider
backlink-check backlink-check
clark-crawler
coccocbot-web
cognitiveseo
crawl.sogou.com crawl.sogou.com
eCatch dataforseo.com
facebookscraper domainsproject.org
gopher evc-batch
internetVista monitor heritrix
lwp-trivial muhstik-scan
meanpathbot
netEstate NE Crawler netEstate NE Crawler
page scorer
pcBrowser pcBrowser
plumanalytics polaris version
probe-image-size satoristudio.net
seobility scalaj-http
seostar scan.lol
sitechecker.pro serpstatbot
siteripz siteripz
sp_auditbot tAkeOut
spyfu
trendiction.de trendiction.de
webprosbot webgains-bot
x09Mozilla webmeup-crawler
xpymep1.exe webpros.com
zgrab x22Mozilla
zauba.io

318
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker AIBOT
80legs
ALittle Client ALittle Client
ASPSeek
Abonti
Acunetix Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot Alexibot
Aipbot AllSubmitter
Alligator Anarchie
Anarchy99 Anthill
Apexoo Apexoo
Aspiegel Asterias
Attach
BBBike BBBike
BackDoorBot BDCbot
BLEXBot
BackStreet BackStreet
Backlink-Ceck Badass
Bandit Barkrowler
Battleztar Bazinga Black Hole
Bigfoot
Bitacle
BlackWidow BlackWidow
Blow Blow
BlowFish BlowFish
Boardreader Boardreader
Bolt
BotALot BotALot
Brandprotect
Buck Buck
Bullseye
BunnySlippers
CCBot
CODE87 CODE87
Calculon CazoodleBot
CensysInspect Cegbfeieh
CheeseBot CheTeam
ChinaClaw
Chlooe Chlooe
Citoid Claritybot
Cliqzbot
Cloud mapping
Cocolyzebot
Collector Collector
Copier
CopyRightCheck CopyRightCheck
Copyscape
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Crescent
Curious Curious
Custo
CyotekWebCopy
DBLBot DBLBot
DIIbot
DTS Agent
DatabaseDriverMysqli
Demon Demon
Deusu DigitalPebble
Devil
Dirbuster Dirbuster
Dispatch Disco
DomCopBot Discoverybot
DomainAppender DomainSigmaCrawler
Domains Project Domains Project
EMail Wolf Download Wonder
Ebingbong Dragonfly
Ecxi EasyDL
EroCrawler EirGrabber
ExtLinksBot
Extreme Picture Finder Extreme Picture Finder
FHscan FDM
Fimap FemtosearchBot
Firefox/7.0 Fuzz
FlashGet
Flunky
Freeuploader
FyberSpider
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW GetRight
GalaxyBot GetWeb
Gigabot Getintent
Go!Zilla Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
Grafula GoZilla
Gotit
Grabber
GrapeFX GrapeFX
HMView GrapeshotCrawler
HTMLparser GridBot
HaosouSpider HTTP::Lite
Havij Harvest
Heritrix Heritrix
Humanlinks Hloader
IDBTE4M IDBTE4M
IDBot Image Sucker
Iblog InfoNaviRobot
Id-search Internet Ninja
Image Fetch IstellaBot
InfoTekies JetCar
InterGET
InternetSeer
JamesBOT
Jbrofuzz
Joomla
Jorgee
JustView
Jyxobot Jyxobot
Kenjin Spider Kenjin Spider
Keyword Density
Kozmosbot Kozmosbot
LNSpiderguy LWP::Simple
Lanshanbot LeechGet
LexiBot
Lftp
LibWeb
Likse Likse
LinkpadBot LinkScan
LinkWalker
LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot MFC_Tear_Sample
Lipperhey Spider MIDown tool
MIIxpc MIIxpc
MJ12bot
MQQBrowser
MSIECrawler MSIECrawler
Mag-Net
Magnet
Majestic SEO Majestic SEO
Majestic-SEO Majestic-SEO
Majestic12
Masscan Masscan
Mata Hari MauiBot
Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri Microsoft Data Access
Microsoft URL Control Mister PiX
Minefield Mojeek
Moblie Safari
Mojolicious Mojolicious
MolokaiBot MolokaiBot
Msrabot Morfeus Fucking Scanner
Musobot Mozlila
NPbot
Nameprotect Nameprotect
NearSite Navroad
Needle Needle
Net Vampire Nessus
NetAnts NetAnts
NetSpider NetSpider
NextGenSearchBot NetZIP
Nettrack
Netvibes
Nibbler Nibbler
Nikto Nikto
Nutch NimbleCrawler
Nmap
Not
Nuclei
Octopus
Offline Navigator Offline Navigator
Openvas OpenVAS
OrangeSpider
OutclicksBot OutclicksBot
OutfoxBot
PECL::HTTP
Page Analyzer Page Analyzer
Panscient PageScorer
Papa Foto
Pavuk Pavuk
Petalbot Petalbot
Pi-Monster PictureFinder
Picscout Pimonster
Piepmatz
Pixray Pixray
PleaseCrawl
Pockey
ProWebWalker ProWebWalker
Probethenet
Psbot Psbot
Pu_iN
Pump Pump
PxBroker RankingBot
Quick-Crawler RankingBot2
RankActiveLinkBot RankurBot
RankFlex ReGet
RealDownload Reaper
Recorder Recorder
RedesScrapy RocketCrawler
RepoMonkey
Ripper
Rogerbot Rogerbot
SBIder SBIder
SEOkicks SEOlyticsCrawler
SEOstats
SISTRIX SISTRIX
Scrapy SalesIntelligent
Seekport ScrepyBot
SeoSiteCheckup SemanticJuice
Semrush
SentiBot
Shodan Shodan
Siphon
Site Sucker Site Sucker
Sitebeam SiteCheckerBotCrawler
SlySearch SiteExplorer
Sociscraper SmartDownload
Sogou web spider Spbot
Sosospider SputnikBot
SpaceBison Sqlmap
Spinn3r Sqworm
Sqlworm
Steeler
Stripper Stripper
Sucuri
SuperBot SuperBot
Surfbot SuperHTTP
Suzuran
Swiftbot
Szukacz Szukacz
T0PHackTeam T0PHackTeam
Teleport Teleport
TightTwatBot Telesphoreo
Titan Telesphorep
Toata TheNomad
Trendictionbot Toweyabot
True_Robot Tracemyfile
Turingos Turingos
Turnitin Turnitin
Twice Twice
Typhoeus
UnisterBot UnisterBot
Upflow
V-BOT
Vacuum
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VidibleScraper VoidEYE
Voil Voltron
WISENutbot Wallpapers
WPScan
WWWOFFLE
WallpapersHD
Web Enhancer Web Enhancer
Web Pix Web Pix
Web Sucker Web Sucker
WebAuto WebBandit
WebCollage WebFuck
WebEnhancer WebStripper
WebFetch WebmasterWorldForumBot
WebReaper Webster
WebSauger Whack
WebSucker
Webalta
Webshag
Website Quester
WebsiteExtractor
WebsiteQuester
Whacker
Who.is Bot
WinHTTrack
WiseGuys Robot
Wonderbot
Woobot
Wprecon Wprecon
Xenu Xenu
YoudaoBot YoudaoBot
Zauba Zade
Zitebot ZoomBot
ZyBorg ZoominfoBot
adscanner ZumBot
check1.exe arquivo-web-crawler
cognitiveseo autoemailspider
com.plumanalytics com.plumanalytics
crawler.feedback crawl.sogou.com
crawler4j
dataforseo.com dataforseo.com
domainsproject.org
eCatch eCatch
evc-batch
facebookscraper
gopher gopher
internetVista monitor internetVista monitor
ips-agent iubenda-radar
lwp-request lwp-request
lwp-trivial magpie-crawler
oBot muhstik-scan
pcBrowser pcBrowser
seobility sexsearcher
seocompany.store ubermetrics-technologies.com
sitechecker.pro
spyfu
trendiction.com
trendiction.de
voyagerx.com voyagerx.com
webmeup-crawler webgains-bot
x22Mozilla webpros.com
webprosbot

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
advancedsoftwaresupport.com 2015god.org
bashtime.ru affiliate-fr.com
blog.remote-computer.de azartniy-bonus.com
bristolhotel.com.ua dickssportinggoods.com
elektrozigarettekaufen2.blogse.nl erotische-geschichten-xxl.com
frvo.alptandem.ru g33.org
onlinetvseries.me mejoresfotos.eu
privetsochi.ru syvertsen-da.no
replica-watch.ru thenetinfo.com
virus-schutzmasken.de zakazvzloma.com

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
000free.us 0912701309f8ce.com
00go.com 100searchengines.com
00it.com 123locker.com
12bet.com 202ch.com
12u.info 24x7-server-support.site
2728fb936f0.com 2ads.co.uk
3th.co.in 4k-player.pl
3w24.com 4webmasters.org
4kepics.com 5elementov.ru
5-steps-to-start-business.com 5u.com
6128786.com 66cpwgln.space
7xc4n.com 883zy.com
8xv8.com a.pr-cy.ru
98oi.ru a14download.com
aarbur.com aa8b68101d388c446389283820863176e7.com
add-add.men actices.com
addblueoff.com.ua activecampaign.dreamhosters.com
adobereader-free.ru admeasures.com
ads.tremorhub.com adultactioncam.com
adsref.men affiliate-fr.com
affrh2015.com agadelha.com.br
aibolita.com ahhjf.com
airlandsea.info airlandsea.info
alfa-img.com alevinefesleri.com
aliexpresscashback.club all-number.com
alive-ua.com all-streaming-media.com
allsilver925.co.il allforminecraft.ru
allwomen.info alloysteel.ru
aludecor.info
am-se.com
amanda-porn.ga amanda-porn.ga
anabolics.shop android4fun.org
animali.deagostinipassion.it anonymous-redirect.com
animaltoplist.com aosexkontakte.net
apiitsthirteende-a.akamaihd.net apelsinnik.website
appearance-cool.com apishops.ru
appleid-verification.com apple.com-webbrowsing-security.science
apps-analytics.net apptester.tk
apxeo.info aramaicmedia.org
arcadepages.com
arendas.net arendas.net
arewater.com arewater.com
arquapetrarca.info
artdestshop.eu artdestshop.eu
atofilms.com articlesdirectoryme.info
audiofree.ru ascat.porn
autoplate.info asophoto.com
autowebmarket.com.ua auto.rusvile.lt
aviapanda.ru
aviav.eu aviav.eu
avitocars.ru avon-novosib.ru
avramstroy.ru
avtoarenda.by
avtointeres.ru
axbocz.net axbocz.net
ayerbo.xhost.ro azarttoto.com
azbukadiets.ru azazu.ru
azinoofficial777.ru azinoofficial777.ru
azte.ch baixar-musicas-gratis.com
bankofthewext.com bang-hotties.com
basisches-wasser.net best-coupon-offer.com
bbtec.net bestdraws.com
bestimagecoollibrary.com
bestmobilityscooterstoday.com
bestsexyblog.com
beta.hotkeys.com beta.hotkeys.com
betterscooter.com betonka.pro
bibys.com betune.onlinewebshop.net
bildsuche.ru bidr.trellian.com
bimatoprost-careprost.com.ua biglistofwebsites.com
bioinnovate.co binomo.kz
bitcoins-live.ru bio-optomarket.ru
bizcheapjerseyswholesalechina.com birzha-truda.eu
blackbabesporn.com bjanshee.ru
blackhatworth.com bluejays-jerseys.us
blog-hits.com boazpower.com
bloggerads.net
bluesman.nu
bnt-team.com
bochemit.com.ua bochemit.com.ua
bookmark4you.com bodybuilding-shop.biz
brandov.ru boleznikogi.com
brendbutik.ru boo-arts.com
brewdom.ru bookhome.info
buchananshardware.com bookmark4you.com.biz
bumble.cheapwebsitehoster.com bouda.kvalitne.cz
buqyxa.rincian.info buehne-fuer-menschenrechte.de
bus-offer.com bumascloset.com
businesn.men buyhoverboard.com
buyessay3.blogspot.ru buynorxx.com
buzzonclick.com camel-beach.com
c-gainsbourg.com candlewooddentalcentre.com.au
cactussoft.biz canzoni.ru
calc-for-credit.ru carina-sy.de
californianews.cf cartechnic.ru
callejondelpozo.es
camdenmemorials.com
canrioloadm.gq
cartierlove.xyz
cartierloveringreplica.xyz
cayado.snn.gr
cbb1smartlist12.click cbb1smartlist12.click
cheapmoncler.pw ccjp.eu
cheapsergiorossi.online cezartabac.ro
checktext.org cfsrating.sonicwall.com
cheapjerseysbizwholesale.us
check-host.net
chee-by.biz
chiptuninger.com
chocolateslim-u-srbiji.com chocolateslim-u-srbiji.com
christianlouboutinreplica.win
christianlouboutinshoes.xyz christianlouboutinshoes.xyz
chuckguilford.com cicaki.net
ciekawinki.pl cienum.fr
claytransformations.info clarithromycin500mg.com
clickprotects.com clean-start.net
cobaltpro.ru clothesforcash.com
com-secure.download compliance-george.top
community.allhiphop.com compliance-julia.top
compliance-jess.top counterbot.com
conboy.us countercrazy.com
contentlook.co covid-schutzmasken.de
converse.ddsoldes.fr cracksplanet.com
coprofam.org cranly.net
cosmeticswomens-womensports.rhcloud.com credit.co.ua
country-chic.ru crest-poloski.ru
creativehutindia.com crystalslot.com
credit-online.ws cyprusbuyproperties.com
csgo4.win daneshetabiat.com
custom-product-labels.com darkages.info
d0t.ru datadepths.com
dairyindia.in davebestdeals.com
deanmoore.ie daymusam.com
detective01.ru decenttools.com
dev78.dioniqlabb.se delfin-aqua.com.ua
dentfidemountpreach.tk
dichvuvesinhhanoi.com dichvuvesinhhanoi.com
discovertreasure-a.akamaihd.net digest-project.ru
divci-hry.info digital-video-processing.com
divku.ru
dms-sw.ru dms-sw.ru
dokumentalkino.net documentsite.net
domaincheck.io dokfilms.net
domik-derevne.ru domain-submit.info
downloadmefiranaratb1972.xpg.com.br domainsphoto.com
drev.biz doska-vsem.ru
driving.kiev.ua e-collantes.com
droidlook.net e-commerce-seo.com
drpornogratisx.xxx electrik-avenue.com
duplicashapp.com elkacentr.ru
dvd-famille.com eluxer.net
earn-from-articles.com evening-dating-club.info
econom.co everflora.ru
efnor-ac.com extremal-blog.com
egovaleo.it extremepornos.net
ekto.ee ezigarettenkaufen1.pagina.gr
electricwheelchairsarea.com face.hostingx.eu
elektrozigaretten1.tumblr.com farmprofi.net
eloconcream.blogspot.com
energydiet24.ru
enginebay.ru
engines-usa.com
eralph.tk
etur.ru
eugenevaultstorage.com
extlabs.io
extlinks.com
f07.de
f5mtrack.com
factorynightclub.com
fanpagerobot.com
faracontrol.ir
fashion-mk.net fashion-mk.net
fashionindeed.ml ferrotodo.com
favornews.com fillmewithhappiness.com
fickluder69.com findpics.pw
filmania-x.ru firesub.pl
findsexguide.com fitnesspiks.com
finuse.com fivedwld.cf
fishingwholesale.us floridamobilebillboards.com
fix-website-errors.com foodcrafts.website
flashahead.info forex21.ru
floridamhca.org forsex.info
fordsonmajbor.cf fortevidyoze.net
fortwosmartcar.pw fotopop.club
foto-weinberger.at fr.netlog.com
foxtechfpv.com freegamesplay.online
free-deals.faith freewhatsappload.com
freemags.cc frustrated-favorable.gq
freewlan.info g33.org
fsakhalin.ru galleryawesome.com
fuel-gas.com game300.ru
galeon.com
games.kolossale.ru games.kolossale.ru
gateway.zscalertwo.net
gay-file.com gay-file.com
gay-site.store germetiki.com.ua
gelezki.com getdot.ru
gemara.com gilsonchiro.xyz
globalscam.ga globalsurfari.com
glopages.ru goodhumor24.com
goblacked.com googlemare.com
good-mummy.ru gratis-sexkontakte.com
grand-chlen.ru greatfind-a.akamaihd.net
greatdealshop.com greenshop.su
guod.me gribkovye-zabolevaniya.com
hasfun.com growshop.es
hdseriale.pl gsasearchengineranker.xyz
hdwallpapers-free.com hacktougroup.ru
handsandlegs.ru
harmonyglen.us
hccoder.info
hdfreeporno.net
healing-dysplasia.ru
help.tpu.ru
helpmymacfaster.trade helpmymacfaster.trade
helvetia.com.ua hentaiheroes.com
hermesreplica.win
heroz.fr
hit-kino.ru
hitcpm.com
home-task.com
home.myplaycity.com home.myplaycity.com
homedecoguide.info hotblog.top
hotel-mkad.ru houtings.xyz
hotsocialz.com hoztorg-opt.ru
ideibiznesa2015.ru hplaserjetpdriver8y.pen.io
igrovyeavtomaty777.ru htmlcorner.com
i-midias.net.br
iamsport.org
ideashome.id
ihtec2019.org
iideaidekonkatu.info iideaidekonkatu.info
incomekey.net ilmexico.com
indetiske.ya.ru iloveitaly.ru
indo-export.ru imagecoolpub.com
infostatsvc.com imagez.co
innodgfdriverhm.aircus.com imagui.eu
inspiring-desperate.tk incolors.club
integritylandscapeservices.com increasewwwtraffic.info
indiakino.net
indianmedicaltourismshop.com
interesnie-faktu.ru
interfucks.net interfucks.net
internetproviderstucson.com iphantom.com
investyb.com isistaylorporn.info
iskalko.ru jabimgo.pw
islamtoday.co.za
ispaniya-costa-blanca.ru
istripper.com
javitas.info javitas.info
jerseywholesalechinabiz.com jennyfire.ru
jetsli.de johnnyhaley.top
jewelryandfiligree.com johnrobertsoninc.com
jrcigars.com juliadiets.com
junglenet-a.akamaihd.net juliaworld.net
kandidos.com justkillingti.me
kanimage.com jyrxd.com
karting196.ru kabbalah-red-bracelets.com
ketoanhanoi.info karusel-market.ru
kiwe-analytics.com kasino-money.pw
katjimej.blog.fc2.com
kazan.zrus.org
kazrent.com
kidd.reunionwatch.com
kidskunst.info
kihi.gdn
kino-fun.ru
kleine-titten.biz kleine-titten.biz
kletkimehan.ru klikbonus.com
knigonosha.net kongoultry.net
konoplisemena.com koral.se
kraljeva-sutjeska.com kredit-pod-zalog-krasnodar.ru
kustanay.kz kretpicf.pw
landscapebackgrounds.blogspot.com ladov.ru
laparfumotec.com landscaping.center
lapolis.it laxdrills.com
lawyersinfo.org lechenie-gemorroya.com
laxob.com lemon-ade.site
leadn.pl lescinq.com
lennatin.info letto.by
lernur.net
lerporn.info
lexiangwan.com lexiangwan.com
light.ifmo.ru lifeinsurancekb.com
lion.conboy.us likesuccess.com
lion.playtap.us lion.misba.us
loginduepunti.it liupis.com
lonerangergames.com lkbennettoutlet.online
love-baby.cz lmrauction.com
low-format.ru loadingpages.me
ltvperf.com lomb.co
m292.info losangeles-ads.com
mackeeper-land-672695126.us-east-1.elb.amazonaws.com lutherstable.org
magazintiande.ru m0r0zk0-krava.ru
maladot.com macotool.com
marcowebonyodziezowe.pl make-money-online.com
mebelcomplekt.ru makenahartlin.com
meendo-free-traffic.ga mall.uk
mikozstop.com matb3aa.com
misandesign.se matpre.top
missclub.info max-eclat.men
maximpartnerspr.com
maxthon.com
mebel-alait.ru
mediafresh.online
medicines-choice.com
medictube.ru
meduza-consult.ru
megaapteka.ru
menstennisforums.com
mex-annushka.ru
minegam.com
minet.club
mirmedinfo.ru
misswell.net
mister-shop.com mister-shop.com
mixtapetorrent.com
mjchamonix.org mjchamonix.org
mlvc4zzw.space mmofreegames.online
mobot.site mobioffertrck.com
mockupui.com modnie-futbolki.net
mojowhois.com moivestiy.biz
monarhs.info mojpreskumanie.com
monclerjacketsoutlet.win monclercheap.xyz
morocco-nomad-excursions.com moncleroutletonline.xyz
moskva.nodup.ru moscow-region.ru
mountainstream.ms mosdverka.ru
mp3downloadhq.com mrpornogratis.xxx
mrsdalloways.com muznachas-service.ru
msk-diplomat.com mycaf.it
mtmtv.info mydirtyhobby.com
mug-na-chas-moscow.ru myfreemp3.eu
mural.co myonigroup.com
muschisexbilder.com naj-filmy24.pl
my-floor.in.ua namecrumilchlet.tk
mybackgroundlandscape.blogspot.com neobux-bg.info
mydownlodablefiles.com networkcheck.xyz
namenectar.com newfilmsonline.ru
nastroyke.net newstudio.tv
national-today-winning-winner.club nhl17coins.exblog.jp
needtosellmyhousefast.com nodup.ru
nerudlogistik.ru norththeface.store
netfacet.net nosecret.com.ua
netoil.no o-o-6-o-o.com
ngps1.ru o-o-6-o-o.ru
nightvision746.weebly.com oakleyglassesonline.us
nikhilbahl.com offer.camp
nikkiewart.ru office-windows.ru
nisuturnetdgu.tk oklogistic.ru
njkmznnb.ru olympescort.com
nmrk.ru one.net.in
nodding-passion.tk ops.picscout.com
nofreezingmac.click outshop.ru
nrv.co.za ownshop.cf
o.light.d0t.ru papasdelivery.ru
o333o.com partner-cdn.men
obiavo.uz pastaleads.com
onemantrip.com photokitchendesign.com
online247.ml piuminiita.com
online7777.com pk-pomosch.ru
orion-code-access.net pokerniydom.ru
oynat.info polytopesexempt.com
page2rss.com pons-presse.com
paighambot.com popmarker.com
patterntrader-en.com popserve.adscpm.net
pattersonsweb.com pornhubforum.tk
pay2me.pl porno-play.net
paydayonlinecom.com porno-video-chati.ru
pc-services.ru pornophoto.xyz
pdn-4.com pornosmola.info
peekyou.com pozdrawleniya.com
perederni.net pr0fit-b0x.com
perm.xrus.org professionaldieselcare.com
perso.wanadoo.es profitfx.online
php-market.ru prpops.com
picture-group.com psvita.ru
pillscheap24h.com pts163.ru
pinup-casino1.ru pxhdwsm.com
pixelrz.com quickbuck.com
pixshark.com rabotaetvse.ru
plastgran.com rankia.com
ploenjitmedia.azurewebsites.net razleton.com
plohaya-kreditnaya-istoriya.ru
poem-paying.gq
pokemongooo.ml
pornhub-ru.com
porno-chaman.info
pornohub.me
pornotubexxx.name
pornpost.in
portnoff.od.ua
potolokelekor.ru
powenlite24.ru
priscilarodrigues.com.br
prod2016.com
profitsport.club
prosperent.com
pssucai.info
pufip.com
pushdata.sendpulse.com
r-e-f-e-r-e-r.com
rankscanner.com
rating-casino2021.ru
realresultslist.com realresultslist.com
redhotfreebies.co.uk realting-moscow.ru
reelheroes.net redbottomheels.xyz
registrydomainservices.com reklama-i-rabota.ru
remont-ustanovka-tehniki.ru research.ifmo.ru
renhacklids.tk rfserial.net
rent2spb.ru ric.info
ribieiendom.no robertefuller.com
ring4rhino.com rockingclicks.com
rollercoin.com rockprogblog.com
rucrypt.com rostov.xrus.org
ruex.org.ua rrutw.com
s1z.ru rubanners.com
s8-nowy-wygraj.comli.com ruscoininvest.company
saecsa.co rusexy.xyz
santasgift.ml s.lollypopgaming.com
sanyuprojects.com sad-torg.com.ua
sax-sex.com samo-soznanie.ru
samolet.fr
sashagreyblog.ga
savefrom.com
saveriopiazza.it
scanner-andrew.top
scanner-don.top
scanner-elena.top scanner-elena.top
scanner-viktor.top
sdi-pme.com
search.alot.com search.alot.com
seccioncontrabajo.com seksvideoonlain.com
securityallianceservices.com semp.net
seeingmeerkat.com sendearnings.com
sell-fb-group-here.com seo-tools-optimizing.com
seocheki.net seocheki.net
seoexperimenty.ru seoprofiler.com
seorussian.ru seorussian.ru
seozoom.it serdcenebolit.com
servethis.com sexgalleries.top
sex-tracker.com sexpornotales.com
sexvporno.ru sexy-pings.com
shemalegalls.blogporn.in shemalegalls.blogporn.in
shinikiev.com.ua shlyahten.ru
shop-electron.ru shohanb.com
shopperifymac.com simplesite.com
shoppingmiracles.co.uk
site-auditor.online
site3.free-share-buttons.com
sitebeam.net
sitevaluation.com
skatestick.bid
skutecznetabletkinaporostwlosow.pl
slavic-magic.ru
small-game.com small-game.com
sms2x2.ru smallseotools.com
social-buttons.com smart-scripts.com
social-buttons.xyz smartpet.ru
social-search.me snip.tw
spammen.de snjatie-geroinovoy-lomki.ru
spitfiremusic.com social-s-hhh.xyz
spravka130.ru sockshare.net
stal-rulon.ru speechfoodie.com
statustroll.com spinazdrav.ru
store-rx.com start.myplaycity.com
sunflowerdrawingpaintings.blogspot.com stockspmb.info
surcentro.com suggest-keywords.com
susanholtphotography.com sundrugstore.com
svolze.com support.nopeas.sk
swingerseiten.com surintech.ac.th
t3chtonic.com swinginwithme.ru
tabletkinaodchudzanie.com.pl tagil.zrus.org
tacbelarus.ru taihouse.ru
tampabaywatch.org takethatad.com
tasteidea.com tarad.com
telefonsexkostenlos.tk telefonsexkostenlos.tk
telegraf.by templates.franklinfire.co
telemetryverification.net thewomenlife.com
terraclicks.com titangel-vietnam.com
tetracsaudi.com tizanidineandcipro.blogspot.com
thefotosgratis.eu tizanidinecapsules.blogspot.com
thegameriders.com tizanidinehclsideeffects.blogspot.com
theheroes.ru tizanidinemusclerelaxant.blogspot.com
theporndude.com tmearegion26.com
timhost.ru toonfamilies.net
tizanidinezanaflex.blogspot.com topads.men
topanasex.com topvidos.ru
touchmods.fr tradgardspartner.se
tracksurf.daooda.com trafficnetzwerk.de
tradedeals.biz trafficstars.com
traffic2money.com truck-land.life
truemfilelj.gq tubeline.biz
turn-up-life.life turist-strani.ru
tvoystartup.ru tversvet.ru
ua-company.ru tvnewsclips.info
ublaze.ru ukrtvir.com.ua
udayavani.com unlimitdocs.net
urdoot.win upupa.net
uroffer.link urengoy.pro
utiblog.fr utrolive.ru
uzporno.mobi v24s.net
vacuumscleaner.com vancleefreplica.pw
validdomain.xyz vapomnoncri.tk
vavilone.com varbergsvind.se
vbabule.net vchulkax.com
veerotech.com vellings.info
venerologiya.com ventelnos.com
via-energy-order.com vesnatehno.ru
video-camer.com videochat.cafe
video-production.com videtubs.pl
vinsit.ru vip-file.com
visa-china.ru vip-parfumeria.ru
vodaodessa.com vipms.ru
vodkoved.ru vipsiterip.org
vriel.batcave.net virtuagirl.com
wallpapers-best.com vizitki.net
warningzscaler.heraeus.com vk-mus.ru
watch-movies.ru vkmusics.ru
vksex.ru
voloo.ru
vut.com.ru
vvon.co.uk
watchmyfb.pl
watracker.net
watsonrealtycorp.com
waycash.net
waysbetter.cn waysbetter.cn
websites-reviews.com webmonetizer.net
wechatdownload10.com webs.com
whiteproduct.com webscouter.net
wordkeyhelper.com webshoppermac.com
writersgroup580.web.fc2.com website-speed-check.site
wrz0iuebwhp5fg.freeddns.com wholesalejerseychinashop.com
wufak.com whos.amung.us
www.arenda-yeisk.ru wnoz.de
www.kabbalah-red-bracelets.com www.event-tracking.com
www.labves.ru xbaboon.com
x-diesel.com xblognetwork.com
xchangetrak.com xchat26.myfreecams.com
xdoza.com xep.info
xfluro.com
xgftnlrt.bloger.index.hr xgftnlrt.bloger.index.hr
xjrul.com xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
xn----7sbahjd3btneuw1joc.xn--p1ai
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
xn----9sbebi2bvzr7h.xn--p1ai
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai
xn--90acenikpebbdd4f6d.xn--p1ai xn--b1addnj3cah.xn--p1ai
xn--l1aengat.xn--p1ai xn--d1abj0abs9d.in.ua
xn--oogle-wmc.com xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
xolodremont.ru xnxx-n.com
xportvusbdriver8i.snack.ws
xuki.us
xvideospornoru.com
xxx-treker.ru xxx-treker.ru
yellocloud.be yebocasino.com
yougame.biz yubikk.info
youtuhe.com zaimite.ru
za-music.mymobiplanet.com zdesoboi.com
zapiszto.pl zeroredirect2.com
zazagames.org zeroredirect5.com
zhongwenlink.com
zhorapankratov7.blogspot.com
zigarettenonl.canalblog.com zigarettenonl.canalblog.com
zionstar.net zigarettenonlinekaufen2.bloog.pl
zixizop.net.ru zkjovpdgxivg.ga
znakomstvaonlain.ru zolotoy-lis.ru
zoogdisany.com zs2vm.top
zrizvtrnpale.tk zverokruh-shop.cz
zvuker.net

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00it.com 0c47f8422d3f.com
108shot.com 101billion.com
123any.com 1314dh.com
12masterov.com 1webmaster.ml
5kstore.com 2pxg8bcf.top
7a2rnuey1tw9ar.ru 2rich4bitches.com
991mostfm.co.id 3xforum.ro
a14download.com 4senses.co
activepr.ru 5tey7463.weebly.com
aa625d84f1587749c1ab011d6f269f7d64.com
adloads.net adloads.net
adzerg.com aliexpresscashback.club
affrh2015.com amazon-adsystem.com
alevinefesleri.com amyfoxfitness.com
algarveglobal.com
allmarketsnewdayli.gdn
alt-servis.ru
androids-store.com
angigreene.com angigreene.com
anonymousfox.co angkortours.vn
apartmentratings.com anmysite.com
apiwebwebgetcom-a.akamaihd.net anonymizeme.pro
arcadepages.com appsecurityr.com
as5000.com arcadeplayhouse.com
arenda-yeisk.ru
asiavirtualsolutions.com asiavirtualsolutions.com
asscenihotosidea.blogspot.com asociatia-tipografilor-transilvania.ro
astana.xxxkaz.org astrochicks.com
auto-moto-elektronika.cz bankcrediti.ru
autolombard-krasnodar.ru banki76.ru
autoseo-traffic.com bashtime.ru
avtolombard-krasnodar.com bazaronline24.ru
avtolombard-krasnodar.ru becuo.com
b2b-lounge.com best-seo-solution.com
beauty-clinic.ru bestcoin.cc
berdasovivan.ru betterhdporn.com
bestchoice.cf bingo8888.com
bestwrinklecreamnow.com bioscorp.ru
bet365.com birzha-truda.eu
bimmerpost.com boazpower.com
blackcurranthumidifiers.site bookmaker-bet.com
blog.axant.it brainxs.us
blogsfunky672.weebly.com buscarfoto.com
bumascloset.com buytizanidineonlinenoprescription.blogspot.com
candcstuccoandstone.com cartechnic.ru
cherrypointplace.ca ceotrk.com
chatseo.com
cheapwebsitehoster.com
checktext.org
clayvasedesigns.tk clayvasedesigns.tk
clickso.com cleanersoft.com
community.allhiphop.com clevernt.com
clksite.com
compliance-andrew.top compliance-andrew.top
compliance-jack.top compliance-irvin.top
cookielawblog.wordpress.com comsysnet.com
countbertwistdisp26.soup.io copenergo.ru
credit.co.ua copypaste.traderzplanet.in
cutt.us countercrazy.com
cxpromote.com crafthubs.com
cymbaltaandtizanidine.blogspot.com crirussian.ru
d2jsp.org csgo4.win
detki-opt.ru d0t.ru
deutschland123.de dataloading.net
dispo.de davebestdeals.com
dmmspy.com deanmoore.ie
dnsrsearch.com devochki.top
domainroam.win ebooknovel.club
doxysexy.com elektrischezigarette1.blog.pl
drpornogratisx.xxx elektrischezigaretten2.beeplog.com
duawitchrarato.tk en.altezza.travel
ebonyporn.site erectile.bid
elektrozigarette1.dreamwidth.org extlabs.io
energydiet24.ru eyessurgery.ru
eng-lyrics.com ezigarettenkaufen1.pagina.gr
essay-writing.work falopicm.pw
expresstoplivo.ru fatfasts-4tmz.com
extrabot.com ferrotodo.com
ezigarettenkaufen2.dreamwidth.org fikasound.tk
ezigarettenshop2.postbit.com financeloan.us
family1st.ca findclan.org
fickluder69.com finuse.com
filesclub.net floridamhca.org
focalink.com forum20.smailik.org
forexunion.net foxydeal.com
forms-mtm.ru free-deals.faith
formulaantiuban.com free-social-buttons.com
forumprofi.de freecamdollars.com
frbizlist.com freejabs.com
freelifetimefuckbook.com frequiry.com
fsakhalin.ru ftns.ru
funnymama.com funnymama.com
gamebackyard.com gaming-journal.com
gamingspark.com gheus.altervista.org
gfaq.ru gloverid.site
gigixo.com golden-catalog.pro
girlgamerdaily.com gotorussia.com
gomusix.com green-tea.tv
graphicwe.org greenshop.su
guiadeserraazul.com growmyfunds.ca
hacktougroup.ru guardlink.org
hcate.com hamptonoaks.ca
hesteel.pl harmonyglen.us
hitsbox.info help.tpu.ru
hobbyhuren24.net helvetia.com.ua
home-task.com hostcritique.com
hotxnights.info hotblognetwork.com
hoztorg-opt.ru howtoclean.club
humanorightswatch.org iblogpress.xyz
hut1.ru ikritikimou.gr
i-midias.net.br importchinacoach-teach.com
i-service.kz invivo.hu
ilovevitaly.ru jav-idol.com
imagerydatabase.com journeydownthescale.info
istizanidineanarcoticdrug.blogspot.com karadene.com
ivanovo.zrus.org karpun-iris.ru
jrpmakati.com ketrzyn.pl
jumptap.com keywordbasket.com
kalb.ru keywordsdoctor.com
kamalsinha.com keywordsking.com
kazinogames.lv keywordteam.net
khadastoafarde.tk kurbappeal.info
krasivye-devushki.net kwzf.net
kretpicf.pw laserpen.club
ktotut.net lenvred.org
ku6.com lexiangwan.com
ledis.top limads.men
linkarena.com lionking-1994.blogspot.com
linkdebrideur.xyz llastbuy.ru
lowephotos.info loadingpages.me
lsex.xyz localflirtbuddies.com
mabdoola.blogspot.com longadventure.com
makis.nu longlifelomilomi.info
mantramusic.ru louboutinreplica.xyz
mearns-tractors.co.uk lunamedia.co
mebelcomplekt.ru mackeeper-center.club
minegam.com magicalfind-a.akamaihd.net
mini-modus.ru mainlinehobby.net
missvietnam.org mansiondelrio.co
monetizationking.net mathgym.com.au
monetizer.com-01.site max-eclat.men
moomi-daeri.com maxthon.com
mowser.com mazda-roadsters.com
mrlmedia.net mebel-alait.ru
muycerdas.xxx meetingrainstorm.bid
mwtpludn.review mini.7zap.com
myfreetutorials.com minneapoliscopiers.com
myiptest.com mobicover.com.ua
myyour.eu montredemarque.nl
nina.az moscow.online-podarki.com
nisuturnetdgu.tk moviemail-online.co.uk
nodup.ru moxo.com
o-o-8-o-o.ru music7s.me
obnal.org my-aladin.com
mydownlodablefiles.com
mymobilemoneypages.com
nac-bearings.ru
narosty.com
net-radar.com
nevansk.ru
nicovideo.jp
nippon-bearings.ru
nonameread45.live
northfacestore.online
notaria-desalas.com
odoratus.net
oivcvx.website
ok-ua.info ok-ua.info
old-rock.com open-odyssey.org
online-zaymy.ru ops.picscout.com
onlinebay.ru oyster-green.com
p-business.ru panicatack.com
paceform.com pewit.pw
page2rss.com pinapchik.com
peekyou.com pinwallpaper.xyz
perso.wanadoo.es piratecams.com
petrovka-online.com pix24x7.com
piccshare.com play-movie.pl
pizza-tycoon.com plugingeorgia.com
pomoc-drogowa.cba.pl pooleroadmedicalcentre.co.uk
popunder.ru porno-play.net
popup.matchmaker.com poshiv-chehol.ru
pornofeuer.com preventheadacheguide.info
pornofiljmi.com privhosting.com
pornokorol.com proftests.net
pornonik.com promalp-universal.ru
pornotubs.com prosperent.com
pravoholding.ru purchasepillsnorx.com
privetsochi.ru puzo2arbuza.ru
profolan.pl queerspace.com
prostitutki-novgoroda.org rcb101.ru
pukaporn.com reckonstat.info
raisedseo.com renecaovilla.online
razvratnoe.org reyel1985.webnode.fr
recinziireale.com rff-cfal.info
redirectme.net rock-cafe.info
remont-mobile-phones.ru rubbed.us
robotixix.com ruclicks.com
rogervivierforsale.com sankt-peterburg.nodup.ru
ru-dety.ru scanner-fred.top
satoristudio.net scanner-ivan.top
sax-sex.com screentoolkit.com
sbricur.com scripted.com
scrapinghub.com sex-pr.net
search-error.com
sexkontakte-seite.com sexkontakte-seite.com
sexkvartal.com share-buttons-for-free.com
sexxdate.net sharebutton.net
sexyebonyteen.com shinikiev.com.ua
sibvitr.ru simon3.ru
sitechecker.pro simply.net
sitiz.club sinestesia.host.sk
slujbauborki.ru softxaker.ru
smartpet.ru spywareit.com
socialtrade.biz
soheavyblog.com
sr-rekneskap.no sr-rekneskap.no
suggest-keywords.com steamoff.net
superkanpo.com store-rx.com
telemetryverification.net svarbit.com
the-torrent-tracker.blogspot.com taihouse.ru
thewebsitetemplate.info tasteidea.com
tizanidinehcl4mgisitanarcotic.blogspot.com tedxrj.com
topcar-krasnodar.ru tizanidinecapsules.blogspot.com
traffic2money.com tobeyouday.win
trafficfactory.biz torture.ml
tri-slona.org tozup.com
turizm.bz track112.site
twelvevisionspartyofcolorado.com trafficnetzwerk.de
twitlinks.com truemfilelj.gq
tytoona.com udsgame.online
ugguk.online unpredictable.ga
ultramart.biz uptime-alpha.net
unblocksit.es upupa.net
uogonline.com vavilone.com
upstore.me vip.51.la
versaut.xxx-cam.webcam voloslove.ru
videochat.cafe vsdshnik.com
videotuber.ru vsexkontakte.net
viven.host.sk vtc.pw
vkak.ru vucms.com
vzubah.com warningwar.ru
waysbetter.cn webs.com
webix.biz wemarketing.se
weblo.com wemedinc.com
website-datenbank.de wholesalejerseychinaoutlet.com
weburlopener.com wleuaprpxuvr.ga
weburok.com world-mmo.com
wesharepics.site x-diesel.biz
wildnatureimages.com xfire.com
wineration.com xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
winwotgold.pl xn--80abgj3a5acid6ghs.top
worldlovers.ru zaidia.xhost.ro
x-lime.com zaimite.ru
xboxster.ru zapiszto.pl
xjrul.com zero1.it
xkaz.org zetgie.com.pl
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xn--24-glceagatoq7c2a6ioc.xn--p1ai
xportvusbdriver8i.snack.ws
youtuhe.com
za-music.mymobiplanet.com
zarabiaj-dzis.pl
zastroyka.org
zdesformula.ru
zeroredirect6.com
zscaler.net
zygophyceous.womanstars.site

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.201 0; 10.1.0.11 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2874 ### Version: V4.2022.01.2875
### Updated: Mon Jan 24 10:03:20 UTC 2022 ### Updated: Mon Jan 24 22:01:46 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2874 ### Version: V4.2022.01.2875
### Updated: Mon Jan 24 10:03:20 UTC 2022 ### Updated: Mon Jan 24 22:01:46 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##