vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 03:29:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.09.3352 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-09-15 22:07:19 +00:00
parent 7ba4b92b2e
commit bedaea455c
46 changed files with 95896 additions and 95896 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.09.3351 #### Version: V4.2022.09.3352
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.09.3351 #### Version: V4.2022.09.3352
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.09.3351 #### Version: V4.2022.09.3352
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11854
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.09.3351 #### Version: V4.2022.09.3352
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11858
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11858
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11858
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11858
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

11858
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

11858
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

11858
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3351 3352

11858
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11858
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.122.164.223 0; 20.253.130.253 0;

11858
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

11858
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

11858
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

11858
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.122.164.223 0; 20.253.130.253 0;

11858
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.122.164.223 0; 20.253.130.253 0;

11858
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 1; 10.1.1.2 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BetaBot Abonti
Cloud mapping Domains Project
LWP::Simple EMail Siphon
MIIxpc Extractor
Sucker IlseBot
Telesphoreo Mojeek
V-BOT SputnikBot
Wprecon SurveyBot
instabid Upflow
page scorer lwp-trivial

288
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404checker
80legs
ADmantX
AIBOT AIBOT
Abonti ALittle Client
Aboundexbot Aboundex
Acunetix AhrefsBot
AiHitBot
Aipbot Aipbot
Alligator Alligator
Anarchie Anarchie
Anarchy99 Anarchy
Ankit
Apexoo Apexoo
Aspiegel
Asterias Asterias
Atomseobot Attach
BackStreet AwarioRssBot
Backlink-Ceck BackDoorBot
Badass Barkrowler
BatchFTP BatchFTP
Battleztar Bazinga
Bigfoot Bigfoot
Blow
Boardreader
Bolt
BotALot
Brandprotect Brandprotect
Bullseye BuiltWith
CODE87 CATExplorador
Calculon
Cegbfeieh Cegbfeieh
CensysInspect
CheTeam CheTeam
CheeseBot
CherryPicker CherryPicker
Chlooe
Citoid Citoid
Cloud mapping Claritybot
Collector Collector
Copier Copier
Copyscape CopyRightCheck
Cosmos Craftbot
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
CrunchBot Crescent
Curious Curious
DBLBot
DSearch
DTS Agent DTS Agent
DataCha0s Devil
Digincore
DigitalPebble
Discoverybot
Dispatch Dispatch
DittoSpyder
DnyzBot
DomCopBot
DomainCrawler DomainCrawler
Dotbot Domains Project
Download Wonder
Dragonfly Dragonfly
Drip
ECCP/1.0 ECCP/1.0
EMail Wolf EirGrabber
Ecxi
Evil Evil
Express WebPictures Express WebPictures
Extractor
Extreme Picture Finder Extreme Picture Finder
EyeNetIE
Ezooms Ezooms
FDM FDM
FemtosearchBot Flunky
Fimap
FlashGet
Freeuploader Freeuploader
Fuzz FrontPage
FyberSpider FyberSpider
GalaxyBot Fyrebot
Genieo
GermCrawler
GetWeb GetWeb
Gigabot
Go!Zilla Go!Zilla
Grafula GoZilla
Hloader GrabNet
Humanlinks GridBot
IDBot HMView
IRLbot HTTP::Lite
HTTrack
HaosouSpider
HonoluluBot
Iblog Iblog
IlseBot
Image Sucker Image Sucker
IndeedBot
Intelliseek Intelliseek
InterGET
IstellaBot
JOC Web Spider
JamesBOT
JennyBot
JetCar JetCar
Jetty
Joomla Joomla
Jorgee Keyword Density
JustView
Jyxobot
Kinza Kinza
Kozmosbot Larbin
LNSpiderguy LeechFTP
LeechGet LeechGet
LexiBot LexiBot
Libwhisker Libwhisker
Likse Lightspeedsystems
LinkScan LinkScan
LinkWalker LinksManager
LinqiaScrapeBot LinqiaMetadataDownloaderBot
Litemage_walker LinqiaRSSBot
Lipperhey
Lmspider Lmspider
Ltx71 Ltx71
MIDown tool MIDown tool
MIIxpc
MJ12bot MJ12bot
MQQBrowser MSFrontPage
MSIECrawler
MTRobot
Mag-Net Mag-Net
Magnet Magnet
Mail.RU_Bot
Majestic SEO
MarkMonitor MarkMonitor
MarkWatch MarkWatch
Mass Downloader
MauiBot
Mb2345Browser Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot Minefield
Mister PiX Mojeek
Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
NICErsPRO Msrabot
Name Intelligence NPbot
Nameprotect
Needle Needle
NetLyzer
NetSpider
NetZIP NetZIP
Nettrack
Netvibes
NextGenSearchBot
Nibbler
Nikto Nikto
Ninja NimbleCrawler
Not Nmap
Nutch
Octopus
Offline Navigator Offline Navigator
OnCrawl OpenVAS
OpenLinkProfiler Openfind
Openvas
OrangeBot OrangeBot
OutfoxBot
PECL::HTTP PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageGrabber Pandalytics
PageScorer Petalbot
PeoplePal Piepmatz
PictureFinder
Pimonster Pimonster
Pixray
PleaseCrawl PleaseCrawl
Pockey
ProPowerBot ProPowerBot
PyCurl Probethenet
QueryN Metasearch Pu_iN
RankActiveLinkBot Pump
RankingBot2 RSSingBot
RealDownload Re-re
RebelMouse
Recorder Recorder
RedesScrapy
RepoMonkey
Ripper Ripper
SEOkicks SBIder
SEOlyticsCrawler SMTBot
SEOstats
Scrapy
ScrepyBot
SearchmetricsBot
Seekport
Semrush Semrush
SentiBot SentiBot
SeoSiteCheckup
SeobilityBot
Shodan
Siphon Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
Snake SiteExplorer
Sogou web spider SiteLockSpider
SpaceBison Sitevigil
Snapbot
Sociscraper
Sottopop
Spammen Spammen
Spbot Spanner
SputnikBot SputnikBot
Sqlmap
Sqworm Sqworm
Steeler Sucuri
SuperHTTP SuperBot
SurveyBot
Swiftbot Swiftbot
T8Abot
Teleport Teleport
TeleportPro TeleportPro
Telesoft
Telesphoreo
Telesphorep
TheNomad TheNomad
Thumbor Toweyabot
TightTwatBot
Titan
Trendiction Trendiction
True_Robot Turnitin
Twice TwengaBot
Typhoeus Typhoeus
URLy Warning URLy.Warning
UnisterBot Upflow
VB Project VB Project
VCI
Vacuum Vacuum
VelenPublicWebCrawler
VeriCiteCrawler
VoidEYE
Voil
Voltron Voltron
WPScan WASALive-Bot
WWW-Mechanize WWW-Collector-E
Wallpapers Wallpapers
Wallpapers/3.0 WallpapersHD
WeSEE Web Collage
Web Auto Web Enhancer
Web Fetch
Web Pix
WebCollage WebCollage
WebFetch WebFetch
WebFuck WebFuck
Web Pix WebGo IS
Web Sucker WebImageCollector
WebCollage
WebFetch
WebLeacher WebLeacher
WebPix
WebReaper WebReaper
WebSucker WebSauger
WebWhacker Website Quester
Webshag
WebsiteExtractor WebsiteExtractor
Webster
Whack Whack
Whacker
Whatweb Whatweb
Widow Widow
WinHTTrack
Wotbox Wotbox
Wprecon
Xaldon WebSpider Xaldon WebSpider
YoudaoBot Xaldon_WebSpider
ZmEu Zauba
Zitebot
arquivo-web-crawler
arquivo.pt
cah.io.community cah.io.community
crawl.sogou.com check1.exe
clark-crawler
crawler.feedback crawler.feedback
demandbase-bot crawler4j
domainsproject.org eCatch
heritrix internetVista monitor
iubenda-radar iubenda-radar
linkdexbot linkdexbot
lwp-request
lwp-trivial lwp-trivial
magpie-crawler
mediawords mediawords
netEstate NE Crawler muhstik-scan
oBot probe-image-size
ripz
satoristudio.net satoristudio.net
scalaj-http scalaj-http
scan.lol scan.lol
seobility sexsearcher
sitechecker.pro sitechecker.pro
spyfu sogouspider
sysscan
trendiction.com trendiction.com
webgains-bot trendiction.de
webmeup-crawler ubermetrics-technologies.com
webpros.com webprosbot
x09Mozilla x22Mozilla
zgrab xpymep1.exe

288
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider
404checker 404checker
80legs 80legs
ADmantX AIBOT
Abonti ALittle Client
AhrefsBot ASPSeek
Acunetix
AiHitBot AiHitBot
Aipbot Alligator
Aspiegel Anarchie
AwarioRssBot Anthill
BBBike Atomseobot
BDFetch
BLEXBot
BackDoorBot
BackStreet
Backlink-Ceck Backlink-Ceck
Badass BacklinkCrawler
Bandit
Barkrowler
BatchFTP BatchFTP
Bitacle Bitacle
BlackWidow
Blackboard
Bolt
BotALot
Brandprotect Brandprotect
Brandwatch
Buck Buck
Buddy
BuiltBotTough BuiltBotTough
BuzzSumo BuiltWith
CCBot BunnySlippers
CODE87 CODE87
Calculon CazoodleBot
CensysInspect CensysInspect
CheTeam
CheeseBot CheeseBot
CherryPicker CherryPicker
Chlooe Cliqzbot
Claritybot Cogentbot
Cocolyzebot
Cosmos
Craftbot
CrazyWebCrawler
CrunchBot CrunchBot
DBLBot Custo
DSearch DIIbot
DTS Agent DTS Agent
DatabaseDriverMysqli DataCha0s
DigitalPebble Devil
Digincore
Disco Disco
DittoSpyder
DnBCrawler-Analytics
DnyzBot DnyzBot
DomCopBot DomCopBot
DomainAppender
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot Dragonfly
Domains Project Drip
Download Wonder
Ebingbong Ebingbong
Ecxi
EirGrabber EirGrabber
EroCrawler Evil
Exabot
ExtLinksBot
Extractor Extractor
FHscan Extreme Picture Finder
Ezooms
Fimap Fimap
Firefox/7.0 Flunky
Foobot
Freeuploader
FrontPage
Fuzz Fuzz
Fyrebot Fyrebot
G-i-g-a-b-o-t
GT::WWW
GalaxyBot
Genieo
GetRight
Gigabot Gigabot
GoZilla Go!Zilla
GrabNet Gotit
Grabber Grabber
GridBot
HEADMasterSEO
HTMLparser HTMLparser
Haansoft Haansoft
HaosouSpider Havij
Hloader Heritrix
HonoluluBot
Humanlinks Humanlinks
IRLbot
Iblog Iblog
Internet Ninja Indy Library
InfoNaviRobot
InfoTekies
Intelliseek
InternetSeer InternetSeer
Iskanie Keyword Density
Jbrofuzz LNSpiderguy
JennyBot LWP::Simple
JikeSpider Lanshanbot
Jorgee
Keybot Translation-Search-Machine
Kinza
Larbin Larbin
Leap
LeechFTP LeechFTP
LeechGet
Lftp Lftp
Libwhisker LibWeb
LieBaoFast LieBaoFast
Likse Lightspeedsystems
LinkScan LinkScan
LinkextractorPro LinkWalker
LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
Litemage_walker Litemage_walker
Ltx71 MIDown tool
MIIxpc
MSIECrawler
Magnet Magnet
Majestic SEO Mata Hari
Majestic-SEO
MarkWatch
MauiBot
Mb2345Browser Mb2345Browser
MeanPath Bot
Mediatoolkitbot Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri Metauri
Microsoft Data Access
Microsoft URL Control
Minefield Minefield
Mojeek Moblie Safari
MolokaiBot MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila
Mr.4x3 Mr.4x3
NICErsPRO
Name Intelligence Name Intelligence
NearSite NearSite
Needle
Nessus Nessus
NetLyzer Net Vampire
NetSpider NetMechanic
Nettrack NextGenSearchBot
Netvibes
Nibbler
Niki-bot
Nimbostratus
Nmap Nmap
Not Not
Nutch Nuclei
Offline Explorer Offline Navigator
OnCrawl
OpenLinkProfiler
OpenVAS OpenVAS
Openfind Openfind
Openvas
OrangeBot OrangeBot
OutfoxBot OrangeSpider
POE-Component-Client-HTTP PHPCrawl
Page Analyzer Page Analyzer
PageGrabber
PageThing.com
Papa Foto Papa Foto
Pockey PeoplePal
Pu_iN Pixray
Quick-Crawler ProWebWalker
RSSingBot Pump
RankingBot2 PxBroker
RankurBot RankingBot
Ripper ReGet
RebelMouse
RepoMonkey
RocketCrawler
Rogerbot Rogerbot
SBIder SEOstats
SEOkicks
SEOprofiler
SISTRIX SISTRIX
SMTBot SMTBot
SalesIntelligent SalesIntelligent
ScanAlert ScanAlert
ScoutJet ScoutJet
ScrepyBot ScrepyBot
Searchestate Seekport
SearchmetricsBot
SemanticJuice
SentiBot SentiBot
SeoSiteCheckup
SeobilityBot SeobilityBot
Shodan Shodan
Siphon Siphon
Site Sucker SiteSnagger
SiteCheckerBotCrawler
Sitebeam Sitebeam
SlySearch SlySearch
SmartDownload SmartDownload
Snake
Snapbot Snapbot
Snoopy Snoopy
SocialRankIOBot
Sosospider Sosospider
SpaceBison Sottopop
Spbot Spammen
SpankBot
Spinn3r
Sqlmap
Steeler
Sucker
SuperBot
SuperHTTP SuperHTTP
Suzuran Surfbot
SurveyBot
Swiftbot Swiftbot
Telesphoreo T0PHackTeam
Telesphorep TeleportPro
Titan TheNomad
Toata Toata
Toweyabot Trendiction
Tracemyfile
Trendictionbot
Turingos Turingos
Turnitin Turnitin
Twice TwengaBot
URLy.Warning VeriCiteCrawler
VB Project VidibleScraper
VCI Voil
Vacuum Voltron
Vagabondo WBSearchBot
VoidEYE WISENutbot
WASALive-Bot WWW-Mechanize
WEBDAV
WWWOFFLE WWWOFFLE
WallpapersHD WallpapersHD
WeSEE
Web Enhancer
Web Fuck Web Fuck
Web Pix
Web Sauger
WebAuto
WebEnhancer WebEnhancer
WebFetch WebFetch
WebImageCollector WebImageCollector
WebPix WebLeacher
WebReaper
WebSauger
WebStripper WebStripper
WebmasterWorldForumBot WebSucker
WebZIP
Webshag Webshag
Website Quester Website Quester
Whack
Whatweb
Widow
Wonderbot
Woobot Woobot
Xaldon_WebSpider Wotbox
Xenu Xenu
YoudaoBot
Zade Zade
Zauba
Zitebot
ZmEu
ZoomBot
ZoominfoBot ZoominfoBot
adscanner arquivo-web-crawler
archive.org_bot arquivo.pt
autoemailspider autoemailspider
backlink-check check1.exe
cah.io.community coccocbot
clark-crawler com.plumanalytics
cognitiveseo
crawl.sogou.com
crawler4j crawler4j
dataforseo.com
demandbase-bot
eCatch eCatch
evc-batch
facebookscraper facebookscraper
heritrix gopher
instabid
internetVista monitor internetVista monitor
iubenda-radar
linkdexbot linkdexbot
lwp-request lwp-request
lwp-trivial meanpathbot
muhstik-scan pcBrowser
oBot satoristudio.net
polaris version
ripz
scalaj-http scalaj-http
seostar seobility
seoscanners
serpstatbot serpstatbot
sitechecker.pro
sysscan
trendiction.com
trendiction.de
ubermetrics-technologies.com
webgains-bot webgains-bot
webmeup-crawler webmeup-crawler
x22Mozilla x22Mozilla
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
adclickthru.net 8lufu.com
apiwoodensealcom-a.akamaihd.net aa14ab57a3339c4064bd9ae6fad7495b5f.com
apps-analytics.net gymi.name
artdeko.info historichometeam.com
e-commerce-seo.com paphoselectricianandplumber.com
fxund.us scanner-brian.top
gotwebsite1.com scoopquest.com
muzaporn.com stairliftsarea.com
sharebutton.org tytoona.com
xlovecam.com vtc.pw

928
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
101billion.com 1-99seo.com
1314dh.com 101lesbian.xyz
18ps.ru 10bet.com
1webmaster.ml 1xbet4you.com
2015god.org 23kw.ru
24x7-server-support.site 2drittel.de
2ads.co.uk 2rich4bitches.com
2x2fan.ru 34.gs
3th.co.in
45en.ru 45en.ru
4kepics.com 4webmasters.org
4ur.click 51.la
888.com 7fon.ru
aa14ab57a3339c4064bd9ae6fad7495b5f.com 7search.com
aa8b68101d388c446389283820863176e7.com 7zap.com
aab94f698f36684c5a852a2ef272e031bb.com aa08daf7e13b6345e09e92f771507fa5f4.com
aad1f4acb0a373420d9b0c4202d38d94fa.com aa625d84f1587749c1ab011d6f269f7d64.com
aa8780bb28a1de4eb5bff33c28a218a930.com
aasoldes.fr
abbanreddy.com
acads.net
acortarurl.es acortarurl.es
actionnooz.com actulite.com
addray.pro adnotbad.com
ads.tremorhub.com ads.tremorhub.com
adultfullhd.com adultwebhosting.info
afora.ru aero2.ru
agahinameh.com
ahmedabadwebs.com ahmedabadwebs.com
akama.com aktivator-windows10.blogspot.com
allboard.xobor.de akusherok.ru
allforminecraft.ru alldezservice.kz
allknow.info aloofly.com
alot.com alphacarolinas.com
alpharma.net amateurlivechat.org
amateurmatch.com amyfoxfitness.com
anaksma.info ananumous.ru
anapa-inns.ru
anaseracresar.tk anaseracresar.tk
anatomiy.com
angel1777.kz angel1777.kz
angigreene.com
apartment.ru apartment.ru
apioasisspacenet-a.akamaihd.net apper.de
apispringsmartne-a.akamaihd.net appiq.mobi
appfixing.space arendakvartir.kz
approvedlocal.co.za artdeko.info
arcadeplayhouse.com articlesdirectoryme.info
arendovalka.xyz autolombard-krasnodar.ru
artparquet.ru autovideobroadcast.com
asrvvv-a.akamaihd.net aviav.co
asseenontvonline.ru avtoarenda.by
aussie-prizes.com avtolombard-krasnodar.com
avirasecureserver.com azadnegar.com
azartniy-bonus.com
b2b-lounge.com
bala.getenjoyment.net
balans.shahterworld.org balans.shahterworld.org
balook.com balla-boo.se
banan.tv banksrf.ru
bannerconnect.net bard-real.com.ua
beget.tech bausparen.bz.it
benchmarkcommunications.co.uk bavsac.com
bestempresas.es beatifullandscape.co
bestbrainprod.win
besthomemadepornsites.com besthomemadepornsites.com
bestofferswalkmydogouteveryday.gq bestmarriages.com
bestpriceninja.com bestmouthwash.club
besttorrentknifta.weebly.com bestserials.com
betterscooter.com bezlimitko.xyz
bfz.biz bildsuche.ru
bget.ru bio-japan.net
bidr.trellian.com bitcoin-ua.top
bimatoprost-careprost.com bitcoins-live.ru
bio-market.kz bizlist.com.de
bioca.org blackplanet.com
bitcoinremote.com blog.f00kclan.de
bkgr.se blogrankers.com
bkns.vn bmw-ark.ru
blockworld.ru bobba.dzaba.com
bloke.com boleznikogi.com
boazpower.com boobsimge.com
bobinoz.com brakehawk.com
bryansk.zrus.org bristolhotel.com.ua
brothers-smaller.ru
budilneg.xyz budilneg.xyz
buketeg.xyz builtwith.com
buntube.net bukleteg.xyz
burger-tycoon.com burger-imperia.com
bus-offer.com buy-loft.ru
buttons-for-your-website.com bytimedance.ru
call-of-duty.info c-english.ru
callcafe.info camdolls.com
canacopegdl.com canzoni.ru
candlewooddentalcentre.com.au cardul.ru
candypeople.se cartierreplica.pw
capacitacionyaprendizaje.com cartierreplica.top
carapics.com cascadelink.org
carina-sy.de casinos4dummies.co.uk
carmuffler.net cbox.ws
carson.getenjoyment.net cbozhe.com
cartierlove2u.com cenoval.ru
cartierlove2u.xyz ceotrk.com
cartierring.xyz cercacamion.it
casablancamanor.co.za
cayado.snn.gr
cdnnetwok.xyz
celejihad.info
cete.ru
chatroulette.world
cheapjerseysa.com
checkm8.com
choosecuisine.com choosecuisine.com
circlesl.com cialis-samples.com
clash-clans.ru cienum.fr
clickprotects.com cjmc.info
club-lukojl.ru clickcash.com
cocolyze.com clickso.com
coderstate.com clicksor.com
coindirect.io clipartnew.com
com-onlinesupport.site cmd.kz
com-secure.download cognitiveseo.com
community.allhiphop.com collegeessay19.blogspot.ru
compliance-julia.top colobit.biz
comsysnet.com com-cleaner.systems
confib.ifmo.ru com-onlinesupport.host
cookie-law-enforcement-ee.xyz compliance-alex.xyz
compliance-barak.xyz
compliance-checker-7.info
compliance-fred.xyz
compliance-nelson.top
conboy.us
cookie-law-enforcement-ff.xyz cookie-law-enforcement-ff.xyz
cosmeticswomens-womensports.rhcloud.com coolbar.pro
countercrazy.com cornomase.win
cowlmash.com cougfan.info
cpajunkies.com
crackzplanet.com
credit-online.ws
customboxes4less.com
customergrowthsystems.com customergrowthsystems.com
czeshop.info cypernhuset.se
daptravel.com dalavia.ru
darmebel.com.ua dame-ns.kz
datadepths.com daretodonate.co
dataloading.net darkbooks.org
dating-time-now.com day-news.info
dealighted.com db.speedup.gdn
demian.kz diesel-parts28.ru
detalizaciya-tut.biz dipstar.org
dickssportinggoods.com div.as
digilander.libero.it dmmspy.com
dms-sw.ru
dneprsvet.com.ua
docs4all.com docs4all.com
domain-tracker.com docsportal.net
domainsatcost.com doctormakarova.ru
dominterior.org dollartree.info
draniki.org domain2008.com
drhomes.biz dosugrostov.site
downloader12.ru
downtuptv.gq
duawitchrarato.tk duawitchrarato.tk
e-commerce-seo1.com dumpsmania24.com
ebonyporn.site e-biznes.info
ebooknovel.club easyukraine.com
ekspertmed.com ed-shop01.ru
ednorxmed.com
eksprognoz.ru
elektrischezigaretten1.blogse.nl
elektrozigaretten1.postbit.com elektrozigaretten1.postbit.com
empathica.com elvenmachine.com
empis.magix.net embedle.com
englate.com enhand.se
eropho.net erectile.bid
esnm.ru eropho.com
essay-edu.biz erotag.com
estibot.com espaceinventoristes.com
exct.net eu-cookie-law.blogspot.com
extlabs.io eupornstar.info
ezigarettekaufen2.yolasite.com exlarseva.webblog.es
ezigaretteshop.webs.com face.hostingx.eu
facecup.top family1st.ca
familyphysician.ru familyphysician.ru
famix.xyz fastfixing.tech
fanrto.com fearcrow.com
fapgon.com fetishinside.com
femmesdenudees.com ff30236ddef1465f88547e760973d70a.com
ferrotodo.com figuringmoneyout.com
fertilitetsradgivningen.se finteks.ru
fetlifeblog.com flagcounter.me
fighrofacciufreesig.ga flirt4free.com
filmfanatic.com foodcrafts.website
financepoints.eu forensicpsychiatry.ru
findsexguide.com forex-procto.ru
folowsite.com foto-weinberger.at
fortunejack.com fotzen-ficken.com
forum.tvmir.org foxjuegos.com
forum20.smailik.org foxydeal.com
free-social-buttons.com free-deals.faith
freemaintenancesysforpcandmac.top free-floating-buttons.com
freewlan.info freesitetest.com
funtoonez.com fuck-paid-share-buttons.xyz
fxgallery.com funnel.co.za
fx-brokers-review.com
gabeshop.ru gabeshop.ru
gamblingnerd.com galaxys6manual.info
gamedayassist.com gazobeton-p.com.ua
genericlowlatencyasiodriverhq.aircus.com gentamicineyedrops.blogspot.com
gepezz.info germes-trans.com
ggiaro.com get-free-traffic-now.com
gilsonchiro.xyz ghostvisitor.com
girlsatgames.ru glasof.es
glopages.ru gnuetella.com
googglet.com go2mike.ru
goodwriterssales.com
googlsucks.com googlsucks.com
gosreg.amchs.ru greenshop.su
gotorussia.com growmyfunds.ca
gowreckdiving.com gsasearchenginerankerdiscount.com
gwhwpxbw.bloger.index.hr gsbs.com.ua
hdapp1008-a.akamaihd.net hackers-crackers.tk
hdpixion.com hamptonoaks.ca
heatpower.ru hdhc.ru
heliko.no hdwalls.xyz
herehloadibs.cf headpress.ru
hetmanship.xyz helvetia.com.ua
himazin.info hexpilot.com
histock.info historichometeam.com
hiwibyh.bugs3.com hmywwogw.bloger.index.hr
hobbyhuren24.net housekuba.org
homedecorpicture.us
homeinns.com
horoshieokna.com
host-tracker.com
hostcritique.com
hosting-tracker.com
hotenergy.ru
house.sieraddns.com
hoztorg-opt.ru hoztorg-opt.ru
hptwaakw.blog.fc2.com humanorightswatch.org
htmlcorner.com hustoon.over-blog.com
humanelydrew.com hvd-store.com
husky-shop.cz i-service.kz
ibb.com.ua idealtits.net
immigrational.info ideashome.id
import-sales.com ido3.com
importchinacoach-teach.com iflycapetown.co.za
incolors.club igadgetsworld.com
indiasourcemart.in igrovyeavtomaty777.ru
infektsii.com ihc2015.info
infogame.name imgarcade.com
internetproviderstucson.com infospot.pt
investpamm.ru intelhdgraphicsgtdrive6w.metroblog.com
iomoio.net intellego.info
isotoner.com investzalog.ru
istizanidineanarcoticdrug.blogspot.com invivo.hu
itis4you.com ipornox.xxx
itservicesthatworkforyou.com iqoption-bin.com
izoll.ru iqoption.pro
javcoast.com irunfar.com
jbl-charge.info islamtoday.co.za
jerseywholesalechinabiz.com izhevsk.xrus.org
jofucipiku.tk j33x.com
journalhome.com jackpotchances.com
jovencitas.gratis jasonpartington.com
jpcycles.com javidol.site
jrpmakati.com javitas.info
justdating.online jennyfire.ru
kabbalah-red-bracelets.com jerseyschinabizwholesale.us
kalandranis.gr jetsli.de
katushka.net jjbabskoe.ru
keyword-suggestions.com job.icivil.ir
kinoduh.ru johannesburgsingles.co.za
kinoplen.ru jwss.cc
kinosed.net kamen-e.ru
kinozapas.org keenoutlet.online
kirov.zrus.org kerwinandcariza.com
klin3952.ru keywordspay.com
komputernaya-pomosh-moscow.ru keywordsuggests.com
kreativperlen.ch knigonosha.net
kriokomora.info knogg.net
kursy-ege.ru koopilka.com
labvis.host.sk koral.se
lasvegaslockandsafe.com kostenlos-sexvideos.com
lazymanyoga.com kunstaktien.info
leftofcentrist.com kvartiry-remont.ucoz.ru
lightinghomes.net laapp.com
limtu.ifmo.ru ladsup.com
littleberry.ru lanasshop.ru
lonely-mature.com landscapebackgrounds.blogspot.com
lsp-awak-perikanan.com lapitec.eu
lulea-auktionsverk.se letmacwork.world
mackeeper-land-672695126.us-east-1.elb.amazonaws.com letsart.ru
mafcards.ru lifehacĸer.com
make-money-online.com lifeinsurancekb.com
masterseek.com liky.co.ua
matb3aa.com liupis.com
mathgym.com.au liverpool.gsofootball.com
matsdale.com livingcanarias.com
megagrabber.ru lkbennettstore.online
metabar.ru lmrauction.com
mezaruk.info loadingpages.me
middlerush-a.akamaihd.net low-format.ru
minegam.com lowephotos.info
mini-modus.ru lyrics.home-task.com
missvietnam.org m5home.ru
machicon-akihabara.info
mactechinfo.info
magazin-pics.ru
magnetic-bracelets.ru
makedo.ru
meddesk.ru
medicovi.com
medkritika.ru
medtherapy.ru
meetlocalchicks.com
members.ghanaweb.com
militarysale.pro
minharevisao.com
mixx.com
mobifunapp.weebly.com
mobplayer.net
mockupui.com
mojpregled.com mojpregled.com
moscow.xrus.org moonci.ru
most-kerch.org morlat.altervista.org
mtmtv.info morocco-nomad-excursions.com
muizre.ru moscow-region.ru
mulberryoutletonlineeu.com most.gov.iq
mustwineblog.com moz.com
muz-baza.net mp3ringtone.info
muz-tracker.net msk.afora.ru
mxgetcode.com musicspire.online
mydirtyhobby.com muznachas-service.ru
mydownlodablefiles.com mylesosibirsk.ru
mylida.org
myliveblog.ru
myxdate.info
mzdish.site mzdish.site
najaden.no narco24.me
narkologiya-peterburg.ru natali-forex.com
negociosdasha.com naturalshair.site
news-readers.ru naughtyconnect.com
newsperuse.com negral.pluto.ro
new-apps.ru
nextbackgroundcheck.gq nextbackgroundcheck.gq
nikitsyringedrivelg.pen.io nfvsz.com
nikkiewart.ru
nofreezingmac.work nofreezingmac.work
ntic.fr norththeface.store
nufaq.com nudo.ca
oecnhs.info nuker.com
offer.wpsecurity.website oakridgemo.com
omsk.xrus.org odywpjtw.bloger.index.hr
onko-24.com okout.ru
ooomeru.ru olympescort.com
open-odyssey.org openfrost.net
openstat.com oranga.host.sk
ordernorxx.com
orion-v.com
osagonline.ru osagonline.ru
ourville.info ostroike.org
pacificair.com ourtherapy.ru
paidonlinesites.com palocco.it
paleohub.info par-fallen.ga
pandarastore.top partner-cdn.men
parajumpersstore.online partners-ship.pro
partner-pop.men partybunny.ru
partner-print.men pavlodar.xkaz.org
pc4download.co percin.biz.ly
perso.wanadoo.es phuketscreen.com
photokitchendesign.com pixshark.com
pic2fly.com pizdopletka.club
picurams.pw playfortuna-play.ru
pihl.se playmsn.com
pijoto.net popup-fdm.xyz
pinup-casino1.ru pornogratisdiario.com
piski.top preparevideosafesystem4unow.site
pizda.lol profit-opportunity.com
pl.id-forex.com promover.org
play-movie.pl prospekt-st.ru
pod-muzyku.club proxyradar.com
poisk-zakona.ru puzo2arbuza.ru
polybuild.ru pxhdwsm.com
porndairy.in
pornoblood.com
pornohd1080.online
pornoxxx.com.mx
porto.abuilder.net
powitania.pl
predominant-invent.tk
progress-upakovka.ru
prointer.net.ua
projefrio.com.br
proxyelite.biz
pssucai.info
qld10000.net qld10000.net
quit-smoking.ga quickchange.cc
quizzitch.net qwesa.ru
rabotaetvse.ru radiogambling.com
radiodigital.co realtytimes.com
rangjued.com redirlock.com
rankingchart.de reelheroes.net
remorcicomerciale.ro reimageplus.com
retreatia.com remedyotc.com
rollercoin.com resant.ru
rome2rio.com rewardit.com
ros-ctm.ru ringtonepartner.com
saletool.ru rniaeba.ga
sbricur.com royal-betting.net
scanner-donald.top rukino.org
scanner-irvin.top rus-teh.narod.ru
scenicmissouri.us rusenvironmental.net
screentoolkit.com saecsa.co
searchencrypt.com sanidumps.com
searchwebknow-a.akamaihd.net sashagreyblog.ga
seccioncontrabajo.com sbetodiodnye-lampy.ru
senger.atspace.co.uk sbornik-zakonov.ru
scanner-fred.top
scansafe.net
scrapy.org
sdelai-prosto.ru
searchlock.com
seasaltwithfood.com
seinterface.com
seo-2-0.com seo-2-0.com
seo-smm.kz seoheap.com
seocheki.net seorankinglinks.xyz
servingnotice.com service-core.ru
serw.clicksor.com sex-dating.co
sevendays.com.ua sexyali.com
sex-tracker.de shlyahten.ru
sexblog.pw
sexpornotales.com
shakhtar-doneck.ru
shillyourcoins.com
shopwme.ru shopwme.ru
sibdevice.ru shtora66.ru
siha.de sientalyric.co
silvercash.com sildenafilcitratemed.com
sims-sims.ru sims-sims.ru
sinestesia.host.sk
siteonomy.com
sitevaluation.com sitevaluation.com
sizeplus.work skutecznetabletkinaporostwlosow.pl
sladkoevideo.com slavkokacunko.de
smart-balancewheel.com
smartshoppymac.com smartshoppymac.com
sofit-dmd.ru smsactivator.ru
sogimlecal.tk soc-econom-problems.ru
sohoindia.net social-vestnik.ru
solmarket.by soft-program.com
space4update.pw solnplast.ru
sport7777.net soochi.co
spywarenuker.com srdrvp.com
srecorder.com srgwebmail.nl
stairliftsarea.com stairliftstrue.com
staynplay.net steame.ru
stroicol.net stop-gepatit.te.ua
stpolice.com
stroilka.info stroilka.info
stuff-about-money.com sundrugstore.com
styro.ru supermama.top
supernew.org surfbuyermac.com
surgut.zrus.org surflinksmedical.com
suture.co tandvardshuset.net
swimpool.ca taqplayer.info
swinger-mobil.net tedy.su
taraz.xkaz.org the-usa-games.blogspot.com
taximytishi.ru thecoolimages.net
tds-advert002.info thexart.club
teenforporn.com threecolumnblogger.com
telegramdownload10.com ticketsys.inetwd.com
terraclicks.com tivolibasket.it
theautoprofit.ml tizanidine4mgstreetprice.blogspot.com
thebestweightlosspills.ovh tizanidineandcipro.blogspot.com
theendivechronicles.com tizanidinehydrochloride2mg.blogspot.com
thegioixekhach.com tnaionline.org
themestotal.com topcar-krasnodar.ru
thepornsex.org topmira.com
theprofitsmaker.net topvidos.ru
thin.me.pn track112.site
time-japan.ru traffic-club.info
tizanidineduringpregnancy.blogspot.com treasuretrack-a.akamaihd.net
tizanidinehcl2mgsideeffects.blogspot.com trubywriting.com
tizanidinevsvalium.blogspot.com tvoystartup.ru
torrent-to-magnet.com u-cheats.ru
torrents.life uchebavchehii.ru
tradedeals.biz uchil.net
trailer.cinemaflix.website
truck-rece.life
truemfilelj.gq
try-rx.com
tubeline.biz
tupper-posuda.ru
twelvevisionspartyofcolorado.com
uchetunet.su
udayavani.com udayavani.com
ufa.xrus.org ukrtvory.in.ua
ukrtextbook.com universals.com.ua
unacittaconte.org unmaroll.ya.ru
unece.org upstore.me
unimodemhalfduplefw.pen.io usswrite.com
ussearche.cf utrolive.ru
vbtracker.net vapomnoncri.tk
vektorpress.ru vbikse.com
vellings.info vchulkah.net
ventelnos.com ventelnos.com
vertaform.com verymes.xyz
video-camer.com via-energy-cumpara.com
videojam.tv videogamesecrets.com
videositename.com videosbox.ru
vids18.site vinylvault.co.uk
vip-file.com voloo.ru
vipcallsgirls.com volume-pills.biz
virtuagirl.com vostoktrade.info
vkontarkte.com vpnmouse.com
vmnmvzsmn.over-blog.com vzglyadriv.kg
voditeltrezviy.ru wahicbefa31.soup.io
vzube.com walkme.com
watch-movies.ru watchmygf.net
wcb.su waterefficiency.co
waycash.net
web-betting.ru
webenlace.com.ar webenlace.com.ar
webextract.profound.net webinstantservice.com
webpromotion.ae webnode.me
website-speed-up.site websiteaccountant.de
websitevaluebot.com wedding-salon.net
weburok.com wheelchairliftsarea.com
weightatraining.com woman-orgasm.ru
whatsappdownload10.com wordkeyhelper.com
wholesalecheapjerseysfree.com wrona.it
windowssearch-exp.com www.timer4web.com
wma-x.com x-diesel.biz
wyniki-lista.pl xclicks.net
x-musics.com xn----btbdvdh4aafrfciljm6k.xn--p1ai
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai xn--d1aifoe0a9a.top
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai xn--h1aakne2ba.xn--p1ai
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai xolodremont.ru
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai xxx-treker.ru
xn--90acenikpebbdd4f6d.xn--p1ai xxxtubesafari.com
xportvusbdriver8i.snack.ws yoshkarola.zrus.org
xsion.net yougame.biz
xzlive.com yourporngay.com
yachts-cruise.info yousense.info
yaoguangdj.com yubikk.info
yellowstonevisitortours.com
youandcredit.ru
yun56.co yun56.co
znaturaloriginal.com yur-p.ru
zoogdiesney.com zajm-zalog-krasnodar.ru
zophim.me zazagames.org
zolotoy-lis.ru
zrus.org
zygophyceous.womanstars.site

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
007angels.com 007angels.com
03e.info 1millionusd.xyz
0lovespells0.blogspot.com 2020iscoming.info
178evakuator178.ru admeasures.com
1j7740kd.website adssafeprotected.com
214jaluwobapef.cf adtech.de
40cg.com adultgalls.com
4rent.ru adverster.com
5-steps-to-start-business.com agardomains.com
7minuteworkout.com
8kisses.eu
actulite.com
adinterax.com
adloads.net
adultfriendfinder.com
ageofclones.com
agysya.ru agysya.ru
allforminecraft.ru alarmobninsk.ru
amateurlivechat.org albumsuper.info
ampower.me alfa9.com
aproposde.com allwidewallpapers.com
arcadeplayhouse.com alyeskaresort.com
asseenontv.ru animaltoplist.com
astana.xxxkaz.org anonymizeme.pro
availit.weebly.com apiluckyleapnet-a.akamaihd.net
banki76.ru apioasisspacenet-a.akamaihd.net
apiqualitinknet-a.akamaihd.net
appearance-cool.com
arenanews.com.ua
asmxsatadriverin.aircus.com
atmagroup.ru
aviav.eu
aviva-limoux.com
ayakino.net
azinoofficial777.ru
b2b-lounge.com
babieca.com
bagcionderlab.com
barbourjackets.online
bbtec.net
bedcapdealers.com
beetpics.pw beetpics.pw
berlininsl.com bestcoin.cc
betune.onlinewebshop.net bird1.ru
bitcoinremote.com bjgugu.net.ua
biznesluxe.ru blog.f00kclan.de
blacktwhite.com bolegapakistan.com
blagovest-med.ru brandbucket.com
blogig.org brandov.ru
boeuklad.com brianhenry.co
braip.com.br bristolhostel.com
buyhoverboard.com buketeg.xyz
carrauterie.be business-made-fun.com
cartiertrinity.xyz camel-beach.com
cascadelink.org candypeople.se
cezartabac.ro carmuffler.net
cfcl.co.uk carson.getenjoyment.net
checkhit.com cartierreplica.top
cialischstgerts.com catalogs-parts.com
clayvasedesigns.tk chatroulette.world
clicksor.com cheapestjerseysonwholesale.com
clickzzs.nl checktext.org
cmd.kz citetick.com
cool-wedding.net cognitiveseo.com
coolingoods.com commoncrawl.org
cosmetic.donna7753191.ru compliance-mary.top
custom-product-labels.com conocer-sanabria.com
cvety24.by cookie-law-enforcement-ee.xyz
czat.wp.pl cornerstone-countertops.com
dealwifi.com dangphoto.trade
decoratinghomes.ga
dentalpearls.com.au
descargar-musica-gratis.net descargar-musica-gratis.net
detsky-nabytek.info directivepub.com
dvr.biz.ua dneprsvet.com.ua
eaplay.ru dogbreedspicture.net
eleimgo.pw doska-vsem.ru
elektrozigarette2.webs.com ec-file.info
elektrozigaretten2.yn.lt ednorxmed.com
elmifarhangi.com ekosmetyki.net.pl
enews.tech eloxal.ru
enginebay.ru elvenar.com
enskedesquashclub.se escort-russian.com
eropho.com exci.ru
erotik-kostenlos.net ezigarettenshop1.yolasite.com
etur.ru f-loaded.de
european-torches.ru fashionavenuegame.com
evaashop.ru fba-mexico.com
everytalk.tv fetlifeblog.com
expresstoplivo.ru fighrofacciufreesig.ga
failingmarriege.blogspot.com forex-procto.ru
fanpagerobot.com formulaantiuban.com
fast-torrent.ru foxweber.com
fetroshok.ru frcls.fr
fickenbumsen.net freeuploader.com
flirt4free.com freshdz.com
forex.osobye.ru
forpackningsutveckling.se
forum20.smailik.org
free-social-buttons.com
fungirlsgames.net fungirlsgames.net
gcup.ru fz139.ttk.ru
gemara.com gameplexcity.com
geoads.com gayxperience.com
getlamborghini.ga gg-arena.ru
girlporn.ru gilbertbanda.net
girlspicsa.com googst2.ru
global-ics.co.za graphics8.info
gosarhivrt.ru
grand-chlen.ru
gsasearchenginerankerdiscount.com gsasearchenginerankerdiscount.com
guarrasdelporno.xxx gsbs.com.ua
gstatey.net
guruofcasino.com guruofcasino.com
histock.info hazardky.net
hkladys.com hdwallpapers-free.com
hully.altervista.org healing-dysplasia.ru
iccornacircri.cf home.myplaycity.com
ideashome.id hosting-tracker.com
images-free.net housemilan.ru
incanto.in.ua ifmo.ru
iqoption.com innodgfdriverhm.aircus.com
itservicesthatworkforyou.com interesnie-faktu.ru
jagg.info interfucks.net
jerseywholesalebizchina.com internet-apteka.ru
jyrxd.com invest-pamm.ru
karaganda.xkaz.org investzalog.ru
khadastoafarde.tk ispac.org
it-max.com.ua
jav-way.site
javitas.info
jrpmakati.com
kamagragelusa.net
karadene.com
kino-rating.ru
kinosed.net kinosed.net
klin3952.ru kongruan.com
konteiner24.com kotaku.com
krassh.ru lasvegaslockandsafe.com
law-check-eight.xyz leadwayau.com
lecbter-relationships.ga leboard.ru
livingcanarias.com lion.misba.us
lovelycraftyhome.com lirunet.ru
lsp-awak-perikanan.com littlecity.ch
lunchrock.co machicon-akihabara.info
makenahartlin.com masserect.com
marmotstore.online matsdale.com
mazda-roadsters.com medmajor.ru
mcnamaratech.com meget.co.za
members.ghanaweb.com meteostate.com
mil-stak.com mex-annushka.ru
milblueprint.com michaelkorsoutlet.store
mindbox.co.za morepoweronmac.trade
mir-business-24.ru moscow.xrus.org
monclerjacketsoutlet.win mouselink.co
monitorwebsitespeed.com mrmoneymustache.com
moomi-daeri.com multgo.ru
mp3downloadhq.com myyour.eu
musicpro.monster napalm51.nut.cc
musicspire.online naturalshair.site
myfreecams.com neobux-bg.info
netvouz.com novgorod.xrus.org
nodup.ru nrjmobile.fr
npoet.ru nsatc.net
ohmyrings.com oksrv.com
omoikiri-japan.ru onblastblog.online
onlainbesplatno.ru onlinadverts.com
onlinefilmz.net opinionreelle.com
ororodnik.goodbb.ru oren-cats.ru
owohho.com orgasmus-virtual.com
paintingplanet.ru orhonit.com
palvira.com.ua osagonline.ru
panchro.co.uk panicatack.com
pandroid.co pastaleads.com
parajumpersstore.online pekori.to
partybunny.ru pensplan4u.com
php-market.ru peugeot-club.org
pics-group.com photo-clip.ru
pic-re.blogspot.com
picmoonco.pw
pinapchik.com
pinsdaddy.com pinsdaddy.com
pladform.ru pizda.lol
pooleroadmedicalcentre.co.uk pochemychka.net
popunder.ru pons-presse.com
popup-fdm.xyz porno-video-chati.ru
pornhub-ru.com pozdrawleniya.com
porno-dojki.net proekt-mos.ru
pornobrazzers.biz prostitutki-tolyatti.org
pornoelita.info prostitutki-tyumeni.org
pornosexrolik.com quality-traffic.com
potoideas.us queerspace.com
praisong.net quick-seeker.com
prostitutki-astana.org razamicroelectronics.com
prostitutki-kiev.org royal-betting.net
rankscanner.com ru-mediaget.ru
rating-casino2021.ru rukino.org
rednise.com russia-tao.ru
refererx.com saitevpatorii.com
reklamuss.ru sajatvelemeny.com
relayblog.com sammyweaver.com
restorator-msk.ru seo-2-0.com
rethinkwasteni.info sexad.net
roma-kukareku.livejournal.com shareyards.com
roznica.com.ua shoesonlinebuy.xyz
salutmontreal.com shopvilleroyboch.com.ua
sammlungfotos.online sildenafilcitratemed.com
scanner-jack.top siteexpress.co.il
scanner-josh.top
scanner-nelson.top
schoolfiles.net
scoopquest.com
scrnet.biz.ua
searchinterneat-a.akamaihd.net
seorankinglinks.xyz
serpstat.com
sexobzor.info
sexy-screen-savers.com
shinikiev.com.ua
silverage.ru
simple-image.com.ua
smadihome.com
smartbalanceworld.com
snegozaderzhatel.ru snegozaderzhatel.ru
sotkal.lark.ru snow.nvr163.com
souvenir.cc socialseet.ru
souvenirua.com softlinesolutions.me
soviet-portal.do.am solution4u.com
speechfoodie.com spamnuker.com
sstroy44.ru sstroy44.ru
stop-zavisimost.com staynplay.net
stroydetali.ru stockspmb.info
surintech.ac.th stop-gepatit.te.ua
swinon.site stroymonolit.su
teslathemes.com symphonyintegratedhealthcare.com
theguardlan.com syndicate.fun
taihouse.ru
talant-factory.ru
tds-advert002.info
tdsing.ru
testbotprocessor44.com
threecolumnblogger.com
timetorelax.biz
titelhelden.eu
tizanidinecapsules.blogspot.com tizanidinecapsules.blogspot.com
tizanidinehcl4mgisitanarcotic.blogspot.com tizanidinecost.blogspot.com
tizanidineoral.blogspot.com tizanidineinteractions.blogspot.com
tizanidinevsvalium.blogspot.com tizanidinevssoma.blogspot.com
tizanidinewithdrawal.blogspot.com tonerbox.kz
toolsky.com triumf-realty.ru
track-rankings.online trustedhealthtips.com
ttrraacckkrr.com twelvevisionspartyofcolorado.com
twinderbella.com twojebook.pl
udayavani.com uggbootsoutletsale.us
ukrobstep.com ukkelberg.no
ukrup.com uogonline.com
urlcut.ru viagengrarx.com
usadacha.net viagra-soft.ru
valueclick.com viddyoze.com
vgoloveboli.net videochat.ph
videochaty.ru videospornogratisx.net
videositename.com vipms.ru
vigrx-original.ru vkmusics.ru
visa-pasport.ru vltai.com
volgograd.xrus.org watch-movies.ru
vvpg.ru web.cvut.cz
vykupavto-krasnodar.ru webjam.com
w3data.co website-audit.com.ua
walkme.com wedding0venues.tk
weblibrary.win wemedinc.com
weekes.biz.tc westen-v.life
white-truck.life works.if.ua
wleuaprpxuvr.ga xjrul.com
www.jbetting.com xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xclicks.net xn--80aanaardaperhcem4a6i.com
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf yourothersite.com
xn----ctbbcjd3dbsehgi.xn--p1ai yourporn.com
xn----dtbndd4ae7eub.top zakonobosago.ru
xpresscare.ru zoominfo.com
xrus.org zophim.me
yellowads.men
yoursearch.me
zajm-zalog-krasnodar.ru
zarepta.com

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.138 0; 10.1.1.2 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.09.3351 ### Version: V4.2022.09.3352
### Updated: Thu Sep 15 10:03:28 UTC 2022 ### Updated: Thu Sep 15 22:01:31 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.09.3351 ### Version: V4.2022.09.3352
### Updated: Thu Sep 15 10:03:28 UTC 2022 ### Updated: Thu Sep 15 22:01:31 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##