vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 15:34:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.01.2885 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-01-29 10:09:13 +00:00
parent 2b4216f655
commit 8c5f43e9d9
46 changed files with 86077 additions and 86077 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2884 #### Version: V4.2022.01.2885
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2884 #### Version: V4.2022.01.2885
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.01.2884 #### Version: V4.2022.01.2885
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10622
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.01.2884 #### Version: V4.2022.01.2885
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10626
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10626
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10626
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10626
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

10626
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

10626
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

10626
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2884 2885

10626
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10626
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.115.165.181 0; 52.156.71.130 0;

10626
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

10626
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

10626
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

10626
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.115.165.181 0; 52.156.71.130 0;

10626
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.115.165.181 0; 52.156.71.130 0;

10626
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 1; 10.1.0.215 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
ADmantX AllSubmitter
Devil Asterias
IndeedBot BDFetch
JetCar Lipperhey
Keyword Density Rankivabot
Nameprotect SeoSiteCheckup
Nuclei SuperBot
Probethenet Website Quester
Wprecon Webster
plumanalytics Zeus

306
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404checker
404enemy 404enemy
ADmantX ADmantX
AIBOT AIBOT
Abonti ALittle Client
Aboundex
Aboundexbot
AhrefsBot AhrefsBot
Aipbot
Alligator Alligator
AlphaBot Anarchie
Anarchy Anarchy
Anarchy99
Atomseobot Atomseobot
Attach Attach
AwarioRssBot AwarioRssBot
BackDoorBot AwarioSmartBot
Backlink-Ceck BLEXBot
Battleztar Bazinga Barkrowler
BatchFTP
Bitacle Bitacle
Blow
Bolt Bolt
BotALot Buck
Buddy Buddy
BuiltBotTough
BuiltWith BuiltWith
Bullseye
BunnySlippers
BuzzSumo BuzzSumo
CATExplorador
CCBot CCBot
CODE87
Calculon Calculon
CheTeam CheTeam
CheeseBot
CherryPicker
Chlooe
Citoid Citoid
Cloud mapping Claritybot
Collector Cliqzbot
Copier Cogentbot
CopyRightCheck
Cosmos Cosmos
Crawling at Home Project Craftbot
CrazyWebCrawler Curious
CrunchBot Custo
CyotekWebCopy CyotekWebCopy
DSearch
DTS Agent DTS Agent
Demon Demon
Digincore Deusu
DigitalPebble
Dirbuster Dirbuster
Discobot
Dispatch Dispatch
DittoSpyder DnBCrawler-Analytics
DomainCrawler DnyzBot
DomCopBot
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project Domains Project
Drip Download Wonder
EMail Wolf ECCP/1.0
EasyDL EMail Siphon
Ebingbong
Ecxi Ecxi
EirGrabber EirGrabber
EroCrawler EroCrawler
Exabot Evil
Express WebPictures Express WebPictures
ExtLinksBot
Extractor
Extreme Picture Finder Extreme Picture Finder
EyeNetIE
Ezooms Ezooms
FDM
FemtosearchBot
Firefox/7.0
Flunky Flunky
Freeuploader FrontPage
G-i-g-a-b-o-t GT::WWW
Getintent GermCrawler
Gigabot GetRight
Gotit GetWeb
Grafula Go!Zilla
HEADMasterSEO Go-Ahead-Got-It
HMView GoZilla
HTMLparser GrabNet
HTTP::Lite Grabber
Haansoft GrapeFX
Harvest HTTrack
HaosouSpider
Havij
Heritrix Heritrix
Hloader Hloader
HonoluluBot HybridBot
IDBTE4M
IDBot IDBot
IRLbot
Iblog Iblog
IndeedBot Id-search
InfoNaviRobot Image Sucker
InterGET InterGET
Iria Internet Ninja
JOC Web Spider InternetSeer
IstellaBot
JamesBOT JamesBOT
JikeSpider Jbrofuzz
Joomla Jyxobot
JustView LexiBot
Kenjin Spider
Kozmosbot
LNSpiderguy
Lftp Lftp
LibWeb LibWeb
Lightspeedsystems Libwhisker
Likse Likse
Linkdexbot
LinkextractorPro LinkextractorPro
LinkpadBot LinqiaMetadataDownloaderBot
LinqiaRSSBot MQQBrowser
LinqiaScrapeBot
Ltx71
MIDown tool
MJ12bot
MSFrontPage MSFrontPage
MSIECrawler
Mag-Net
Magnet Magnet
Mail.RU_Bot
Majestic SEO Majestic SEO
Majestic-SEO Majestic-SEO
Mata Hari Masscan
Meanpathbot MeanPath Bot
MegaIndex.ru Metauri
MicroMessenger Microsoft Data Access
Microsoft URL Control
Minefield
Moblie Safari Moblie Safari
Mr.4x3 Morfeus Fucking Scanner
NearSite Mozlila
Nessus Msrabot
NetLyzer Musobot
NetMechanic NPbot
NetZIP Nameprotect
Navroad
NetSpider
Netcraft Netcraft
Netvibes Nettrack
Not Niki-bot
Nimbostratus
Nmap
Nutch Nutch
Offline Navigator Octopus
Openvas OnCrawl
OutfoxBot OpenLinkProfiler
Page Analyzer OrangeBot
PageGrabber
PageThing.com PageThing.com
Pandalytics Pandalytics
Panscient Pavuk
Papa Foto
PeoplePal
Pi-Monster Pi-Monster
Picscout
Picsearch Picsearch
Pimonster Pimonster
Pockey PleaseCrawl
ProPowerBot ProPowerBot
Probethenet PxBroker
Psbot PyCurl
Pu_iN
Pump
QueryN Metasearch QueryN Metasearch
Quick-Crawler RSSingBot
RankActiveLinkBot RankActiveLinkBot
Rankivabot RankingBot
RankurBot
ReGet ReGet
RealDownload RepoMonkey
Rogerbot Ripper
SBIder SEOkicks-Robot
SEOlyticsCrawler
SEOstats
SISTRIX
SMTBot SMTBot
ScanAlert
Scanbot Scanbot
ScreenerBot Semrush
ScrepyBot SemrushBot
SearchmetricsBot SeobilityBot
Siphon Shodan
SiteCheckerBotCrawler Site Sucker
SiteSnagger SiteLockSpider
SiteSucker Sitebeam
Siteimprove Siteimprove
Sitevigil
Snake
Sociscraper Sociscraper
Sottopop Sogou web spider
SpankBot
Spanner Spanner
Sqlmap SputnikBot
Sqlworm Sqworm
Steeler Sucuri
Sucker
SuperHTTP SuperHTTP
Surfbot T8Abot
Suzuran
Teleport Teleport
Telesoft Telesphoreo
Thumbor Telesphorep
TightTwatBot Toata
Titan Trendiction
Trendictionbot
True_Robot
Turnitin
TwengaBot TwengaBot
Twice
Typhoeus
URLy.Warning URLy.Warning
V-BOT UnisterBot
VB Project
VCI VCI
Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VoidEYE VoidEYE
WISENutbot Voil
WBSearchBot
WPScan
WWW-Mechanize WWW-Mechanize
WWWOFFLE
Wallpapers
Wallpapers/3.0 Wallpapers/3.0
Web Auto WallpapersHD
Web Pix Web Collage
Web Sauger
Web Sucker Web Sucker
WebBandit WebAuto
WebGo IS WebEnhancer
WebFuck
WebImageCollector WebImageCollector
WebLeacher WebReaper
WebSucker WebSauger
WebWhacker WebWhacker
WebZIP WebZIP
Webalta Webalta
Webshag Webshag
WebsiteExtractor WebsiteExtractor
Whack WebsiteQuester
Widow
WinHTTrack WinHTTrack
Wonderbot
Woobot Woobot
Wotbox Wprecon
Xaldon_WebSpider
Zade Zade
Zeus ZoomBot
Zitebot
ZmEu
ZyBorg
adscanner
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
cah.io.community arquivo.pt
coccocbot-web coccocbot-web
cognitiveseo cognitiveseo
com.plumanalytics
crawler.feedback
crawler4j
dataforseo.com dataforseo.com
demandbase-bot demandbase-bot
eCatch evc-batch
gopher gopher
heritrix heritrix
instabid
internetVista monitor internetVista monitor
netEstate NE Crawler lwp-request
oBot meanpathbot
plumanalytics mediawords
pcBrowser
probe-image-size probe-image-size
s1z.ru satoristudio.net
scalaj-http scalaj-http
scan.lol
seobility
seoscanners seoscanners
sexsearcher sexsearcher
siteripz spyfu
sysscan sysscan
trendiction.de webprosbot
ubermetrics-technologies.com x09Mozilla
webgains-bot xpymep1.exe
webmeup-crawler zauba.io

310
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404checker
80legs 80legs
AIBOT ADmantX
ALittle Client ALittle Client
Aboundexbot Aboundex
Acunetix AfD-Verbotsverfahren
Anarchie AiHitBot
Anarchy Alexibot
Anarchy99 Alligator
AlphaBot
Ankit
Aspiegel Aspiegel
Asterias AwarioSmartBot
BDFetch BDFetch
BackDoorBot BLEXBot
BackStreet BackStreet
BackWeb
BacklinkCrawler BacklinkCrawler
Barkrowler
Battleztar Bazinga Battleztar Bazinga
Bitacle BlowFish
BlackWidow Boardreader
Blackboard Brandwatch
Blow Buddy
BotALot Bullseye
BuiltWith BunnySlippers
BuzzSumo BuzzSumo
CATExplorador CCBot
Cegbfeieh Calculon
CheeseBot CazoodleBot
ChinaClaw CheTeam
Chlooe
Cliqzbot Cliqzbot
Cloud mapping
Cogentbot
Collector
Copier
CopyRightCheck CopyRightCheck
Copyscape Craftbot
Crawling at Home Project CrazyWebCrawler
CrunchBot Crescent
Custo
CyotekWebCopy
DBLBot DBLBot
DIIbot DIIbot
DSearch DSearch
Demon DTS Agent
DigitalPebble DigitalPebble
Dirbuster
Discobot Discobot
DnyzBot DittoSpyder
DomainAppender DnBCrawler-Analytics
DomainCrawler
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project Domains Project
Dragonfly Download Wonder
ECCP/1.0 ECCP/1.0
EMail Siphon EMail Siphon
EMail Wolf
EasyDL EasyDL
Ebingbong
Ecxi
EirGrabber EirGrabber
Evil Express WebPictures
Exabot ExtLinksBot
Ezooms ExtractorPro
FDM EyeNetIE
FHscan FemtosearchBot
Fimap Fimap
Flunky FlashGet
Freeuploader Freeuploader
Fyrebot FrontPage
GT::WWW G-i-g-a-b-o-t
GermCrawler Genieo
GetWeb Gigabot
Getintent Go!Zilla
Go-Ahead-Got-It Gotit
Grafula
GrapeFX
GrapeshotCrawler
HEADMasterSEO HEADMasterSEO
HaosouSpider HMView
Havij Haansoft
Heritrix Heritrix
Hloader Humanlinks
HonoluluBot HybridBot
IDBTE4M
IRLbot IRLbot
Image Fetch Iblog
Image Sucker Id-search
InfoNaviRobot InfoNaviRobot
InterGET InternetSeer
Internet Ninja Iskanie
IstellaBot JennyBot
JOC Web Spider JetCar
Jetty JikeSpider
Jorgee Joomla
Kinza Kinza
LNSpiderguy LNSpiderguy
LWP::Simple
Larbin
Leap
LeechFTP LeechFTP
LeechGet LeechGet
LexiBot LexiBot
Lftp Lftp
Libwhisker LibWeb
Lightspeedsystems Lightspeedsystems
LinkScan
Linkbot Linkbot
Linkdexbot
LinkextractorPro
LinksManager LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
Lipperhey Lipperhey
Lmspider Lipperhey Spider
Litemage_walker
Ltx71 Ltx71
MIDown tool
MJ12bot MJ12bot
MQQBrowser
MSFrontPage MSFrontPage
Mag-Net Mag-Net
Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic-SEO Majestic SEO
Majestic12 Majestic12
MarkMonitor MarkMonitor
MauiBot MarkWatch
Mata Hari
MeanPath Bot
Meanpathbot
MegaIndex.ru MegaIndex.ru
MolokaiBot Microsoft URL Control
Mr.4x3 Mister PiX
Msrabot Moblie Safari
NICErsPRO Mojeek
Nessus Morfeus Fucking Scanner
NetLyzer Mozlila
NetSpider Net Vampire
Nettrack NetZIP
Netvibes Netvibes
NimbleCrawler Nikto
Not
Octopus
Offline Navigator
OpenLinkProfiler
Openfind
Openvas Openvas
OrangeBot
OrangeSpider
OutclicksBot OutclicksBot
OutfoxBot OutfoxBot
PECL::HTTP PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageGrabber PageGrabber
PageThing.com PageScorer
Papa Foto Pandalytics
Petalbot Pi-Monster
Picscout Picscout
Psbot Picsearch
Pu_iN Pockey
PxBroker ProPowerBot
QueryN Metasearch Probethenet
RankActive PyCurl
RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankingBot RankFlex
Rankivabot ReGet
Re-re RebelMouse
Reaper
Ripper Ripper
RocketCrawler
Rogerbot Rogerbot
SBIder
SEOkicks SEOkicks
SEOkicks-Robot SMTBot
SalesIntelligent
ScreenerBot ScreenerBot
SearchmetricsBot Searchestate
SemrushBot Seekport
Site Sucker SemanticJuice
SiteExplorer SiteLockSpider
SiteSnagger
Sitebeam Sitebeam
Sitevigil Sitevigil
SmartDownload SlySearch
Snoopy Sogou web spider
Sociscraper
Sosospider Sosospider
Sottopop Sottopop
SpaceBison
Spanner Spanner
Spbot Spbot
Sqworm SputnikBot
Sqlmap
Steeler Steeler
Stripper Stripper
Sucker Sucker
Sucuri
SuperBot SuperBot
SuperHTTP Surfbot
SurveyBot
Swiftbot Swiftbot
Teleport TheNomad
TeleportPro Thumbor
Telesoft
Telesphoreo
Telesphorep
TightTwatBot TightTwatBot
Toata
Toweyabot
Tracemyfile Tracemyfile
Turnitin Trendictionbot
TurnitinBot TurnitinBot
Typhoeus TwengaBot
UnisterBot URLy Warning
V-BOT
VCI VCI
Vacuum
VelenPublicWebCrawler VelenPublicWebCrawler
VidibleScraper VeriCiteCrawler
Voil Voltron
WBSearchBot WASALive-Bot
WISENutbot WEBDAV
WWW-Collector-E WPScan
WWW::Mechanize Wallpapers
WallpapersHD WeSEE
Web Collage Web Collage
Web Enhancer Web Fuck
Web Sauger
Web Sucker
WebAuto WebAuto
WebBandit WebBandit
WebCollage WebCollage
WebEnhancer WebFuck
WebImageCollector WebGo IS
WebLeacher
WebStripper WebStripper
WebSucker
WebZIP
Webalta Webalta
WebmasterWorldForumBot Webshag
WebsiteExtractor Website Quester
WebsiteQuester
Whacker Whacker
WiseGuys Robot Widow
Wonderbot
Woobot
Wprecon Wprecon
YoudaoBot Zermelo
Zeus
ZmEu ZmEu
adscanner arquivo-web-crawler
arquivo.pt arquivo.pt
cah.io.community check1.exe
clark-crawler coccocbot-web
cognitiveseo cognitiveseo
com.plumanalytics crawler4j
crawl.sogou.com
demandbase-bot
domainsproject.org domainsproject.org
gopher gopher
heritrix instabid
isitwp.com internetVista monitor
ips-agent
lwp-request
lwp-trivial lwp-trivial
magpie-crawler magpie-crawler
meanpathbot
mediawords mediawords
netEstate NE Crawler netEstate NE Crawler
plumanalytics page scorer
polaris version
probe-image-size probe-image-size
scalaj-http ripz
scan.lol s1z.ru
seoscanners seocompany.store
seostar
serpstatbot serpstatbot
siteripz sexsearcher
sogouspider sogouspider
sp_auditbot webprosbot
ubermetrics-technologies.com
x22Mozilla x22Mozilla
zgrab zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
anapa-inns.ru falcon-images.blogspot.com
centre-indigo.org.ua kladrus.ru
domain.webkeyit.com poppen-nw.net
drugstoreforyou.com pornogig.com
infowarcraft.ru razamicroelectronics.com
moetomnenie.com speedup-my.site
portside.cc tizanidinedosage.blogspot.com
torontoplumbinggroup.com unece.org
videtubs.pl url-extractor.xyz
vksaver-all.ru waterpurifier.club

952
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0lovespells0.blogspot.com 100searchengines.com
11235813.webzdarma.cz 123kuma.com
123movies.love 123locker.com
2020iscoming.info 1kdailyprofit.me
24x7-server-support.site 1webmaster.ml
273-fz.ru 24-ak.ru
404.mn 3rbseyes.com
4replicawatch.net 57883.net
5000-cotydzien.com 7zap.com
883zy.com 8lufu.com
abcdeg.xyz 9icmzvn6.website
abiente.ru aa9d046aab36af4ff182f097f840430d51.com
ad-words.ru absolute-s.ru
adexprts.com abusora.com
adloads.net academiacsmendoza.org
admeasures.com acheterviagrafr24.com
adprotect.net adf.ly
adultnet.in adult-shop.com.ua
advertisingtag.net agadelha.com.br
affiliate-fr.com aghanyna.com
afftrack001.com akita.kz
afmuseum.com albumsuper.info
afora.ru alfabot.xyz
aktivator-windows10.blogspot.com all-number.com
alive-ua.com allbizne.co.ua
alldownload.pw allornamenti.com
andpolice.com aludecor.info
animebox.com.ua aosexkontakte.net
anonymizeme.pro ap.senai.br
apelsinnik.website apessay.com
apiitsthirteende-a.akamaihd.net apiholdingmypage-a.akamaihd.net
apimegabrowsebiz-a.akamaihd.net apimountainbikei-a.akamaihd.net
apispringsmartne-a.akamaihd.net apper.de
appleid-verification.com appfastplay.com
arcadeplayhouse.com approved.su
arius.tech apptester.tk
asscenihotosidea.blogspot.com arabgirls.adultgalls.com
aversis.net arewater.com
arquapetrarca.info
artefakct.com
as5000.com
atley.eu.pn
atmovs.com
aucoinhomes.com
aussie-prizes.com
autolombard-krasnodar.ru
automobile-spec.com
autovideobroadcast.com
aviav.ru.com aviav.ru.com
avitocars.ru avkzarabotok.info
avon-novosib.ru avtoarenda.by
avtointeres.ru avtocredit-legko.ru
avtolombard-krasnodar.com avtovolop.ru
awency.com awency.com
ayakino.net azazaporn.com
ayanaspa.com b00kmarks.com
backuperwebcam.weebly.com b2b-lounge.com
bannerbank.ru backlink4u.net
bannerpower.com backlinkwatch.com
bannerspace.com bag77.ru
bavsac.com balkanfarma.org
bayareaaeroclub.org balkanfarma.ru
best-coupon-offer.com begalka.xyz
bestbuy.ca belstaffstore.online
bestofpicture.com bestofupload.info
bestpriceninja.com bestssaker.com
beta.hotkeys.com biketank.ga
bharatdefencekavach.com bin-brokers.com
biglistofwebsites.com
bingoporno.com bingoporno.com
blogos.kz binomo.kz
bluejays-jerseys.us biographiya.com
bolitgorlo.net bizzliving.com
bombla.org blogporn.in
bonvillan.com blogrankers.com
boom-celebs.com bluesalt.co
bracketsmackdown.com bnt-team.com
braip.com.br boo-arts.com
brandbucket.com bosman.pluto.ro
breastaugmentation.co.za bpro1.top
brewdom.ru buyessayonline19.blogspot.ru
browsepulse-a.akamaihd.net buypanicdisorderpill.com
budpost.com.ua buzzsumo.com
buntube.net bvlgaribracelet.xyz
bupropion-sr-150-mg.us cactussoft.biz
buyessay3.blogspot.ru callstevens.com
buypuppies.ca cartierlovebraceletreplica.xyz
byme.se cartierloveringreplica.xyz
callawaygolfstore.online
cardsharp1.ru
carson.getenjoyment.net
cartujano-pre.de
casite-513049.cloudaccess.net casite-513049.cloudaccess.net
cb.iphantom.com cdnanalytics.xyz
cenoval.ru cdncash.org
cerev.info cfjump.com
cezartabac.ro charmstroy.info
chatroulette.video cheap-pills-norx.com
chatroulette.world cheapmoncler.xyz
cheapjerseysbizwholesale.us chocolateslim-en-france.com
chiblackhawks-jerseys.com cialischsrx.com
chocolateslim-original.com cialischstgerts.com
choice-credit.ru cigarpass.com
cjmc.info citysecurity.nu
clean-virus-mac.com cleanallvirus.com
cleanpcnow.com club-musics.ru
club-lukojl.ru com-onlinesupport.host
comeondog.info com-secure.download
compliance-andrew.xyz compliance-barak.top
compliance-don.top compliance-irvin.xyz
compliance-donald.xyz compliance-john.top
compliance-george.top
compliance-julia.top
compliance-viktor.top
comsysnet.com
cookie-law-enforcement-ff.xyz
cookielawblog.wordpress.com
coop-gamers.ru coop-gamers.ru
copypaste.traderzplanet.in coprofam.org
cougfan.info cornerstone-countertops.com
crackzplanet.com covid-schutzmasken.de
d2jsp.org craftinsta.ru
dailystrength.org crystalslot.com
dame-ns.kz customergrowthsystems.com
dengi-pod-zalog-nedvizhimosti.ru czeshop.info
design-lands.ru damedingel.ya.ru
dignitasdata.se dandiyabeats.in
data-mining.tk
dev78.dioniqlabb.se
digest-project.ru
digital-sale.su
dinosaurus.site
dirtpics.pw
diusyawiga.tk diusyawiga.tk
divatraffic.com djonwatch.ru
docs4all.com dneprsvet.com.ua
docstoc.com docstoc.com
dojki-hd.com dollartree.info
doyouknowtheword-flummox.ml domainanalyzing.xyz
dpihatinh.gov.vn domik-derevne.ru
driving.kiev.ua downtuptv.gq
dynainbox.com drivotracker.com
eager-nash.188-93-233-196.plesk.page droidlook.net
editmedios.com drpornogratisx.xxx
edudocs.net duawitchrarato.tk
ekobata.ru e90post.com
elektronischezigarettekaufen1.beeplog.com ebonyporn.site
elektrozigarette1.dreamwidth.org ebooknovel.club
elitedollars.com efkt.jp
embedle.com elite-sex-finders.com
enbersoft.com emerson-rus.ru
everflora.ru empathica.com
evidencecleanergold.com eropho.net
exchanges-bet.com etur.ru
ezigaretteshop2.mywapblog.com evehemming.blogspot.com.au
f012.de evogarage.com
f07.de excitacion.info
faithe.top extads.net
famix.xyz eyelike.com.ua
fast-torrent.ru f00kclan.de
fast-wordpress-start.com familyphysician.ru
favornews.com farmprofi.net
fickenbumsen.net feargames.ru
filkhbr.com fermersovet.ru
filmbokep69.com fetroshok.ru
filmetricsasia.com fickkontakte.org
filunika.com.ru fidelityfunding.com
finejewelryshop.ru filesvine.com
finuse.com fillmewithhappiness.com
findpics.pw
fiuxy.com
fix-website-errors.com fix-website-errors.com
flex4launch.ru flowwwers.com
flowersbazar.com footballfarrago.com
fotopop.club fotopop.club
fototravel.eu fr-bearings.ru
futbolkisales.ru free-share-buttons.com
gallery.rennlist.com freecamdollars.com
gambarxkata.co freelifetimefuckbook.com
gamblingnerd.com freewareseek.com
game-mmorpg.net front.to
gameprimary.com funnymama.com
funtoonez.com
fx-brokers-review.com
fym.com.ru
galaxyflowers.ru
galeria-zdjec.com
gambarkataku.co
gamevalue7.weebly.com
gaming-journal.com gaming-journal.com
genericviagrasildenafiled.net gay-site.store
gepezz.info gaygalls.net
gfaq.ru gazporno.com
girlspicsa.com gdcentre.ru
go2album.com geileweiber.tk
golaya.pw genetworx.com
goldandcard.ru get-seo-domain.com
gomusix.com getmyads24.com
goodhousekeeping.com ghostvisitor.com
goodhumor24.com gktt.ru
googlemare.com glogow.pl
graphicwe.org gne8.com
green-tea.tv golden-catalog.pro
gsasearchengineranker.top goodwinmetals.co
gsmtlf.ru goodwriterssales.com
googglet.com
gotwebsite1.com
gribkovye-zabolevaniya.com
grupografico-pilar.com.ar
guidefs.ru
guigyverpo.cf guigyverpo.cf
guitar-master.org gwagka.com
hdhc.ru gyffu.com
hdimagegallery.net hacktougroup.ru
health-medical-portal.info handicapvansarea.com
herehloadibs.cf hasfun.com
hifidesign.ru hccoder.info
high-speed1.net hdseriale.pl
hitsbox.info headpharmacy.com
hobbyhuren-datenbank.com helicalpile.us
homemade.gq hikesearch.net
hotxnights.info hobild.net
howtoclean.club hostcritique.com
ideawheel.com hosting-tracker.com
igithab.com hotenergy.ru
igrovyeavtomaty777.ru hoverboard360.nl
imagine-ex.co hoztorg-opt.ru
immobiliaremassaro.com humbmosquina.tk
indetiske.ya.ru iklysha.ml
infektsii.com imagecoolpub.com
innoslicon.com imagefinder.site
inome.com.ua incomekey.net
intelhdgraphicsgtdrive6w.metroblog.com infospot.pt
intellekt21.ru invitefashion.com
irkutsk.online-podarki.com iqs.biz.ua
islamtoday.co.za isitpaleo.info
isotoner.com itis4you.com
ispac.org ivanstroi.ru
ivoiretechnocom.ci jacago.com
jabimgo.pw jav-fetish.site
janerikholst.se jaxcube.info
jbl-charge.info jikoman.info
job.icivil.ir kaidalibor.de
jobgirl24.ru kakablog.net
johannesburgsingles.co.za kartiniresto.com
joyceblog.top kasino-money.pw
karusel-market.ru kaz.kz
kazka.ru kazan.xrus.org
keki.info keki.info
kiskinhouse.com kichenaid.ru
kinocccp.net
kinomaniatv.pl
komputers-best.ru komputers-best.ru
koronirealestate.gr kountrylife.com
kovrenok.ru krasivoe-hd.com
kupit-adenu.ru ku6.com
lacasamorett.com kvartiry-remont.ucoz.ru
letmacworkfaster.site labafydjxa.su
lflash.ru lapolis.it
life.biz.ua law-check-nine.xyz
lifehacĸer.com leboard.ru
linkdebrideur.xyz ledpolice.ru
liver-chintai.org legalrc.biz
lombardfinder.ru letto.by
lifeinsurancekb.com
lightinghomes.net
linkwithin.com
lion.bolegapakistan.com
lion.snapmap.us
lionking-1994.blogspot.com
liquimondo.com
localmatchbook.com
lomb.co
longadventure.com
losangeles-ads.com losangeles-ads.com
lubetube.com lostfilm-online.ru
lulea-auktionsverk.se louboutinbooties.xyz
macdamaged.space louboutinreplica.xyz
mackeeper-land-672695126.us-east-1.elb.amazonaws.com lowephotos.info
macnewtech.com lyrster.com
magento-crew.net magicdiet.gq
mansparskats.com manipulyator-peterburg.ru
max-eclat.men maranbrinfo.com.br
maxxtor.eu matpre.top
mecash.ru maximilitary.ru
medizinreisen.de maxthon.com
megakino.net me-ke.com
members.ghanaweb.com medosmotr-ufa.ru
message-warning.net meet-flirt-dating.com
mikozstop.com mensandals.xyz
militarysale.pro merryhouse.co.uk
minecraft-neo.ru mhtr.be
mini.7zap.com michaelkorssaleoutletonline.net
missis.top michellblog.online
misswell.net mikrobiologies.ru
mjchamonix.org mir-betting.ru
mobsfun.net missclub.info
modenamebel.ru misterjtbarbers.com
moesonce.com mmostrike.ru
moneyzzz.ru mompussy.net
morf.snn.gr moncleroutletonline.xyz
most.gov.iq money-every-day.com
mouselink.co motherboard.vice.com
mrpornogratis.xxx mototsikl.org
mydearest.co mulberryoutletonlineeu.com
na15.ru muschisexbilder.com
nash-krym.info musicas.baixar-musicas-gratis.com
naughtyconnect.com musirc.com
negociosdasha.com mustat.com
negral.pluto.ro muycerdas.xxx
networkad.net myra.top
new-apps.ru naturalshair.site
nextbackgroundcheck.gq netanalytics.xyz
nfljerseyscheapbiz.us newsperuse.com
nightvision746.weebly.com nfljerseys.online
njkmznnb.ru nfljerseysforsalewholesaler.com
nofreezingmac.work nikitsyringedrivelg.pen.io
nosecret.com.ua norththeface.store
nottyu.xyz nottyu.xyz
novatech.vn odywpjtw.bloger.index.hr
oakleyglassesonline.us
obiavo.net
offside2.5v.pl
oil-td.ru oil-td.ru
okmedia.sk onemactrckr.com
onemantrip.com online-sbank.ru
onko-24.com onlinemegax.com
online247.ml onlineslotmaschine.com
online7777.com onlyforemont.ru
onlinemeetingnow.com ooomeru.ru
openfrost.com oren-cats.ru
openfrost.net ozas.net
openstat.com palocco.it
orel-reshka.net pammik.ru
osnova3.ru panamaforbeginners.com
p-business.ru parfusale.se
paperwritingservice17.blogspot.ru park.above.com
paphoselectricianandplumber.com partners-ship.pro
parajumpersstore.online partnersafe.men
parlament.biz pattersonsweb.com
partner-high.men pawli.eu
perm-profnastil.ru pb-dv.ru
phelissota.xyz penisvergrotendepillennl.ovh
photokitchendesign.com pepperstyle.ru
phpdevops.com photorepair.ru
phrcialiled.com phrcialiled.com
pila.pl pic2fly.com
piter.xrus.org pierrehardysale.online
pizdeishn.com pinkduck.ga
pizza-tycoon.com pinwallpaper.top
pk-services.ru pixell.club
plastjulgranar.se play-mp3.com
plohaya-kreditnaya-istoriya.ru
pochtovyi-index.ru pochtovyi-index.ru
podshipniki-nsk.ru pogruztehnik.ru
pokemon-go-play.online porndairy.in
pokemongooo.ml pornoblood.com
pony-business.com pornofiljmi.com
popunder.net pornoklad.ru
porngalleries.top pornokorol.com
pornokajf.com pornophoto.xyz
pornosemki.info pornotubexxx.name
preparevideosafesystem4unow.space postclass.com
princeadvantagesales.com powitania.pl
psa48.ru praisong.net
pukaporn.com privatamateure.com
puzzleweb.ru prod2016.com
questionmarque.ch promoforum.ru
quickloanbank.com psoriasis-file.trade
r-control.ru purplesphere.in
rabot.host.sk pussysaga.com
radiodigital.co q-moto.ru
rangoman.date quality-traffic.com
rankia.com qwertty.net
realmonte.net raavidesigns.com
realting-moscow.ru randalljhoward.com
redirlock.com rankings-analytics.com
regionshop.biz rapidgator-porn.ga
registrydomainservices.com rating-bestcasino.com
repeatlogo.co.uk razorweb-a.akamaihd.net
revistaindustria.com reklama-i-rabota.ru
rff-cfal.info remont-comp-pomosh.ru
ritlweb.com remont-mobile-phones.ru
royal-betting.net remorcicomerciale.ro
rozalli.com rennlist.com
rucrypt.com rentehno.ru
rusprostitute.com resellerclub.com
sabizonline.com resultshub-a.akamaihd.net
samolet.fr reversing.cc
santasgift.ml ringporno.com
sbf441.com rockma.se
sc-specialhost.com roleforum.ru
scanner-don.top russkoe-zdorovie.ru
scanner-jane.top rustic-quiver.win
scanner-mark.top rvi.biz
scanner-viktor.top sady-urala.ru
school-diplomat.ru sampleletters.net
scottbywater.com santechnik.jimdo.com
screpy.com sbt-aqua.ru
scrnet.biz.ua sbtdesign.co.uk
searchinquire.com scanmarine.info
searchkut.com search-goo.com
seccioncontrabajo.com semp.net
seeresultshub-a.akamaihd.net seotoolsagency.com
semalt.com seozoom.it
semprofile.com serpstat.com
seoanalyses.com
seobility.net
seocdvig.ru
seojokes.net
serviporno.com serviporno.com
seryeznie-znakomstva.ru sex-znakomstva.online
sex-tracker.de
sexad.net sexad.net
sexfreepornoxxx.com sexpornotales.com
sexystrippe.info sexs-foto.com
sharebutton.net shillyourcoins.com
sharpchallenge.com shoppytoolmac.com
shemalegalls.blogporn.in shopsellcardsdumps.com
shisha-swag.de simple-share-buttons.com
shmetall.com.ua sitechecker.pro
shodanhq.com skanninge.se
shopfishing.com.ua
sibdevice.ru
sierraapps.com
siteexpress.co.il
skladvaz.ru
slavkokacunko.de
slayerlife.com
slonechka.ru slonechka.ru
smallseotools.com smadihome.com
social-fun.ru social-button.xyz
socialsignals24.com sodexo.com
soft1.ru songplanet.ru
softxaker.ru soserfis.com
souvenirua.com spacash.com
spabali.org
space2019.top
spb-plitka.ru
sport-video-obzor.ru sport-video-obzor.ru
starwars.wikia.com stefanbakosab.se
stats-collector.org stockquotes.wooeb.com
stevemonsen.com strana-krasoty.ru
stroy-portal22.ru streamin.to
summerlinhomes411.info stretchmate.net
superfish.com stroiminsk.org
supermama.top strongsignal-a.akamaihd.net
supermesta.ru superkanpo.com
svbur.ru symbaloo.com
swinger-mobil.net tattomedia.com
tackletarts.co telefonsexsofort.tk
td-l-market.ru teletype.in
textads.men tfxiq.com
thaisamkok.com thebluenoodle.com
thegolfclub.info thegalerie.eu
thewebsitetemplate.info theprofitsmaker.net
thiegs.reco.ws titangel-vietnam.com
titan-cloud.life tizanidineduringpregnancy.blogspot.com
tizanidine4mgstreetprice.blogspot.com tizanidinepills.blogspot.com
tizanidineandcipro.blogspot.com tizanidinevssoma.blogspot.com
tizanidinedruginteractions.blogspot.com torrentdownloadhub.com
tizanidinehcl4mgtab.blogspot.com traffixer.com
tizanidinerecreationaluse.blogspot.com truebeauty.cc
tizanidinesleepaid.blogspot.com tversvet.ru
tizanidineusedfor.blogspot.com twodollarshows.com
tizanidinevscyclobenzaprine.blogspot.com
top-karkas.ru
topanasex.com
topdownloads.ru
toproadrunner5.info
topsiteminecraft.com
toyota.7zap.com
traxdom.ru
truck-addzilla.life
trucri.me
trudogolik.net
tryrating.com
twitlinks.com
uchebavchehii.ru
uchil.net uchil.net
ukrobstep.com udsgame.online
ukrtextbook.com ufa.xrus.org
umnovocaminho.com unacittaconte.org
uptime-alpha.net unitexindia.com
ural-buldozer.ru unrealcommander.org
urengoy.pro upupa.net
utrolive.ru url-extractor.xyz
vclicks.net urlopener.blogspot.com.au
veerotech.com vashsvet.com
vellings.info vegan-foods.us
veopornogratis.xxx vekzdorov.ru
videochat.life velen.io
videooko.weebly.com veles.shop
velobikestock.com
via-energy-cumpara.com
video-camer.com
video-production.com
videochat.mx
videospornogratisx.net videospornogratisx.net
visa-pasport.ru vip-file.com
volgograd.xrus.org visionwell.com.cn
voloo.ru viveresaniesnelli.it
voluumtrk.com vodkoved.ru
vonradio.com vozbujdenie.com
vseigru.one vsexkontakte.net
vuclip.com vzlomfb.com
vvon.co.uk vzlomtw.com
vykup-avto-krasnodar.ru warningwar.ru
vysigy.su watchmygf.net
w3javascript.com webads.co.nz
wanker.us webcamdevochka.com
webmasterhome.cn weblo.com
webscouter.net webnode.me
website-audit.com.ua webradiology.ru
webtherapy.ru weddingdresses.xyz
weburlopener.com wheelchairliftsarea.com
westvilletowingservices.co.za while.cheapwebsitehoster.com
white-truck.life wholesalejerseychinashop.com
wholinkstome.com wholesalejerseys-cheapest.com
williamrobsonproperty.com woman-tampon.ru
wjgony.com works.if.ua
wonderfulflowers.biz www.get-free-traffic-now.com
world-mmo.com x-diesel.biz
writingservices17.blogspot.ru xaijo.com
wufak.com xdoza.com
www.wohnkabinen-shop.de xn----7sbifcamovvfggw9d.xn--p1ai
xerox-douglas.cf xn----itbkqkfiq.xn--p1ai
xjlottery.com xn--80abgj3a5acid6ghs.top
xmnb.net xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai xn--80atua3d.xn--p1ai
xn--90acenikpebbdd4f6d.xn--p1ai xn--d1acah0c.xn--p1ai
xn--b1ag5cfn.xn--p1ai xn--e1afanlbnfckd7c3d.xn--p1ai
xn--b1agm2d.net xuki.us
xn--c1acygb.xn--p1ai xwatt.ru
xn--lifehacer-1rb.com youjizz.vc
xn--oogle-wmc.com youradulthosting.com
xxlargepop.com youtube-downloader.savetubevideo.com
y8games-free.com youtubologia.it
youporn-forum.ga yunque.pluto.ro
yourdesires.ru zarabiaj-dzis.pl
youresponsive.com zeroredirect2.com
zaim-pod-zalog-krasnodar.ru zeroredirect5.com
zhorapankratov7.blogspot.com znaturaloriginal.com
zigarettenonl.canalblog.com zojirushi-products.ru
zigarettenonlinekaufen1.bloog.pl
zigarettenonlinekaufen2.drupalgardens.com
zocaparj.kz

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
03p.info 2daytrendingnews.com
100searchengines.com 51.la
10bet.com a.frcls.fr
123movies.love adclickthru.net
2000k.ru admatic.com.tr
20pascals.nl adrunnr.com
28n2gl3wfyb0.ru ads-seo.men
2ads.co.uk affordablewebsitesandmobileapps.com
3th.co.in akuhni.by
404.mn alltheviews.com
5kstore.com alvtank.se
76brighton.co.uk anaseracresar.tk
7xc4n.com anime.dougasouko.com
absolute-s.ru appfixing.space
acmebtn.ml aquarium-pro.ru
actionnooz.com aramaicmedia.org
advertex.info asrv-a.akamoihd.net
aero2.ru avtolombard-krasnodar.com
agysya.ru backlink4u.net
albumsuper.info badavit.com.ua
alert-fdm.xyz bedandbreakfast.com
algarveglobal.com best-coupon-offer.com
allsilver925.co.il betune.onlinewebshop.net
apiitsthirteende-a.akamaihd.net bibys.com
appearance-cool.com bin-brokers.com
apple.com-cleaner.systems bkgr.se
appleid-verification.com
ascat.porn
avek.ru
babs.com.ua
back.dog
baikaleminer.com
balois.worldbreak.com
bambi.ck.ua
beauby.ru
bestfortraders.com
bestofpicture.com
bestssaker.com
bestwrinklecreamnow.com
bio.trade-jp.net
blackhatworth.com
blog.f00kclan.de
blog.koorg.ru
bloggen.be
boazpower.com boazpower.com
brus-vsem.ru bochemit.com.ua
businesxxl.com bolezniorganov.ru
cashkitten-a.akamaihd.net bonky.biz
centrumcoachingu.com bookhome.info
cialisovercounteratwalmartusa.com brainxs.us
bristolhostel.com
brokergid.ru
brothers-smaller.ru
btnativenav.com
buyfriend.ru
bytimedance.ru
cakesplus.com.au
call-of-duty.info
canoncdriverq3.pen.io
cardsharp1.ru
chastnoeporno.com
chee-by.biz
clodo.ru
club-musics.ru club-musics.ru
club-samodelkin.ru
com-secure.download
compliance-checker-7.info compliance-checker-7.info
conboy.us compliance-fred.top
coolwallpapers-hd.com confib.ifmo.ru
craftinsta.ru copro.pw
decorazilla.com cosmeticswomens-womensports.rhcloud.com
delta-line.men costablue.xyz
dev.citetick.com customboxes4less.com
diffbot.com customchocolate.business-for-home.com
dinosaurus.site ddpills.com
dojki-hd.com dealitright.click
dolohen.com demian.kz
doxyporno.com drugs-no-rx.info
duawitchrarato.tk dumuelave.xyz
e90post.com elektrischezigaretten1.blogse.nl
easync.io elite-sex-finders.com
ekn-art.se elitedollars.com
elektrischezigarette2.onsugar.com englishdictionaryfree.com
elektrozigarettekaufen2.blogse.nl enternet.ee
empathica.com erotik0049.com
eng-lyrics.com estibot.com
eu-cookie-law.info euroskat.ru
everypony.ru exci.ru
extractorandburner.com excitacao.com
fantasticpixcool.com exlarseva.webblog.es
fast-wordpress-start.com eyemagination.com
feorina.ru ezigarettekaufen2.yolasite.com
ff30236ddef1465f88547e760973d70a.com fellowshipoftheminds.com
fickenprivat.info fermersovet.ru
filesdatabase.net find1friend.com
foodcrafts.website florida-tourism.net
fordsonmajbor.cf footbalive.org
forum-engineering.ru formularz-konkurs.tk
frcls.fr forum.doctissimo.fr
free-deals.faith forums.d2jsp.org
freshmac.space fotopop.club
gabeshop.ru funnel.co.za
ge0ip.org fxtips.ru
getaclueamerica.com gardene.ru
girls-ufa.ru gazobeton-p.com.ua
girlsfuckdick.com geilehausfrauen.net
goldandcard.ru germetiki.com.ua
gonextmedia.com getlamborghini.ga
guod.me gidonline.one
guruofcasino.com goodwriterssales.com
haikuware.com gosreg.amchs.ru
hamilton.ca greatzip.com
happy.new.yeartwit.com guarrasdelporno.xxx
hdpixion.com guigyverpo.cf
heatpower.ru hayate.biz
hitsbox.info hdseriale.pl
hoporno.com heliko.no
howlongdoestizanidinestayinyoursystem.blogspot.com hetmanship.xyz
hoztorg-opt.ru hmmm.cz
ido3.com homedecoguide.info
ilikevitaly.com homemature.net
ilte.info houseofrose.com
imabase.com howmuchdoestizanidinecost.blogspot.com
imagez.co howopen.ru
hyipmanager.in
igadgetsworld.com
ilmen.net
imagecoolpub.com
imagui.eu imagui.eu
iphantom.com imfamous.info
iplogger.org increasewwwtraffic.info
jerseyssportsshop.com interesnie-faktu.ru
joessmogtestonly.com io9.com
journeydownthescale.info istizanidinelikexanax.blogspot.com
joyceblog.top japan-bearings.ru
keywordhouse.com javspace.net
khadastoafarde.tk joingames.org
kletkimehan.ru karaganda.xkaz.org
kreativperlen.ch khafre.us
kupiproday.com.ua kimcurlrvsms.com
kuzinsp.ru kinohall.ru
lacloop.info kliksaya.com
ladsup.com krasnodar.ru
laserpen.club krassh.ru
law-check-seven.xyz laapp.com
legionalpha.com law-check-nine.xyz
lennatin.info lawyersinfo.org
lexiangwan.com li-er.ru
lionking-1994.blogspot.com limads.men
lockerz.com locimge.pw
lodki-pvh.dp.ua locustdesign.co
lombardfinder.ru lovasszovetseg.hu
lostfilm-online.ru manyresultshub-a.akamaihd.net
lunamedia.co matsdale.com
m5home.ru mbiologi.ru
macdamaged.space medizinreisen.de
mafa-free.com mensandals.xyz
makis.nu mobilierland.com
marcowebonyodziezowe.pl morocco-nomad-excursions.com
menetie.ru moskva.nodup.ru
metalonly.info
mexicotravelnet.com
mindbox.co.za
mnimmigrantrights.net
monetizationking.net
moonci.ru
mp3s.club mp3s.club
mrwhite.biz mundoaberrante.com
music7s.me najaden.no
mygameplus.com namecrumilchlet.tk
narkologiya-orel.ru narco24.me
nash-krym.info narosty.com
novosti-hi-tech.ru nfljerseys.online
nsatc.net obiavo.com
nuit-artisanale.com offer.camp
ofanda.com offergroup.info
onlinebay.ru onlineslotmaschine.com
onlinetvseries.me orsonet.ru
openmediasoft.com outpersonals.com
openmultipleurl.com paceform.com
optikremont.ru pandarastore.top
ossmalta.com parajumpersstore.online
owathemes.com parvezmia.xyz
panchro.co.uk pateaswing.com
partnerads.men peekyou.com
pastaleads.com petedrummond.com
patterntrader-en.com pinup-casino1.ru
pertlocogasilk.tk plaff-go.ru
phrcialiled.com podshipniki-nsk.ru
piratecams.com pogosh.com
poffet.net porn-w.org
popmarker.com porno-dojki.net
porno-transsexuals.ru pornoreino.com
pornoelita.info posible.net
powenlite24.ru positive2b.ru
q-moto.ru primedice.com
rainbowice.ru privatov-zapisi.ru
rarbg.to profit-opportunity.com
realitykings.com profitwithalex.info
rednise.com pulseonclick.com
refads.pro qwarckoine.com
saitevpatorii.com rebuildermedical.com
sankt-peterburg.nodup.ru reeyanaturopathy.com
sape.top registrationdomainsite.com
scanner-nelson.top remontgruzovik.ru
searchaddis.com rollercoin.com
searchengineranker.email rrutw.com
seasonvar.ru saveriopiazza.it
segol.tv scanner-barak.top
seoanalyses.com scopich.com
seobility.net search.1and1.com
seoprofiler.com searchinterneat-a.akamaihd.net
seryeznie-znakomstva.ru sethrollins.net
sex-sex-sex5.com sex-sex-sex5.com
sexkontakteao.info sex-tracker.de
sexpornotales.com sexad.net
sexreliz.com sexiporno.net
sharebutton.net sexspornotub.com
shoppingmiracles.co.uk sextracker.be
skutecznetabletkinaporostwlosow.pl shtaketniki.ru
slavkokacunko.de sigmund-freud.co.uk
social-search.me silktide.com
socialmadesimple.com snyatie-lomki-v-stacionare.ru
spammen.de soc-econom-problems.ru
studiokamyk.com.pl socialbuttons.xyz
suhanpacktech.com socialmonkee.com
sunflowerdrawingpaintings.blogspot.com spbchampionat.ru
supermama.top sribno.net
superstats.com streamin.to
svetlotorg.ru subj.ukr-lit.com
svnuppsalaorebro.se tako3.com
svtrd.com tdsing.ru
swinon.site technika-remont.ru
taboola.com tedxrj.com
telsis.com telefonsexsofort.tk
timdreby.com thepornsex.org
titangel-vietnam.com thesmartsearch.net
tizanidine4mgprice.blogspot.com tizanidinecapsules.blogspot.com
tizanidineduringpregnancy.blogspot.com tizanidinehydrochloride4mgstreetvalue.blogspot.com
top-study.work topsy.com
track-rankings.online twittrading.com
tracksz.co uasb.ru
ttsq.fr
ucoz.ru
uhdtv.website uhdtv.website
ultimatesetnewfreeallsoftupgradesystems.pw ussearche.cf
unmaroll.ya.ru vduplo.ru
varikozdok.ru venerologiya.com
venta-prom.ru virtuagirl.com
vizitki.net vzglyadriv.kg
vrotike.ru wallpapersist.com
vzube.com webalan.ru
watch-movies.ru webinstantservice.com
webcamtalk.net webjam.com
wholesalejerseychinaoutlet.com website-speed-up.site
wholesalejerseyscheapjerseys.us.com woman-tampon.ru
windowssearch-exp.com womensterritory.ru
wineitudes.wordpress.com woweb.com.ua
withstandingheartwarming.com www.solartek.ru
ww1943.ru xev.ru
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
xn--80ab4aa2g.xn--p1ai xn--80ab4aa2g.xn--p1ai
yhit.press xn--h1aakne2ba.xn--p1ai
yoluxuryevents.com xsion.net
xxart.ru
yellowads.men
yougotanewdomain.com
youradexchange.com youradexchange.com
yuarra.pluto.ro youresponsive.com
zeg-distribution.com ytmnd.com
zog.link zastenchivosti.net
zscaler.net zeroredirect11.com
zionstar.net
znaniyapolza.ru
zygophyceous.womanstars.site

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.205 0; 10.1.0.215 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2884 ### Version: V4.2022.01.2885
### Updated: Fri Jan 28 22:01:46 UTC 2022 ### Updated: Sat Jan 29 10:03:22 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2884 ### Version: V4.2022.01.2885
### Updated: Fri Jan 28 22:01:46 UTC 2022 ### Updated: Sat Jan 29 10:03:22 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##