vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 06:15:06 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.08.3319 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-08-29 10:09:19 +00:00
parent 12982c9d9e
commit 3a76dc125d
46 changed files with 93607 additions and 93607 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.08.3318 #### Version: V4.2022.08.3319
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.08.3318 #### Version: V4.2022.08.3319
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.08.3318 #### Version: V4.2022.08.3319
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11564
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.08.3318 #### Version: V4.2022.08.3319
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11568
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11568
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11568
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11568
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 1; 10.1.0.155 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 0; 10.1.0.155 0;
127.0.0.1 0; 127.0.0.1 0;

11568
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 1; 10.1.0.155 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 0; 10.1.0.155 0;
127.0.0.1 0; 127.0.0.1 0;

11568
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 1; 10.1.0.155 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 0; 10.1.0.155 0;
127.0.0.1 0; 127.0.0.1 0;

11568
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3318 3319

11568
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11568
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.43.222.152 0; 20.12.206.88 0;

11568
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 1; 10.1.0.155 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 0; 10.1.0.155 0;
127.0.0.1 0; 127.0.0.1 0;

11568
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 1; 10.1.0.155 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 0; 10.1.0.155 0;
127.0.0.1 0; 127.0.0.1 0;

11568
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 1; 10.1.0.155 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 0; 10.1.0.155 0;
127.0.0.1 0; 127.0.0.1 0;

11568
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.43.222.152 0; 20.12.206.88 0;

11568
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.43.222.152 0; 20.12.206.88 0;

11568
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 1; 10.1.0.155 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Intelliseek Bitacle
InterGET Citoid
NetAnts Fimap
RankingBot2 Id-search
RocketCrawler LexiBot
Scrapy OutclicksBot
Sosospider Pavuk
WWW-Mechanize Xenu
WebSucker pcBrowser
Webster sitechecker.pro

314
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 404checker
404enemy Aboundex
80legs AfD-Verbotsverfahren
ALittle Client
ASPSeek
Aboundexbot
Acunetix
AhrefsBot AhrefsBot
AllSubmitter AllSubmitter
AlphaBot Alligator
AwarioRssBot Anarchie
Ankit
Apexoo
Aspiegel
AwarioSmartBot
BDCbot BDCbot
BDFetch BackStreet
BLEXBot BackWeb
BackDoorBot Backlink-Ceck
Bandit BacklinkCrawler
Barkrowler BetaBot
Bitacle BlackWidow
Black Hole
Blackboard Blackboard
Blow
Boardreader Boardreader
Bolt
BotALot BotALot
Buck Buck
Buddy BuiltWith
Bullseye
BunnySlippers
BuzzSumo BuzzSumo
CazoodleBot Cegbfeieh
CheTeam CensysInspect
CheeseBot
CherryPicker
ChinaClaw
Chlooe
Citoid Citoid
Claritybot Claritybot
Cliqzbot Cliqzbot
Copier Cloud mapping
CopyRightCheck Craftbot
Cosmos
CrazyWebCrawler CrazyWebCrawler
CyotekWebCopy CrunchBot
DIIbot Custo
DSearch DBLBot
DataCha0s DataCha0s
Dirbuster Demon
Disco DomCopBot
Discoverybot
DittoSpyder
DomainAppender DomainAppender
DomainCrawler
DomainSigmaCrawler
Domains Project
Dotbot
Dragonfly Dragonfly
Drip ECCP/1.0
EasyDL
Ecxi Ecxi
EroCrawler EroCrawler
Evil Evil
Exabot ExtLinksBot
Extractor
Extreme Picture Finder
FDM FDM
FlashGet Foobot
Freeuploader G-i-g-a-b-o-t
FrontPage GoZilla
GalaxyBot Grabber
GetWeb GrapeshotCrawler
Gigabot HEADMasterSEO
Gotit
GrabNet
GrapeFX
HMView
HTTP::Lite HTTP::Lite
HTTrack HTTrack
HaosouSpider Havij
IDBot Heritrix
Iblog HonoluluBot
Id-search IRLbot
IlseBot
Image Sucker
Indy Library Indy Library
InfoTekies InfoTekies
Internet Ninja Intelliseek
Iria
Iskanie Iskanie
JetCar
Jetty Jetty
JikeSpider Jorgee
Kozmosbot Kinza
LNSpiderguy LWP::Simple
Lanshanbot Leap
Larbin Lightspeedsystems
LeechFTP LinkextractorPro
Lftp
Libwhisker
LieBaoFast
LinkWalker
LinksManager
LinqiaRSSBot LinqiaRSSBot
Lipperhey
Lipperhey Spider
Litemage_walker Litemage_walker
Ltx71 MIDown tool
MIIxpc MIIxpc
MJ12bot MQQBrowser
MSFrontPage
MSIECrawler MSIECrawler
Mag-Net
Magnet Magnet
Mail.RU_Bot
Majestic-SEO Majestic-SEO
MarkWatch
Mass Downloader
Masscan
Mata Hari Mata Hari
Mb2345Browser MeanPath Bot
Meanpathbot
Mediatoolkitbot Mediatoolkitbot
Microsoft Data Access Microsoft Data Access
Microsoft URL Control
Mister PiX
Moblie Safari Moblie Safari
Mojeek MolokaiBot
Mojolicious Morfeus Fucking Scanner
Mr.4x3
NICErsPRO NICErsPRO
NPbot
Nameprotect Nameprotect
Nessus
NetAnts NetAnts
NetLyzer NetLyzer
NetSpider NetMechanic
Nettrack NextGenSearchBot
Netvibes
Nibbler Nibbler
Niki-bot Niki-bot
Nikto Nmap
NimbleCrawler Not
Nimbostratus Nuclei
Ninja Offline Explorer
Nutch
Offline Navigator Offline Navigator
OnCrawl
OpenVAS OpenVAS
PECL::HTTP Openvas
OrangeBot
OutfoxBot
PHPCrawl
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageScorer PageThing.com
Panscient Pandalytics
Papa Foto Pavuk
Pi-Monster PeoplePal
Picscout
Piepmatz Piepmatz
PleaseCrawl PleaseCrawl
Pockey
ProPowerBot
ProWebWalker ProWebWalker
Pu_iN Pu_iN
PyCurl PyCurl
QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot
RankActive
RankActiveLinkBot
RankFlex RankFlex
RankingBot RankingBot
RankurBot RankurBot
Reaper Re-re
Ripper ReGet
RocketCrawler RealDownload
Recorder
Rogerbot Rogerbot
SEOkicks SEOkicks-Robot
SEOlyticsCrawler SEOstats
SEOprofiler SISTRIX
ScanAlert ScanAlert
Scrapy ScoutJet
ScreenerBot ScrepyBot
Searchestate Seekport
SemanticJuice SemanticJuice
SemrushBot Semrush
SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteSucker
SiteLockSpider
Sitebeam Sitebeam
Siteimprove Siteimprove
SlySearch
Snake
Snapbot Snapbot
SocialRankIOBot Sociscraper
Sogou web spider
Sosospider Sosospider
SpaceBison Sottopop
Spammen Spammen
SpankBot
Spbot Spbot
Spinn3r
SputnikBot
Sqlmap
Sqworm Sqworm
Sucker Steeler
SuperHTTP Surfbot
Swiftbot SurveyBot
T0PHackTeam Suzuran
T8Abot
Teleport Teleport
TeleportPro Telesoft
Telesphoreo
Telesphorep
Toata Toata
Toweyabot Toweyabot
Tracemyfile Tracemyfile
Trendiction Trendiction
Trendictionbot
Turnitin Turnitin
TurnitinBot TurnitinBot
TwengaBot TwengaBot
URLy Warning Typhoeus
V-BOT UnisterBot
Upflow
VB Project VB Project
Vagabondo VCI
Vacuum
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie Voltron
VoidEYE WASALive-Bot
WBSearchBot
WEBDAV
WPScan WPScan
WWW-Mechanize WWW-Mechanize
WWW::Mechanize Wallpapers/3.0
Wallpapers
Web Auto
Web Collage Web Collage
Web Enhancer
Web Fuck
WebCopier WebCopier
WebEnhancer
WebFetch
WebImageCollector WebImageCollector
WebPix
WebSauger
WebStripper
WebSucker WebSucker
Webalta Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
WebsiteExtractor WebsiteQuester
Whacker
Whatweb
Who.is Bot Who.is Bot
Widow Widow
WinHTTrack WiseGuys Robot
Woobot Xenu
Wprecon YoudaoBot
Xaldon WebSpider Zauba
Zade Zermelo
ZoominfoBot Zeus
ZyBorg Zitebot
arquivo.pt ZumBot
backlink-check archive.org_bot
crawler.feedback arquivo-web-crawler
check1.exe
demandbase-bot
domainsproject.org
eCatch
evc-batch
facebookscraper facebookscraper
gopher instabid
isitwp.com iubenda-radar
lwp-request linkdexbot
magpie-crawler magpie-crawler
netEstate NE Crawler muhstik-scan
page scorer pcBrowser
plumanalytics polaris version
s1z.ru probe-image-size
satoristudio.net scan.lol
seocompany.store
seoscanners
seostar seostar
sexsearcher serpstatbot
spyfu sogouspider
sp_auditbot
tAkeOut tAkeOut
trendiction.de
voyagerx.com
webmeup-crawler webmeup-crawler
webpros.com xpymep1.exe
webprosbot
x09Mozilla
zauba.io zauba.io
zgrab

322
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 360Spider
404enemy
AIBOT
ALittle Client
Aboundex Aboundex
Aboundexbot Aboundexbot
Acunetix Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AiHitBot AhrefsBot
Apexoo AllSubmitter
Asterias AlphaBot
Anarchie
Ankit
Anthill
Attach Attach
BLEXBot AwarioSmartBot
BDFetch
BackDoorBot BackDoorBot
Badass BackStreet
BackWeb
Bandit Bandit
Battleztar Bazinga Barkrowler
Blow Bigfoot
Bullseye Black Hole
BlackWidow
BlowFish
Brandprotect
Buck
Buddy
BuiltWith
CODE87 CODE87
Calculon CSHttp
CheeseBot Cegbfeieh
ChinaClaw CensysInspect
CheTeam
Chlooe Chlooe
Citoid Cosmos
Cliqzbot
Cloud mapping
CopyRightCheck
Copyscape
Craftbot
Crawling at Home Project Crawling at Home Project
Curious CrunchBot
CyotekWebCopy CyotekWebCopy
DTS Agent DatabaseDriverMysqli
Demon Demon
Devil Devil
Digincore
DigitalPebble DigitalPebble
Disco
Discobot Discobot
Discoverybot DnyzBot
Dispatch
DittoSpyder
DomCopBot
DomainAppender DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Dotbot Dotbot
Download Wonder
Dragonfly
ECCP/1.0 ECCP/1.0
EirGrabber EMail Siphon
ExtractorPro EasyDL
Ebingbong
ExtLinksBot
Extreme Picture Finder Extreme Picture Finder
EyeNetIE
Ezooms Ezooms
FDM
FHscan FHscan
Fimap Fimap
Firefox/7.0 Firefox/7.0
FlashGet
Flunky
Foobot
Freeuploader Freeuploader
FyberSpider FrontPage
Fyrebot Fuzz
G-i-g-a-b-o-t
GT::WWW GT::WWW
Genieo GalaxyBot
GetRight Grafula
GetWeb
Getintent
GoZilla
Gotit
Grabber
GrapeFX GrapeFX
GrapeshotCrawler
HMView HMView
HTMLparser
HTTP::Lite HTTP::Lite
Haansoft HTTrack
HaosouSpider
Harvest Harvest
Heritrix
Hloader Hloader
Humanlinks HonoluluBot
HybridBot IDBot
IRLbot IRLbot
Iblog
Id-search Id-search
Image Fetch
IndeedBot IndeedBot
Indy Library Indy Library
InfoTekies InfoTekies
Intelliseek JamesBOT
InternetSeer
IstellaBot
JetCar JetCar
Jetty Jetty
JikeSpider
Joomla Joomla
Jorgee
Kenjin Spider
Keyword Density
Kinza Kinza
LWP::Simple LWP::Simple
LexiBot Lanshanbot
Larbin
Leap
LeechGet
LibWeb LibWeb
Likse Likse
LinkpadBot LinksManager
Litemage_walker LinqiaScrapeBot
Ltx71
MIIxpc MIIxpc
MTRobot MJ12bot
Majestic12 MQQBrowser
MarkMonitor MSFrontPage
Mass Downloader Magnet
MauiBot Mail.RU_Bot
Majestic-SEO
MarkWatch
Masscan
Mb2345Browser Mb2345Browser
Meanpathbot Mediatoolkitbot
Microsoft Data Access MicroMessenger
Mister PiX Moblie Safari
Mojeek Mr.4x3
MolokaiBot
Mozlila
Msrabot Msrabot
NICErsPRO
NearSite NearSite
Net Vampire NetSpider
NetLyzer Niki-bot
Netcraft Nikto
Netvibes Nimbostratus
Nibbler
Ninja
Nmap Nmap
Nuclei
Octopus
Offline Explorer Offline Explorer
OpenLinkProfiler Offline Navigator
Openvas OnCrawl
OrangeBot Openfind
OutfoxBot OrangeSpider
PHPCrawl OutclicksBot
POE-Component-Client-HTTP POE-Component-Client-HTTP
Pandalytics PageScorer
Panscient Panscient
Papa Foto
PeoplePal
Petalbot
Pi-Monster Pi-Monster
Picscout Pixray
Picsearch PleaseCrawl
Pimonster
Pockey Pockey
ProPowerBot
Probethenet Probethenet
Psbot Pu_iN
Pump PyCurl
Quick-Crawler QueryN Metasearch
RankActiveLinkBot RSSingBot
RankurBot RankFlex
Reaper RankingBot
RankingBot2
ReGet
Recorder Recorder
RepoMonkey
Ripper
SBIder
SEOkicks-Robot SEOkicks-Robot
SEOstats
SMTBot SMTBot
SalesIntelligent Scanbot
ScoutJet ScrepyBot
Screaming Seekport
Semrush Semrush
SemrushBot
SentiBot
SeoSiteCheckup
SeobilityBot
Seomoz Seomoz
Shodan Shodan
Site Sucker SiteCheckerBotCrawler
Siteimprove SiteExplorer
Sitevigil SiteLockSpider
SlySearch SiteSnagger
Snapbot SmartDownload
Sociscraper Snoopy
SpaceBison Sogou web spider
Spanner Sosospider
Spinn3r SpankBot
Sqlmap Spbot
Sqworm SputnikBot
Stripper Stripper
Sucker Sucker
SuperHTTP SuperHTTP
Surfbot
SurveyBot
Suzuran
Szukacz Szukacz
T8Abot Teleport
TeleportPro Telesphorep
Titan TightTwatBot
Toata
Trendiction
Trendictionbot Trendictionbot
Turingos Turingos
URLy Warning Turnitin
URLy.Warning TwengaBot
UnisterBot
VB Project
VCI VCI
Vagabondo VelenPublicWebCrawler
VidibleScraper VeriCiteCrawler
Virusdie WWW-Collector-E
Voil Wallpapers/3.0
Voltron WallpapersHD
WBSearchBot Web Enhancer
WWW-Mechanize Web Pix
WWW::Mechanize
WWWOFFLE
Web Collage
Web Fetch
Web Fuck
Web Sauger
WebAuto WebAuto
WebBandit WebEnhancer
WebCopier WebFuck
WebFetch
WebGo IS
WebImageCollector WebImageCollector
WebLeacher WebPix
WebReaper
WebSucker WebSucker
Webalta WebWhacker
WebmasterWorldForumBot Webshag
WebsiteQuester WebsiteExtractor
Whacker
Who.is Bot
Widow Widow
WinHTTrack
WiseGuys Robot WiseGuys Robot
Wonderbot Wonderbot
Xaldon_WebSpider Wotbox
Zade Wprecon
Zauba YoudaoBot
Zermelo Zermelo
ZoomBot
ZoominfoBot ZoominfoBot
ZyBorg ZumBot
adscanner arquivo-web-crawler
archive.org_bot arquivo.pt
autoemailspider autoemailspider
cah.io.community coccocbot
cognitiveseo cognitiveseo
crawler.feedback crawler.feedback
evc-batch
facebookscraper
gopher gopher
heritrix
instabid instabid
internetVista monitor linkdexbot
isitwp.com meanpathbot
iubenda-radar muhstik-scan
magpie-crawler page scorer
mediawords plumanalytics
oBot probe-image-size
pcBrowser ripz
scan.lol scan.lol
seobility seobility
seocompany.store seocompany.store
seoscanners
seostar
serpstatbot serpstatbot
sexsearcher
sitechecker.pro sitechecker.pro
sp_auditbot sogouspider
spyfu
sysscan
trendiction.com
trendiction.de trendiction.de
ubermetrics-technologies.com
voyagerx.com
webmeup-crawler webmeup-crawler
webpros.com
webprosbot
x09Mozilla
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
bluesalt.co asophoto.com
daptravel.com coolyarddecorations.com
eyelike.com.ua eralph.tk
intellektmedia.at fm-upgrade.ru
le-clos-des-alouettes.com fusoradio.info
marcogrup.com jmat.cn
musicas.baixar-musicas-gratis.com sad-torg.com.ua
porno-play.net seo-traffic-ranking.info
stats-public.grammarly.io themeforest.net
webinstantservice.com trafficshaper.com

926
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0daymusic.org 00webcams.com
101flag.ru 1kdailyprofit.me
123locker.com 1millionusd.xyz
12bet.com 2pxg8bcf.top
1qingdao.com 3xforum.ro
1webmaster.ml 4senses.co
2x2fan.ru 5-steps-to-start-business.com
45en.ru 55wmz.ru
4k-player.pl 6hopping.com
4kpics.rocks 777-club.ru
7minuteworkout.com 7wind.ru
8lufu.com aa8b68101d388c446389283820863176e7.com
a96527gi.beget.tech ad-words.ru
addray.pro add-add.men
ads.gold adsref.men
adssafeprotected.com adtech.de
advokateg.ru adtology.com
agahinameh.com advancedsoftwaresupport.com
akvamaster.dp.ua adverster.com
albuteroli.com advocatemsk.ru
aibolita.com
aksonural.ru
ald2014.org ald2014.org
alert-fdm.xyz
alfapro.ru alfapro.ru
alienwheel.es algerianembassy.co.in
alienwheels.de am15.net
all-number.com amazingninja.com
allboard.xobor.de amung.us
allknow.info anmysite.com
allmarketsnewdayli.gdn apptester.tk
allsilver925.co.il arabsexxxtube.com
alphahoverboards.com asiengirls.net
amehdaily.com asrv-a.akamaihd.net
amospalla.es atdedinotuho.tk
ample-awards-today.us atout-energie-69.com
ampower.me automobile-spec.com
apartment.ru autonew.biz
apiadanaknet-a.akamaihd.net
apiappenableinfo-a.akamaihd.net
apiwebwebgetcom-a.akamaihd.net
app-ready.xyz
apple.com-cleaner.systems
appleid-verification.com
arcadeplayhouse.com
arclk.net
artavenuegardenstudios.com
ascat.porn
atley.eu.pn
atmovs.com
autodo.info
autovideobroadcast.com autovideobroadcast.com
aversis.net avek.ru
avkzarabotok.com aviav.co
ayeartoforget.com aviav.ru.com
bad-stars.net azlex.uz
bazaronline24.ru b-styles.xyz
beachpics.com baikaleminer.com
beauty-b0x.pl banan.tv
belinka.com.ua bestbrainprod.win
bestadbid.com bestpornuha.com
bestimagecoollibrary.com bildsuche.ru
bestwaystogetpaid.us bitcoin-ua.top
bestwebsitesawards.com
bingoporno.com
bio-market.kz
biocn.dx.am
bitcoinremote.com
bitnote.co bitnote.co
blockety.co biznesrealnost.ru
blackhatworth.com
blog.yam.com blog.yam.com
bloggerads.net blogos.kz
bmsco.co boeuklad.com
boleznikogi.com bolezniorganov.ru
bonux.nextview.ru bonanza-fortune.men
bonzybuddy.com bookmark4you.com
boostmyppc.com bosefux.esy.es
brainboosting.club brateg.xyz
bristolhotel.com.ua brillianty.info
brothers-smaller.ru btvn.ru
browserprotecter.com buyclomidonlaine.com
buy-cheap-online.info bvlgariwallet.xyz
buyantiviralwp.com cah.io.community
buyessaynow.biz callawaygolfoutlet.online
buzz-porno.info
buzzsumo.com
buzzurl.jp
cacheimages.com
callawaygolfstore.online callawaygolfstore.online
callstevens.com camdenmemorials.com
carabela.com.do camsex.xxx-cam.webcam
cartierlovebracelet.xyz canzoni.ru
cartierlovestore.com carina-sy.de
cdncash.com carmuffler.net
cementaresearch.se cartechnic.ru
chatroulette.world cartierreplica.xyz
cheap-trusted-backlinks.com cb.iphantom.com
cienum.fr cbb1smartlist12.click
click2pawn.com cctva.tv
club-lukojl.ru cenokos.ru
cocyq.inwtrade.com chastnoeporno.com
compliance-irvin.top chatroulette.life
compliance-ivan.xyz cheap-pills-norx.com
compliance-josh.top cheapmoncler.pw
compliance-julia.top children-learningreading.info
cookielawblog.wordpress.com ci.ua
cool-wedding.net cialis-samples.com
cornomase.win clean-start.net
cosmetic.donna7753191.ru cleansearch.net
cottageofgrace.com clickaider.com
counter.yadro.ru clickzzs.nl
compliance-walt.top
compucelunlock.net
computernetworksonline.com
cookie-law-enforcement-hh.xyz
coop-gamers.ru
cornerstone-countertops.com
cpajunkies.com cpajunkies.com
crackzplanet.com crafthubs.com
crd.clan.su crazyboost.pro
credit-online.ws culturevie.info
cupday.com cwetochki.ru
customcatchcan.com dailystrength.org
cymbaltaandtizanidine.blogspot.com daisye.top
dame-ns.kz dangphoto.trade
dayibiao.com datract.com
decorazilla.com dear-diary.ru
delayreferat.ru decorationspcs.com
deshevo-nedorogo.ru demian.kz
design-lands.ru diamond-necklace.info
dignitasdata.se diegolopezcastan.com
dimaka.info diffbot.com
divci-hry.info digitalassetmanagement.site
dktr.ru digitalfaq.com
dneprsvet.com.ua diminishedvalueoforegon.com
doctissimo.fr dipstar.org
dokumentalkino.net dirtpics.pw
domaincdn.xyz discountbarbour.online
donvito.unas.cz divatraffic.com
dosug-lux.ru dnepropetrovsk.mistr-x.org
downloadeer.net dnsrsearch.com
drillsaw.ru docs4all.com
drpornogratisx.xxx domain2008.com
drugs-no-rx.info domination.ml
druzhbany.ru dostavimvdom.ru
dstroy.su dostavka-v-krym.com
e-biznes.info download-wallpaper.net
e-commerce-seo.com driving.kiev.ua
e90post.com dustyorate.com
dvr.biz.ua
e-avon.ru
easytuningshop.ru
ebooknovel.club
ecommerce-seo.com ecommerce-seo.com
econom.co ed-shop01.ru
educatemetv.com edwinkonijn.com.au
egvar.net
ekspresihati.info
eleimgo.pw eleimgo.pw
elektrischezigarette2.devhub.com elektrischezigarette2.onsugar.com
empirepoker.com elektronischezi.livejournal.com
en.altezza.travel eredijovon.com
epngo.bz essay-writing.work
escortplius.com eugenevaultstorage.com
eshop.md euronis-free.com
estelight.ru exoneration-project.us
europages.com.ru expertblog.info
excaliburfilms.com ezigarettekaufen.myblog.de
exmasters.com ezigarettenkaufen2.dreamwidth.org
extremal-blog.com
ezigaretteshop2.mywapblog.com
ezigaretteshop2.vefblog.net
f012.de
f1nder.org f1nder.org
factorynightclub.com familyphysician.ru
fanpagerobot.com fashion-stickers.ru
farmingworm.com
fast-torrent.ru fast-torrent.ru
fenoyl.batcave.net fbdownloader.com
feeriaclub.ru
fickenprivat.info fickenprivat.info
fickkontaktehobbyhuren.com
fighrofacciufreesig.ga fighrofacciufreesig.ga
findthe.pet find1friend.com
finemanteam.com fordsonmajbor.cf
fireads.men forensicpsychiatry.ru
firstsiteguide.com forex21.ru
fiuxy.com forpostlock.ru
fix-website-errors.com free-share-buttons.blogspot.com
flipper.top free411games.com
footbalive.org freejabs.com
forex.osobye.ru freshsuperbloop.com
formulaf1results.blogspot.com fuel-gas.com
formularz-konkurs.tk galeon.com
forum20.smailik.org
foto-telok.net
frankofficial.ru
free-fb-traffic.com
free-stock-illustration.com
free-today.com
freecamdollars.com
froggytube.com
fullfileaccess.com
fungirlsgames.net
galaxy-family.ru
galeria-zdjec.com galeria-zdjec.com
gallictures.com gallerylisting.com
gambarxkata.co gameonasia.com
gamblingnerd.com gardene.ru
gaypornmovie.net gaytube.com
getaclueamerica.com getprismatic.com
gktt.ru gheus.altervista.org
glasof.es gigapeta.com
go2mike.ru glavprofit.ru
goforexvps.com glopages.ru
golden-catalog.pro goodwinmetals.co
googleseo.com.tr googglet.com
gorabagrata.ru gorodservis.ru
grand-chlen.ru gratuitbaise.com
greatzip.com grtyi.com
gribkovye-zabolevaniya.com guitar-master.org
gsasearchengineranker.space hdseriale.pl
gta-club.ru hit-men.men
gungamesz.com hitsbox.info
gymi.name homedecoguide.info
hdimagegallery.net hotblog.top
hentaiheroes.com hotdl.in
highland-homes.com hotel-mkad.ru
highstairs-a.akamaihd.net
homemade.gq
hostcritique.com
hothot.ru
hotshoppymac.com hotshoppymac.com
hoverboard360.es housemilan.ru
hspline.com howlongdoestizanidinestayinyoursystem.blogspot.com
hubbble.com images.gyffu.com
iflycapetown.co.za imediadesk.com
igadgetsworld.com img.wallpaperstock.net
imperia31.ru in-tandem.co
increasewwwtraffic.info indianmedicaltourismshop.com
infobanks.ru inet-traffic.com
iomoio.net infospot.pt
ipornox.xxx investpamm.ru
ipostroika.ru itis4you.com
izismile.com
jav-fetish.site jav-fetish.site
javrip.net javrip.net
jillepille.com joingames.org
kakadu-interior.com.ua juliadiets.com
kerch.site jwcialislrt.com
keywordsuggest.org jyvopys.com
khadastoafarde.tk kamagragelusa.net
kerei.ru
keywordhouse.com
kimcurlrvsms.com kimcurlrvsms.com
konpax.com kino-fun.ru
krynica.info kino2018.club
kskjco.club kinobaks.com
laexotic.com kinobest.pl
lakomka.com.ua kinostorm.net
landoftracking.com kosmetyki.tm.pl
laparfumotec.com krasivoe-hd.net
kw21.org
labplus.ru
lankarns.com
law-check-eight.xyz
law-check-nine.xyz
lawyersinfo.org
laxdrills.com
layola.biz.tc layola.biz.tc
ldrtrack.com ledpolice.ru
li-er.ru letmacworkfaster.site
lida-ru.com letslowbefast.site
liffa.ru lezbiyanki.net
linkbolic.com lifebyleese.com
lirunet.ru lineshops.biz
lisque.batcave.net linksharingt.com
listiseltemournlan.gq liran-locks.com
liver-chintai.org littlecity.ch
lizace.com live-sexchat.ru
ljusihus.se locatellicorretor.com.br
loadopia.com locksmith.jp
lob.com.ru lombardfinder.ru
lol-smurfs.com louboutinreplica.xyz
lostaruban.com lubetube.com
m5home.ru magicalfind-a.akamaihd.net
magicdiet.gq mail.allnews24.in
maglid.ru manyresultshub-a.akamaihd.net
mansiondelrio.co marcoislandvacations.net
masterseek.com maxxtor.eu
mathgym.com.au mazda-roadsters.com
maxthon.com mcnamaratech.com
mcadamssupplyco.com med-zdorovie.com.ua
megakino.net mega-bony-2017.pl
members.ghanaweb.com megagrabber.ru
midst.eu mejoresfotos.eu
milblueprint.com metascephe.com
moesexy.com mielec.pl
moetomnenie.com minharevisao.com
mojaocena.com mobile-appster.ru
mobilemedia.md
mokrayakiska.com mokrayakiska.com
monarhs.info monarchfind-a.akamaihd.net
monitorwebsitespeed.com monetizationking.net
morpicert.pw morepoweronmac.trade
mrpornogratis.xxx morocco-nomad-excursions.com
mttwtrack.com moyakuhnia.ru
mturkcontent.com moyaterapiya.ru
music7s.me mulberryoutletonlineeu.com
musicktab.com muschisexbilder.com
musicspire.online
muz-baza.net
muz-shoes.ru
muznachas-service.ru
mvpicton.co.uk
mybackgroundlandscape.blogspot.com mybackgroundlandscape.blogspot.com
mylovelibrabry.com myfreecams.com
myplaycity.com myperiod.club
mypornfree.ru mypets.by
myseoconsultant.com narco24.me
na15.ru navalwiki.info
nacap.ru
naj-filmy24.pl
narkologiya-voronezh.ru
nationalbreakdown.com
negral.pluto.ro negral.pluto.ro
netallergy.ru nero-us.com
nibbler.silktide.com nerudlogistik.ru
niroo.info nikkiewart.ru
nordvpn.com
novgorod.xrus.org novgorod.xrus.org
nufaq.com novodigs.com
nvformula.ru o-o-8-o-o.com
nw-servis.ru o-o-8-o-o.ru
offer.camp obiavo.ru
offers.bycontext.com onemantrip.com
offf.info online-hit.info
one.net.in onlinedomains.ru
oneminutesite.it onlythegames.com
onko-24.com ontargetseo.us
online-sbank.ru orgasmatrix.com
onlyporno.ru orgasmus-virtual.com
ooo-gotovie.ru page2rss.com
opinionreelle.com paramountmarble.co.uk
optibuymac.com pcimforum.com
ostrovtaxi.ru pdns.download
ownshop.cf petsblogroll.com
ozoz.it photorepair.ru
paighambot.com photostudiolightings.com
paintingplanet.ru pic2fly.com
panouri-solare-acoperis.com picsfair.com
paparazzistudios.com.au pix24x7.com
paphoselectricianandplumber.com pkr1hand.com
pavlodar.xkaz.org pladform.ru
perederni.net
photo.houseofgaga.ru
photokitchendesign.com
phuketscreen.com
picurams.pw
plastgran.com plastgran.com
plastweb.ru plastjulgranar.se
pochemychka.net playmsn.com
poffet.net podshipniki-ntn.ru
polybuild.ru pornodojd.ru
pornhive.org
pornofeuer.com pornofeuer.com
pornogratisdiario.com pornokorol.com
pornokajf.com pornosee.info
pornoreino.com pornotubexxx.name
porto.abuilder.net praisong.net
pozdrawleniya.com profit-opportunity.com
pr0fit-b0x.com profitwithalex.info
priceg.com prohoster.info
privacylocationforloc.com promgirldresses.xyz
proekt-mos.ru prospekt-st.ru
professionaldieselcare.com puserving.com
proxyelite.biz quality-traffic.com
pussyfleet.com radiogambling.com
puzo2arbuza.ru rankings-analytics.com
qpypcx.com realresultslist.com
queerspace.com referencemoi.com
rada.ru registrydomainservices.com
raisedseo.com remontbiz.ru
rasteniya-vs-zombi.ru replicalouboutin.xyz
rating-bestcasino.com ring4rhino.com
relatodelpresente.com.ar ringtonepartner.com
remontgruzovik.ru rique.host.sk
remote-dba.de riralmolamsaca.tk
retailwith.com rniaeba.ga
rezeptiblud.ru robot-forex.biz
ritlweb.com rockprogblog.com
runtnc.net rossmark.ru
rus-teh.narod.ru royalads.net
ruscoininvest.company rrutw.com
ruspdd.com ru-mediaget.ru
sampleletters.net ruex.org.ua
sanjosestartups.com runstocks.com
santasgift.ml sabizonline.com
sarm.tk sadaholding.com
scanner-george.top saddiechoua.com
scanner-jessica.top salmonfishingsacramentoriver.com
scanner-john.top sandhillsonline.com
scanner-julianna.top sarf3omlat.com
saveindex.xyz
savetubevideo.com
sbprabooks.com
scanner-mary.top scanner-mary.top
schalke04fc.info scanner-walt.top
scmor.ilxc.cc scoopquest.com
searchimpression.com score-ads.men
searchlock.com scrapinghub.com
secret.xn--oogle-wmc.com sdi-pme.com
see-your-website-here.com sdsjweb.com
seo-2-0.com search.pch.com
searchaddis.com
sei80.com
selectads.men
seo-tools-optimizing.com
seorussian.ru
seotoolsagency.com seotoolsagency.com
sergiorossistore.online seozoom.it
servisural.ru serdcenebolit.com
seryeznie-znakomstva.ru serviporno.com
sex-pr.net sfd-chess.ru
sex-watch.com shariki-zuma-lines.ru
sexflirtbook.com shiksabd.com
sexfreepornoxxx.com shlyahten.ru
sexkontakteao.info shopcheermakeup.info
sexkvartal.com sibtest.ru
sexpartygirls.net sigmund-freud.co.uk
sexphoto.site signx.info
sextracker.com sildenafil-tadalafil.info
sexyebonyteen.com silvercash.com
sexystrippe.info sim-service.net
sexyteens.hol.es sklad-24.ru
sharebutton.to skyway24.ru
shemalegalls.blogporn.in sms2x2.ru
signoredom.com snip.to
slkrm.ru socas.pluto.ro
slonechka.ru social-buttons.com
smichovbike.cz sockshares.tv
smokewithrabbits.com soft1.ru
snowplanes.com sotkal.lark.ru
soaksoak.ru speechfoodie.com
softlinesolutions.me squidoo.com
solinf.co ssconstruction.co
spinnerco.ca stairliftstrue.com
spylog.com stoki.ru
stardevine.com stpolice.com
start.myplaycity.com
steamoff.net
steelmaster.lv
stevemonsen.com
streetfooduncovered.com
stromerrealty.com
studentguide.ru
sucsesofinspiration.com sucsesofinspiration.com
superlist.biz superkanpo.com
tabakur77.com supermodni.com.ua
taxi-v-eisk.ru superoboi.com.ua
terraclicks.com tabletkinaodchudzanie.com.pl
tgtclick.com tasteidea.com
thaisamkok.com td-l-market.ru
thedownloadfromwarez.blogspot.com technopellet.gr
tedxrj.com
teenporn18.net
telefonsexi.com
telegraf.by
the-torrent-tracker.blogspot.com
thebestphotos.eu
thecoolimages.net
thefds.net thefds.net
thegolfclub.info theplacetoupdating.pw
thexart.club thepornsex.org
thetoiletpaper.com
tizanidine4mgstreetprice.blogspot.com tizanidine4mgstreetprice.blogspot.com
tizanidinehydrochloride4mgstreetvalue.blogspot.com tizanidinedrugtest.blogspot.com
tizanidineinfo.blogspot.com tizanidinehcl2mgtablet.blogspot.com
tizanidinevsvalium.blogspot.com tizanidineorflexeril.blogspot.com
toon-families.com tonivedu.it
top-karkas.ru top1-seo-service.com
topanasex.com tovaroboom.vast.ru
topquality.cf trackzapper.com
toy-shop.top traffic2cash.org
tpu.ru trafficjunky.net
tracklead.net trafficmonetize.org
traffmonster.info
triplepanda.xyz triplepanda.xyz
tripper.de
truemfilelj.gq truemfilelj.gq
tryrating.com turkeyreport.tk
tubeline.biz tversvet.ru
twinderbella.com tvnewsclips.info
ua.tc uchil.net
uac.net.au ukrtextbook.com
ucsol.ru um-razum.ru
ultramart.biz umnovocaminho.com
umityangin.net uptime.com
unacittaconte.org uytmaster.ru
uptimebot.net v-doc.co
urdoot.win validdomain.xyz
userequip.com vasileostrovsky-rayon.ru
vacuumcleanerguru.com veopornogratis.xxx
validccseller.com vepad.com
vapeface.club video-chat.cn
vclicks.net videochat.tv.br
vesnatehno.ru videospornogratisx.net
villacoloniale.com
villakohlanta.nu villakohlanta.nu
vipcallsgirls.com visa-pasport.ru
vitanail.ru visitcambridge.org
vkgaleria.com vita.com.hr
vitalads.net
vkonche.com
vodaodessa.com vodaodessa.com
voloomoney.com voronezh.xrus.org
vut.com.ru vseuznaem.com
vysigy.su vvpg.ru
watchdogs-2.ru vzglyadriv.kg
watracker.net vzubah.com
webmasterhome.cn vzube.com
webscouter.net we-are-gamers.com
website-audit.com.ua web-betting.ru
website-datenbank.de webadvance.club
wedding0venues.tk webalan.ru
westen-z.life webjam.com
wildcattube.com whatsappdownload10.com
womama.ru winner7777.net
wovis.site writingservices17.blogspot.ru
www.solartek.ru wrz0iuebwhp5fg.freeddns.com
x-lime.com x-musics.com
xblognetwork.com
xchangetrak.com
xgames-04.com
xjlottery.com
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai xn----btbdvdh4aafrfciljm6k.xn--p1ai
xn--d1abj0abs9d.in.ua xn--1-8sbcpb0bdm8k6a.xn--p1ai
xn--d1aifoe0a9a.top xn--80ahvj9e.xn--p1ai
xsfetish.org xn--80ak6aa92e.com
xxxtube69.com xn--c1acygb.xn--p1ai
yellowfootprints.com xn--lifehacer-1rb.com
yellowproxy.net xnxx699.com
xzlive.com
yaoguangdj.com
yougame.biz yougame.biz
yuarra.pluto.ro youghbould.wordpress.com
zambini.ru yourads.website
zarepta.com youraticles.pl
zawyna.ua yourporn.com
yousense.info
yurgorod.ru
zaapplesales.blogspot.com
zaimhelp.ru
zajm-pod-zalog-nedvizhimosti.ru
zarabiaj-dzis.pl
zarabotok--doma.ru
zetgie.com.pl
zhuravlev.info zhuravlev.info
znakom.sibtest.ru zigarettenonlinekaufen.tumblr.com
zocaparj.kz zigzog.ru
zojirushi-products.ru znakomstvaonlain.ru
zoogdisany.com zskdla.site
zoompegs.com

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1millionusd.xyz 00webcams.com
256bit.by 23kw.ru
4ureyesonly.com 2ads.co.uk
51.la 7makemoneyonline.com
76brighton.co.uk
91abcw.com 91abcw.com
a.pr-cy.ru 9med.net
aab94f698f36684c5a852a2ef272e031bb.com addbin.men
aac52006ec82a24e08b665f4db2b5013f7.com agadelha.com.br
acads.net ahrefs.com
adobereader-free.ru alessandraleone.com
alfabot.xyz animal36.com
alkoravto.ru apilinkswiftco-a.akamaihd.net
anal-acrobats.com artparquet.ru
anjalika.co.in asia-forum.ru
appearance-cool.com auto-moto-elektronika.cz
arewater.com autobudpostach.club
arkkivoltti.net autovideobroadcast.com
arraty.altervista.org azartmix.com
asociatia-tipografilor-transilvania.ro azinoofficial777.ru
aussie-prizes.com bestimagecoollibrary.com
auto-zapchasti.org bestwebsiteawards.com
ayakino.net bidr.trellian.com
backlinkwatch.com bloggerads.net
beachdriveblog.com
belstaffstore.online
betwinservice.com
biglistofwebsites.com
bizcheapjerseyswholesalechina.com
bjorkbacken.nu
blog.koorg.ru
bluesalt.co bluesalt.co
bmwhighperformers.com bookhome.info
bochemit.com.ua bradleylive.xyz
boo-arts.com break-the-chains.com
boom-celebs.com brewdom.ru
bristolhotel.com.ua cartierlove.xyz
bryansk.zrus.org cb.iphantom.com
burkesales.com clickzzs.nl
cartierlove2u.xyz coindirect.io
cartierreplica.top compliance-ivan.xyz
cejewelry.xyz compliance-josh.top
cheapjerseysfootballshop.com compliance-walt.top
children-learningreading.info cutt.us
cialisndbrx.com dignitasdata.se
co.lumb.co dojki365.online
coccoc.com download-wallpaper.net
collegerentals.com e-kwiaciarz.pl
compliance-irvin.xyz e90post.com
cpabegins.ru ecommerce-seo.org
cvety24.by
dandiyabeats.in
davebestdeals.com
decorationspcs.com
demian.kz
detskie-zabavi.ru
dirtpics.pw
div.as
dlya-android.org
dogbreedspicture.net
dogclothing.store
dogsrun.net
domain.webkeyit.com
dosugrostov.site
doxyporno.com
e-commerce-seo.com
eandsgallery.com
efkt.jp efkt.jp
electrik-avenue.com elektronischezigarettekaufen1.beeplog.com
electro-prom.com elektrozigarette2.webs.com
elektrischezigaretten2.beeplog.com elvenmachine.com
elektrozigarette1.dreamwidth.org emerson-rus.ru
erectile.bid enbersoft.com
europeanwatches.ru eurosamodelki.ru
extractorandburner.com ewebarticle.info
favornews.com f00kclan.de
fdzone.org f5mtrack.com
fighrofacciufreesig.ga fanoboi.com
film-one.ru fast-wordpress-start.com
filmfanatic.com ferretsoft.com
free-share-buttons.com fertilitetsradgivningen.se
free-traffic.xyz fettefrauen.net
free411games.com fikasound.tk
freemaintenancesysforpcandmac.top filesvine.com
freevpn.space findacheaplawyers.com
freshsuperbloop.com fingerprintjs.com
funnymama.com floridamobilebillboards.com
game300.ru forex-procto.ru
gdebestkupit.ru fr.netlog.com
generalporn.org freenom.link
geoads.com fxtips.ru
germes-trans.com g.starmoe.xyz
galaxy-family.ru
gameplexcity.com
gaming-journal.com
gateway.zscalerone.net
gaytube.com
genericviagrasildenafiled.net
golden-praga.ru
goodhousekeeping.com goodhousekeeping.com
gourcy.altervista.org gox.com.ua
gpirate.com gratuitxblcodes.com
hccoder.info greatzip.com
hdimagelib.com green-tea.tv
homelygarden.com gsmtlf.ru
hoporno.com hawaiisurf.com
hostcritique.com hoholikik.club
hostnow.men homemature.net
humanelydrew.com hopeonthestreet.co.uk
humbmosquina.tk hotshoppymac.com
imagez.co hustoon.over-blog.com
imk.com.ua ibb.com.ua
intermesh.net ilovevitaly.xyz
itservicesthatworkforyou.com imagefinder.site
james13prix.info img.wallpaperstock.net
javlibrary.cc imperiafilm.ru
jerseyssportsshop.com insider.pro
johnrobertsoninc.com intimshop-fantasy.ru
jolic2.com invest-pamm.ru
justkillingti.me iplusbit.blogspot.co.za
karachev-city.ru istripper.com
karting196.ru itbc.kiev.ua
keywordsuggests.com ix20.ru
kredit-pod-zalog-krasnodar.ru javstock.com
lacapilla.info jaxcube.info
larutti.ru jerseywholesalebizchina.com
law-check-nine.xyz justdating.online
lerporn.info kanimage.com
liveinternet.ru kerei.ru
luckyshop.net.ua kinoduh.ru
mansparskats.com konica.kz
methodsmarketing.com koral.se
microsoftportal.net kraljeva-sutjeska.com
miniads.ca kwzf.net
morf.snn.gr lecbter-relationships.ga
mycouponizemac.com lenvred.org
mygameplus.com lignofix.ua
na15.ru linkpulse.com
newfilmsonline.ru lion.bolegapakistan.com
lonerangergames.com
lubetube.com
macdamaged.space
machicon-ueno.info
manve.info
map028.com
matematikus.info
max-p.men
meble-bogart.info
megahdporno.net
metarip.ru
misswell.net
mixtapetorrent.com
mncrftpcs.com
moivestiy.biz
mrcsa.com.au
mrlmedia.net
my-cash-bot.co
myprintscreen.com
natali-forex.com
new7ob.com
nfljerseyscheapchinabiz.com nfljerseyscheapchinabiz.com
notaria-desalas.com nikhilbahl.com
npoet.ru nofreezingmac.click
nvformula.ru nucia.biz.ly
odywpjtw.bloger.index.hr o-o-11-o-o.com
olvanto.ru okeinfo.online
onclkads.com onemantrip.com
oneminutesite.it onlainbesplatno.ru
openmediasoft.com onlinadverts.com
openstat.com online-hit.info
paintingplanet.ru online.ktc45.ru
partnerline.men online247.ml
pcboa.se open-odyssey.org
pdns.download openlibrary.org
perl.dp.ua partner-trustworthy.men
php-market.ru pc-test.net
picsearch.com pdns.cz
pinsdaddy.com pekori.to
picswe.com
pizda.lol
pontiacsolstice.info
popads.net popads.net
popugauka.ru pornoinn.com
porn-w.org pornoklad.net
porngalleries.top pornoted.com
pornhive.org pornotubexxx.name
porno-play.net pornowarp.info
pornokorol.com portadd.men
printdirectforless.com
privat-girl.net
privetsochi.ru privetsochi.ru
prlog.ru prlog.ru
prodess.ru prospekt-st.ru
prodvigator.ua prostitutki-astana.org
promover.org pwwysydh.com
prostitutki-yaroslavlya.org ranksays.com
pskcijdc.bloger.index.hr razorweb-a.akamaihd.net
psvita.ru redbottomheels.xyz
raavidesigns.com reklamuss.ru
rangoman.date rennlist.com
rapidsites.pro replicaclub.ru
reckonstat.info
research.ifmo.ru
rewardit.com
ringtonepartner.com
risparmiocasa.bz.it risparmiocasa.bz.it
rosbalt.com.ua rn-to-bsn.com
ryetaw.com rozalli.com
sack.net ruf777.com
samolet.fr rusoft-zone.ru
saugatuck.com s1z.ru
sbtdesign.co.uk sabaapress.com
scanner-mark.top scanner-andrew.top
scanner-willy.top scanner-walter.top
semprofile.com scrapinghub.com
sexsearch.com semrush.com
sexy-pings.com seo-smm.kz
shopcheermakeup.info servicecenter.co.ua
shtora66.ru servingnotice.com
simon3.ru sexpartygirls.net
siteimprove.com sexspornotub.com
smallseotools.com sheerseo.com
snip.tw skutecznetabletkinaporostwlosow.pl
socialmonkee.com sky-mine.ru
spaceshipad.com snjack.info
splendorsearch-a.akamaihd.net snjatie-geroinovoy-lomki.ru
soft-program.com
soft1.ru
souvenirua.com
spanking.to
stair.registrydomainservices.com
starpages.net starpages.net
startufa.ru stroilka.info
stauga.altervista.org studiofaca.com
suhanpacktech.com studiokamyk.com.pl
supermodni.com.ua sugarlyflex.pw
svarkagid.com tabletkinaodchudzanie.com.pl
svensk-poesi.com
tarad.com
tattooreligion.ru
teastory.co
tedxrj.com tedxrj.com
tizanidinedosageforsleep.blogspot.com telemetryverification.net
tizanidineshowupondrugtest.blogspot.com tengohydar.tk
tobeyouday.win thegalerie.eu
tour-line.net thin.me.pn
trafficnetzwerk.de tizanidinestreetprice.blogspot.com
trustl.life tizanidinevsvalium.blogspot.com
ua-company.ru tootoo.to
ustion.ru toursmaps.com
trafficcentr.xyz
trafficmania.com
traffictrade.life
triumf-realty.ru
try-rx.com
turizmus.us
typer.one
unmaroll.ya.ru
urlopener.com
uvozdeckych.info uvozdeckych.info
vashsvet.com vacances-voyages.info
vezuviy.su viagraneggrx.com
viagengrarx.com video--production.com
video-chat.love vkak.ru
videochat.ph voditeltrezviy.ru
vilingstore.net w3data.co
vipps.com.my wardreapptokone.tk
viven.host.sk website-speed-check.site
voprosotvet24.ru wma-x.com
webjam.com wordpresscore.com
website-speed-up.site ww1943.ru
websiteaccountant.de xgftnlrt.bloger.index.hr
weburok.com xingzi-vision.com
westum.se xitjw.info
whatsappbot.flyland.ru xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
word-vorlagen.xyz xn--80ahvj9e.xn--p1ai
wordpress-crew.net xn--80aikhbrhr.net
wosik-dach.service-for-web.de xolodremont.ru
writingservices17.blogspot.ru yellowads.men
www.get-free-traffic-now.com yoursearch.me
x-diesel.info ytmnd.com
xaijo.com zakon-ob-obrazovanii.ru
xn----9sbebi2bvzr7h.xn--p1ai zaloadi.ru
yebocasino.co.za zigarettenonlinekaufen1.blox.pl
yoluxuryevents.com zygophyceous.womanstars.site
yoshkarola.zrus.org
youdesigner.kz
yousense.info
youthreaders.com
youtubedownload.org
zagadki.in.ua
zdesformula.ru
zeleznobeton.ru
zero1.it
znakom.sibtest.ru

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.92 0; 10.1.0.155 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.08.3318 ### Version: V4.2022.08.3319
### Updated: Sun Aug 28 22:01:18 UTC 2022 ### Updated: Mon Aug 29 10:03:26 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.08.3318 ### Version: V4.2022.08.3319
### Updated: Sun Aug 28 22:01:18 UTC 2022 ### Updated: Mon Aug 29 10:03:26 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##