vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.12.3507 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-12-06 10:18:40 +00:00
parent 08125f1a64
commit ff370b9aff
46 changed files with 91012 additions and 91012 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3506 #### Version: V4.2022.12.3507
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3506 #### Version: V4.2022.12.3507
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.12.3506 #### Version: V4.2022.12.3507
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11242
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.12.3506 #### Version: V4.2022.12.3507
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11246
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11246
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11246
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11246
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 1; 10.1.0.165 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 0; 10.1.0.165 0;
127.0.0.1 0; 127.0.0.1 0;

11246
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 1; 10.1.0.165 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 0; 10.1.0.165 0;
127.0.0.1 0; 127.0.0.1 0;

11246
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 1; 10.1.0.165 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 0; 10.1.0.165 0;
127.0.0.1 0; 127.0.0.1 0;

11246
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3506 3507

11246
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11246
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.228.93.51 0; 40.88.132.124 0;

11246
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 1; 10.1.0.165 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 0; 10.1.0.165 0;
127.0.0.1 0; 127.0.0.1 0;

11246
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 1; 10.1.0.165 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 0; 10.1.0.165 0;
127.0.0.1 0; 127.0.0.1 0;

11246
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 1; 10.1.0.165 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 0; 10.1.0.165 0;
127.0.0.1 0; 127.0.0.1 0;

11246
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.228.93.51 0; 40.88.132.124 0;

11246
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.228.93.51 0; 40.88.132.124 0;

11246
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 1; 10.1.0.165 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Anarchie CODE87
Bullseye Lipperhey
Exabot Lmspider
FlashGet MIDown tool
Netvibes Nmap
PageScorer RankurBot
Pockey Rogerbot
RSSingBot Wallpapers/3.0
TeleportPro WebGo IS
Web Collage cognitiveseo

300
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404enemy 404checker
80legs
ADmantX ADmantX
AIBOT ALittle Client
ASPSeek ASPSeek
Abonti Abonti
AiHitBot Aboundex
Alexibot Aboundexbot
AhrefsBot
AllSubmitter AllSubmitter
AlphaBot Anarchie
Anarchy Anarchy
Anthill Anarchy99
Ankit
Apexoo Apexoo
Aspiegel Aspiegel
Asterias BDCbot
Attach BDFetch
AwarioRssBot
AwarioSmartBot
BLEXBot BLEXBot
BackDoorBot
BackWeb
Backlink-Ceck
BatchFTP BatchFTP
Battleztar Bazinga Battleztar Bazinga
BetaBot Bigfoot
BlackWidow Black Hole
Blackboard
BlowFish
Boardreader
Bolt Bolt
Buck Brandprotect
BunnySlippers BuiltBotTough
CazoodleBot BuzzSumo
CCBot
Cegbfeieh Cegbfeieh
CensysInspect CheTeam
CherryPicker
ChinaClaw ChinaClaw
Citoid Citoid
Claritybot Cliqzbot
Cloud mapping
Cogentbot Cogentbot
Copier
Copyscape Copyscape
Cosmos Cosmos
Craftbot CrazyWebCrawler
Crescent Crescent
CrunchBot
Custo
DBLBot
DSearch
Demon
Deusu
Devil
Digincore Digincore
Dirbuster
Discoverybot Discoverybot
DnyzBot DnBCrawler-Analytics
DomCopBot DomainSigmaCrawler
DomainAppender Download Wonder
DomainCrawler EMail Wolf
Dotbot Ecxi
Dragonfly Evil
ECCP/1.0
Exabot Exabot
Express WebPictures
ExtLinksBot
Extractor Extractor
ExtractorPro
FDM FDM
FHscan FemtosearchBot
Fimap Fimap
Firefox/7.0 Firefox/7.0
Flunky Freeuploader
Foobot FrontPage
Fuzz FyberSpider
G-i-g-a-b-o-t Fyrebot
GT::WWW GT::WWW
GalaxyBot
Genieo Genieo
GermCrawler GetRight
GetWeb
Getintent
Gigabot Gigabot
Go-Ahead-Got-It Go!Zilla
Gotit GoZilla
GrabNet
Grabber Grabber
GrapeshotCrawler
HEADMasterSEO HEADMasterSEO
HMView HTMLparser
HTTrack HTTP::Lite
Haansoft
Havij
Hloader
HonoluluBot HonoluluBot
Humanlinks
HybridBot HybridBot
IDBTE4M
Id-search
IlseBot
Image Fetch
Intelliseek
Internet Ninja
InternetSeer
Iria
JOC Web Spider JOC Web Spider
JamesBOT Jbrofuzz
JennyBot Jetty
JetCar JikeSpider
Joomla
Jorgee Jorgee
Keybot Translation-Search-Machine Jyxobot
Keyword Density
Kinza
Kozmosbot Kozmosbot
LNSpiderguy LNSpiderguy
Lanshanbot
Larbin Larbin
LexiBot LeechGet
Lftp Libwhisker
LieBaoFast Likse
LinkWalker
Linkbot Linkbot
LinkextractorPro
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaScrapeBot
LinqiaRSSBot Lipperhey
Litemage_walker Lipperhey Spider
Lmspider
MQQBrowser
MSIECrawler MSIECrawler
Mag-Net MTRobot
Majestic-SEO Majestic12
MarkMonitor
MarkWatch
Masscan Masscan
MauiBot MauiBot
Mb2345Browser Meanpathbot
MegaIndex.ru Microsoft Data Access
MicroMessenger Microsoft URL Control
Moblie Safari Moblie Safari
Mojeek Mojolicious
MolokaiBot MolokaiBot
Nameprotect Mr.4x3
Navroad Msrabot
Netcraft Musobot
NPbot
Name Intelligence
Needle
Net Vampire
Nettrack
Nibbler Nibbler
Niki-bot Not
Nimbostratus Nutch
Offline Explorer Offline Explorer
OnCrawl OnCrawl
OpenLinkProfiler
OpenVAS
Openfind Openfind
Openvas
OutfoxBot OutfoxBot
POE-Component-Client-HTTP PHPCrawl
PageAnalyzer PageAnalyzer
PageGrabber
PageScorer
PageThing.com PageThing.com
Pandalytics Pavuk
Petalbot
Pi-Monster Pi-Monster
Picscout Picscout
Pimonster Picsearch
Pixray PictureFinder
PleaseCrawl Pu_iN
ProWebWalker Quick-Crawler
Psbot
RSSingBot RSSingBot
RankActive RankActive
RankActiveLinkBot
RankingBot RankingBot
RankingBot2 Rankivabot
Re-re
ReGet
RealDownload RealDownload
Recorder Recorder
Ripper RedesScrapy
RocketCrawler
Rogerbot Rogerbot
SEOkicks SEOkicks
SEOprofiler
SEOstats
SISTRIX SISTRIX
SMTBot ScanAlert
Scanbot ScoutJet
Scrapy
Screaming
ScreenerBot
ScrepyBot
SearchmetricsBot SearchmetricsBot
Seekport Seekport
SemrushBot SemanticJuice
SeoSiteCheckup SeoSiteCheckup
SeobilityBot Seomoz
Shodan Shodan
Siphon Siphon
SiteLockSpider Site Sucker
SiteSucker SiteSucker
Siteimprove Sitebeam
Sitevigil
SlySearch
SmartDownload
Snapbot
SocialRankIOBot SocialRankIOBot
Sottopop
Spammen
SpankBot
Spbot Spbot
SputnikBot SputnikBot
Sqlmap Sqlmap
Sqlworm Sucker
Sucuri SurveyBot
SuperBot Suzuran
Surfbot Swiftbot
Szukacz Szukacz
T0PHackTeam
Teleport
TeleportPro TeleportPro
Telesphoreo Telesoft
Thumbor
Titan Titan
Toweyabot Toata
Trendictionbot Trendictionbot
Turnitin
TurnitinBot TurnitinBot
TwengaBot TwengaBot
Typhoeus Twice
URLy Warning
Upflow Upflow
VCI
Vacuum Vacuum
Vagabondo
VeriCiteCrawler VeriCiteCrawler
VoidEYE WPScan
WWW-Collector-E
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
WWWOFFLE WeSEE
Wallpapers/3.0 WebAuto
Web Auto
Web Fetch
Web Fuck
Web Sauger
Web Sucker
WebEnhancer WebEnhancer
WebFetch WebFetch
WebImageCollector
WebPix
WebReaper WebReaper
WebSauger WebSauger
WebStripper WebSucker
Webalta WebWhacker
WebmasterWorldForumBot
Webshag Webshag
Website Quester
WebsiteQuester
Whacker
Widow Widow
Wprecon WiseGuys Robot
Zauba Wonderbot
Zeus Woobot
Xaldon WebSpider
YoudaoBot
ZmEu
ZoomBot
ZyBorg ZyBorg
archive.org_bot
autoemailspider autoemailspider
backlink-check
cah.io.community
check1.exe check1.exe
clark-crawler cognitiveseo
com.plumanalytics facebookscraper
evc-batch
gopher gopher
heritrix
instabid instabid
iubenda-radar ips-agent
lwp-request lwp-request
muhstik-scan
netEstate NE Crawler netEstate NE Crawler
oBot
page scorer page scorer
scalaj-http
scan.lol
seobility seobility
seocompany.store seocompany.store
sexsearcher seostar
sitechecker.pro
siteripz
sp_auditbot sp_auditbot
spyfu
sysscan
tAkeOut tAkeOut
voyagerx.com trendiction.com
webgains-bot trendiction.de
webmeup-crawler
webpros.com webpros.com
webprosbot xpymep1.exe
x22Mozilla

304
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
80legs 80legs
AIBOT
ALittle Client ALittle Client
Aboundex ASPSeek
Acunetix Abonti
AfD-Verbotsverfahren Aboundexbot
AhrefsBot AiHitBot
Anarchie Alexibot
Anthill AllSubmitter
Alligator
Anarchy
Ankit
Attach
AwarioRssBot AwarioRssBot
BBBike
BLEXBot
BackDoorBot BackDoorBot
Backlink-Ceck BackStreet
Badass
Barkrowler
Battleztar Bazinga Battleztar Bazinga
BetaBot Bigfoot
Black Hole Bitacle
BlackWidow
Blackboard Blackboard
BlowFish
BotALot
Brandwatch Brandwatch
Buck
Buddy
BuiltWith
Bullseye
BunnySlippers BunnySlippers
CATExplorador CATExplorador
CCBot CazoodleBot
CODE87 CheTeam
CensysInspect CherryPicker
Chlooe Cliqzbot
Claritybot Cloud mapping
Cocolyzebot
Cogentbot Cogentbot
CopyRightCheck Copier
Cosmos Copyscape
Craftbot
Crawling at Home Project Crawling at Home Project
Crescent CrazyWebCrawler
Curious
Custo Custo
DBLBot
DIIbot DIIbot
DTS Agent
DatabaseDriverMysqli
Deusu Deusu
DigitalPebble Devil
Dirbuster
Disco
Discobot Discobot
Dispatch Dispatch
DittoSpyder DittoSpyder
DnBCrawler-Analytics DomCopBot
DnyzBot
DomainCrawler
DomainSigmaCrawler
Domains Project
Dotbot Dotbot
Download Wonder
Dragonfly Dragonfly
Drip ECCP/1.0
EMail Siphon EMail Siphon
EasyDL EasyDL
Ebingbong EirGrabber
Ecxi ExtractorPro
EroCrawler Extreme Picture Finder
Evil
ExtLinksBot
FDM FDM
FHscan
FemtosearchBot FemtosearchBot
Fimap Fimap
Firefox/7.0 Firefox/7.0
FlashGet
FrontPage
FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t
GT::WWW GT::WWW
GetRight Getintent
GetWeb Gigabot
Gotit Go!Zilla
Go-Ahead-Got-It
Grabber Grabber
Grafula GrapeFX
HEADMasterSEO HTMLparser
HMView HTTrack
Haansoft
HaosouSpider
Harvest
Havij Havij
Heritrix Heritrix
Hloader Hloader
IDBTE4M HonoluluBot
Id-search IRLbot
Indy Library IlseBot
Intelliseek Intelliseek
InterGET Internet Ninja
IstellaBot JOC Web Spider
Jbrofuzz JamesBOT
JennyBot JennyBot
JikeSpider JikeSpider
JustView Joomla
Kenjin Spider
Keyword Density
Kinza
Kozmosbot Kozmosbot
LNSpiderguy Leap
Lanshanbot LeechGet
LeechFTP Lightspeedsystems
LexiBot
Lftp
LibWeb
LieBaoFast
LinkextractorPro LinkextractorPro
LinkpadBot
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot Lipperhey
MFC_Tear_Sample Litemage_walker
MIDown tool Lmspider
MIIxpc Ltx71
MSFrontPage MJ12bot
MSIECrawler Magnet
MTRobot Majestic SEO
Majestic-SEO Majestic-SEO
Mass Downloader
Mb2345Browser Mb2345Browser
Metauri MeanPath Bot
MicroMessenger Meanpathbot
Minefield Microsoft Data Access
Mister PiX Mister PiX
Mojeek
MolokaiBot MolokaiBot
Msrabot Morfeus Fucking Scanner
NPbot
Nameprotect Nameprotect
Needle
Nessus Nessus
NetLyzer Net Vampire
NetSpider NetSpider
Nettrack Nettrack
Netvibes
Niki-bot Niki-bot
NimbleCrawler NimbleCrawler
Nutch Nimbostratus
Openfind OpenVAS
OrangeBot
OutfoxBot
PECL::HTTP
PHPCrawl PHPCrawl
PageScorer PageAnalyzer
Pandalytics PageGrabber
Papa Foto Papa Foto
PictureFinder Pavuk
Pi-Monster
Picscout
Piepmatz Piepmatz
Pimonster Pimonster
Pixray Pixray
Pockey Pockey
ProPowerBot
Probethenet
Psbot Psbot
Pu_iN Pu_iN
Pump
PyCurl
QueryN Metasearch
Quick-Crawler Quick-Crawler
RankActive
RankFlex RankFlex
RankingBot RankingBot
RankingBot2
Rankivabot
Re-re Re-re
ReGet
Recorder Recorder
RepoMonkey RepoMonkey
Ripper
RocketCrawler RocketCrawler
SEOkicks SBIder
SEOlyticsCrawler
SEOstats SEOstats
SMTBot SISTRIX
SalesIntelligent Searchestate
Scanbot SearchmetricsBot
ScoutJet Semrush
Scrapy
Screaming
Seekport
SemanticJuice
SemrushBot
SentiBot SentiBot
Seomoz SeoSiteCheckup
Siphon
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer SiteExplorer
Siteimprove SiteLockSpider
SlySearch Sitevigil
SmartDownload Snake
Sociscraper
Sosospider Sosospider
SpaceBison Sqworm
Spbot Steeler
Sucuri SuperBot
SurveyBot Suzuran
T8Abot
Teleport Teleport
TeleportPro
Telesoft Telesoft
Telesphoreo Telesphorep
Thumbor TheNomad
TightTwatBot TightTwatBot
Trendiction Titan
Toweyabot
True_Robot
Turingos Turingos
Turnitin
TurnitinBot TurnitinBot
TwengaBot
Twice Twice
Upflow UnisterBot
VelenPublicWebCrawler VB Project
WEBDAV VeriCiteCrawler
Virusdie
Voltron
WISENutbot
WPScan WPScan
WWW-Mechanize WWW-Mechanize
WWWOFFLE WWWOFFLE
WeSEE Web Auto
Web Collage Web Sauger
Web Enhancer
Web Pix
Web Sucker Web Sucker
WebAuto
WebBandit WebBandit
WebImageCollector WebFetch
WebLeacher WebPix
WebReaper WebZIP
WebSauger WebmasterWorldForumBot
WebSucker Webshag
Webalta
WebsiteExtractor
Webster
Whacker
Whatweb Whatweb
Who.is Bot
WinHTTrack WinHTTrack
WiseGuys Robot
Wonderbot Wonderbot
Woobot
Wotbox Wotbox
Wprecon Wprecon
Xaldon_WebSpider Zade
Zeus Zermelo
ZmEu Zitebot
ZoomBot
ZoominfoBot ZoominfoBot
ZumBot
adscanner adscanner
archive.org_bot arquivo.pt
autoemailspider
backlink-check
cah.io.community cah.io.community
check1.exe coccocbot
com.plumanalytics com.plumanalytics
crawler.feedback crawler.feedback
crawler4j crawler4j
facebookscraper eCatch
internetVista monitor evc-batch
linkdexbot gopher
lwp-trivial instabid
ips-agent
magpie-crawler magpie-crawler
meanpathbot muhstik-scan
mediawords netEstate NE Crawler
page scorer
pcBrowser
plumanalytics plumanalytics
probe-image-size
ripz ripz
satoristudio.net satoristudio.net
scalaj-http scalaj-http
seocompany.store scan.lol
seoscanners seoscanners
seostar seostar
serpstatbot sogouspider
sexsearcher
sitechecker.pro
sp_auditbot sp_auditbot
sysscan
trendiction.com trendiction.com
trendiction.de zgrab
ubermetrics-technologies.com
voyagerx.com
webpros.com
x22Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
amt-k.ru azbukafree.com
angigreene.com christianlouboutinshoes.xyz
apibrowsesmartne-a.akamaihd.net docsportal.net
asscenihotosidea.blogspot.co.za dwomlink.info
koversite.info pihl.se
mmoguider.ru psoriasis-file.trade
parlament.biz scanner-john.top
screentoolkit.com taraz.xkaz.org
streetfooduncovered.com vrnelectro.ru
uchebavchehii.ru xn--80ab4aa2g.xn--p1ai

912
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
007angels.com 00webcams.com
03p.info 1-99seo.com
1-free-share-buttons.com 1688.com
11235813.webzdarma.cz 2daytrendingnews.com
178evakuator178.ru 2itech.ru
1millionusd.xyz 300richmond.co.nz
20pascals.nl 3dracergames.com
2drittel.de 3xforum.ro
3w24.com 888.com
7yue.org a.pr-cy.ru
7zap.com aa9d046aab36af4ff182f097f840430d51.com
991mostfm.co.id abcdeg.xyz
9med.net abctoppictures.net
a96527gi.beget.tech above.com
aa9bd78f328a6a41279d0fad0a88df1901.com acheterviagrafr24.com
abcdefh.xyz adpremium.org
abovetherivernc.com adserver-e7.com
actionnooz.com adsfresh.men
adcash.com advocatemsk.ru
adclickservice.com advokateg.ru
adclickthru.net agadelha.com.br
adelly.bg alert-fdm.xyz
ads-seo.men alert-hgd.xyz
alevinefesleri.com allblogroll.com
alfapro.ru alphacarolinas.com
alienwheel.es am15.net
alive-ua.com amazon-seo-service.com
allfinweb.com andpolice.com
allpdfmags.net android-style.com
allwidewallpapers.com androids-store.com
alyeskaresort.com animalia-life.club
ample-awards-today.us anime.dougasouko.com
animal-drawings.com apartment.ru
animaltoplist.com api.stathat.com
apartmentbay.ru apparel-offer.com
apiallgeniusinfo-a.akamaihd.net appearance-cool.com
articlesdirectoryme.info appfixing.space
appleid-verification.com
approved.su
apxeo.info
ardimobileinfo.ml
arquapetrarca.info
artpicso.com artpicso.com
atelielembrancaqueencanta.com.br as5000.com
atleticpharm.org asrvvv-a.akamaihd.net
aussie-prizes.com atlant-auto.info
auto4style.ru avkzarabotok.info
autotrends.today
autovideobroadcast.com
avon-severozapad.ru avon-severozapad.ru
awency.com avtointeres.ru
azadnegar.com azadnegar.com
backgroundpictures.net babyfactory.fr
baksman.com
balkanfarma.org balkanfarma.org
batanga.net balla-boo.se
beget.tech banan.tv
belsetirehafi.tk beauty-b0x.pl
best-seo-software.xyz benchmarkcommunications.co.uk
best-seo-solution.com best-way.men
bestcoin.cc bestbrainprod.win
bestofferswalkmydogouteveryday.gq
besttorrentknifta.weebly.com besttorrentknifta.weebly.com
bezsporno.ru betwinservice.com
bharatdefencekavach.com bimatoprost-careprost.com.ua
bio.trade-jp.net bio-market.kz
biocn.dx.am bioscorp.ru
bitnote.co bizru.info
bjgugu.net.ua blackle.com
blog.f00kclan.de bloggerads.net
blog.yam.com blogos.kz
bloggedporn.com blogrankers.com
bloke.com blogs.rediff.com
bookmark4you.com.biz blogsfunky672.weebly.com
boole.onlinewebshop.net bmw-ark.ru
bosefux.esy.es bobinoz.com
brains2.biz boeuklad.com
brandimensions.com brainzpod.win
bristolhotel.com.ua brandehk.dk
brimstonehillfortress.org
burn-fat.ga
bus-offer.com bus-offer.com
calvet.altervista.org buytizanidineonline.blogspot.com
buzzurl.jp
bvlgaribracelet.xyz
bzero1jewelry.net
c-gainsbourg.com
callstevens.com
cand.jp
candy-glam-hp.com
canzoni.ru
capsderfudd.tk
carabela.com.do
carina-sy.de
cartierlovestore.com
cartierring.xyz cartierring.xyz
cashkitten-a.akamaihd.net cartiertrinity.xyz
casite-513049.cloudaccess.net cbb1smartlist12.click
cbozhe.com cbcseward.com
cdncash.org cdn.walkme.com
censys.io cenoval.ru
chatroulette.life cheapmoncler.pw
chlooe.com chelyabinsk.xrus.org
christianlouboutinoutlet.win christianlouboutinreplica.win
christianlouboutinshoes.xyz cialischstgerts.com
cialis-samples.com cialisovercounteratwalmartusa.com
ciproandtizanidine.blogspot.com cindymatches.com
cleansearch.net clicksor.net
clickbank.net
clipartnew.com clipartnew.com
co.lumb.co clmforexeu.com
compliance-andrew.xyz cntravelre.com
compliance-don.xyz colehaanoutlet.store
compliance-olga.top com-supportcenter.website
compliance-walter.top comissionka.net
copypaste.traderzplanet.in compucelunlock.net
cosmeticswomens-womensports.rhcloud.com conboy.us
country-chic.ru connectingsingles.com
crynet.cc connectionstrenth.com
cubook.supernew.org consorzioilmosaico.org
copblock.org
cranly.net
crazyboost.pro
customcollegeessays.net
d-black.bz d-black.bz
dafatiri.com dafatiri.com
darkbooks.org daptravel.com
date-withme.com data.vtc.pw
dengi-pod-zalog-nedvizhimosti.ru davidovic.info
digital-sale.su daydream-studio.ru
dirtpics.pw dcj-nn.ru
discovertreasurenow.com decenttools.com
div.as detskie-konstruktory.ru
domaincdn.xyz detskie-zabavi.ru
donna7753191.ru diesel-parts28.ru
dorratex.tn digital-video-processing.com
download-of-the-warez.blogspot.com dojki-hd.com
domainsatcost.com
dotmass.top
doxysexy.com
doyouknowtheword-flummox.ml doyouknowtheword-flummox.ml
dprtb.com drugstoreforyou.com
drunkenstepfather.com druzhininevgeniy63.blogspot.com
drunkmoms.net
drupa.com
druzhbany.ru
dtnlyss.com
dynainbox.com dynainbox.com
e-commerce-seo.com e-commerce-seo.com
easycommerce.cf easycommerce.cf
efkt.jp ecommerce-seo.org
ekn-art.se educatemetv.com
elektir.ru eduserver.net
elektronischezigarettekaufen2.tumblr.com elektronischezigarettekaufen1.beeplog.com
elite-sex-finders.com elstal.com.pl
elitepcgames.com en.altezza.travel
elvel.com.ua englishdictionaryfree.com
emergencyneeds.org eralph.tk
energydiet24.ru erot.co
enginebay.ru
esnm.ru esnm.ru
esopini.com
eu-cookie-law.info eu-cookie-law.info
eurosamodelki.ru
everypony.ru
evidencecleanergold.com
evogarage.com
exchangeit.gq
exct.net exct.net
exoneration-project.us extlabs.io
extremal-blog.com
eyemagination.com eyemagination.com
ezigarettekaufen2.mpbloggar.se facebook-mobile.xyz
ezigarettenkaufen1.deviantart.com fanoboi.com
familienzahnaerzte.com
fast-wordpress-start.com fast-wordpress-start.com
ferretsoft.com fetlifeblog.com
fetroshok.ru filefilter.weebly.com
findanysex.com fillmewithhappiness.com
fishtauto.ru
fiuxy.com
fiverr.com
fix-website-errors.com fix-website-errors.com
flooringinstallation-edmonton.com floridamobilebillboards.com
forensicpsychiatry.ru focalink.com
formulaf1results.blogspot.com forex21.ru
formularz-konkurs.tk forum69.info
fortevidyoze.net forums.d2jsp.org
forum.poker4life.ru foto-basa.com
forum20.smailik.org frbizlist.com
forzeronly.com free-laptop-reward.com
fr.netlog.com free-share-buttons.top
free-fbook-traffic.com freelinkbuilding.website.tk
free-stock-illustration.com freevpn.space
freegamesplay.online gambarkatabaru.com
funcrushgames.com gamedayhouse.com
funnel.co.za gameplexcity.com
fz139.ttk.ru gaming-journal.com
gabeshop.ru gaytube.com
gael-s.ru gazobeton-p.com.ua
galaxy-family.ru geckoandfly.com
gallerylisting.com gemgrab-a.akamaihd.net
games.kolossale.ru gepezz.info
gamezblox.com germes-trans.com
gardene.ru
geileweiber.tk
gelezki.com
get-seo-domain.com
getlaid-xxxhookupdirect.com
getmyads24.com getmyads24.com
getoutofdebtfree.org gettpromos.com
gidonline.one
gilbertbanda.net
gk-atlant.info
gk170.ru
glcomputers.ru glcomputers.ru
globalscam.ga gloverid.site
gribokstop.com goforexvps.com
grizzlysgrill.com gojiberriess.apishops.ru
gsasearchengineranker.top googlemare.com
gsbs.com.ua googst2.ru
hanink.biz.ly gorabagrata.ru
hasshe.com gpms.org.my
hdwalls.xyz greamimgo.pw
hdxnxxtube.mobi greenzaim.ru
hermesbracelets.xyz gsasearchengineranker.pw
hesteel.pl gsasearchengineranker.site
hidefiles.org gwagka.com
histock.info hacktougroup.ru
hkdiiohi.skyrock.com harmonyglen.us
hatedriveapart.com
hermesbirkinhandbagoutlets.com
hermesreplica.win
hifidesign.ru
historichometeam.com
hitmuzik.ru
homedecoguide.info
homeinns.com
homemature.net homemature.net
honyaku.yahoofs.jp homik.pw
hopeonthestreet.co.uk
hoste.octopis.com hoste.octopis.com
hotsocialz.com houseofrose.com
howtoclean.club hoverboard360.at
hptwaakw.blog.fc2.com id-forex.com
i-service.kz idealtits.net
ihtec2019.org igru-xbox.net
ilbe.club igtools.club
imagez.co ilmen.net
iminent.com impisr.ru
impresagaia.it in-tandem.co
inboxdollars.com inboundlinks.win
instasexyblog.com indiakino.net
intelhdgraphicsgtdrive6w.metroblog.com innoslicon.com
invest-pamm.ru interesnie-faktu.ru
investmac.com interior-stickers.ru
investyb.com iptool.xyz
ipchicken.com irunfar.com
iqupdatetmz.win isitwp.com
istanbulit.com isoveti.ru
ixora.pro istizanidineanopiate.blogspot.com
iyasimasennka.com istripper.com
jacago.com it-max.com.ua
itch.io
ivearchenceinflu.cf
jasonpartington.com jasonpartington.com
jennyfire.ru
jetsli.de jetsli.de
joessmogtestonly.com jwcialislrt.com
johnnyhaley.top kamalsinha.com
journalhome.com kandidos.com
justucalling32211123456789.tk karpun-iris.ru
kabbalah-red-bracelets.com karusel-market.ru
kartiniresto.com keki.info
kerei.ru kerei.ru
keyhantercume.com keywords-monitoring-success.com
keywordblocks.com keywordsdoctor.com
keywordsuggests.com
kinky-fetishes.com kinky-fetishes.com
kino-rating.ru kino2018.cc
kinobaks.com kino2018.club
kinostorm.net knogg.net
kinotorka.ru kometars.xyz
klejonka.info kostenlos-sexvideos.com
klikbonus.com kosynka-games.ru
kovesszucs.atw.hu krasula.pp.ua
kretpicf.pw ladov.ru
kriokomora.info
kupit-adenu.ru
lalalove.ru
landinez.co
landreferat.ru
landscaping.center
larger.io larger.io
law-check-nine.xyz laserpen.club
lawyers.cafe lasvegaslockandsafe.com
legalrc.biz legionalpha.com
letmacworkfaster.site light.ifmo.ru
life-instyle.com lion.bolegapakistan.com
life.biz.ua lion.playtap.us
linkbux.ru
liquimondo.com liquimondo.com
livejasmin.com littleberry.ru
livetsomudvekslingstudent.bloggersdelight.dk magento-crew.net
lkbennettstore.online magnetpress.sk
lol-smurfs.com makedo.ru
lombia.co manipulyator-peterburg.ru
longadventure.com mararoom.ru
love-baby.cz marktforschung-stuttgart.com
lunamedia.co matb3aa.com
lunchrock.co mebel-vstroika.ru
m3gadownload.pl medizinreisen.de
magnetic-bracelets.ru meet-flirt-dating.com
mainlinehobby.net megahdporno.net
manyresultshub-a.akamaihd.net megakino.net
marcoislandvacations.net meteostate.com
marmotstore.online microsearch.ru
maslenka.kz mir-business-24.ru
masturbate.co.uk mmog-play.ru
maxthon.com mncrftpcs.com
mearns-tractors.co.uk mnogabukaff.net
meble-bogart.info mobsfun.net
medtherapy.ru monarchfind-a.akamaihd.net
meetlocalchicks.com moncleroutletonline.xyz
members.ghanaweb.com monsterdivx.com
menhealed.net moviemail-online.co.uk
mexicosleevegastrectomy.com moviezbonkerssk.cf
michaelkorsoutletstore.net moz.com
midst.eu mp3ritm.top
mielec.pl msk.afora.ru
mlf.hordo.win mug-na-chas-moscow.ru
mnimmigrantrights.net muzaporn.com
moetomnenie.com my-bc.ru
mompussy.net mydearest.co
monclerjacketsoutlet.pw mydirtyhobby.com
montazhnic.ru naperehresti.info
morlat.altervista.org narkologiya-voronezh.ru
mowser.com naturtreenspicerx.pw
mural.co net-profits.xyz
muz-baza.net newhairstylesformen2014.com
mybinaryoptionsrobot.com newstaffadsshop.club
myonigroup.com nhl17coinsforps3.gratisblog.biz
myperiod.club nikitabuch.com
napalm51.nut.cc notaria-desalas.com
naturalbreakthroughsresearch.com
new-post.tk
newfilmsonline.ru
news-speaker.com
newyorkhotelsmotels.info
nfljerseysforsalewholesaler.com
ngps1.ru
nicefloor.co.uk
niki-mlt.ru
nmrk.ru
no-rx.info
noclegonline.info
nordstar.pro
ntic.fr ntic.fr
nullrefer.com obiavo.net
nyfinance.ml of-ireland.info
offside2.5v.pl offtime.ru
olympescort.com old-rock.com
onlinewritingjobs17.blogspot.ru omsk.xrus.org
openlibrary.org oneshotdate.com
orion-code-access.net online247.ml
paccohichetoti.ml onlinefilmz.net
paintingplanet.ru onlyporno.ru
onthemarch.co
ops.picscout.com
orion-v.com
osb.se11.ru
otbelivanie-zubov.com
ozas.net
par-fallen.ga par-fallen.ga
parlament.biz paydayloanslocal.com
pechikamini.ru pdamods.ru
penisvergrotendepillennl.ovh pensplan4u.com
peugeot-club.org pepperstyle.ru
phpdevops.com petedrummond.com
piccshare.com photosaga.info
picmoonco.pw php-market.ru
picswe.com picquery.com
pix-hd.com playlott.com
plastgran.com ploenjitmedia.azurewebsites.net
pod-muzyku.club plohaya-kreditnaya-istoriya.ru
poffet.net pogodnyyeavarii.gq
poiskzakona.ru
poligon.com poligon.com
popup.matchmaker.com polytopesexempt.com
pornokajf.com popserve.adscpm.net
pornosee.info porn-w.org
pornoted.com porndairy.in
powitania.pl porno-video-chati.ru
pp-budpostach.com.ua pornoelita.info
privetsochi.ru pozdrawleniya.ru
profitfx.online prohoster.info
prospekt-st.ru promoheads.com
prostitutki-novgoroda.org proxyelite.biz
prostitutki-odessa.org pulseonclick.com
puteshestvennik.com puserving.com
quickchange.cc pussysaga.com
q-moto.ru
qld10000.net
quangcaons.com
rankexperience.com rankexperience.com
ranking2017.ga ranksays.com
razvratnoe.org ranksonic.info
rcb101.ru razleton.com
realnye-otzyvy.info referencemoi.com
refudiatethissarah.info registry-cleaner.net
registrysweeper.com registrydomainservices.com
remybutler.fr rentaremotecomputer.com
rep-am.com ru-dety.ru
restaurantlescampi.com runstocks.com
rewardit.com rvtv.ru
richinvestmonitor.com sa-rewards.co.za
ring4rhino.com sadaholding.com
royalvegascasino.com saecsa.co
ruclicks.com sakhboard.ru
rus-pornuha.com saveriopiazza.it
sabizonline.com sbwealthsolutions.ca
sajatvelemeny.com scanner-margo.top
sale-japan.com scanner-mary.top
salutmontreal.com scanner-nelson.top
sanatorrii.ru scanner-willy.top
sbdl.no screentoolkit.com
scanner-ivan.top scrnet.biz.ua
scanner-jessica.top se-welding.ru
searchinterneat-a.akamaihd.net seonetwizard.com
searchkut.com sevendays.com.ua
searchwebknow-a.akamaihd.net sex-pr.net
selectads.men sexad.net
seoexperimenty.ru sextracker.be
seoheap.com sexyteens.hol.es
seorankinglinks.us ship-marvel.co.ua
serpstat.com signal03.ru
sex-znakomstva.online simplesite.com
sexkontakteao.info sinel.info
shijian.ac.cn sispe.com.br
shitting.pro
shlyahten.ru
shoppingjequiti.com.br
shopwme.ru
shtaketniki.kz
sicfor.bcu.cc
simon3.ru
site-speed-checker.site site-speed-checker.site
slowmac.tech slavkokacunko.de
snyatie-lomki-v-stacionare.ru slujbauborki.ru
social-fun.ru soaksoak.ru
social-vestnik.ru social-buttons.xyz
socialbutton.xyz
socialsignals24.com socialsignals24.com
sosdepotdebilan.com sockshares.tv
speechfoodie.com soft-program.com
spidtest.org soft1.ru
stard.shop sonata-arctica.wz.cz
statistici.ro spabali.org
steelmaster.lv specialfinanceoffers.com
sterva.cc spl63.fr
stoki.ru sport-video-obzor.ru
stopnarco.ru spymac.net
strana-solnca.ru streha-metalko.si
strongsignal-a.akamaihd.net sudexpert66.ru
stuffpride.com sumo.com
sugarkun.com surgut.zrus.org
sureone.pro swingerseiten.com
surfbuyermac.com syndicate.fun
swinger-mobil.net takeprofitsystem.com
swsociety.se tasteidea.com
sygraem.com tattooha.com
tampabaywatch.org tengohydar.tk
telegraf.by tetracsaudi.com
template-kid.com tgtclick.com
thaismartloan.com thedownloadfromwarez.blogspot.com
thecounter.com theendivechronicles.com
thegalerie.eu timer4web.com
thegolfclub.info tizanidineandcipro.blogspot.com
thepantonpractice.co.uk tizanidinedosageforsleep.blogspot.com
titan-cloud.life tizanidinehcl2mg.blogspot.com
tizanidine4mg.blogspot.com tizanidinesideeffects.blogspot.com
tizanidinehclsideeffects.blogspot.com tizanidinevsvalium.blogspot.com
tizanidinesleepaid.blogspot.com tmm-kurs.ru
top-study.work tmtrck.com
topvidos.ru top-deal.com.pl
tracklead.net topdownloads.ru
tracksurf.daooda.com topsiteminecraft.com
trafficmp.com toursmaps.com
truck-addzilla.life tpu.ru
track112.site
trackzapper.com
tradgardspartner.se
traxdom.ru
trustaffs.com trustaffs.com
ttrraacckkrr.com tupper-shop.ru
ttsq.fr turizm.bz
turn-up-life.life turkeyreport.tk
turvgori.ru txxx.com
tvteleport.ru u17795.netangels.ru
ublaze.ru udsgame.online
ugguk.online
ukrtvory.in.ua ukrtvory.in.ua
ultimateclassicrock.com ultimateclassicrock.com
ultimatesetnewfreeallsoftupgradesystems.pw unblocksit.es
unimodemhalfduplefw.pen.io univerfiles.com
unitygame3d.com universals.com.ua
uptime-delta.net
uptimechecker.com uptimechecker.com
urccvfmc.bloger.index.hr urll.eu
vashsvet.com videojam.tv
vavilone.com videooko.weebly.com
vchulkah.net videositename.com
videonsk.com vilingstore.net
vinylvault.co.uk vip-file.com
vipsiterip.org vip-parfumeria.ru
viven.host.sk vitalads.net
vrotike.ru voloo.ru
vsesubwaysurfers.com voluumtrk.com
vseuznaem.com vseuznaem.com
vzlomtw.com warning.or.kr
w3data.co
w7s.ru
warningwar.ru
web-analytics.date
webcamtalk.net
webextract.profound.net
website-speed-check.site website-speed-check.site
website-speed-up.site wellcome2slovenia.ru
weclipart.com whengirlsgowild.com
wejdz-tu.pl wikes.20fr.com
while.cheapwebsitehoster.com wildworld.site
wonderfulflowers.biz wufak.com
wordkeyhelper.com www.solartek.ru
worst-sites.online wygraj-teraz.com
wurr.voila.net xaylapdiendanang.com
www.bookmaker-bets.com xchat26.myfreecams.com
x5market.ru xitjw.info
xblog.in xmlinde.com
xcc24.pl xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xcombear.ru xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xlovecam.com xn----9sbubg3ambdfl1j.xn--p1ai
xn--80aafb2a.xn--p1ai
xn--80aikhbrhr.net xn--80aikhbrhr.net
xn--b1agm2d.net xsion.net
xn--d1acah0c.xn--p1ai
xn--d1aifoe0a9a.top
xnxxandxvideos.com
xxxnatelefon.ru xxxnatelefon.ru
yatrk.xyz yoopsie.com
yhit.press youbrainboost.asia
youdao.com youradexchange.com
youporn-forum.ga youtubologia.it
yourmovies.pl yunque.pluto.ro
yubikk.info zaobao.com.sg
yugk.net zarenica.net
yurgorod.ru zastenchivosti.net
zahvat.ru zed21.net
zajm-pod-zalog-nedvizhimosti.ru zeroredirect7.com
zawyna.ua zvooq.eu
zetmaster.ru zzbroya.com.ua
zigarettenonlinekaufen.tumblr.com
zootravel.com
zytpirwai.net

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00it.com
1-free-share-buttons.com
13tabs.com 13tabs.com
3w24.com 2020iscoming.info
7ooo.ru 214jaluwobapef.cf
7xc4n.com 2x2fan.ru
a96527gi.beget.tech 4replicawatch.net
adsloads.com 5kstore.com
adventureparkcostarica.com 83net.jp
agadelha.com.br 9icmzvn6.website
agapovdg.ru absugars.com
alessandraleone.com adidas.frwebs.fr
alpinism.ru agreda.pluto.ro
anlimebel.kiev.ua alekseevec.ru
aosheng-tech.com alfapro.ru
apartmentbay.ru allnews24.in
apilinkswiftco-a.akamaihd.net alltheviews.com
apple.com-webbrowsing-security.review amateurlivechat.org
autochoixspinelli.com aosexkontakte.net
avitocars.ru apelsinnik.website
azbukadiets.ru artdestshop.eu
babieca.com asrv-a.akamaihd.net
bbsoldes.fr astrochicks.com
beatifullandscape.co autoseo-traffic.com
berdasovivan.ru aviva-limoux.com
axbocz.net
azbuka-mo.ru
azlex.uz
bang-hotties.com
bankiem.pl
bedandbreakfast.com
bestadbid.com
besthoro.ru besthoro.ru
bet-prognoz.com bestofpicture.com
blackle.com betune.onlinewebshop.net
bluesalt.co bif-ru.info
bolegapakistan.com bjetjt.com
boom-celebs.com bloglag.com
bracketsmackdown.com bonkers.name
brimstonehillfortress.org boo-arts.com
buscarfoto.com braindaily.xyz
business-suggested.tk braip.com.br
businesxxl.com brateg.xyz
buzzonclick.com bristolhotel.com.ua
calstaterealty.us bugshoot.cn
candcstuccoandstone.com buyhoverboard.com
candlehandmade.com buyk.host.sk
cdncash.org cactussoft.biz
cementaresearch.se camdolls.com
chinajerseyswholesalecoupons.com cardsharp1.ru
clickpapa.com cartierlovestore.xyz
clickzzs.nl casinorewards.info
collegeessay19.blogspot.ru cellfun.mobi
countbertwistdisp26.soup.io centraletermice.us
d2jsp.org cezartabac.ro
demenageur.com cfacarrosserie74.com
diarioaconcagua.com cheapcigarettesc.info
dignitasdata.se cjs.com.ru
dirtpics.pw classicakuhni.ru
dolohen.com clmforexeu.com
domainanalyzing.xyz com-onlinesupport.host
drunkenstepfather.com com-secure.download
compliance-checker.info
compliance-john.top
converse.ddsoldes.fr
coolgamechannel.com
cornomase.win
counterbot.com
crackzplanet.com
cryptoswap.biz
cubook.supernew.org
customsua.com.ua
ddlmega.net
deadlinkchecker.com
dellalimov.com
deluxedumps.com
deplim.com
dogbreedspicture.net
druzhininevgeniy63.blogspot.com druzhininevgeniy63.blogspot.com
e-buyeasy.com e-c.al
eduinfosite.com e705.net
elektrozigaretten1.webs.com en.home-task.com
enskedesquashclub.se enbersoft.com
envaseslotusama.com eralph.tk
erotik-kostenlos.net essay-writing.work
expolicenciaslatam.co fanoboi.com
fanpagerobot.com fatmaelgarny.com
fidelityfunding.com finstroy.net
flex4launch.ru foodgid.net
flytourisme.org forminecrafters.ru
free-social-buttons.com forums.toucharcade.com
free-today.com gamebackyard.com
gaz-voshod.ru gameplexcity.com
gazobeton-p.com.ua gamezblox.com
gazporno.com gay.adultgalls.com
gdebestkupit.ru ghazel.ru
gonextmedia.com go2jump.org
goodhumor24.com gsbs.com.ua
googst2.ru hosting-tracker.com
greatdealshop.com hotcore.info
greatgrace.ru hotgirlhdwallpaper.com
gsasearchengineranker.site housediz.com
guarrasdelporno.xxx humanorightswatch.org
hannasolution.ru ilmen.net
hdpixion.com
healmytrauma.info
hentai-manga.porn
highstairs-a.akamaihd.net
hobbyhuren-datenbank.com
hol.es
homeart.space
hotdl.in
ilo134uloh.com
ilovevitaly.xyz ilovevitaly.xyz
imagefinder.site
imagui.eu
incest-ru.com
indetiske.ya.ru indetiske.ya.ru
instasexyblog.com indo-export.ru
investsuccess.org isoveti.ru
ispaniya-costa-blanca.ru jerseyswholesalechinalimited.com
itrevolution.cf jikoman.info
itsdp3.com kemerovo.zrus.org
jennyfire.ru kinohit1.ru
jewelryandfiligree.com labvis.host.sk
jmat.cn lalalove.ru
ketrzyn.pl lasvegaslockandsafe.com
kinky-fetishes.com lezbiyanki.net
knowyournextmove.com linkredirect.biz
koral.se liupis.com
kretpicf.pw livingcanarias.com
lacapilla.info lonfon.xyz
lankarns.com lostfilm-online.ru
letolove.ru lovi-moment.com.ua
lignofix.ua luchshie-topcasino.ru
limads.men luciddiagnostics.in
link.ac magiadinamibia.blogspot.com
liumimgx.pw magicdiet.gq
louboutinreplica.pw mantramusic.ru
magazintiande.ru mararoom.ru
maglid.ru marcoislandvacations.net
magnetpress.sk max-p.men
maladot.com members.ghanaweb.com
map028.com meteostate.com
marktforschung-stuttgart.com michaelkorsoutletstore.net
medicineseasybuy.com militarysale.pro
methodsmarketing.com missclub.info
mexicosleevegastrectomy.com monetizationking.net
micasainvest.com monetizer.com-01.site
millionare.com moroccosurfadventures.com
mobot.site moskva.nodup.ru
moesen-ficken.com mustwineblog.com
mowser.com mygameplus.com
music7s.me myplaycity.com
mydoctorok.ru nac-bearings.ru
myghillie.info narkologiya-belgorod.ru
myxdate.info newyorkhotelsmotels.info
narkologiya-penza.ru notaria-desalas.com
narutonaruto.ru obnal.org
net-radar.com offtime.ru
nudepatch.net optom-deshevo.ru
nyfinance.ml paparazzistudios.com.au
oakleyglassesonline.us pdns.cz
obiavo.by pendelprognos.se
odesproperty.com poptool.net
onlinadverts.com popup-hgd.xyz
online-x.ru pornogad.com
onlinetvseries.me pornotubexxx.name
open-odyssey.org pornoxxx.com.mx
oranga.host.sk posible.net
osb.se11.ru powitania.pl
page2rss.com pp-budpostach.com.ua
pensplan.com prostitutki-kharkova.org
pills24h.com prostitutki-yaroslavlya.org
pipki.r.acdnpro.com qwertty.net
playboyfiles.xblog.in rankia.com
polytopesexempt.com ranksonic.com
pooleroadmedicalcentre.co.uk razleton.com
porno-chaman.info razyboard.com
porno-video-chati.ru recordpage-a.akamaihd.net
pornosemki.info
potoideas.us
praisong.net
priora-2.com
promgirldresses.xyz
pulseonclick.com
pussyspace.net
puzzleweb.ru
pxhdwsm.com
quizzitch.net
rasteniya-vs-zombi.ru
refudiatethissarah.info
reimageplus.com
reklama-i-rabota.ru
remont-fridge-tv.ru remont-fridge-tv.ru
responsive-test.net research.ifmo.ru
retailwith.com rewardit.com
rewardpoll.com ring4rhino.com
rollercoin.com rosbalt.com.ua
rostov.xrus.org rostov.xrus.org
ru-dety.ru sack.net
russia-tao.ru sajatvelemeny.com
ryetaw.com sanyuprojects.com
sady-urala.ru savefrom.com
saecsa.co sbt-aqua.ru
scanmyphones.com scanner-donald.top
scanner-brian.top scanner-irvin.top
securesmrt-dt.com scripted.com
sexcamamateurchat.com search.1and1.com
sexytrend.ru seo-smm.kz
shemalegalls.blogporn.in seo18.su
sherlock.se sexspornotub.com
shmetall.com.ua sheerseo.com
shohanb.com shop.acim.org
shopcheermakeup.info shopwme.ru
sibdevice.ru simpoed.ufop.br
signal03.ru sitechecker.pro
site-speed-checker.site siteripz.net
siteaero.com slonechka.ru
skachat-besplatno-obrazcy.ru smartpet.ru
smartadserver.com smsactivator.ru
smstraf.ru snow.nvr163.com
snip.to snts.shell-pmr.ru
socialmadesimple.com sortthemesitesby.com
socialseet.ru srgwebmail.nl
softxaker.ru stanthonyscatholicchurch.org
spb.afora.ru stockquotes.wooeb.com
steebook.com stoki.ru
studiokamyk.com.pl symphonyintegratedhealthcare.com
surflinksmedical.com telefonsexsofort.tk
svarbit.com telesvoboda.ru
taihouse.ru the-usa-games.blogspot.com
tattomedia.com thiegs.reco.ws
tengohydar.tk timeallnews.ru
texbaza.by tn811.us
thepornsex.org toloka.hurtom.com
titslove.yopoint.in tonivedu.it
tizanidinehcl2mg.blogspot.com tracxn.com
tizanidinehcl2mgtablet.blogspot.com truck-addzilla.life
tizanidinerestlesslegsyndrome.blogspot.com ttrraacckkrr.com
torrents.life upstore.me
traxdom.ru urll.eu
trk-4.net varikozdok.ru
turist-strani.ru vedomstvo.net
ultramart.biz
umaseh.com
umityangin.net
unacittaconte.org
url-extractor.xyz
velen.io velen.io
viagra.pp.ua venta-prom.ru
video--production.com vigrx-original.ru
video-production.com vkmusics.ru
videochat.mx vksex.ru
videojam.tv voloo.ru
vitalads.net vostoktrade.info
voluumtracker1.com vzlomfb.com
wcb.su vzubah.com
webkeyit.com walkme.com
webtherapy.ru warning.or.kr
wemarketing.se webmasterhome.cn
wholesalecheapjerseysfree.com webmonetizer.net
wholesalenfljerseys.us.com westum.se
winwotgold.pl whatzmyip.net
woodyguthrie.se wholesalejerseys-cheapest.com
ws.ampower.me winterclassichockeyjerseys.com
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai woman-tampon.ru
xn--d1acah0c.xn--p1ai womensplay.net
xn--e1aggki3c.xn--80adxhks writersgroup580.web.fc2.com
yourmovies.pl xdoza.com
zigarettenonl.canalblog.com xn--b1ag5cfn.xn--p1ai
zkjovpdgxivg.ga xn--l1aengat.xn--p1ai
znaniyapolza.ru xxx-cam.webcam
xxxguitars.com
youraticles.pl
yourtemplatefinder.com
youtuhe.com
zaimhelp.ru
zambini.ru
zeroredirect5.com
zigarettenonlinekaufen1.bloog.pl
zone-kev717.info
zzlgxh.com

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.58 0; 10.1.0.165 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3506 ### Version: V4.2022.12.3507
### Updated: Sun Dec 4 10:03:30 UTC 2022 ### Updated: Tue Dec 6 10:12:42 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3506 ### Version: V4.2022.12.3507
### Updated: Sun Dec 4 10:03:30 UTC 2022 ### Updated: Tue Dec 6 10:12:42 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##