454 commits

Author	SHA1	Message	Date
Mitchell Krog	ec919d888f	Bot (+) Quick-Crawler ... Referrer (+) scrapinghub.com Closes: #181	2018-06-08 10:28:26 +02:00
Travis	cdc76609d9	V3.2018.06.1112 [ci skip]	2018-06-07 11:21:22 +02:00
Mitchell Krog	ad5750f32d	Bot (+) Referrer (+) ... - PXBroker / db.speedup.gdn Closes: #180	2018-06-07 11:12:56 +02:00
Travis	a9ef73b12b	V3.2018.06.1109 [ci skip]	2018-06-06 16:46:52 +02:00
Mitchell Krog	fe2647c0e6	User-Agent + - Mr.4x3 ... Closes: #177	2018-06-06 16:36:40 +02:00
Mitchell Krog	99a41f81e3	Remove "NYU" already covered by "scan.lol" ... -REF: ab1c59ad02 (commitcomment-29255444)	2018-06-06 08:40:45 +02:00
Mitchell Krog	48c011e68e	Merge pull request #172 from JayBizzle/patch-2 ... Remove user agent false positive	2018-05-17 07:50:38 +02:00
Travis	3d6e497acf	V3.2018.05.1106 [ci skip]	2018-05-17 07:48:21 +02:00
Mark Beech	f5fdf6ec02	Update bad-user-agents.list	2018-05-16 21:44:27 +01:00
Mark Beech	ee80803bd8	Remove genuine websites	2018-05-16 20:21:03 +01:00
Travis	d41078c8ab	V3.2018.05.1103 [ci skip]	2018-05-11 13:21:30 +02:00
Mitchell Krog	ef74e9773d	Referrers (+) ... seocdvig.ru seorussian.ru	2018-05-11 13:12:02 +02:00
Travis	d89f99c8d1	V3.2018.05.1102 [ci skip]	2018-05-09 10:31:26 +02:00
Mitchell Krog	84b709cfb3	Referrers (+) ... scat.porn unlimitdocs.net eduserver.net filesdatabase.net docs4all.com store-rx.com baditri.com pornonik.com filesclub.net office2web.com officedocuments.net xtrafficplus.com incomekey.net kinosed.net	2018-05-09 10:23:06 +02:00
Mitchell Krog	b907c6bf39	Merge pull request #169 from lukeb/hotfix/eliasonrealty ... Remove false positive of legitimate Real Estate office in Wisconsin	2018-05-08 10:52:35 +02:00
Travis	0d98b74989	V3.2018.05.1100 [ci skip]	2018-05-08 10:23:24 +02:00
Mitchell Krog	5b885236d1	Referrers (+) ... eduinfosite.com popugaychiki.com bird1.ru brillianty.info 08800.top fordsonmajbor.cf of-ireland.info slavia.info thenews-today.info ivearchenceinflu.cf m292.info parajumpersjakkesalgnorge.info beremenyashka.com docs4all.com documentbase.net edudocs.net scat.porn	2018-05-08 10:16:08 +02:00
Luke Brookhart	6ebf236b2b	Remove false positive of legitimate Real Estate office in Wisconsin	2018-05-07 09:41:28 -04:00
Travis	fd87312e39	V3.2018.04.1098 [ci skip]	2018-04-21 13:23:48 +02:00
Mitchell Krog	23fb3f62c6	Referrers (+) / Bots (+)	2018-04-21 13:17:11 +02:00
Travis	b773a1ed17	V3.2018.04.1097 [ci skip]	2018-04-20 13:27:50 +02:00
Mitchell Krog	fcce5b719b	Referrers (+)	2018-04-20 13:20:51 +02:00
Travis	1d091ecc17	V3.2018.04.1096 [ci skip]	2018-04-20 11:50:53 +02:00
Mitchell Krog	261e32cb36	Referrers (+)	2018-04-20 11:44:05 +02:00
Travis	5d04fa114d	V3.2018.04.1095 [ci skip]	2018-04-20 11:23:39 +02:00
Mitchell Krog	b4795ed886	Bots (+) ... OutclicksBot	2018-04-20 11:16:50 +02:00
Travis	9ead97e9d6	V3.2018.04.1094 [ci skip]	2018-04-20 09:28:10 +02:00
Mitchell Krog	4f83090d8c	Bots (+) ... Zauba Search for Research; http://www.zauba.io/; admin@zauba.io - Does not request robots.txt	2018-04-20 09:21:32 +02:00
Travis	83596094ae	V3.2018.04.1091 [ci skip]	2018-04-14 16:58:58 +02:00
Mitchell Krog	5a75d55c5e	Merge pull request #153 from StarkRavingZA/master ... Request Re-Addition of RankingBot	2018-04-14 16:55:41 +02:00
Travis	1611ce1591	V3.2018.04.1090 [ci skip]	2018-04-13 09:00:19 +02:00
Mitchell Krog	c56c381dd3	Bots (+) ... IndeedBot Closes: #155	2018-04-13 08:53:37 +02:00
Travis	330934f171	V3.2018.04.1089 [ci skip]	2018-04-10 13:15:07 +02:00
Mitchell Krog	4b789167b7	Referrers (+) ... dollartree.info magnetic-bracelets.ru coaching-netz.info unlimitdocs.net formulaantiuban.com regionshop.biz animalia-life.club kicknights.gq kurbappeal.info ontargetseo.us beatifullandscape.co deplim.com leftofcentrist.com foto-basa.com machicon-akihabara.info gloverid.site canvas.gloverid.site hoztorg-opt.ru rfid-locker.co svolze.com mailemedicinals.com incomekey.net xtrafficplus.com homeart.space docsportal.net documentsite.net	2018-04-10 13:08:09 +02:00
Travis	bae8a933c6	V3.2018.04.1082 [ci skip]	2018-04-03 09:41:40 +02:00
Mitchell Krog	98623076e4	Bots (+) ... crawler4j	2018-04-03 09:35:30 +02:00
Travis	6052fd38f3	V3.2018.04.1081 [ci skip]	2018-04-03 09:35:30 +02:00
Mitchell Krog	30e570448c	Referrers (+) / Bots (+) ... REFERRERS ontargetseo.us intim-uslugi.info regionshop.biz urldelivery.com 2pxg8bcf.top pixshark.com BOTS crawler4j	2018-04-03 09:29:12 +02:00
Travis	6724f64c03	V3.2018.04.1080 [ci skip]	2018-04-02 15:36:05 +02:00
Mitchell Krog	bc4902f5ab	Referrers (+) ... tds-advert005.info beatifullandscape.co	2018-04-02 15:30:07 +02:00
StarkRavingZA	97fa50811e	Request Re-Addition of RankingBot	2018-03-31 16:47:40 +02:00
Travis	8ab7b68053	V3.2018.03.1078 [ci skip]	2018-03-31 12:27:12 +02:00
Mitchell Krog	aec2eeaaa8	Referrers (+) / Bots (+) ... REFERRERS veselokloun.ru edudocs.net exdocsfiles.com pillscheap24h.com animenime.ru sundrugstore.com seocheckupx.com game300.ru hccoder.info buntube.net filesclub.net saneyes.com compiko.info bitcoinpile.com docs4all.com infodocsportal.com mixx.com vzube.com yun56.co BOTS scan.lol NYU MauiBot	2018-03-31 12:21:10 +02:00
Travis	25fd2ae663	V3.2018.03.1074 [ci skip]	2018-03-29 09:23:06 +02:00
Mitchell Krog	4341d84d24	Bots (+) Closes: #146 ... crawler - crawler.feedback@gmail.com	2018-03-29 09:17:12 +02:00
Travis	4963244255	V3.2018.03.1073 [ci skip]	2018-03-27 11:40:03 +02:00
StarkRavingZA	fcdb5bf11e	Add new User Agents and Referer	2018-03-27 11:19:09 +02:00
Travis	a1aa66d827	V3.2018.03.1071 [ci skip]	2018-03-27 10:50:47 +02:00
Mitchell Krog	e3dd0f85c2	Referrers (+) ... hoztorg-opt.ru sildenafil-tadalafil.info exdocsfiles.com 1688.com slavia.info wanker.us allwidewallpapers.com documentbase.net picturesmania.com goodhumor24.com staticfs.host aibolita.com beachtoday.ru goodhousekeeping.com	2018-03-27 10:44:58 +02:00
Travis	0ce362a7dd	V3.2018.03.1070 [ci skip]	2018-03-26 09:37:33 +02:00