ASPSeek
Cogentbot
Dirbuster
Jbrofuzz
OrangeSpider
Pu_iN
Scrapy
VeriCiteCrawler
WebStripper
webprosbot