BBBike
CensysInspect
CheeseBot
Custo
HTTP::Lite
LinkpadBot
Openfind
PageAnalyzer
Web Fetch
WebStripper