Luke Stephens (@hakluke)

Jan 3, 2020

3 min read

Introducing Hakrawler: A Fast Web Crawler for Hackers

Hakrawler Output Example



  • Easily chainable with other tools (accepts hostnames from stdin, dumps plain URLs to stdout using the -plain tag)
  • Collects URLs by crawling each page the application, following links
  • Collects URLs from wayback machine
  • Collects URLs from robots.txt
  • Collects URLs from sitemap.xml
  • Discovers new domains and subdomains belonging to the target as it finds them during the crawling process
  • Written in Golang
  • Variable scope can be set to narrow down or expand results
  • Can export results into files containing raw HTTP requests, which may be parsed by other tools such as SQLMap



Pentester | Hubby | Musician

Love podcasts or audiobooks? Learn on the go with our new app.