Spondulas is browser emulator and parser designed to retrieve web pages for hunting malware. It supports generation of browser user agents, GET/POST requests, and SOCKS5 proxy. It can be used to parse HTML files sent via e-mail. Monitor mode allows a website to be monitored at intervals to discover changes in DNS or content over time. Autolog mode creates an investigation file that documents redirection chains. The retrieved web pages are parsed for links and reported to an output file. More information is available on the wiki.
Support for GET and POST methods
Parsing of retrieved pages to extract and categorize links
Support for HTTP and HTTPS methods
Support for non-standard port numbers
Support for the submission of cookies
Support for SOCKS5 proxy using TOR
Support for pipe-lining (AJAX)
Monitor mode to poll a website looking for changes in DNS or body content
Input mode to parse local HTML files, e.g., e-mailed forms
Automatic conversion of GZIP and Chunked encoding
Automatic IP address Look-up
Selection or generation of User Agent Strings
Automatic creation of an investigation file
