LinkChecker
LinkChecker is a Python script for checking your HTML documents for broken links. Features include: recursive checking, multithreaded, output can be colored or normal text, HTML, SQL, CSV or a GML sitemap graph, additionally reports download time for HTML pages, HTTP/1.1 and 1.0, HTTPS, FTP, mailto:, Gopher, Telnet and local file links are supported, Javascript links are currently ignored, restrict link checking to your local domain, HTTP proxy support, give username/password for HTTP and FTP authorization, robots.txt exclusion protocol support, and internationalization support.