Excellent device to creep my website and also aid me locate dead link and also unlinked documents
I have a rather large heritage website with essentially hundreds of PDFs that are occasionally making up in a data source, yet usually are simply web links on the web page, and also are saved in the majority of every directory site on the website.
I have created a php spider to adhere to all the web links on my website, and afterwards I am contrasting that versus a dump of the directory site framework, yet exists something less complicated?
There are numerous items from Microsys, specifically their A1 Sitemap Generator and also A1 Website Analyzer that will certainly creep your internet site and also record every little thing you can perhaps visualize concerning it.
That consists of busted web links, yet additionally a table sight of all your web pages so you can contrast points like the same
If you are making use of windows 7 the most effective device is IIS7's SEO Toolkit 1.0. It is free and also you can download it absolutely free.
The device will certainly check any kind of website and also inform you where every one of the dead links are, what web pages require to long to load, what web pages have missing out on titles, replicate titles, very same for search phrases and also summaries, and also what web pages have actually damaged HTML.
I'm a large follower of
check.ll and also do:
Here's what my check.ll documents resembles
# linklint -doc . -delay 0 -http -htmlonly -limit 4000 -net -host www.example.com -timeout 10
That does a crawl of
www.example.com and also creates HTML documents with cross - referenced records wherefore is damaged, missing out on, etc