I understand, when you are reading this blog post, you have a requirement to identify all the URL’s of the website. This is tool is a simple and cool link checker or crawler, helps to satisfy the need (most of).
What is Xenu ?
Xenu Link Sleuth is a free tool will help to crawl to the website, also this tool helps to find the broken links on the site. The tool has option export the report into HTML & Tab separated, this helps to analyze the data much better.
What outputs the tool can provide ?
- Identify all the links of the page
- Identify the broken links on the website
- Identify the redirect url’s
- Identify the type of resources (image/png, application/javascript, text/html, etc.,) used on the website.
- Identify the external links of the website.
Why Sitemap will not help to identify the links of the site ?
Sitemap is a structured (hierarchical) list of pages indented for Web Crawlers and Search Engine. The website will have complete control on what links need to be available sitemap. The search engine like Google, don’t have to crawl entire site rather crawl only the links exposed in the sitemap.
At times when evaluating a website, we may need to find all available pages and links in website, this time you can use crawlers like Xenu to find all the links of the website, the links that are working and those are not working.
What are the other similar tools in the market ?
My Journey With Technology! 