WorthWhileSoftware.com

Download and archive entire websites such as forums, galleries, online books and articles. Website downloader can perform website login and handle cookies. Copy and store websites on portable media such USB and CD.

Convert dynamic database driven websites into static. Present websites to clients in offices with no internet access. All links in the website can be converted to relative. Upload the downloaded website to mirrors and servers. Save webserver resources by not using server side code. Download and convert websites from localhost, intranet, internet etc. Get everything from small to giant websites with many thousands of pages. Quickly download websites using multiple connections. Use include and exclude filters to control which files to crawl and list. Follow all kinds of references including redirects, frames, javascripts and images. Automate download of websites at times convenient to you, e.g. during the night.

Download entire websites and browse them offline later. Features a highly configurable crawler; decide which file types to download, which directories to crawl etc. Crawler can also perform login and maintain sessions and cookies.

Using simultaneous connections entire websites can be downloaded very quickly. Links can be checked and converted to HTTP or local disk file system. Take websites with you on the road utilizing USB / CD / DVD or any portable media. It is possible to use a task scheduler to automate download of websites at times convenient to you.

Create Text, HTML, RSS and XML sitemaps for your website.

Create text, HTML, RSS and XML sitemaps to help search engines like Google and Yahoo to crawl and index your website. Crawler is feature rich supporting many website crawling options. Configure amount of simultaneous connections to use. Supports crawler filters, robots.txt, custom connection and read timeout values, removal of session IDs, scanning of javascript and css files, proxy setup, website login, and various other options. Alias paths during scan, e.g. http://www.example.com and http://example.com. Scan websites from multiple start paths, useful for websites not fully crosslinked. Scan local and online websites on internet, localhost, LAN, CD-ROM and disks. Scan static and dynamic websites such as portals, online stores, blogs and forums. View reports on broken and redirected links (whereto and wherefrom). Rich template support for HTML sitemaps.

Generate sitemap files for ASP.Net controls. Supports splitting and compressing XML sitemaps. Can set and calculate priority, change frequency and last modified in XML sitemaps. Change root path used in generated text, HTML, RSS and XML sitemaps, useful in case you have scanned a mirror or localhost website copy. Integrated FTP sitemap upload. Can also ping and notify search engines of sitemap changes. Command line support includes load project, scan website, build sitemap, FTP upload and ping search engines.

Rich template support for generating sitemaps and near-endless scanning options. The scanning engine stores vast amounts of data so that you can, among other things, view full report on broken and redirected links (wherefrom and whereto). This data is e.g. used when generating Google XML sitmaps (e.g. when defining priority).