Search for software:
|
| Site Spider 1.0.0 - Spider a Site Intelligently |
 |
| Site Spider |
|
|
| Spiders a site and creates output files containing (1) a list of URLs in the site, (2) a Google sitemap, (3) an XML summary file with title, description, and keywords for each page, and (4) links to URLs external to the spidered domain. Robots.txt compliant. |
|
|
|
|
| Keywords: bot, data mining, download, free, intelligence, internet, probe, robot, spider, tool, web |
|
|
