Crawl a Web site with this free ActiveX spidering component. Advanced features include caching, "avoid" patterns, and robots.txt compliance. Automatically handles HTTPS URLs. Includes safety measures for avoiding large / infinite size Web pages or infinitely growing-in-length URLs. Includes a wind-down features for crawling a maximum number of pages per site.
| Price | USD 0 |
|---|---|
| License | Free |
| File Size | 976.5 kB |
| Version | 2.2.2 |
| Operating System | Windows, Windows 2000, Windows XP |
| System Requirements | None |
