Crawler4j

Crawler4j is an open source Java crawler which provides a simple interface for the Web crawling. You can setup a multi-threaded web crawler in 5 minutes. You can create a crawler class that extends WebCrawler and it decides which URLs should be crawled and handles the downloaded page.
Price USD 0
License Free
File Size 89.37 kB
Version 3.3
Operating System Windows 2003, Windows Vista, Windows, Windows 7, Windows XP
System Requirements None