SourceForge.net hosts over 100,000 Open Source projects. You may find what you're looking for by searching our site directory .
This is the infocrawler project ("infocrawler")
This project is hosted by SourceForge.net. The project team describes it as:
InfoCrawler allows you to crawl and index various types of documents, accessing data from various resources: Intranets, public WEB sites, local or remote file systems. For product information please see our website at http://www.infocrawler.org/
SourceForge.net is the world's largest provider of hosting for Open Source software development projects. SourceForge.net provides a variety of services to projects, including a download mirror network, code hosting (like Git, Mercurial, and Subversion), and tools to support discussion and support. These services are provided to projects and their end-users free-of-charge.
Of benefit to users, Open Source software is licensed so you can download and use the software free-of-charge. The source code for this software is made available free-of-charge, you (or a programmer you hire) can make changes to this software to better meet your needs, and you can release your changed code back to the community passing the benefit on to other users.
To join this project, please contact the project administrators of this project, as shown on the project summary page.
This page is the default project web page supplied by SourceForge.net. If you are a member of this project, you can deploy your own project web site as per our site documentation.
If you are a developer interested in this project, please consider reaching out to the project admin (per the "Join this project" section, above) to offer your assistance.