Project Description
Simple and very efficient multithreaded web crawler with pipeline based processing written in C#. Contains HTML, Text, PDF, and IFilter document processors and language detection(Google). Easy to add pipeline steps to extract, use and alter information.

NCrawler targets both .Net v.3.5 and v.4.0

Last edited Aug 6, 2010 at 9:17 PM by EsbenCarlsen, version 11