357b334 Python threads are very slow. Instead, start one process per host to

Authored and Committed by mdomsch 18 years ago
    Python threads are very slow.  Instead, start one process per host to
    crawl, which is much faster.  Because I never figured out how to get
    the forked child process database connection to work, instead spit out
    a bash-like set of commands, which can be piped into bash.  e.g.
    
    ./crawler -c dev-postgres.cfg --include-private | bash
    
        
file modified
+48 -304