[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: apple2clones site
> I used to use a program called Pagesucker to snarf sites. It was
> shareware.
SiteSucker and PageSucker grab the index and thats it. Even with
robots.txt ignored. The apple2clones site was a dynamic driven site,
most of the content on it was also from other sources. So those other
sources are mostly still out there too.
Doing Page/SiteSucker anymore seems to not work on many sites anymore.
They come back right away as done.