[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: apple2clones site

On Oct 14, 2:29 am, a2avia...@gmail.com wrote:
> > I used to use a program called Pagesucker to snarf sites.  It was
> > shareware.
>
> SiteSucker and PageSucker grab the index and thats it. Even with
> robots.txt ignored. The apple2clones site was a dynamic driven site,
> most of the content on it was also from other sources. So those other
> sources are mostly still out there too.
>
> Doing Page/SiteSucker anymore seems to not work on many sites anymore.
> They come back right away as done.

That is because they use Drupal or some other community board type
software. Apple2Clones.com was done in Drupal.