Ground.ecn download stats

[apple2pd@garp3.icaen.uiowa.edu (ground.ecn AppleII Librarian)]

October 19, 2000

Jeff Fink asked me to do some download statistics on ground.ecn.uiowa.edu.
This report summarizes _only_ FTP downloads (not http access).

I have run some summary stats for file downloads from ground 1996-2000.
This turned into a challenge which I couldn't resist conquering.  I have
created a Top100 list (actually top 107 since I'm using all files
downloaded 300 or more times).  Some stats are a little misleading since a
multipart file (ie some binscii files) is counted once for each part.  The
Top100 file is available on ground in the main apple2 directory.  It is
also posted as a separate message on csa2.

Challenges:  FTP log files        1996    6,254,776 bytes    83506 records
                                  1997   35,007,752 bytes   442134 records
                                  1998   56,929,482 bytes   688684 records
                                  1999   39,239,234 bytes   456728 records
                                  2000   76,906,213 bytes   859367 records

Processing time with SAS 6.12 on HP B132L workstation: 36 minutes

I eliminated 'duplicates' (file downloaded more than once on a given
machineID) and a few files (name <4 chars (files in the lost+found)) or
any of the 'FILE.LIST' files.

Top 107 list of file downloads from ground Feb 1996 - Sep 2000

                          Ground Usage - FTP downloads
 Filename                    1996    1997    1998    1999    2000    Total

 index.html                    12     176     138      94     199      619
 aboutdivdos.txt                .     190     139      85     145      559
 ARCHIVE.SUFFIX.INFO.txt        .     170     177      88     116      551
 spyhunter.1of3                17     208     141      91      94      551
 ...
 Lambda.txt                    21     102      68      43      67      301
 ProTYPE11.hqx                  .      69      71      65      96      301
 text.adventures               49      60      64      51      76      300


 Total downloads            72830  381949  478553  406191  740070  2079593

 Total filenames = 32978

As you can see from the Top100 file, over 2 million file downloads have
occurred in the past 5 years.  In the first 9 months of this year over 740
thousand downloads have occurred.  We may reach 1 million downloads for
the year 2000 by the end of December.  (That is on average over 2700
files/day!)

A complete list of the 32978 files in the summary is 600 pages, single
spaced. (Only 2,785,445 bytes.)


Determining the number of users who have downloaded files from the ground
archives is nearly impossible.  The FTP logs contain 'machine id' instead
of userid or login names.  There are over 27000 unique machine IDs.  But
many of these represent (bogus) generated IDs. There are thousands of IDs
that begin with [cisco,cvx,dial,du,host,ip,kmm,pm,port,ppp,proxy,oct,
remote,rock,slip,sf,tc,tp,ts,user] to name a few.  AOL represents another
situation because most aol users are identified as xxxxxxxx.ipt.aol.com
(where xxxxxxxx is a RANDOM alphanumeric string not attached to a
particular user).

So instead of users, I tried to identify 'sites' by stripping off the
leading level of each machid (ie all chars up to and including the first
'.', except when the machid was a IP number (www.xxx.yyy.zzz).  Thus all
AOL users were reduced to one site.  Same for ppp logins from particular
service provider, etc.  This reduced the unique 'sites' to 13560. Still, 
4265 'sites' had downloaded only one file.  So only 9295 sites have 
downloaded more than one file.  4382 sites have only downloaded between 2
and 5 files, thus 4913 sites have downloaded 6 of more files.  3584 sites
have downloaded 10 of more files.  61 sites have downloaded over 10,000
files apiece.  25 sites have downloaded over 20,000 files.

Given that some 'sites' represent more than 1 user, and assuming only
those 'sites' with 6 or more downloads represent an active apple2 user,
I estimate the number of apple2 users who have actively used
ground.ecn.uiowa.edu to be over 5000 from all over the world.

A complete list of the 13560 sites in the summary is 247 pages, single
spaced. (Only 1,145,520 bytes.)  The top100 list would include sites which
have downloaded 5340 or more files.  I would consider most of sites as
private mirrors of (parts of) ground.ecn

Site              Rank    1996    1997    1998    1999    2000   Total

ex-pressnet.com      1       .       .       .   21513   13878   35391
dialnet.net          2      23   30416    4373       .       .   34812
omah.uswest.net      3       .       .       .     193   33411   33604
...
idx.com.au          99       .    5411       .       1       .    5412
deltasoft.com      100       .       .    5340       .       .    5340
...
                          ====    ====    ====    ====    ====    ====
  N = 13560              72830  381949  478553  406191  740088 2079611


I believe these statistics demonstrate that ground.ecn.uiowa.edu continues
to be one of the premier apple2 sites for a world-wide group of users.

Thank you to Jeff Fink (author of Silvern Castle adventure game) for
nudging me into this project.  And a special thank you to apple2 users
world-side for supporting ground.ecn.uiowa.edu

  --Steve

-- 

 --Steve  (apple2pd@ground.ecn.uiowa.edu)