National Capital FreeNet
Ottawa, Canada
Contents
This presentation will show the various means by which NCF measures what is actually happening online. Many of NCF's statistical measures are updated daily and available online to NCF members and information providers, giving the NCF community a way to see its own interests.I will present some tables, graphs, and interpretations of actual NCF online statistics gathered over the past 16 months. Effects of Board changes in NCF policy and practice may be detected in usage changes visible in many of the graphs.
I will also draw on survey data from the 1995 AGM on "perceived benefit" and donation levels by participants, show the "showcounts" system that presents usage statistics for any online menu, and present data on what NCF's 57,000 registered members find most popular.
The FreeNet still uses a heavily modified FreePort menu package originally written at Case Western Reserve University. NCF plans to add full graphical access to its content via the World Wide Web this fall.
Raw NCF facts
To help put NCF statistics in perspective, here are some raw facts about our operation:
Member activity:
The FreePort software currently in use at NCF requires a people to login with a userid to have access to the system; this gives us the unique ability to gather statistics by userid, not just by number of accesses. We can distinguish between a few people using a service many times and many people using a service a few times.
Since we also record donation data by userid, we have the opportunity to correlate funding with service use.
<<< What's New and What's Popular at the NCF >>>
(go popular)
1 About this guide
What's new:
2 Menus...
3 NCF newsgroups...
What's popular:
4 NCF newsgroups...
5 Special Interest Groups (SIGs)...
6 Usenet newsgroups...
7 Menus...
8 See also: NCF usage statistics...Each choice leads to a new menu of the top 18 items in the category, com piled from data collected over the previous seven days.
Online, all the lists pointed to by this "What's Popular" menu are not just passive text snapshots of the new or popular items; they are "live" menus. Users may type a number and proceed directly to the displayed menu or news group.
At any FreePort menu, the online "showcounts" command displays more detailed statistics about the menus. A showcounts example is included later in this paper.
New menus
The online list of new menus shows the newest and most recently modified menus on NCF since the last time the user looked at the list. The list looked something like this last week:
<<< New NCF Menus (since 96/Jan/15) >>>
1 SPEAK OUT! ... HELP OUT!... (96/Aug/12)
2 Help with World Wide Web... (96/Aug/11)
3 Help with HTML... (96/Aug/11)
4 Help with Lynx... (96/Aug/11)
5 Help Desk Task and Documentation Task Force... (96/Aug/11)
6 Scouts Canada, National Capital Region... (96/Aug/11)
7 The Courier, August/September, 1996... (96/Aug/11)
8 Hobbies SIGs -- Part 2... (96/Aug/11)
9 Public Access Sites... (96/Aug/10)
10 FreePlan Development Menu... (96/Aug/10)
11 Ottawa-Carleton Regional Transit Commission... (96/Aug/10)
12 Canadian Transportation Agency (CTA)... (96/Aug/10)
13 Carleton Board of Education (CBE)... (96/Aug/10)
14 The Ottawa Citizen... (96/Aug/10)
15 Community Networking Information... (96/Aug/09)
16 Help from selected Internet Resources... (96/Aug/09)
17 Roadmap96 Week 6 Lessons... (96/Aug/09)
18 1994 ROADMAP Internet Workshop... (96/Aug/09)
Of course, if more than about 18 menus change in one day, this list only presents you with the top 18. An obvious improvement to this raw list, and the others, would be a way for users to restrict the listing to items of interest.
New news groups
The online list of new news groups shows the newest NCF news groups created since the last time a user looked at the list. The list looked something like this last week:
<<< New NCF Newsgroups (since 94/Jan/01) >>>
1 ncf.sigs.language-lovers >>> (96/Aug/13)
2 ncf.sigs.mad-magazine >>> (96/Aug/06)
3 ncf.sigs.popular-education >>> (96/Aug/06)
4 ncf.sigs.games.command-and-conquer >>> (96/Aug/06)
5 ncf.religions.jrsp >>> (96/Aug/02)
6 ncf.sigs.dare-to-be-different >>> (96/Jul/30)
7 ncf.ca.cfuwo >>> (96/Jul/27)
8 ncf.sigs.computer.java >>> (96/Jul/24)
9 ncf.sigs.people.bells-corners >>> (96/Jul/24)
10 ncf.medias.chuo-gb >>> (96/Jul/20)
11 ncf.ca.mfrc >>> (96/Jul/06)
12 ncf.ca.mfrc-youth >>> (96/Jul/06)
13 ncf.sigs.music.country >>> (96/Jun/21)
14 ncf.government.oc-transpo >>> (96/Jun/19)
15 ncf.sub.public-relations >>> (96/Jun/14)
16 ncf.sigs.sports.walking-hiking >>> (96/Jun/13)
17 ncf.sigs.business.real-estate >>> (96/Jun/10)NCF is fairly undemanding in responding to a request for a new newsgroup. We try to screen out frivolous requests or requests that are already served by an existing news group; but, beyond those simple guidelines, pretty much anything goes.
Popular NCF menus
Here is the online list of the most popular menus, based on the count of users who saw that menu at least once during one or more online sessions with NCF in the past week:
<<< NCF Menus: Menus with the Most Users >>> (Aug7-Aug14)
1 National Capital FreeNet -- Main Menu... 12698 (16%)
2 E-Mail... 5649 (7%)
3 Communications Centre... 5500 (6%)
4 Chat with others at NCF (IRC)... 2270 (2%)
5 NCF and Usenet Newsgroups... 2196 (2%)
6 Manage and Transfer Files... 2100 (2%)
7 World Wide Web (WWW)... 1760 (2%)
8 File Management... 1571 (1%)
9 Search Tools... 1551 (1%)
10 Public Discussion... 1358 (1%)
11 People Finder... 1348 (1%)
12 Special Interest Groups (SIGs)... 1190 (1%)
13 Regional Newsgroups: Ottawa... 842 (1%)
14 Gopher... 800 (1%)
15 Transfer Files: Download from NCF... 690 (0%)
16 Using the National Capital FreeNet... 667 (0%)
17 Telnet to Other Sites... 602 (0%)
18 Telnet to Community Networks... 591 (0%)
Naturally the Main Menu is the most popular menu, since users must always start with this menu when they first connect to NCF!
Multiple viewings of the same menu by a userid only count as one menu "user" in this ranking list. The count of users seeing the main menu is therefore the count of unique users of NCF in the past week: 12,698. A summer weekly count of 12,698 users is a bit below the annual average of 13,000 unique members per week mentioned earlier.
Showcounts - detailed menu statistics
Every menu item selection on NCF is logged as it is made. The past seven days of statistics for any menu are available simply by typing the command showcounts while looking at the menu. Past weeks of data may also be available if the menu hasn't moved in the overall menu hierarchy.
Here is the online showcounts snapshot of the NCF main menu:
All Guest --Registered Users--
----Uses--- Uses Uses Users Ratio Admin Menu entry--
445 (1%) 21% 79% 286 1.2 1% 1 About the National Capital Fre
1396 (4%) 42% 58% 624 1.3 1% 2 Using the NCF / Become a membe
3377 (9%) 12% 88% 1402 2.1 0% 3 Search tools...
21468 (59%) 3% 97% 5033 4.1 0% 4 Communications centre...
4104 (11%) 9% 91% 1180 3.2 1% 5 Public discussion...
522 (1%) 18% 82% 320 1.3 1% 6 Community and Government...
335 (1%) 19% 81% 203 1.3 2% 7 Non-government organizations..
685 (2%) 19% 81% 305 1.8 3% 8 Schools, colleges and universi
1047 (3%) 17% 83% 469 1.9 1% 9 Media...
554 (2%) 23% 77% 358 1.2 0% 10 Libraries...
2467 (7%) 19% 81% 1039 1.9 0% 11 Special interest groups...
142 (0%) 20% 80% 88 1.3 1% 12 Menu principal francais...
84 (0%) 27% 73% 54 1.1 6% 13 How to make a donation to keep
The first time a menu is visited during a session with NCF, that visit is recorded as a "use" of the menu. Subsequent visits to the same menu during the same session are not counted as separate uses.
Popular news groups
The online list of most popular NCF news groups ranks all NCF news groups by the number of members that read them. This is not equivalent to the number of times the groups were accessed, since some users check the same groups over and over in a session, watching new articles arrive and responding to them immediately.
The NCF popular news group list looked like this last week:
<<< NCF Newsgroups: Most Popular >>> (Aug7-Aug14)
1 NCF Announcements >>> 1030 (7%)
2 Internet Relay Chat SIG (ncf.sigs.irc) >>> 521 (3%)
3 NCF General Public discussion >>> 355 (2%)
4 NCF Help Desk -- Questions & Answers >>> 310 (2%)
5 Computer Games SIG >>> 175 (1%)
6 ncf.sigs.computer.java >>> 173 (1%)
7 ncf.sigs.hobby.restaurant >>> 166 (1%)
8 ncf.sigs.health.ask-doctor >>> 155 (1%)
9 Comedy SIG >>> 152 (1%)
10 PC Technical Support SIG >>> 141 (0%)
11 Dog Lovers SIG >>> 126 (0%)
12 ncf.sigs.arts.alanis >>> 120 (0%)
13 Gay, Lesbian, Bisexual SIG >>> 120 (0%)
14 Poetry SIG >>> 117 (0%)
15 ncf.sigs.radio-tv.sailor-moon >>> 116 (0%)
16 Macintosh Users SIG >>> 114 (0%)
17 Stock and Commodity Trading SIG >>> 113 (0%)
18 DOS Windows SIG >>> 109 (0%)
Newer news groups haven't yet had a descriptive title added to the news group database, so some of the news groups above appear as their Usenet names.
A high member count is not necessarily an indication of the number of the groups vibrancy or the number of unexpired articles you might find in the group. The most popular group, the NCF Announcements group, is per- manently set in a user's list of favourite news groups; however, it is a moderated group and we purposefully keep the number of postings there to a minimum.
Special Interest Groups
Special Interest Groups (SIGs) are a particular class of menu wholly created by community interest. The news groups in these menus are particularly active; they make up almost half of NCF's total 725 local newsgroups. We track the member popularity of these 350+ news groups in a separate online menu:
<<< SIG Newsgroups: Most Popular >>> (Aug7-Aug14)
1 Internet Relay Chat SIG (ncf.sigs.irc) >>> 521 (4%)
2 Computer Games SIG >>> 175 (1%)
3 ncf.sigs.computer.java >>> 173 (1%)
4 ncf.sigs.hobby.restaurant >>> 166 (1%)
5 ncf.sigs.health.ask-doctor >>> 155 (1%)
6 Comedy SIG >>> 152 (1%)
7 PC Technical Support SIG >>> 141 (1%)
8 Dog Lovers SIG >>> 126 (1%)
9 ncf.sigs.arts.alanis >>> 120 (1%)
10 Gay, Lesbian, Bisexual SIG >>> 120 (1%)
11 Poetry SIG >>> 117 (1%)
12 ncf.sigs.radio-tv.sailor-moon >>> 116 (1%)
13 Macintosh Users SIG >>> 114 (1%)
14 Stock and Commodity Trading SIG >>> 113 (1%)
15 DOS Windows SIG >>> 109 (1%)
16 ncf.sigs.futurist.x-files >>> 106 (0%)
17 Star Trek SIG >>> 105 (0%)
18 Punk SIG >>> 102 (0%)
Despite the small number of weekly readers, SIG news groups contain particularly active discussions. Here is a count of current on-disk articles for the most vibrant SIG news groups (this data is not currently available online):
7065 radio-tv.sailor-moon
2454 irc
940 techno-rave
899 kids
617 star-trek-rpg
577 youth
417 arts.anime
400 star-trek
394 health.childbirth
384 lifestyle.singles-30-some
380 arts.alanis
339 punk
327 pc-tech
320 games.computer-games
311 religion.christian
292 arts.poetry
281 feminism
280 radio-tv.letterman
252 mens-issues
252 propaganda+media
246 windows-dos
... ...
32549 total articles for all ncf.sigs.* news groups
56210 total articles for all ncf.* news groups
The Sailor Moon news group, read by a modest 116 users last week, shows a whopping 7,065 unexpired articles on disk, all posted within the past two months! The IRC SIG's 521 news group readers have also managed to contribute a respectable 2,454 articles over the past two months.
Here is a showcounts example, from the Sailor Moon special interest group menu:
All Guest --Registered Users--
--Uses----Uses Uses Users Ratio Admin Menu entry--
7 (1%) 0% 100% 7 1.0 0% 1 About the Sailor Moon SIG
971 (73%) 14% 86% 97 8.6 0% 2 The Sailor Moon SIG >>>
345 (26%) 17% 83% 36 7.9 0% 3 Usenet "alt.fan.sailor-mo
Newsgroups referenced by this menu:
Percentage of All Guest --Registered Users--
Registered Users Uses Uses Uses Users Ratio Admin
ncf.sigs.radio-tv.sailor-moon
Opened using this menu 84% 971 14% 86% 97 8.6 0%
Opened using any method 2179 14% 86% 116 16.2 0%
alt.fan.sailor-moon
Opened using this menu 71% 345 17% 83% 36 7.9 0%
Opened using any method 617 15% 85% 51 10.3 0%
Note the statistics collected on the news groups mentioned in the menu. News groups may be opened "singly", using the menu item here, or using a multiple-group news reader where many groups may be read at once. By subtracting the "opened using this menu" figure from the "opened using any method", the number of people who use the multiple-group news reader for each group may be deduced.
Most users this past week used this menu to read these two news groups: 97 out of 116 and 36 out of 51.
Here is a snapshot, available online each day under "go usage", of the most used NCF services of the past week, ranked by the number of NCF members that used them at least once during the week:
All Guest --Registered Users--
----Uses---- Uses Uses Users Ratio Adm Item/Service-----
93190 (20%) 4% 96% 12606 7.1 1% Service: FreePort-menu-system
96909 (21%) 3% 97% 12595 7.4 1% menu.main
49754 (11%) 0% 100% 9577 5.2 1% Service: mail-mr
129307 (27%) 5% 95% 6517 18.8 1% Service: nr/mgnr-newsreader
9920 (2%) 0% 100% 3041 3.3 1% Service: mail-send
13585 (3%) 0% 100% 3007 4.5 1% Service: lynx-web-browser
20689 (4%) 0% 100% 2009 10.3 0% Service: irc
23566 (5%) 0% 100% 1562 15.1 1% Service: telnet-other
18024 (4%) 2% 98% 1490 11.8 1% Service: who
2894 (1%) 3% 97% 994 2.8 1% Service: time-remaining
1916 (0%) 14% 86% 948 1.7 1% Service: userInfo--get-from-name
2971 (1%) 6% 94% 748 3.7 2% Service: userInfo--get-from-ID
3410 (1%) 0% 100% 720 4.7 1% Service: telnet-comnet
2059 (0%) 1% 99% 390 5.2 3% Service: mail-BBelm
787 (0%) 27% 73% 385 1.5 1% Service: help-menu
1491 (0%) 0% 100% 299 5.0 3% Service: one-line-msg
614 (0%) 1% 99% 122 5.0 7% Service: mail-from
238 (0%) 39% 61% 89 1.6 4% Service: search-titles
238 (0%) 0% 100% 71 3.4 1% Service: chat
100 (0%) 1% 99% 31 3.2 35% Service: showcounts
1 (0%) 0% 100% 1 1.0 0% Service: dg_showcount
The count of uses of a service is incremented every time a member selects a service.
We see that the FreePort menu system itself, and the presentation of the main NCF menu, are both compulsory for all users, and their use by members is the same as the number of unique users over the past week. This sets the upper bound on how many users could have used any NCF service this past week.
The most frequently user-selected NCF service, reading news, is not the service used by the largest number of users, which is the mail system. Part of this difference is caused by users selecting individual news groups from a FreePort menu, where each selection counts as a "use", instead of starting up the news reader and reading many different news groups at the same time.
Note which services are used more than seven times per week (more than once per day). In addition to the news reader, several other services were also used more than once per day, such as Internet Relay Chat, the "who is online" command, and outgoing telnet. Interestingly enough, member use of the mail system does not average out to be daily; it is used only about 5.2 uses per member per week.
NCF members are limited to a maximum of two hours of peak usage per day. The popular "time-remaining" command gives the amount of peak-hours time left.
http://www.ncf.carleton.ca/ip/freenet/ops/team/tech-dir/stats/index.html
Trends in NCF usage can be inferred from some of this data. These graphs are built from Unix wtmp records collected since March 13, 1995.
Usage and Registration Graphs
Where a graph is titled "Days since March 13, 1995", the "X" (horizontal) axis is the number of days since that date. (GnuPlot didn't let me re-label the axis the way I wanted to!) Based on the March 13, 1995 start date and the date of the graph itself (labelled at the top), you can estimate dates in the interior of the graph.
Total Member Online Minutes
is the sum of all member online time for each day. It is the sum total of all sessions by all non-guest users for the day. It is measure of "system utilization" by members. The higher the number, the more the system was being used to service online members. The decrease in system use visible in the early part of the graph corresponds with the mid-April 1995 implementation of two-hour online time limits. Online usage by members declined instead of shifting to non-peak hours.Total Guest Online Minutes
is the sum of all guest online time for each day. It is measure of "system utilization" by guests only (not by members). The higher the number, the more the system was being used to service online guests. Some members started using the guest account to avoid the time limits on their own accounts; this may account for the increase in guest usage after mid-April 1995.Daily Guest Connections
is the count of guest login connections (Sessions) for each day. Some members started using the guest account to avoid the time limits on their own accounts; this may account for the increase in guest usage after mid-April 1995.Members Daily Minutes Online
is an average of the total number of minutes spent online in each day by each member. This is simply the sum of all online session times each day divided by the number of unique users that day. The mid- April 1995 drop in average daily minutes online corresponded with the introduction of online time limits.Members Average Session Length
shows the average length of a login session for NCF members for two types of connections:Weekly Unique UsersAnnex: members using a dial-up/library (Annex) line
Telnet: members using telnet connections
Note how the ease with which telnet users can connect and disconnect without waiting for a phone line permits them to have shorter sessions. The mid-April 1995 drop in average session length corresponded with the introduction of online time limits.
shows the number of unique users of the system in the weeks preceding the given date, for several "windows" of numbers of weeks. (See the next section for an explanation of "window" sampling.) For example, the line labelled "6 Wk" graphs the number of unique users of the system in the 6 weeks preceding each date on the horizontal axis.Daily Unique Users
is the same idea as above, but the "window" is days wide instead of weeks. For example, "6 Day" graphs the number of unique users of the system in the 6 days preceding each date on the horizontal axis. Note the decrease in number of unique users during the 1995 December holiday season.Registrations and Activations
shows the weekly number of users requesting accounts through the online registration process ("Registrations"), and the number of users who actually follow-through with sending the signed registration form to the office and actually get an account ("Activations"). Since the activations are processed by the office, changes in the office (holidays, staff turn over, etc.) may delay or otherwise shift activations into "clumps" in some weeks.Pledge and Donation
graphs the average pledge and donation amount, per user, in each week. The Pledge amount is the amount of money a prospective member says will accompany his/her registration form at the time s/he fills out the online registration. The Donation amount is the amount of money actually received by the office from new members. (Donations arriving outside the registration process are not covered here.) The Pledge amount is averaged among all people completing the online registration that week. The donation amount is averaged among all people having their accounts validated (enabled) by the office that week. (The "clumping" factor mentioned above also applies to donations.) Note that between several days and several weeks elapse between the time a Registration form is filled out online and the account is actually Activated, so anything that affects people's willingness to donate will have an effect first in the Pledge graph and some time later in the Donations graph.NCF Statistics - window sampling
The window sample graphs show the number of unique users seen by the system for several "windows" of different lengths. One graph shows windows of differing numbers of weeks, the other shows windows of differing numbers of days. The window is a length of time preceding a date on the horizontal axis.
For example, the line labelled "6 Wk" in the Weekly graph indicates a 6 Week long window. For each date on the "X" (horizontal) axis, the unique users logging in to NCF in the 6 week window preceding that date is counted. That count becomes the "6 Wk" "Y" (vertical) axis value for that date.
Similarly, for the "6 Day" line in the Daily graph, each Y axis value is the number of unique users of the system in the 6 days preceding each date on the X axis.
Thus, the Y axis shows the count of unique users in the window preceding each X axis date. For example, the Y axis value of the rightmost point on the graph is the count of unique users in the window preceding the date the graph was made.
The longer the window, the more unique users register in the counts. This lets us layer several windows onto the same graph. Note that the 7 day window in the Daily graph is identical to the 1 week window in the Weekly graph.
End Effects:
For dates on the left end of the graph, some window lengths extend before the graph start, and this has interesting effects.
For example, a 6 week window calculated for an X axis date of April 1, 1995 would have a window start date back in February 1995; but, the graph data only begins on March 13, 1995. A 6-week window cannot reflect a full 6 weeks of users until 6 weeks after March 13th. Similarly, the 12-week window won't show a full 12-week count of users until 12 weeks after March 13th.
Until the start date of a window falls after March 13, the window's Y axis value reflects simply the count of unique users between March 13 and the Y axis date. The longer windows take longer for the start date to pass March 13; this produces the smoothly rising slopes on the right hand side of the graph for each window size.
As each window's start date passes March 13, the Y axis graph for that window size stops simply increasing and reflects a true count of unique users in the window.
Most people, if they counted how many people swam across the river, would never think about building bridges.The Ottawa population was not asking for a FreeNet in 1992. Today, NCF has 58,000 members registered; 13,000 use the system every week; 33,251 have used it this year.
- Ronald Altman
Just as the community network itself didn't exist once, many of the popular services that people will come to use on a community network aren't yet offered or don't exist. Let us not be misled by online usage statistics; they simply measure what people do, not what they might do, given the opportunity.
We have a good idea of what is popular on NCF; but, that doesn't necessarily indicate how important these items and services are to the members. Ultimately, we have to ask the members what they think the important NCF content and service features are.
Ian! Allen (idallen@freenet.carleton.ca) is an independent consultant currently providing technical direction to the National Capital FreeNet in Ottawa. He has been using shared online communication services since 1976 and the Internet since about 1981.
Ian has fifteen years' experience as Systems Administrator of academic and community online systems, where responsiveness and a keen ability to listen are key job requirements.
Ian has an honours B.A. in Psychology and a Master's degree in Computer Science, both from the University of Waterloo.
Credits
Jim Elder (jim@freenet.carleton.ca) wrote the modifications to the FreePort menu software that provide NCF with showcounts and "most popular" statistics.