Andrew S. Patrick, Alex Black, & Thomas E. Whalen
Network Services & Interface Design Laboratory
Communications Research Centre, Industry Canada
The National Capital FreeNet (NCF) in Ottawa Canada is one of the largest and most successful community online services in the world. An examination of this system may provide valuable information for planning future services. A survey was completed by 1073 NCF users and the results indicate that the NCF users are not a specialized group in the community. Although there is a large gender imbalance, they are not exclusively male (especially when compared to other similar systems). There is a broad range of ages represented on the system, and a wide variety of education levels. Income levels for NCF users, especially household incomes, are quite comparable to the local region. NCF users are not necessarily highly experienced in computers or computer networks. In areas where there are discrepancies with the local regional data, data from other studies suggests that the NCF is less extreme than other comparable systems. NCF users are quite satisfied with their system. Their levels of satisfaction are not strongly related to the demographic variables measured here, but the amount of computer experience seems to be the strongest factor (in the negative direction).
The National Capital FreeNet (NCF) in Ottawa Canada is one of the largest and most successful community networks in the world (Beamish, 1995). The NCF was launched in February of 1992 and by the Fall of 1994 it had grown to 29,000 registered users, with approximately 12,000 members using the service each week. Members of the system enjoy such services as electronic mail, local and international (Usenet) discussion groups, library search and information retrieval services, and connections to other systems on the Internet. An examination of this system and how it is used may provide valuable information for planning future services.
The NCF does not require any personal information, other than an address, when members register for an account. Thus, the operators of the NCF know little about their users. The current research involved developing a survey of NCF users to gather information about the characteristics of the users, their use of the system, and the impact the services have on their lives. This report presents the results for some of the user characteristics variables and users’ satisfaction with the system. Future reports will discuss the usage and impact data.
To properly examine the characteristics of NCF users it is necessary to research comparison groups. The Statistics Canada Census data from 1991 (Statistics Canada, 1992) will be used to provide a comparison group from the local regional community. There has also been research conducted on a similar FreeNet system in Cleveland, Ohio (Anderson, 1992), and comparisons will be made to users of that system. Amsterdam also has a FreeNet system called the "Digital City", and a recent research report (Schalken & Tops, 1994) provides some useful comparison data.
Finally, Pitkow & Recker (1994) reported a survey of users of the Internet World Wide Web (WWW) service, and it will also be used as a comparison. The WWW is a multimedia information service on the Internet, and it is one of the fastest growing and most popular services. The WWW service requires a fairly expensive connection to the Internet and modern computer equipment to use all the multimedia features. The WWW service provides an interesting contrast to the NCF system that is very inexpensive and requires only the most basic computer equipment, or no equipment at all.
Materials The survey consisted of 74 questions in four categories: demographics (e.g., age, gender, etc.), use of the FreeNet (e.g., using e-mail, using discussion groups, etc.), the impact that the NCF had on the subjects’ lives (e.g., watching more, less, or the same amount of TV, socializing more, less, or the same, etc.), and satisfaction with the system. The current analysis only includes some of the demographic variables and a question concerning satisfaction with the FreeNet.
Two versions of the survey were used for this study. One version was written in HTML, a mark-up language used for the World Wide Web (WWW) on the Internet. This survey was used with the Lynx software so that users could complete the survey online using a form-filling interface (see Figure 1 for an example). When each survey was completed the results were transferred automatically to a computer in our laboratory for processing. The second version of the survey was in plain text, and this was sent to subjects via e-mail upon request. Subjects could complete the survey using a text editor and return the survey via e-mail.
The survey was made available to all NCF users on a self-selection basis. The survey was accessible from the main menu of the system and notices inviting users to complete the survey were sent to the most active discussion groups on the system (e.g., computer games, men’s issues, youths) and posted in the announcements areas. Responses were collected during the months of September, October, and November 1994.
In addition, we used a method of "random encouragement" where a sample of 3138 user IDs was prepared from those users who had not completed the survey themselves. Each of these users was sent a personal invitation via e-mail to complete the survey. A total of 294 users (9.37%) responded to this encouragement by completing the survey.
After removal of duplicates, a total of 1073 survey responses were available for analysis (51 via the e-mail system and the remainder via the WWW interface). This sample is approximately 3.7% of the total number of NCF registered users at the time of the study. However, not all the NCF users are active at any one time, and this sample size is approximately 8.9% of the estimated 12,000 users who were active on the system during the period of the survey.
Experience with the NCF
One method of measuring the success of the sampling is to examine the amount of experience users had with the system. Subjects were asked "How long have you been using the NCF (months)", and the distribution of responses can be seen in Figure 2. On average, users had 11.2 months of experience with the system (N=1046), and there was a broad range of experience from a few months to more than 2 years. This suggests that both new and veteran NCF users completed the survey.
Members’ Location
In order to determine how many of the users were from the local area, the survey asked: "Do you live in the National Capital area (Eastern Ontario or Western Quebec)?" 1050 respondents answered this question and 85.3% indicated they did live in the local region. In contrast, Anderson (1992) reported that only 24% of the users she surveyed came from the local Cleveland region. In Amsterdam, only 45% of the users surveyed came from the local region. These results suggest that the NCF is unique because its membership is primarily from the local region. It also indicates that comparisons with census data from the local region will be appropriate.
Gender
1052 users answered a question about their gender and 81.8% of the respondents were male (18.2% female). The census data for the local region shows that only 49% of the population is male. This suggests that females are under-represented on the NCF, and might indicate that NCF users are not representative of the community.
It is important, however, to compare these results with those of other systems. The study of Cleveland Free-Net users reported 83% males, while the Digital City data showed 91% males. Thus, when compared to other FreeNet systems the NCF is more gender balanced. Further, the survey of WWW users showed 90% males. So there may be gender inequality in the NCF, but it is not as extreme as other FreeNet systems or the Internet WWW service.
Quarterman & Carl-Mitchell (1995) recently reported some interesting results on gender inequality on the Internet. For their study they surveyed organizations connected to the Internet and asked: "For network users who can send electronic mail from inside to outside your domain, please give percentages for the following genders: male, female." Their survey of 1,468 organizations indicated that 64% of users with mail connections to the Internet were male. These results suggest that when it comes to access to the Internet (at least for electronic mail), there is more gender equality. It is important to note the difference between ACCESS to a service and USE of the service. These results from organizations connected to the Internet suggest that more women have access to the Internet than actually use it.
Age
The average age of respondents to the NCF survey was 34.3 years (N=1053). The distribution of ages is summarized in Figure 3 where the percentage of people falling in each age range is illustrated. It is clear that there is a broad range from pre-teens to senior citizens. The most frequent age category was from 20-24 years.
A comparison of the age distribution with the results from the census of the local region is shown in Figure 4. These results indicate that the NCF under-represents people in the extremely low and high age ranges (less than 15 and greater than 65). There also appears to be an over-representation of the young and middle age ranges, particularly in the 15-19 and 20-24 categories.
This comparison can best be seen when difference scores are plotted, as in Figure 5. Here we see again that the NCF has lower representation in the low and high age ranges, and somewhat higher representation in the young and middle age categories. A simple rule of thumb for evaluating the importance of the differences is to look for discrepancies larger than the expected sampling error for this survey. With an error rate of +-3%, differences of more than 6% would be extremely rare if the survey was repeated many times. Using this criteria, the NCF age distributions differs from the local region only in the very young ages (less than 9 years), and the range of 20-24. In all other categories the distributions are quite similar. Given that literacy is required to use the system, only the difference in the 20-24 age category is unexpected.
Studies of other FreeNet systems have not included a comparison with their local region. Given the fact that their users are not primarily from their local area this may have been appropriate, but it makes comparisons to the current results difficult.
We can compare the ages of NCF users with the ages of users of similar systems. The average age of Cleveland Free-Net users was 31.7 years, nearly 3 years younger than the results for the NCF. The two distributions of ages can be seen in Figure 6. It appears that the Cleveland system has a higher percentage of their users in the ranges 19-24 and 25-34 than the NCF system, while the latter has more representation in the 45-54 category. The differences between these distributions is shown in Figure 7. The most notable difference is a much larger percentage in the 45-54 category for the NCF.
A comparison of ages with the Amsterdam Digital City is shown in Figures 8 and 9. Similar to the comparison with the Cleveland data, the NCF seems to less dominated by users in the young adult ranges (19-24, 25-30), and has more representation from older users (greater than 40).
The average age of WWW users was 31.2 years, and a comparison of age distributions is shown in Figures 10 and 11. Here there is a very large difference in the 25-29 year range, with this category making up nearly 30% of WWW users. Once again, the NCF is less dominated by users in the young adult age ranges and has more representation from users in the older categories.
The age results taken together suggest that the NCF does include users from a broad range of ages. Moreover, any discrepancy with the age distribution in the local population is probably less extreme than would be seen with other similar systems, or the Internet WWW service.
Education
NCF users were asked about their education in terms of the highest level achieved so far, ranging from grade-school to graduate and professional degrees. 1061 answers were collected and the results are summarized in Figure 12. These results show that the most frequent category was people with university degrees, followed by graduate or professional degrees and some college or university.
A comparison of education levels with the census data from the local region is shown in Figure 13. This shows that the NCF users differ most from the local region in the category of university educated (the census data did not distinguish between undergraduate and graduate or professional degrees), with more than 50% of the NCF users being university educated (this is also the largest category for the region). There were also fewer NCF users with a grade school education than might be expected from the regional data.
The only other research that looked at education in a comparable way was the survey of WWW users. A comparison of education levels with the results of that survey are shown in Figure 14. WWW users are also well-educated, with many more users having graduate or professional degrees than was seen on the NCF. WWW users also differ from NCF users in that fewer of them have only grade school or high school educations.
The education results, therefore, are similar to the age results. There are a variety of education backgrounds among NCF users. There is some discrepancy from the local population, but this is less extreme than the pattern that is seen with other online systems.
Occupation Status
The current survey also included a question about occupation status, and the results for 1050 respondents can be seen in Figure 15. 41.4% of the users were salaried employees, while 27.5% classified themselves as students.
The only comparable data from the local census report was the rate of unemployment. NCF users reported a 3.9% rate of unemployment, while the census reported an unemployment rate of 7.3%. There is a 3-year difference between the 1991 census data and the research period for this study and employment rates fluctuate regularly, so it is difficult to draw conclusions from this data.
Personal Income
One of the survey questions asked users to list their annual gross income in Canadian dollars. Only 991 respondents (92% of the sample) answered this question and some indicated a reluctance to disclose personal information. On the other hand, other respondents were glad to see questions about income and looked forward to seeing the results. Data about income levels may be particularly important for designing Information Highway services that users might pay for. Further, it is valuable information for FreeNets that rely on donations for a large part of their funding.
The distribution of income levels is shown in Figure 16. The most common category was income levels less than $9,000 per year. This is surprising since the most frequent occupation status was salaried employee.
A comparison of personal income levels with the regional census data is shown in Figure 17. It is important to note that the regional figures are not corrected for inflation and there is a 3-year gap between the data gathering periods. Average wages and support programs for students and others may have risen in this period. Thus, any indication that NCF incomes are higher than the regional data should be consider in light of the possibility of inflation.
The comparison with the regional income levels shows that NCF users are both better off than the local community (incomes greater than $50,000) and less well off (incomes less than $9,000). There is a corresponding under-representation of people in the middle income ranges ($10,000-$40,000).
Household Income
Only 918 users (85% of the sample) answered a question about household income. This low responding rate may be due to an unwillingness to provide personal information and perhaps an unwillingness by some younger people to ask their parents about income levels. This sample size does allow a reasonable estimation of income levels, however.
The distribution of household income levels is shown in Figure 18. Here the most frequent category is in the middle range of incomes ($40,000-$49,000), followed by households with incomes greater than $99,000 per year.
A comparison with the corresponding regional census data is shown in Figure 19. Similar to the results for personal income, the large discrepancies are at the extremes of the range (less than $9,000 and greater than $70,000). Overall, however, there is a close match in household income levels between the NCF and the local community.
Computer Experience
Users were asked: "How long have you been using computers (years)". 1062 people provided answers to the question and the average was 11.5 years of experience. The distribution of experience is illustrated in Figure 20, and it can be seen that most users had between 6 and 15 years of computer experience.
Thus, NCF users appear to have a large amount of computer experience. However, the question did not ask the type of experience and casual use of computers is very common these days. Many children in grade school can answer correctly that they have been using computers for 5 years or more.
Anderson (1992) asked Cleveland Free-Net users about their computer experience. A comparison of experience levels using her categories is shown in Figure 21. The NCF and the Cleveland Free-Net have nearly identical experience patterns, at least with the limited range of categories that were available for comparison.
Network and Bulletin Board Experience
Users were also asked how long they had been using computer networks and/or Bulletin Boards. The average for the 1046 people who responded was 4 years, and the distribution of responses is shown in Figure 22. Most of the users had less than 2 years of network experience, which is a marked contrast to the large amount of computer experience that was reported. Computer network systems are newer and less common than computers themselves.
The research by Schalken and Tops (1994) on the Digital City in Amsterdam also included a question about computer network experience. In that study 29% of the respondents reported "a lot of experience with computer networks", and 38% reported "some experience". It is difficult to compare numerical data with experience ratings of this type, but these results might suggest that Digital City users have more experience with computer networks. This is not surprising since a large portion of their users access the system remotely, probably using computer networks, while the majority of NCF users are in the local region and may rely on dial-up connections.
Measuring Satisfaction
Finally, the present survey included a general question about satisfaction with the NCF: "Overall, how satisfied are you with the NCF?" Users were asked to indicate one of 5 ratings ranging from "not at all satisfied" to "extremely satisfied". The distribution of ratings in these categories is shown in Figure 23. The majority of the 1061 responders to this question indicated that they were "very" satisfied with the system, and a total of 76.9% described themselves as either "very" or "extremely" satisfied.
Schalken and Tops (1994) also measured satisfaction by asking users to indicated their "general opinion of the Digital City". Users could choose one of 5 ratings ranging from "very negative" through "neutral" to "very positive". This study found that 22.4% of the responders chose "very positive", while 61.9% gave the system a "positive" rating. In total, 84.3% rated the system as "positive" or "very positive".
Anderson (1992) included a number of measures of satisfaction in her study of the Cleveland Free-Net. The most comparable question for the current discussion is one that asked users to indicate how much they agreed or disagreed with as statement that they found the system "usually enjoyable or interesting". 39.4% of those users "strongly agreed" to this statement, while 50.2% "agreed". Thus, a total of 89.5% of the users gave the system a positive rating with this question.
It is difficult to compare the results across systems because the questions were quite different. There is some suggestion, however, that NCF users are somewhat less satisfied with their system than users of the Cleveland Free-Net or the Digital City. Users were given an opportunity to leave general comments about the NCF and these comments were very positive overall. Areas where users did complain about the system include: long delays to access the system via modem, slow system response times, a desire for more Internet services, and some complaints about the user-friendliness of the system.
Factors Related to Satisfaction
The overall satisfaction ratings were examined to determine if any of the demographic variables were related to users’ satisfaction. The satisfaction ratings were converted to numerical values ranging from 1 to 5 and subjected to statistical analyses.
An analysis of the satisfaction ratings given by men and women indicated a small but statistically significant difference (p < .005). Women gave the NCF higher ratings (4.16) than men did (3.98). It is not clear what the cause of this rating difference is and it may be interesting to pursue, especially in the context of the relatively low participation by women on the system.
The satisfaction ratings for each occupation status category are shown in Figure 24. The lowest satisfaction ratings were given by salaried users, while the highest ratings were given by unemployed people and contracted workers. This suggests that the an inability to pay for alternative commercial services might be an important factor for influencing how satisfied people are with the NCF. The correlation between personal income and satisfaction with the NCF was in the negative direction and statistically significant (r=-.13, p < .01). There was no significant correlation with household income (r=-.06).
Other variables that were related to satisfaction were: computer experience (r=-.15), education (r=-.12), network experience (r = -.11), and age (r=-.08). It should be noted that these correlations, while statistically significant, are very modest and account for very little of the variance in the satisfaction ratings.
It is interesting to note that users with more computer and network experience tend to be somewhat less satisfied with the NCF. This may be an important argument for countering claims that the system is only valuable to users with a high degree of interest and experience with computers and computer communications.
The satisfaction results indicate that none of the demographic variables are strongly related to satisfaction. Satisfaction with the NCF is not something that is easily explained by the characteristics of the users.
The main finding of the current research is that the NCF users are not a specialized group in the community. Although there is a large gender imbalance, they are not exclusively male (especially when compared to other similar systems). There is a broad range of ages represented on the system, and a wide variety of education levels. Income levels for NCF users, especially household incomes, are quite comparable to the local region. NCF users are not necessarily highly experienced in computers or computer networks. In areas where there are discrepancies with the local regional data, data from other studies suggests that the NCF is less extreme than other comparable systems.
NCF users are quite satisfied with their system. Their levels of satisfaction are not strongly related to the demographic variables measured here, but the amount of computer experience seems to be the strongest factor (in the negative direction).
The differences found between the NCF users and users of the Cleveland and Amsterdam system are worthy of further exploration. NCF users are more often from the local region, and they have broader characteristics than users of these other systems. What factors give the NCF wider appeal than the systems in these other cities?
There is a need for more research of these types on online systems. We know little about the users and usage of these systems, even though they are being installed regularly throughout the world. The current research did reveal some problems with the methodology that was employed, and future research may want to refine the survey and the collection methods.
Some of the NCF users had problems using the Lynx WWW interface to complete the survey. This interface was new to the NCF system and users were not familiar with its operation. Also, the Lynx program puts high demands on the terminal control functions of the computer interface and some users’ terminal emulation programs were not able to handle this demand. Future research may wish to use a simpler interface or one that is more standard for their system.
Many respondents commented that the survey was too long. Pre-testing in the laboratory indicated that users could complete the survey in 20-30 minutes, but it appears that some users took longer. Future research may wish to prepare multiple shorter surveys that address separate issues (e.g., demographics, service usage, impacts).
There was some reluctance to provide income information in this survey. Some users commented that this information was too personal, while others welcomed the questions and looked forward to the results. The income information provides a valuable comparison with the demographics of the local region, and questions about income should probably be included in future research.
One of the key variables in this analysis was the amount of computer experience. This variable was found to be related to the levels of satisfaction with the system. However, the question that was asked ("How long have you been using computers") provides little information about the type of experience. It does not distinguish between casual users and professional programmers, and even small children may be able to answer that they have many years of experience. Future research should develop more detailed questions about computer experience.
Finally, comparison groups are an important part of this research, but making comparisons can be difficult if the questions and answer options do not match between the different studies. Future research should identify previous studies that can be used for comparisons and ensure that the data that is collected will be comparable. Also, comparisons with local census data are particularly important, and research that uses the same questions and categories found in a census would be beneficial.
Anderson, S.E. (1992). Factors Associated with Usage of a Public Telecomputing System. Curry School of Education, University of Virginia. University Microfilms No. 9324908.
Beamish, A. (1995). Communities on-line: Community-based computer networks. Masters Thesis, Department of Urban Studies and Planning, MIT. URL: http://alberti.mit.edu/arch/4.207/anneb/thesis/toc.html
MIDS. (1995). New Data on the size of the Internet and the matrix. Matrix News, 5(1), January. URL: http://www.tic.com/mids/pressbig.html
Pitkow, J.E., & Recker, M.M. (1994). Using the web as a survey tool: Results from the second WWW user survey. Georgia Institute of Technology Technical Report No. GIT-GVU 94-40. URL: http://www.cc.gatech.edu/gvu/user_surveys/survey-09-1994
Quarterman, J.S., & Carl-Mitchell, S. (1995). Is the Internet all male? Matrix News, 5(5), May. URL: gopher://akasha.tic.com/00/matrix/news/v5/gender.505
Schalken, K., & Tops, P. (1994). The Digital City: A study into the backgrounds and opinions of its residents. Paper presented to the Canadian Community Networks Conference, Ottawa, Canada. URL: http://www.ncf.carleton.ca/freenet/rootdir/menus/freenet/conference2/ conference_papers/dcity.txt
Statistics Canada. (1992). Profile of census tracts in Ottawa-Hull. Parts A & B. 1991 Census of Canada. Ottawa: Industry, Science and Technology Canada.
Andrew Patrick, Ph.D.,Networks Services and Interfaces Laboratory, Communications Research Centre, Ottawa, Canada, http://debra.dgbt.doc.ca, http://www.crc.doc.ca,
For more information, please contact:
Dr. Andrew Patrick at andrew@calvin.dgbt.doc.ca