Proceedings of the AoIR-ASIST 2004 Workshop on Web Science Research Methods
Measuring the use of the Internet in research and development. Empirical evidence from seven European countries
Franz Barjak and Najib Harabi, University of Applied Sciences Solothurn Northwest Switzerland, Riggenbachstrasse 16, CH-4600 Olten, Switzerland. najib.harabi @ fhso.ch
Computer networks and particularly the Internet have effectively changed how many activities in research and development (R&D) are carried out: scientists collect data from on-line databases, through web-based surveys or from remote-controlled instruments. They retrieve information from electronic libraries or through search engines and the WWW; they communicate and collaborate via e-mail. At the same time the new possibilities of computer networks have become increasingly important in concepts of science and technology policies across Europe. However, there has been little effort to assess the importance of the Internet in scientific research practice. Scientific analyses have been carried out above all for the US. Statistical databases that cover the technological inputs for science or private R&D activities do not exist. The proposed paper tries to fill in some of this lacking knowledge. It is based on the results of the European Union funded research project “Statistical Indicators Benchmarking the Information Society (SIBIS)”.
Based on scientific literature, statistical and policy documents an indicator system was set up that differentiates between three thematic domains. For each domain several indicators were developed:
Readiness for computerized and networked science: This domain contains indicators on the computer and network infrastructure in science and the computer skills and IT awareness of scientists.
Use of Internet tools and applications: This domain includes indicators on a variety of tasks in science for which computer networks are employed, such as data collection and data analysis, information retrieval, communication, collaboration and publishing.
Impact of the Internet: Indicators on publications and patents are used to assess the impact of the Internet. As scientific collaborations can also be affected by the use of Internet applications further indicators are included on this issue.
Most indicators were developed from new in the SIBIS project. In order to test the indicators a postal survey was carried out among European scientists from seven countries (Denmark, Germany, Italy, Ireland, the Netherlands, Switzerland, and UK) and five scientific disciplines (astronomy, chemistry, economics, computer science, and psychology). Based on this dataset of close to 1,500 scientists a benchmarking of the importance of the Internet at different levels (e.g. countries, scientific disciplines etc.) was carried out.
The implementation of the indicator system and the benchmarking of the Internet in science among European countries in general provided valid and interpretable results. Overall, the differences in Internet access and use in science can be correlated fairly well with the known differences in scientific output among countries. There is no direct linear relationship between these phenomena, but the overall tendency is clear: countries with highly productive science systems have made more advanced use of Internet tools than countries with less productive science systems.
However, the differences are much more pronounced for the five scientific disciplines of the survey than for the countries. As for readiness, astronomers and computer scientists have a better infrastructure and higher Internet-related skills levels than scientists from the other disciplines. They are also more experienced in using Internet applications. Chemists, economists and psychologists in general have lower scores for the readiness and use indicators than the two other disciplines. It is rather difficult to compare the outcome of scientific work between the different scientific disciplines as they publish in different media which cannot be compared easily. In regard to R&D collaborations it is notable that nearly all astronomers are involved in collaborations. Their collaboration networks are the largest and they have the most co-authors of all scientists. At the opposite end of the scale, economists and psychologists are a lot less involved in R&D collaborations. Computer scientists and chemists are somewhere in between the extremes. Hence, for the scientific disciplines under consideration we can state that there are some parallels for collaboration activities and Internet readiness and use. This is not particularly surprising, as the main benefit of many Internet applications is the lowering of barriers to communication and collaboration.
A major conclusion refers to the general availability of S&T indicators and data. The Frascati Manual without doubt represents a very valuable tool for the assessment of data in R&D. However, its most recent version does still not contain any indication as to how to measure the computer input nor the input of other facilities without which modern science and R&D are unthinkable. These gaps should be closed and they should lead to a more detailed and internationally comparable data assessment of inputs and outputs in public science.
The preliminary and exploratory benchmarking of the SIBIS project provides some evidence that Internet availability and use are positively correlated to the amount of new knowledge produced. We identified countries that are more advanced and other countries that are less advanced on their path towards computerised science. With the developments currently underway, computer networks will permeate science to a greater rather than a lesser degree. It is advisable for the countries that have fallen behind to launch catch-up strategies. However, country differences are a lot less important than differences between scientific disciplines. This suggests that in a united Europe it might be more advisable to tailor at least the infrastructure related part of science policy along the needs of academic disciplines instead of country borders.