Proceedings of the AoIR-ASIST 2004 Workshop on Web Science Research Methods























Changing Evidence and Changing Paradigms: How does technology affect the claims of scientific research?

Irene Berkowitz, Temple University, Philadelphia, PA.

Internet Communication Technologies (ICTs) have a profound affect on our ability to collect, store and transmit data. This in turn has allowed for new forms of secondary, tertiary and even higher order analysis of evidence through a process often referred to as knowledge discovery. The foundation of positivist scientific paradigms relies on empirical methods using carefully documented and detailed observations, and somewhat rigid methodological protocols. The evolution of increasingly sophisticated inferential statistics has provided methods for drawing broad conclusions using observations from limited, but highly controlled data sets. Current research methods using ICTs allow for the development of data sets that are broader and of higher magnitude, but frequently less controlled. One of the questions that must be brought into focus at this point is whether this movement towards reliance on more derivative forms of evidence is affecting basic tenets of scientific method, and what impact this might have on the claims of contemporary scientific research.

Philosophers of Science and post-modern critics of the scientific process have argued that the “process of science” and in particular, laboratory-oriented science, is subject to numerous factors that compromise the positivist scientific approach. Many practicing scientists, on the other hand, suggest that the very nature of the discovery process is fluid and dynamic and that this fact does not compromise the viability and validity of the scientific process, even if protocols are not as rigid as described or don’t necessarily follow an a priori approach. Thus, the conclusion of some scientists is that even though the nature of collecting and analyzing evidence has undergone significant transformation in the past decade due to the enhanced capability provided by ICTs that this process in itself does not change the underlying fundamentals of the scientific process, if an appropriate scientific protocol is followed to collect and analyze data. On the other hand, legal scholars and philosophers, who study the field of evidence, have traditionally preferenced primary evidence over secondary, tertiary (and now even higher-orders of evidence) when attempting to determine “if the facts represent the truth” and would suggest that the nature of the evidence does in fact affect the validity and reliability of conclusions.

But are contemporary methods of conducting research via ICT’s creating changes in scientific paradigms? And if changes are happening, how might they be characterized? What impact does this have on our research claims? And, how do we best address the vulnerabilities in these research methodologies and research designs, or is there really a need to be concerned at this juncture?