7.1.1Scientometrics 1.x – A historical sketch

From the historical viewpoint, scientometrics expresses the development of methods, indicators (metrics) for monitoring and measuring quantitative aspects of scholarly communication. It was originally developed for application to the basic sciences, first within the framework of scientific information. With time elapsing, the increasing demand for indicators in research evaluation resulted in a ‘perspective shift’ (Glänzel, 2006). The main field of application of the metrics was now laid in evaluation and assessment of scientific research. As the first consequence of this shift, both scientometricians and users were faced with a change in application contexts and interpretation of indicators. Indicators became gradually used in contexts for which they never were designed (cf. Journal Impact Factors) and measures of scholars’ communication patterns (cf. author self-citations) were, in the light of the new focus, re-interpreted. Inevitably, first limitations became apparent, uninformed use occurred and earned the attention of both researchers and users. – This was the era of scientometrics 1.0.

Following the pioneering days of the field and its coming of age, a new challenge was issued to the meanwhile established discipline: the necessary extension towards applied sciences, and later on also to the social sciences and humanities (SSH) and technology. The extension of data sources and partially broadening the scope of scientometrics resulted in what can be considered Scientometrics 1.x versions. It has two main characteristics: on one hand the already mentioned “perspective shift” and the trend to the applications to lower levels of aggregation, away from the macro level down to the meso level and increasingly to the evaluation of individual scientists (challenges of individual-level bibliometrics – cf. Wouters et al., 2013). In short, the changes are not only shown in the shift of different targeted samples but also in the scale of scientometric analyses.

The advanced features of Scientometrics 1.x and the challenges from them involve in several issues. The opening and inclusion of new data sources has become an essential prerequisite to meet these challenges. New data sources including proceedings, books, national sources and the web became integrated in the traditional foundation of bibliometrics. Hence also data-related issues arose, including big-data related issues, such as data cleaning, name disambiguation and coping with redundancies. Other issues arising from this broadening the scope of scientometrics were of more conceptual and methodological nature as being closely related to specific cultures in scholarly communication of various fields, notably in the applied, social sciences and the humanities, but also meso- and micro-level specific issues like individual co-authorship, gender, publication in an Open Access (OA) require new qualities of data processing and a higher granularity of information.

Beyond doubt, the traditional scientometric 1.x model had undeniable strengths. First, as to data sources it was based on a dynamic but closed universe: unique, mostly multidisciplinary bibliographic databases such as The ISI Science Citation Index, later on, its successor, Thomson Reuters Web of Science, or Elsevier’s Scopus. This offered a great potential for standardisation and integration of indicators, which, in turn, facilitates comparability of scientometric result. Since it was restricted to the measurement of scholarly communication, it furthermore provides clear definitions of actors, impact and the users of information within this framework (i.e., scholars themselves) and this facilitates the interpretation of scientometric results. Third, because of the general availability of the mostly proprietary data products it shows high level of reproducibility and documentability. Fourth, it proved to work at any level of aggregation and useful in combination with peer review system also at lower levels of aggregation. Finally, mathematical-statistical models for a variety of processes (publication activity, citation impact, co-authorship, citation-based networks, literature growth and evolution, etc.) could successfully be applied to the empirical results.

The other side of the coin are the limitations of the scientometrics 1.x model that should not be ignored. Various opportunities and limitations have been discussed among others by Glänzel and Debackere (2003). Most of those are of methodological or technical nature and concern the use and application of results and indicators. Apart from these, perhaps the most general and conceptual limitation is due to the focus on scholarly communication. However, web-based data sources go, at least in part, already beyond this framework (cf. Google Scholar, web[o]metrics). As an example, shown in a small-scale study, Hoffmann et al. (2014) observed no correlation of online communication activity with any of the more established impact measures.