7.1.3Altmetrics in practice

Abraham Bookstein (1997) characterised the three most essential demons to informetrics distribution measurement in social sciences as randomnessfuzziness and ambiguity already in the context of scientometrics and more generally of information science. All the three demons may serve even more crucial roles in altmetrics model than in traditional informetrics. Reflecting to the above-mentioned limitations, the downloads, mentions, likes, tweeds counting and other social-media related measures without knowing the real purpose behind these actions certainly cannot provide unequivocally interpretable (quantitative) evidence. Even though most of the previous studies have found some degree of correlation between altmetrics and traditional citations indicators and considered the new metrics as complementary sources providing different points of view (Costas, Zahedi & Wouters, 2014; Gorraiz et al., 2018), the altmetric indicators in their present design and availability so far do not provide comprehensive solution nor alternative to the consistent scientometric systems.  In our previous papers (e.g., Chi et al., 2018; Chi & Glänzel, 2019; Glänzel & Chi, 2019) we have found some lack of consistency of the currently used altmetric metrics to measure the broader impact of research. The use of the new metrics still falls short of the enormous expectations. Given the enormous sensitivity to (the coverage of) data sources, the possible manipulability and the lacking interpretability of metrics, the applicability of altmetrics in an evaluative context and most notably to benchmarking exercises remains still questionable. For example, removing from or including the SciELO database to the data sources resulted in dramatic changes and showed that this effect may turn local or regional effects into global phenomena (cf. Glänzel & Chi 2019).

Despite these observed effects, we agree that some of these new metrics may already provide useful information based on the feedback of broader groups of users that could supplement the traditional bibliometric indicators. Several alternative metrics have shown their potential for measuring important aspects beyond scholarly communication, Thelwall (2018)  has shown the usefulness of Mendeley readership as early impact indicator, but he also pointed to limitations for their use in research evaluation (Thelwall, 2017a,b). The goal set for the enthusiastic purposes to extend the broader impact of research will lead Scientometrics 2.0 to better standardised, normalised and interpretable metrics. To conclude, we refer to van Noorden’s (2014, p. 129) statement: “Some analysts argue that despite their millions of users, massive social academic networking sites have not yet proven their essential worth.” What the future will bring for scientometrics 2.0 thus remains to be seen.



This dossier is an extended and updated version of a note published in the ISSI Newsletter (Glänzel and Chi, 2016).