On the Possibilities of Evaluating Properties of Scientific Documents on the Basis of their Citations Count (or again: What Property is Reflected by Citations Count par excellence, after all?). Part 2: Quality?

V. S. Lazarev


The objective is consideration of the recent works on the problem of the phenomenon reflected by citations count of scientific documents with a slant to the papers analyzing the citations count possibilities to reflect quality of cited research. The method: authoritative definitions of quality were under analysis and interpretation of corresponding research publications in the context of the results the mentioned analysis was undertaken. The results: citations count does not reflect quality of cited research as there are not causal relations between quality and citedness. The conclusions: since in practice the results of the citations count for documents value assessment and of peer assessment of their quality may mostly coincide, this makes it possible still to use citations count as a proxy indicator of quality. As for the “negative” citations, they seem to refer to cited items of high value, but of low quality.


citations count; properties; quality; value

Full Text:



Aksnes, D. W., Langfeldt, L., & Wouters, P. (2019). Citations, citation indicators, and research quality: an overview of basic concepts and theories. SAGE Open, January-March, 1-17. doi: 10.1177/2158244019829575

Bornmann, L., & Haunschild, R. (2017). Does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards societal impact? Scientometrics, 2(2), 937-943. doi: 10.1007/s11192-016-2200-2

Cole, J. R., & Cole, S. (1973). Social stratification in science. Chicago, IL: The University of Chicago Press.

Cole, S., & Cole, J. R. (1967). Scientific output and recognition: a study in the operation of the reward system in science. American Sociological Review, 32(3), 377-390.

Encyclopedia Britannica. (1969). (Vol. 18). Chicago: William Benton Publ.

Gove, Ph. B. (Ed.-in-Chief). (1993). Webster’s Third New International Dictionary of the English Language. Unabridged. Cologne: Köneman.

Lawani, S. M., & Bayer, A. E. (1983). Validity of citation criterion for assessing of scientific publication: new evidence with peer assessing. Journal of American Society for Information Science, 34(1), 59-66. doi: 10.1002/asi.4630340109

Lazarev, V. S. (1996). On chaos in bibliometric terminology. Scientometrics, 35(2), 271-277. doi: 10.1007/BF02018485

Lazarev, V. S. (2018). Is it right to consider the level of citations to scientific papers as the indicator of their quality? Scientometrics: methodology, tools, practical application, 88-103. Retrieved from (in Russian, with English abstract)

Lazarev, V. S. (2019). Insufficient definitions or a vaguely grasped notion? On definitions of “impact”. Scholarly Research and Information, 2(1), 63-78. doi: 10.24108/2658-3143-2019-2-1-63-78 (in Russian, with English abstract)

Martin, B. R., & Irvine, J. (1983). Assessing basic research: Some partial indicators of scientific progress in radio astronomy. Research Policy, 12(2), 61-90.

Nicolaisen, J. (2002). The J-shaped distribution of citedness. Journal of Documentation, 58(4), 383-395. doi: 10.1108/00220410210431118

Quality. (2019). In: Your Dictionary. Retrieved from

Radlov, L. (Ed.). (1904). Filosofskiy slovar: logika, psikhologiya, etika, estetika i istoriya filosofii. St.-Petersburg: Brokgauz - Efron. (in Russian)

Ricker, M. (2017). Letter to the Editor: About the quality and impact of scientific articles. Scientometrics, 111(3), 1851-1855. doi: 10.1007/s11192-017-2374-2

Rinia, E. J., van Leeuwen, T. N., van Vuren, H. G., & van Raan, A. F. J. (1998). Comparative analysis of a set of bibliometric indicators and central peer review criteria: Evaluation of condensed matter physics in the Netherlands. Research Policy, 27, 95-107. doi: 10.1016/S0048-7333(98)00026-2

Significant. (2019). In: Dictionary. Retrieved from

Sistemy menedzhmenta kachestva. Osnovnye polozheniya i slovar, ISO 9000:2015, IDT. (2015). Retrieved from

Tahamtan, I., &·Bornmann L. (2019). What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018. Scientometrics, 121(3), 1635-1684. doi: 10.1007/s11192-019-03243-4

Teplitskiy, M., Duede, E., Menietti, M., & Lakhani, K. (2019). What does citations measure: evidence from citers. Proceedings of the 17th Conference of the International Society for Scientometrics and Informetrics, September 2-5, 2019, Rome, Vol. 2, pp. 2440-2441. Retrieved from

Thornley, C., Watkinson, A., Nicholas, D., Volentine, R., Jamali, H. R., Herman, E., … Tenopir, C. (2015). The role of trust and authority in the citation behaviour of researchers. Information Research, 20(3). Retrieved from

Virgo, J. A. (1977). A statistical procedure for evaluating the importance of scientific paper. The Library Quarterly, 47(4), 415-430. doi: 10.1086/620723


  • There are currently no refbacks.