researchers evaluation

Researchers' Evaluation Methods- Limitations of Measurement Methods

The current bibliometric indices receive many controversial reviews regarding its flaws when evaluating researchers, academic institutions and even research fields.

There are two methods of scientific evaluation:

Qualitative Evaluation:

Qualitative methods include association evaluations, panels, evaluation committees, surveys and receiving research grants. The main faults of this method are in over-subjectivity and the lack of a comparative source. Another disadvantage in this approach is that it involves a lot of effort and many resources to manage all of the factors.

Quantitative Evaluation:

Quantitative methods are based on publications and citations metrics calculated by existing sources of information (Google Scholar, SJR, JCR, Web of Science, Scopus and more). Using quantitative methods significantly reduces the work involved in managing information and investing time and budget. Quantitative indices have many disadvantages and limitations, such as:

  • Differences in information coverage of various databases will lead to different results
  • The differences in publication frequency and number of citations in various research fields can make them incomparable.
  • Some databases are not able to automatically recognize self citations
  • Quantitative evaluation does not consider the duration of a researchers’ career (researchers and institutions with longer careers will gain higher indices over the years)

To get the most accurate evaluation results, it is recommended to combine both methods (qualitative and quantitative). The above indicates that additional metrics are required through which a comprehensive scientific evaluation can be performed.

For more information about researcher evaluation, visit the scientific evaluation guide we prepared for you:

 https://library.technion.ac.il/evaluation/

Do you want to know more?

The Technion Libraries are here for you- we are happy to provide assistance in research and academic tutoring. Please contact us if you have any question or request, and send us your comments or ideas for our next newsletter – The Reference & Instruction Department (Elyachar’s Central Library): reflib@technion.ac.il