Introduction to metrics

Metrics are tools used to assess the quality and academic impact of research. Different metrics are used for different research outputs.

Use of metrics

Metrics are used as a signifier of academic impact or engagement. Metrics will contribute to:

  • Reputation of an individual researcher, journal and publisher, research group or institute
  • Career development as they feed into departmental reviews, PBRF, CVs and promotion applications
  • Funding as they are indicative of a researcher's potential impact value
  • Finding collaboration opportunities
  • Strategic publishing decisions when picking an outlet through which to distribute works

A single metric can only tell part of a researcher's impact story, and all metrics have limitations as to what they can articulate about a piece of research. Consequently, using multiple metric tools can help researchers see the full scope of their research impact.

Additionally, metrics should also be used in context. A standalone number does not give the full picture of the article or the author's impact.

Building and maintaining author profiles are essential to boosting the quality of your metrics. Databases and metric tools rely on unique author profiles and IDs (such as ORCID) to monitor and link publications, citations and attention to your name. For more on online profiles and IDs, see Researcher profiles and IDs.

Metrics for authors

Metrics can tell you about your output volume and performance, your prominence in your field and your level of collaboration with internal and external stakeholders.

Key metrics and measures of impact include:

  • Publication number: Total scholarly output of an author
  • Total citations: Total cumulative citation count of an author's works
    • Available via author profiles on Dimensions, Web of Science (citation report function provides further analysis), Scopus and SciVal
  • Field weighted citation impact (FWCI): For the cumulative works of an author, FWCI is the citation count of all items normalised to the average citation count of similar items of the same age, type and subject
  • Collaboration: The cumulative number of items with co-authorship from an author, either international, national, institutional or corporate collaboration
  • Policy mentions: Use of research in government policy documents
    • Available via Sage Policy Profiles after registration with name and/or ORCID or via domain search of government websites.
    • Google site:.govt.nz followed by keywords to domain search New Zealand government sites. As referencing research in policy is not as consistent as in academia, multiple searches may be required to be comprehensive, such as variations on author names, the first author, or the title in quotation marks. Each country will use unique domain suffixes, for example, to search Australian government websites, use .gov.au.
  • Grants and clinical trials: Number of active grants and active clinical trials
  • Author citation indexes: These aim to capture the reuse and referencing of an author's work. Methods vary across indices, often accounting for citations received by individual publications and the author's prolificacy.
    • H-index, available in various databases under your author profile, is calculated from the count of citations to an author’s set of publications. The H-index is the number of papers (h) that have each been cited at least h times.
    • Use this with caution as the value will vary between databases due to their index coverage, and the value is influenced by the length of career and by discipline.
    • i10-index from Google Scholar is the number of papers an author has authored with at least 10 citations.

Metrics for groups

Metrics generated on a research group or institution are from the cumulative works of each individual within the group. Importantly, metrics from institutions are considered in global ranking systems and are indicative of an institution's research impact and its overall prestige.

Key metrics and measures of impact include:

  • Publication number: Total scholarly output of a group
  • Total citations: The total cumulative citation count of a group's works
    • Available on SciVal
  • Field weighted citation impact (FWCI): For the cumulative works of the group, FWCI is the citation count of all items normalised to the average citation count of similar items of the same age, type and subject.
  • Collaboration: The cumulative number of items with co-authorship from a group, either international, national, institutional or corporate collaboration
    • Available on SciVal
  • Institutional-level metrics: Metrics for all authors within an institution can be aggregated into institutional metrics. These form part of the basis for university global rankings, QS World University Rankings or Times Higher Education Rankings

Metrics for research outputs

Metrics for journal articles

The metrics available for articles and conferences depend on where an item is indexed. Different databases have varied, non-exhaustive collections and offer unique tools. Not all articles can be found in every database, and not all articles will have the same metrics. Availability depends on inclusion in the source database's index.

Key metrics include:

  • Field weighted citation impact (FWCI): The citation count normalised to the average citation count of similar items of the same age, type and subject
  • Highly cited papers and hot papers: Highly cited papers are papers published in the last 10 years within the top 1% based on the number of citations received compared to other papers published in the same field in the same year. Hot papers are papers published in the past two years and have been cited enough times in the last two months to place them in the top 0.1%.
  • Alternative metrics: Measurement of use or mention of works outside of academic literature, for example, social media, policy, patents and more
  • Article ranking, percentiles and outputs in top percentiles: Where an article ranks relative to similar publications of the same age and discipline
    • Available on Scopus, SciVal and Altmetric
  • View and/or download counts from publishers and ResearchSpace: The number of times an item has been viewed or downloaded from the publisher's site. This is a possible substitute for items not indexed in large databases.
  • Inclusion on institutional course reading listings
  • Policy mentions: Use of research in government policy documents
    • Available via Sage Policy Profiles after registration with name and/or ORCID or via domain search of government websites.
    • Google site:.govt.nz followed by keywords to domain search New Zealand government sites. As referencing of research in policy is not as consistent as in academia, multiple searches may be required to be comprehensive, such as searching variations on author names, the first author or the title in quotation marks. Each country will use unique domain suffixes, for example, to search Australian government websites, use .gov.au.

Metrics for reports

Commissioned research reports can be cited in other literature, but importantly, they are often commissioned for use outside of academia. The use of a report to inform policy by external groups or the government is indicative of your impact on external stakeholders.

Key metrics and measures of impact include:

  • Citations: Number of times an item has been cited by another item
  • Policy or legislation mentions: Number of times a report has been used in government documents, mentioned in parliament, or included in Cabinet documents. Particularly pertinent if the report is commissioned by a government body.
    • Available on Google Scholar, Altmetric, Sage Policy Profiles (after registration with name and/or ORCID) or via domain search of government websites. Google site:.govt.nz followed by keywords to domain search New Zealand government sites. As referencing research in policy is not as consistent as in academia, multiple searches may be required to be comprehensive, such as variations on author names, the first author, or the title in quotation marks.
  • Prestige of commissioning body: Being commissioned by a prominent group in your field is indicative of your reputation as a researcher.
    • Available on the commissioning body's website or via internet search
  • Inclusion on institutional course reading listings

Metrics for books and chapters

Metric availability for books can be limited due to relatively low indexing in large databases. There are additional metrics for physical holdings and attention received, including reviews and awards.

Key metrics and measures of impact include:

  • Citation count: The number of times an item has been cited by another item
  • Field weighted citation impact (FWCI): The citation count normalised to the average citation count of similar items of the same age, type and subject
    • Available on Scopus and SciVal
  • Alternative metrics: Measurement of use or mention of works outside of academic literature, for example, social media, policy, patents and more
  • Library holdings: Number of libraries that hold a book. In particular, the number of prestigious libraries, within a discipline or otherwise, such as library holdings of THE-ranked universities or Group of Eight universities.
    • Some library holdings are recorded in WorldCat. Note, WorldCat does not include all libraries or complete catalogues.
  • Book/chapter reviews: Reviews can be written for general or scholarly audiences. Find out more about types of book reviews from the University of Chicago’s guide How do I find reviews?
    • Scholarly reviews available on JSTOR using the “reviews” filter and Web of Science using “book reviews” after searching the book/chapter title.
  • Prestige of publisher/editors or collection/series
    • Available on publisher's website
  • Chapter contributions within a prestigious edited book collection: Book included as part of a prestigious book series
  • Awards or best seller listings
    • Available on publisher's website or via internet search
  • Inclusion on institutional course reading listings

Metrics for creative works

Because creative works are disseminated through unique pathways, citation-based metrics are rarely applicable. However, there are other indicators of impact for creative works, some of which include:

  • View and/or download counts: Number of times a work found in an online repository has been accessed
    • Available on ResearchSpace under Statistics, YouTube as view count
  • Sales, commissions and grant number and value
  • Visit numbers for exhibitions/live performances
  • Awards, prizes, best seller listings
  • Prestige of venue, publisher and/or editor
  • Invitations to exhibit/perform
  • Media or public interest
  • Reviews: Can be written for general or scholarly audiences. Find out more about types of reviews in the University of Chicago’s guide How do I find reviews?

Metrics for journals

Journal metrics indicate a journal's prestige based on citations and the quantity of items published, often accounting for a discipline's publishing and citation patterns through weighting. It may be worth considering a journal's performance when choosing where to publish.

Exercise caution when using journal metrics to indicate the value of an article. The prestige of the journal is not necessarily indicative of article quality; rather, it means that, as an author, you are reputable enough to be published in a notable source in your field.

Key metrics and measures of impact include:

  • Journal impact factor (JIF): The average number of citations received by recent articles in that journal
  • Eigenfactor: This algorithm is a prestige metric that reflects the idea that journals are considered influential if they are often cited by other influential journals. Eigenfactor metrics apply to journals indexed in Journal Citation Reports. It uses a five-year citation window (useful for disciplines with a longer time lag before citations are generated).
  • SCImago Journal Rank (SJR): The average number of weighted citations received by a journal in a year, divided by the number of documents from the last three years
  • Source Normalised Impact per Paper (SNIP): The journal’s citation count per paper divided by citation potential in its subject field, where the impact of a single citation will have a higher value in disciplines where citations are less likely, and vice versa.
  • Journal rating lists: These supplement citation-based metrics. Many lists are collated by experts in a field and are specific to a particular discipline. Ideal if discipline is not well indexed by large databases.
  • Google Scholar metrics: Include h-index, h-core and h-median, which contribute to their top journal lists, such as the top 100 publications.

Related links