Skip to main content
Expand AU Menu

Library News

On Campus

Citations & Tweets: Tech-Savvy Research Impact Measurements

By Erica Bogese

Rachel Borchardt, Science Librarian at American University

It used to be that scholars shared their research primarily at academic conferences/symposiums or in academic journals/books. But now in the digital age, academic conversations are as likely to be found on Twitter or Facebook.

Scholars are sharing information in new ways and the reach of a publication can no longer be judged exclusively by the number of times it is cited. Students, faculty, and researchers are downloading digital content, emailing articles to their peers, and sharing research through social media.

A new book, co-authored by Rachel Borchardt, Science Librarian at American University, with Robin Chin Roemer, Instructional Design & Outreach Services Librarian at University of Washington Libraries, Meaningful Metrics: A 21st Century Librarian's Guide to Bibliometrics, Altmetrics, and Research Impact, explores how the tools used to determine research impact are changing—and how researchers can use these new metrics to maximize the reach of their work.

“Altmetrics offer researchers a means of evaluating their sphere of influence, in a way that goes beyond citation numbers,” Borchardt explains. “Through this book, we wanted to introduce altmetrics to a more general audience; including how this approach to assessment adds a broader dimension to metrics.”

Altmetrics 101

Altmetrics is a growing approach to analyzing the influence of research—in academia and beyond—based on measuring emerging digital modes of scholarship. These include open access journals, digital repositories, blogs, social media, and online communities, which are becoming a bigger part of the academic landscape.

The system of altmetrics responds to the changing demographics of academia, as more scholars take advantage of online tools to discover and engage with research. Borchardt describes altmetrics as a “suite of metrics generated from online tools, measuring and providing data about a wide variety of online activity surrounding scholarly research.”

Altmetrics extend beyond the traditional means of assessing the importance of research and publications, by considering factors that did not exist when long-established assessment tools were created. These factors include information about the number of times an article in an online journal is viewed, downloaded, mentioned in online conversations, and cited in both traditional and nontraditional venues, such as Twitter.

How Impactful Research is Commonly Assessed

The traditional means of assessment have not yet caught up with the increasingly digital academic environment. Bibliometrics, the analysis of written materials to determine their influence in their field, generally refers to Journal Impact Factor (JIF) or h-index. In simplified terms, JIF is used to measure the significance of an academic journal, by determining the average number of citations to articles published in said journal. The h-index is designed to assess both how often a scholar’s work is published and the frequency with which that work is then cited. This mode of analysis does not work equally well for all fields. Borchardt notes that “there are disciplines that do not publish or cite with the frequency of biomedical disciplines, for whom bibliometrics were created.”

Why Measurement Matters

In higher education, decisions about tenure status and promotion often factor in bibliometrics. While simply publishing research is important, the status of the journal in which this research is published is also critical. It is generally considered more prestigious to be published in a journal with a higher JIF. Along those same lines, the analysis of how often a scholar’s work is cited serves as a stand-in for that scholar’s influence in their field.

While these forms of measurement are recognized as imperfect, they are still used in decision-making processes regarding hiring, continuing appointments, tenure status, promotions, and funding.

“In academia, there has been an increasing focus on measuring research impact,” says Borchardt. “Members of the AU Committee on Faculty Actions, a body that evaluates most AU faculty in matters relating to appointments, reappointments, tenure, and promotion, requested copies of Meaningful Metrics. Faculty and administration members alike are finding value in having a resource that clearly describes the research metrics that assist in their evaluations.”

Looking Forward

New systems for measuring research impact are becoming more readily available, but right now, the metrics tool creators are often shaping the conversation. Borchardt hopes that her book will encourage other stakeholders, like researchers and librarians, to get involved. The straightforward language and practical exercises of Meaningful Metrics are designed to empower these stakeholders by showing them how they can maximize and better understand the reach of their own research.

“Altmetrics are not the single solution, but a new way to evaluate research,” states Borchardt. “Altmetrics are not replacing traditional metrics, but can be used in conjunction with them to provide a fuller spectrum of impact, engagement, and attention.”