How to Judge Impact Factors
In this short 5-minute video, I cover the essentials of analyzing literature networks and judging the impact and continued relevance of a given paper.
Specifically, I discuss these topics in the context of a recent high-impact publication by Jeremy England at MIT:
Below is an edited transcript of this video. Thanks to AstraGlacialia on Reddit for the feedback and suggestion.
Finding High Impact Papers
A good place to find high-impact papers is an outlet like MIT Technology Review. While they have some grandiose article titles like, Physicist Derives Laws of Thermodynamics For Life Itself, it’s usually attached to a specific paper.
This particular paper is by Jeremy England at MIT. That’s already a good sign. It’s a high-impact university.
Journal Impact Factor
So first, what is the impact of Journal of Chemical Physics? Turns out its impact factor is 2.965. To compare, the universally regarded high-impact journal Nature is 40.137.
You can dig more into what impact factor is and how it’s measured. Just know that we have two journals, one is higher impact than the other across many different ways of judging impacts.
Relevance and Impact Factors
Now that we’ve determined the journal to be good, we next look at the year of publication. The paper was published in 2013. We must ask, is the work stale and are ideas contained in it still relevant? Any journal search engine, like Google Scholar, is especially good for judging this.
This article is cited 215 times, a large number. Across many domains the average number of citations for any given paper is between 1 and 10. Many many aritlces are never cited at all. Being cited 200+ times is very notable. It’s several standard deviations above the average.
The papers that are citing this one are also in high-impact journals like Nature Physical Review. This paper, while starting in a lower impact factor journal has, over time, been cited by higher impact journals. This is a really good sign that a paper is high-impact.
Additionally, we can look at the Journal of Chemical Physics and notice that it has a pretty nice website. Looking at a journal of lower impact, the Journal of Computational and Theoretical Nanoscience, which has an impact factor of about 1.4, you can see visually that the impact is embodied in the look of the website itself.
The Author
Another factor is the author who published the article and may have published others. How those are going as a good sign of the impact of a given paper. Jeremy England is a professor at MIT. High-impact. He has 1300 citations since 2011. The h-index and i10-index numbers are other measures of impact.
To compare him with myself, I published my Ph.D. a little bit after in 2013, but I’m not an active publisher unlike Jeremy. I have about 300 citations dating back to the same year with the highest being cited 107 compared to Jeremy’s 215. This is an indication that he’s an up and coming professor at MIT and he’s publishing high-impact work. To compare him with somebody even further along in their career, my former adviser Alan J. H. McGaughey at CMU has 6000 citations with the highest single paper being cited 377 times.
The number of people that have these credentials is getting smaller smaller to the point where you’re almost to the highest impact person in the field. David G. Cahill at UIUC has nearly 30,000 citations, with his highest cited paper having over 2,400 citations. It’s extremely rare to find a paper with this number of citations.
Work in high-impact journals typically leads to getting cited in high-impact work, so these numbers are not inflated. This is really a tremendous high-impact piece of work. You’re looking at the pinnacle of impact across journal, paper, and individual.
I hope that was helpfu! I am one of the PhD Mentors on this site providing monthly 1-on-1 mentorship. If you would like my monthly guidance for your PhD journey, feel free to sign up on the homepage.
Best,
Jason Larkin
Cofounder & Principle PhD Mentor
PhDMentors.org
Enter your email to get matched to a PhD Mentor immediately:
Space is limited. Get on the list now for priority consideration.