Report

Unheralded AI Researchers

Finding Hidden AI Research Talent

5
min read
Download PDF

/

Author

Authors

Jonathan Mak
Plaintext Group
Researcher
Frank Long
Plaintext Group
Research Lead
Nick Rose
Google
Product Manager
Anna Mitchell
Schmidt Futures
Senior Associate Product Manager

Individual Endorsers

No items found.

All individual endorsers participated in their personal capacity. This report was prepared independently from any political or governmental entity. While the report generally reflects the observations, insights and recommendations of the endorsers, it is not the case that every endorser will agree with everything expressed herein.

Executive Summary

Thesis: Prolific individuals contributing significantly to AI Research exist at a broader range of institutions, not just the most highly ranked universities.

AI conferences are an indicator for quality research.
Academic conference publications are selected through a blind review process, and are a validated indicator of vetted quality research. Dewey, a computational bibliometric review system, allowed analysis and scraping of metadata from ~20,000 publications in top AI conferences. 
Highly ranked universities are at the forefront of AI research, but many schools “punch above their weight” and make significant contributions to the field. 
Despite having a smaller faculty, Rutgers (unranked by US News Top AI Programs) published 38 papers into top AI conferences in 2019 compared to Harvard (ranked #18), who published 35 papers.
Prolific individual researchers drive innovation at less highly ranked schools.
Ming-Hsuan Yang (UC Merced) publishes the 3rd most into top AI Conferences (2013 - 2019), while Milind Tambe (Harvard) is 26th.

AI Conferences are an Indicator for Quality Research

Every year, countless Artificial Intelligence (AI) research papers are published, and it can be hard to distinguish which make a lasting impact. For example, NeurIPS (a highly touted AI conference) paper submissions continue to grow, and in 2020, the number of submissions was greater than 0.5% of all scientific papers published in a year. Academic conference paper submissions, which are currently selected through a blind review process, can act as a validated indicator of vetted research and a potential solution to bolster academics in Artificial Intelligence. This paper highlights undiscovered researchers making an outsized impact on the industry.

Key Insights

Key Insight 1: As Expected, Elite Universities are Often at the Forefront of AI Research

Top ranked universities publish the most into academic AI conferences and receive the most media attention for their efforts. Figure 1 shows that a handful of elite research universities tend to publish the most, and are ordered roughly the same way as they are ranked by typical top AI graduate program lists.

Schools ranked higher on program lists often receive more resources to keep positive momentum going. For example, Carnegie Mellon, which is at the top of the list for university publications (Figure 1), recently received the most AI funding from the NSF. Similarly, large tech companies like Apple, Facebook, and Google recruit from these same top 10 schools. As a result, this positive feedback loop leads to increasing funding and resources to higher ranked schools, and increases the gap between institutions in the top 10 and those outside.

Key Insight 2: A Surprising Number of Schools Outperform their Rankings

Despite having fewer resources, many schools outperform top research powerhouses at elite AI conferences. Beyond the Top 10 schools, a broad range of universities such as Rutgers (unranked), University of Maryland (ranked 16th), and New York University (unranked) as judged by this top AI program list, outperform other institutions with historically more established AI departments and researchers. Figure 2 shows that several of these universities outperform Ivy League institutions in terms of AI Publications into Top Conferences, and the most significant results are highlighted in Table 1. A full list of 2019 publication counts is available here.

US News Rank (Academic Prestige) vs. Paper Publication Count (Conference Success)

While these schools rank lower on the US News Report Top AI Programs and tend to have less resources, they outperform their academic prestige and publish a significant amount.

US News Rank (Academic Prestige) vs. Publication Count:Faculty Ratio (Efficiency)

These same schools are also highly efficient, and publish more despite having a lower faculty count.

Furthermore, despite having a smaller AI department size, smaller schools have a higher publication count to faculty (PCF) ratio, which outperforms even other highly funded institutions (Table 2). Even more importantly, higher ranked institutions like Harvard and University of Maryland consistently receive more media coverage and funding, but have the lowest PCF Ratio and efficiency. As such, schools such as Rutgers and UCSD should be recognized for their prolific publishing profile.

Key Insight 3: Individual Prolific Researchers Contribute Highly to Innovation at Institutions

Mapping prolific AI Research can be traced all the way down to the researcher level.

Lastly, singular and highly prolific researchers that are less well-known exist outside of elite institutions, driving a majority of total paper count. Table 3 highlights the prominent authors that have published from 2013 - 2019 outside of the top 10 institutions. For example, UC Merced’s Ming-Hsuan Yang publishes more into top AI conferences than the highest ranked Harvard researcher, Milind Tambe.

Final Thoughts

Top AI Research is Abundant Outside of Top 10 Institutions

Since the rise of deep learning, big tech and elite universities have crowded out many top AI conferences. However, analysis shows there are top schools outside of typical elite institutions that publish prominently into these conferences. These universities and researchers are able to outperform higher ranked institutions even through resource and funding constraints. Highlighting these researchers will be imperative to bolster and diversify the AI research landscape as well as help provide adequate recognition towards unheralded schools. Elite institutions like Harvard get significantly more interest compared to Rutgers, despite Rutgers’ high publication count into AI Conferences (Figure 3).

This paper presents several institutions and authors that are prominent in their field but may have less public visibility, and how Dewey is able to highlight these prolific researchers. Moving forward, becoming more cognizant of the diverse academic landscape within AI will be helpful in creating new opportunities for institutions and individuals.

Acknowledgements

Thank you to Tom Kalil, Chris Kirchhoff, Kumar Garg, Kyle McEneaney, Tomas Pueyo, Dewey Murdick, Jordan Blashek, Jasmine Sun, and Kasey Luo for their valuable insights in formulating a thesis for this paper.

Continue Reading