Lab Manager | Run Your Lab Like a Business

News

researcher searching for academic papers on Google Scholar

Google Scholar Shows Bias Against Non-English Papers

Research analyzing academic searches reveals non-English papers rarely make top 900 results

by
Rachel Muenz

Rachel Muenz, managing editor for G2 Intelligence, can be reached at rmuenz@g2intelligence.com.

ViewFull Profile.
Learn about ourEditorial Policies.
Register for free to listen to this article
Listen with Speechify
0:00
5:00

If you’ve written a scientific article or conference paper in a language other than English, it may as well not exist on Google Scholar, according to recent research published in Future Internet.

Knowing that academic search engines such as Google Scholar have been optimized to ensure that research papers get optimal ranking in search results, researchers from Universitat Pompeu Fabra’s (UPF’s) Department of Communication (Barcelona, Spain) wanted to explore if the language documents were published in affected their ranking by search algorithms.

According to the study abstract, the researchers used “a reverse engineering research methodology based on a statistical analysis that uses Spearman’s rank correlation coefficient”—a way to measure how strong a monotonic relationship exists between two variables—to answer the question of whether language affects the ranking of scientific documents. The researchers did three types of multilingual searches—by keyword, by year, and by author—which each generated 15,000 results. The final data set was made up of 45,000 items obtained from 45 searches that each produced 1,000 results.

They found that, even when doing a Google Scholar search with results in multiple languages, roughly 90 percent of papers in non-English languages “were being systematically relegated to positions that make them virtually invisible,” suggesting “a bias in multilingual searches conducted in Google Scholar.” For the study, the authors defined poor ranking as anywhere outside of the first 900 positions of the search engine results page.  

High-quality, non-English articles still rank poorly in search

Document quality didn’t appear to be a factor in ranking, the authors noted, pointing out that while articles written in English with numerous citations were ranked higher by Google Scholar, papers in other languages did not follow this pattern, with even highly-cited documents failing to appear among the first 900 results of a search.

This isn’t the first time lack of English language has been shown to be a barrier in science. Recently, survey results published in September 2020 in the journal PLOS ONE explored the negative consequences of English language dominance in science. 

Chief among those consequences was the decision by many to leave the sciences due to a lack of English proficiency.

"When I published this on bioRxiv and tweeted, a lot of people started writing to me with very emotional things like, 'I left science because of English,' 'I cannot graduate with a master's thesis because of English,' 'I thought about studying abroad, but then I had the interview and I froze because of English,' (and) 'I couldn't do it.' Super difficult things," said lead author Valeria Ramírez-Castañeda in a press release. "People are leaving science because of English. It is not something that is isolated."

While the authors of the Google Scholar study acknowledge that their study does have its limitations, they say their findings warrant further research, given the potential consequences to scientists.

“It is more than evident that until this bias is addressed, the chances of being ranked in a multilingual Google Scholar search increase remarkably if the researchers opt for publication in English,” the researchers conclude. They add that being unaware of this bias could lead other researchers to mistakenly believe there are no relevant studies in their native language.