What’s with Wikipedia and women?

Rajini Rao found the Wikipedia article about herself by accident. “I was sort of flattered,” she said. “It was remarkably detailed. I was like, ‘Somebody read my thesis?’”

But a few months later, the physiologist found a banner across the top of the page, indicating that a volunteer contributor to the encyclopedia had expressed doubt about her academic notability.

MADAMSCIENTIST/WIKIMEDIA COMMONS

Rajini Rao

Rao, a full professor at Johns Hopkins School of Medicine, said the comments stung. “I’d rather not have a wiki page than have a page and then some banner across it saying, ‘Hmm, this person is suspect; they’re an imposter.’”

Eventually, after some conversation among Wikipedia’s editors, the banner was removed. But the question about whether Rao’s work was important enough to rate an entry in the world’s largest general-interest encyclopedia mirrors many skirmishes about notability. Social scientists say that these conversations focus disproportionately on women, and they get at issues of bias, sexism and society that the Wikipedia community is trying to resolve.

The gender gap

What trivial question have you asked Alexa, Siri or Google recently? Odds are, the virtual assistant pulled part of its answer from a Wikipedia article. The encyclopedia, which celebrated its 20th anniversary last year, is visited by 1.5 billion unique devices every month. It distributes information even further via voice assistants, Google reference cards, and technologies that mine its vast database for information and for natural language patterns.

A massive volunteer effort and huge donations of time and scholarship are needed to keep the project running and up to date. In addition to the Wikimedia Foundation, which provides servers, software and other infrastructure to run encyclopedias and databases in dozens of languages, the English-language Wikipedia project has about 130,000 active contributors, who call themselves Wikipedians, and 1,000 volunteer administrators.

But significant gaps exist in the coverage. When last surveyed a decade ago, about 85% of English Wikipedia’s editors were men. Most lived in the U.S., the U.K. or India. And of the roughly 1.8 million biographies they jointly have produced, at least four out of every five is about a man.

ZMCCUNE/WIKIMEDIA

Attendees of the 2016 Wikimedia conference included co-Wikimedians of the Year Emily Temple-Wood (left) and Rosie Stephenson-Goodknight (third from left).

The encyclopedia’s gender imbalance is hardly news; the Wikimedia Foundation has acknowledged a problem since that 2011 survey, and efforts to change things have received a great deal of media attention. Wikipedian Rosie Stephenson–Goodknight co-founded a project called Women in Red to write articles about women mentioned in encyclopedia articles on other topics; the project has created over 175,000 articles in five years.

Similar efforts have focused specifically on women scientists. In 2016, Stephenson–Goodknight and Emily Temple–Wood jointly were awarded Wikipedian of the Year by Wikipedia co-founder Jimmy Wales in recognition of Women in Red and of Temple–Wood’s efforts to write new pages about women in science. More recently, British materials scientist Jess Wade and American bioinformatician Maryam Zaringhalam published a call to action in the journal Nature inspired by the dearth of articles about women in science. Advocacy organizations such as 500 Women in Science have run Wikipedia edit-a-thons focused on including more women in the encyclopedia.

According to Stephenson–Goodknight, who recently was elected to the board of the Wikimedia Foundation, “When editors such as Maryam, such as Jess, such as Emily have a platform and their voice can be heard … that has a significant effect.”

WIKIMEDIA COMMONS

Jess Wade

For instance, a Wikimedia Foundation researcher demonstrated in a 2017 paper that efforts such as Temple–Wood’s Women in Science project can have a dramatic impact on the encyclopedia’s coverage gaps and the quality of pages on a topic. Stephenson–Goodnight said, “When you shine a light and talk about a specific topic related to Wikipedia, it has this effect of improving the representation on that topic.”

Yet when biographies of women are added, they sometimes are challenged, as Rao’s was, over whether these women are important enough to merit coverage in a general-interest encyclopedia. In 2018, when physicist Donna Strickland received the Nobel Prize, Wikipedia had no page about her. An article had once existed — but it was deleted from the encyclopedia by community consensus after a volunteer questioned her notability.

Who writes Wikipedia?

The article on Rao was written by a Wikipedian who goes by the name Microglia145. The contributor’s user page is brief and to the point: “I will deliver the unheard stories of notable women in STEM to the public in an unbiased way through Wikipedia.”

It’s common for contributors to focus on specific topics. According to University of North Carolina assistant professor Francesca Tripodi, a sociologist at UNC's school of information and library science who studies how communities use the platform, “Wikipedians are a community, but many of them go into different spaces.”

MACKENZIE LEMIEUX

Mackenzie Lemieux

Some volunteers specialize in contributing in topic areas they care about — and there are projects organizing volunteer efforts on every topic from soap operas to sports, synthesizers to seamounts, and thousands more. Other volunteers focus on formatting citations, finding open-source images for new articles or checking new articles for problems. Administrators, elected by the community, have special privileges to protect or delete pages, block problem users and remove spam.

Postgraduate student Brianna Bibel has been contributing to pages about biochemistry and women in science on and off since becoming inspired by a news story about Wade in 2018. “In school, people would always (say) ‘You don’t want to use Wikipedia, because anyone can edit that,’” Bibel said. Until she read about other scientists’ editing campaigns, “It didn’t really click that ‘anyone’ means you.”

The human behind Microglia145, a medical student at Washington University in St. Louis named Mackenzie Lemieux, also was inspired by reading about Wade. During early pandemic lockdowns, she began writing an article a day about a woman in science or medicine. In this, Lemieux joined a years-old tradition of Wikipedia reformers, women who focused specifically on redressing the encyclopedia’s gender imbalance.

Because it aims to write a general-interest encyclopedia, filled with verifiable facts on notable subjects that are relayed in as neutral a tone as possible, the Wikipedia community has many rules. New editors are apt to run afoul of these rules, especially the ones concerning neutrality; most come to write on topics they’re passionate about, and they tend to gush. And biographies are particularly tricky to write.

“One thing you’ll learn if you edit Wikipedia is it’s easier to do — and make it stick — a biography of a deceased person versus a living person,” said academic librarian Laurie Bridges, who has organized numerous themed edit-a-thons.

Articles about living people are simultaneously more likely to be vanity projects and to be targeted in smear campaigns. Prominent figures’ pages are locked down to prevent nonfactual contributions. Brand new pages of any type, but especially biographies, are checked rigorously by multiple editors for violations of copyright law, defamation or blatant self-promotion, and signs they were written by writers for hire.

While writing one article a day, Lemeiux received a mix of feedback from more seasoned editors. Several praised her commitment and productivity. Others coached her on writing to Wikipedia’s style: Avoid academic titles, such as Dr. Don’t copy lists of awards directly from other websites; that may raise copyright infringement concerns. Above all, editors repeatedly recommended, work hard to maintain a neutral point of view, avoiding words like “prominent,” “innovative” and “pioneering” that sound inherently promotional. Several moved pages she had written from the live encyclopedia into a sandbox called draft space until they could be improved. Several others nominated some of Microglia145’s pages for outright deletion.

Tripodi, the sociologist, was working on her dissertation when she began to attend edit-a-thons aimed at closing the gender gap. She met many volunteers motivated by the same concerns as Microglia145. She also noticed that many contributors complained about seeing their articles flagged for deletion soon after an edit-athon ended — or even, sometimes, while it was ongoing.

“I started thinking, ‘Is this happening proportionally?’” Tripodi said. “‘Are women being targeted, or are women just seen as less notable subjects, even though they’re meeting this threshold for inclusion?’”

She set out to find out.

Deletion discussions

Once a month, working with a software developer, Tripodi scraped a webpage where contributors discuss Wikipedia articles that one or more editors have voted to delete.

When an article is nominated for deletion, it is linked on a page called Articles for Deletion. The community has a week to weigh in. Any editor who is paying attention may make an argument for deleting an article or in its defense.

The conversation, according to Stephenson–Goodknight, focuses on how well an article follows Wikipedia’s rules for neutrality, notability, verifiability and reliable sources. Notability — whether a subject is important enough to rate a Wikipedia page — is often the sticking point. And it’s a little different for professors than for the general public.

David Eppstein, a distinguished professor in computer science at the University of California, Irvine, has contributed over 3,000  articles to Wikipedia, including about 2,000 on women in math and science. “When I started editing on Wikipedia, there was much more of an informal (guideline): Does this person stand out above the  average professor?” Eppstein said. “Nowadays, it’s more formalized in terms of signals of recognition like fellowships or highly cited publications.”

The community has come up with metrics for academic notability that focus on scholarly impact; automatic qualifiers include prestigious awards, named chairs and fellowship in honorific societies. Other criteria include high-impact scholarship, which usually is measured by citation indexes, and press coverage of a scientist’s work.

“You know and I know that all of these things are loaded with bias and inherent problems within the scientific community,” Wade said of the notability guidelines. In addition, because the encyclopedia relies on secondary sources, forbidding editors from doing their own research in the scientific literature, who makes it into the digital pages of Wikipedia is not just up to editors, Wade said.  “You can’t write a Wikipedia page solely on (a scientist’s) academic profile on their university website. You actually need that scientist to be spoken about by other people.”

Eppstein said, “There are quite a few women who meet those criteria and are prominent within their research specialty and have received that kind of recognition, but don’t yet have Wikipedia articles.” He makes a habit of writing about such people when he happens to know about their research.

He also keeps an eye on the Articles for Deletion page, watching for academics incorrectly placed there — but, he says, there are plenty of cases where deletion is the correct outcome. “Oftentimes articles are created for people who, really, the world is not ready for an article for them yet. They’re working, they’re doing good stuff, but they’re not at that level of prominence yet.”

When an article is nominated for deletion, the discussion lasts a week. After the community reaches consensus — determined not by numerical vote but by how well contributors defend an article’s adherence to Wikipedia’s guidelines — an administrator acts to close the discussion and either delete a page or keep it and archive the concerns.

Tripodi, studying these issues, monitored the Articles for Deletion page for more than three years, building a data set she could use to test what she had observed qualitatively at edit-a-thons, that articles about women were more likely to be nominated for deletion than articles about men. She also kept an eye on the outcomes of those conversations.

Over time, she saw a pattern play out. The 22,000 biographies that landed in Articles for Deletion conversations were disproportionately likely to be about women. But pages about women were more likely than pages about men to be retained after discussion. She called these miscategorized.

“They met the criteria for inclusion at a point where they shouldn’t have been (on the Articles for Deletion list) to begin with,” she said. “If there weren’t networks like Women in Red devoted to saving these articles, these articles would have been deleted.”

Saving pages

Some academics described the process of discussing articles for deletion as invigorating, similar to responding to peer reviewers. However, Tripodi said, many others find it demoralizing.

In the summer of 2020, Lemieux wrote an article on psychologist Ayana Jordan, a clinician–researcher who directs a research center at Yale and had been in the news a lot discussing the mental health impacts of the pandemic and systemic racism. As she usually did, Lemieux tweeted about the new article. Responses were positive; Jordan herself was involved in the conversation.

The problems began when Twitter users, clicking through, noticed the article was flagged for deletion. The editor who had nominated it wrote, “Assistant professor … who does not meet the notability standard for academics or (general notability guidelines). The article itself is promotional and CV like.”

Lemieux’s Twitter followers were indignant. One or two pointed out that this happens a lot with new Wikipedia pages describing women in STEM, especially women of color (Jordan is Black). Several, aware that anybody can edit Wikipedia, made their own accounts just to join the discussion page, weighing in in favor of retaining the article.

A JOHNSON/ WIKIMEDIA COMMONS

A photo illustration created for the 2020 event WikiHerStory, which the Wikimedia foundation organized during women’s history month to promote inclusion of women’s history in the encyclopedia, shows contributor Jess Wade, one of three WIkipedians who were highlighted as part of the project.

This behavior annoyed established Wikipedians. Several commenters on the page pointed out that the system, while based in community discussion of the encyclopedia’s rules and guidelines, is not meant to operate as a democracy. The administrator who closed the discussion wrote, “This discussion suffered greatly from outside canvassing. This always makes the task difficult for a closer (of the discussion), but rarely has the desired effect.”

The page was deleted.

“It’s pretty disheartening, after you’ve spent so much time writing a page, to have it deleted,” Lemieux said. She also said that Wikipedia administrators discussed on the Jordan discussion page banning her as a user over recruiting outsiders to stuff the ballot box, which she maintains she did not do.

Skirmishes over academic notability happen regularly, though they don’t usually involve outsiders. Wikipedians argued over astrophysicist Katie Bouman, who became famous when she was photographed reacting to the first-ever image of a black hole; although she became the media face of a 400-physicist team, Slate reported at the time that some editors argued that her scholarly impact did not warrant an article of her own and she should be folded into the article about the black hole itself. Bouman now has her own page.

Wade and other editors receive a lot of negative attention alongside the positive; some fellow contributors are skeptical about the biographies they choose to write. In a backroom conversation on David Eppstein’s user talk page several years ago, one editor posted, “From my perspective, if there are more biographies of women being AfD’d, it’s because there are more biographies of non-notable women being created.”

Another was more blunt: “Ask Jess Wade to write borderline articles about white male academics, and we can AfD them at ease.”

Eppstein argued that responding to Wade’s fame by “poring through her hundreds or thousands of article creations looking for weaker articles to nominate … is 100% a bad thing. We have already driven away at least one other productive member of Women in Red by exactly the same tactics.”

Wade said her articles aren’t flagged so often these days; anyway, when they are, she likes the challenge of defending them. “I quite enjoy (it). You know when you get comments from peer review, and you’re like, ‘Ha! I can take you on’? I quite like that.”

Lemieux said that the argument over Ayana Jordan didn’t impact her motivation as an editor. But it did suggest a next project that intrigued her.

“After I reached 100 pages, I decided to dedicate my time to exploring the extent of racial and gender bias in Wikipedia, specifically for women in STEM,” Lemieux said. She and a friend, Rebecca Zhang, reached out to Tripodi, whose paper had been published in the journal New Media and Society, in July 2020.

Working with Tripodi, Lemieux and Zhang designed a study to test the impact of Google hits, which administrators often use as a quick proxy for notability when weighing whether to nominate an article for deletion. Their study has yet to be peer reviewed, but Lemieux said that among the male subjects of articles nominated for deletion, search notability is a decent predictor of whether the deletion goes through. “But that’s not the case for white women, or men or women of color.”

“Wikipedia, in good faith, intentionally tried to keep some subjectivity in terms of how to apply (notability) criteria, because they weren’t sure what would matter,” Lemieux said. But that subjectivity, she said, has left the door open for societal biases to factor into how pages are assessed. “There needs to be more research about specific areas where bias exists … and how we can begin to improve it.”

“If you’re looking at inequality as like peeling back the layers of an onion,” Tripodi said, “the first layer is that … women don’t get hired for those positions.” Second? “We’re in those positions, but we don’t really get recognized — we don’t get covered by the media, or we don’t get books written about us.” And third: “There’s this tiny subset of women who are getting hired and are getting things written about them — but then aren’t able to stick on this resource.”

The article on Rajini Rao that Lemieux created was flagged for notability (although not nominated for deletion) in the summer of 2021, about a year after Lemieux stopped editing. When Rao noticed the banner at the top of the page, she too took to Twitter. “I was like, ‘What is this? Can anyone help me? Does anyone know what to do?’” she said. “And it was just a wonderful experience.”

Wikipedia editors, including Wade, marshaled arguments in support of Rao’s prominence; a friend in Spain, who Rao said is not even a scientist, logged in to post about how much higher Rao’s h-index was than the average biologist’s. The article remains online.

The way forward

In Wade’s four years of intensive editing, she said, she and fellow activist editors have learned a lot about working with other factions in the Wikipedia contributor community. “I think that everyone is working toward the same aim. It’s just that we need to get to a point where the old-school editing community recognizes that there are women who should be there and aren’t there, and we recognize that not every single woman who ever existed needs to be on Wikipedia.”

In the years since she started writing women into the encyclopedia, it’s gotten easier to find sources in medical and scientific journals, and more diverse scientists are winning awards and fellowships. “All this is kind of happening,” she said. “And I think my role, our role as people who care about this is to go out and just try and make it happen faster.”

Stephenson–Goodknight said, “What all of us, all editors of all genders, can deal with is only those sources that we can find. If society doesn’t provide us with sources … what we can’t do is go back and change history.”

Just how representative is Wikipedia today?

To measure representation of prominent scientists on Wikipedia, ASBMB Today analyzed a list of the 1,052 researchers inducted into the American Association for the Advancement of Science in 2021 and 2022. AAAS membership is one way to meet the encyclopedia’s notability threshold.

The two years’ worth of AAAS fellows included almost twice as many men (676) as women (376). However, a higher proportion of women in the data set (40%) had Wikipedia articles than men (28%) as of January 2022. A small number of prolific page creators, including Jess Wade and David Eppstein, accounted for the majority of the difference in pages about this group of scientists.

The date of article creation also shows the impact of activist editors. From 2002 to 2017, the number of new articles about men in this cohort fluctuated, but each year they outnumbered new articles about women in the cohort. From 2019 onward, the number of new articles about women in the cohort outstripped new articles about men.

Visit github.com/Laurel-O/wikipedia.git for more information on how these data were collected.