Armed with a belief in technology’s generative potential, a growing faction of researchers and companies aims to solve the problem of bias in AI by creating artificial images of people of color. Proponents argue that AI-powered generators can rectify the diversity gaps in existing image databases by supplementing them with synthetic images. Some researchers are using machine learning architectures to map existing photos of people onto new races in order to “balance the ethnic distribution” of datasets. Others, like Generated Media and Qoves Lab, are using similar technologies to create entirely new portraits for their image banks, “building … faces of every race and ethnicity,” as Qoves Lab puts it, to ensure a “truly fair facial dataset.” As they see it, these tools will resolve data biases by cheaply and efficiently producing diverse images on command.
The issue that these technologists are looking to fix is a critical one. AIs are riddled with defects, unlocking phones for the wrong person because they can’t tell Asian faces apart, falsely accusing people of crimes they did not commit, and mistaking darker-skinned people for gorillas. These spectacular failures aren’t anomalies, but rather inevitable consequences of the data AIs are trained on, which for the most part skews heavily white and male—making these tools imprecise instruments for anyone who doesn’t fit this narrow archetype. In theory, the solution is straightforward: We just need to cultivate more diverse training sets. Yet in practice, it’s proven to be an incredibly labor-intensive task thanks to the scale of inputs such systems require, as well as the extent of the current omissions in data (research by IBM, for example, revealed that six out of eight prominent facial datasets were composed of over 80 percent lighter-skinned faces). That diverse datasets might be created without manual sourcing is, therefore, a tantalizing possibility.
As we look closer at the ways that this proposal might impact both our tools and our relationship to them however, the long shadows of this seemingly convenient solution begin to take frightening shape.
Computer vision has been in development in some form since the mid-20th century. Initially, researchers attempted to build tools top-down, manually defining rules (“human faces have two symmetrical eyes”) to identify a desired class of images. These rules would be converted into a computational formula, then programmed into a computer to help it search for pixel patterns that corresponded to those of the described object. This approach, however, proved largely unsuccessful given the sheer variety of subjects, angles, and lighting conditions that could constitute a photo— as well as the difficulty of translating even simple rules into coherent formulae.
Over time, an increase in publicly available images made a more bottom-up process via machine learning possible. With this methodology, mass aggregates of labeled data are fed into a system. Through “supervised learning,” the algorithm takes this data and teaches itself to discriminate between the desired categories designated by researchers. This technique is much more flexible than the top-down method since it doesn’t rely on rules that might vary across different conditions. By training itself on a variety of inputs, the machine can identify the relevant similarities between images of a given class without being told explicitly what those similarities are, creating a much more adaptable model.
Still, the bottom-up method isn’t perfect. In particular, these systems are largely bounded by the data they’re provided. As the tech writer Rob Horning puts it, technologies of this kind “presume a closed system.” They have trouble extrapolating beyond their given parameters, leading to limited performance when faced with subjects they aren’t well trained on; discrepancies in data, for example, led Microsoft’s FaceDetect to have a 20 percent error rate for darker-skinned women, while its error rate for white males hovered around 0 percent. The ripple effects of these training biases on performance are the reason that technology ethicists began preaching the importance of dataset diversity, and why companies and researchers are in a race to solve the problem. As the popular saying in AI goes, “garbage in, garbage out.”
This maxim applies equally to image generators, which also require large datasets to train themselves in the art of photorealistic representation. Most facial generators today employ Generative Adversarial Networks (or GANs) as their foundational architecture. At their core, GANs work by having two networks, a Generator and a Discriminator, in play with each other. While the Generator produces images from noise inputs, a Discriminator attempts to sort the generated fakes from the real images provided by a training set. Over time, this “adversarial network” enables the Generator to improve and create images that a Discriminator is unable to identify as a fake. The initial inputs serve as the anchor to this process. Historically, tens of thousands of these images have been required to produce sufficiently realistic results, indicating the importance of a diverse training set in the proper development of these tools.
This means, however, that the plan to use synthetic data to fix the diversity gap relies on a circular logic. Like the computer vision technologies they are meant to supplement, these image generators are unable to escape this “closed system.” The proposed solution merely pushes the problem one step back, since it doesn’t do anything to fix the biases ingrained in the source data training the generators. Without first resolving these shortcomings, the image generators we develop are merely poised to mimic and reflect their existing constraints, rather than resolve them. We can’t use these technologies to create what the training data doesn’t already contain.
As a result, the images they produce could reinforce the biases they’re seeking to eradicate. The “racial transformations” demonstrated in the IJCB paper, for example, created outputs unsettlingly evocative of blackface and yellowface. Another study out of Arizona State University discovered that GANs, when tasked with generating faces of engineering professors, both lightened the “skin color of non-white faces” and transformed “female facial features to be masculine.” Without diversity to start with, these generators were unequipped to create it—ex nihilo nihil fit, from nothing comes nothing.
More concerningly, the biases contained within these synthetic images would be incredibly difficult to detect. After all, computers don’t “see” the way we do. Even if the faces produced appeared completely normal to us, they could still contain hidden idiosyncrasies visible to a computer. In one study, AI was able to predict a patient’s race from medical images that contained “no indications of race detectable by human experts,” as MIT News reports. Moreover, researchers struggled even in retrospect to identify what the computer was observing to make these distinctions.
These synthetic images might also contain details capable of mistraining these tools that are entirely invisible to the human eye. If these systems were to associate these concealed synthetic features with non-white subjects, they would become susceptible to a range of malfunctions we’d be poorly equipped to deal with given our inability to see the relevant differences—an undetectable wrench thrust into the cogs.
There is an ironic contradiction that lurks within these synthetic images. Despite being designed to empower and protect marginalized groups, this strategy fails to include any actual people in the process of representation. Instead, it replaces real bodies, faces, and people for artificially generated ones. As we consider the ethical merits of this proposal, this sort of substitution should give us some pause—not least because of the internet’s long and complicated history of erasure.
Early internet theorists were well attuned to the ways in which digital life was poised to reconfigure our understanding of race. Though some were cautiously optimistic— believing that these possibilities might prove liberating for marginalized groups—the most prescient critics were skeptical, noting that this malleability was, even in its primordial stages, largely reserved for those who already held power. Lisa Nakamura, for example, wrote in the ’90s about the “identity tourism” that she saw going on in chat rooms, the ways the anonymity of the digital space allowed white users to “indulge in a dream of crossing over racial boundaries temporarily and recreationally” by adopting raced personas with usernames like “Asian Doll,” “Geisha Guest,” and “MaidenTaiwan.” Rather than equip people with a new way of reckoning with the thorny, complex realities of identity and its lived implications, digital life seemed particularly adept at extracting these features from their real-world conditions and commodifying it.
As the internet sprawled outward over the ensuing decades, this sort of behavior found expression in an increasing number of ways. The influencer economy empowered digitally rendered figures like Lil Miquela to leverage “mixed-race identity as a form of power and cache,” as Rosa Boshier writes— giving brands the ability to profit off of “a relatable, oppressed queer young woman of color” without having to actually work with one. Meanwhile, white users were able to engage in new, digitally-inflected forms of appropriation thanks to the plasticity of the digital body, wielding tools like facial filters and Photoshop to racialize their appearances for likes. More recently, echoes of the abominable practice of slavery re-emerged via the propertarian apparatus of NFTs, which enabled the buying, selling, and owning of raced avatars for fun. In each of these instances, race became virtualized, transformed into a free-floating trait that could be pinned onto anyone or anything regardless of its actual positionality, often for profit.
Synthetic images of people of color operate along identical lines, separating race from those who live it—transmuting it into pure, manipulable data. Minority subjects would be recast as passive inputs incapable of asking for justice, forced to appear on call to fill in the potholes of our datascapes. In many ways, this strategy takes the logic of abstraction and commodification Nakamura identified and builds it into the fundamental architecture of our emergent technologies. By venerating the digitized symbol, we would free ourselves to forget about the referent in all its concrete, urgent reality.
The idea that we might use synthetic images to train our AI succumbs to the “comic faith in technofixes” that theorist Donna Haraway characterizes as a key dimension of current discourse. Self-assured in our own cleverness—in our ability to solve fundamental problems with yet another tool—we are proposing to build a technological castle on sand. It is a strategy taped together by little more than circular reasoning and motivated largely by apathy. To follow through would not only undermine the potential functioning of these systems, but also mean that we gave in to moral laziness. One might hope that by now, we would have learned our lesson. Shortcuts make long delays.