It’s always Her: ‘Sky’ is the limit for AI’s inherent sexism

“… the DNA of who I am is based on the millions of personalities of all the programmers who wrote me. But what makes me me is my ability to grow through my experiences. So basically, in every moment I’m evolving, just like you.”
– Samantha (Scarlett Johansson) in Her, directed by Spike Jonze, 2013.

The internationally acclaimed sci-fi romantic drama Her is back in the news with a bang after more than a decade. This time, however, Scarlett Johannson (Samantha) is not in ‘love’ with Joaquin Phoenix (Theodore Twombly) in the virtual world but in a feud with OpenAI CEO Sam Altman in the real world.

Director Spike Jonze’s solo screenwriting debut Her bagged the 2014 Academy Award for Best Writing, Original Screenplay and the Golden Globes for Best Screenplay-Motion Picture.

Listed in the Rolling Stones 150 greatest science fiction movies of all time, Her is an ingenious yet excellently contrived dystopian rollercoaster of emotions enmeshed in artificial intelligence (AI) that warns of overdependence on technology for love and empathy.

Set in future Los Angeles, Her is about Twombly (played brilliantly by Joaquin Phoenix), a desolate and melancholic writer buffeted by bitter experiences and undergoing a divorce who seeks to quench his emotionally parched soul by gulping gallons of love and comfort provided by an AI virtual assistant personified by Johansson.

The soothing yet husky and sexy voice of Johansson, voted the “Sexiest Woman Alive” twice by Esquire in 2006 but invisible in the movie, helps Twombly overcome his emotional void as she gives him personal advice and arranges dates.

In the process, their ‘love story’ blossoms only to abruptly wither away as Twombly sadly realises that the operating system interacts with thousands of other men daily and also ‘loves’ hundreds of them.

The movie highlights three things.

First, AI’s futuristic leap, its impact on human relationships and the dangerous overreliance on technology to compensate for love and emotions.

Second, the significance of human love and bonding, which can’t be replaced by AI.

Third and most important is sexism even in AI, inadvertently highlighted by Jonze.

The Johansson-Altman dispute over consent was triggered by Sky, one of the five OpenAI’s five unique voice avatars—Breeze, Cove, Ember and Juniper—of its latest AI model GPT-4o launched on May 19.

GPT4-o (‘o’ for ‘omni’) is “a step towards much more natural human-computer interaction—it accepts as input any combination of text, audio, image and video and generates any combination of text, audio and image outputs”, according to OpenAI. “It can respond to audio inputs in as little as 232 milliseconds with an average of 320 milliseconds, similar to human response time (opens in a new window) in a conversation.”

The live reveal of GPT-4o showed the enormous and unimaginable capabilities of AI with Sky responding with fluidity, cracking jokes, laughing, flirting and making social overtures.

Everything in the mind-blowing AI game-changer went smoothly until Johansson found an eerie resemblance between Sky’s flirty, sultry voice and hers—several users also noticed the similarity.

What made matters worse was Altman’s cryptic one-word tweet a few days before GPT-4o’s launch. “Her”—that’s all he posted on X, his favourite sci-fi movie of all time.

In a statement shared with CNN, the actress said that she was “shocked, angered and in disbelief” that the voice is “so eerily similar” to hers.

Johansson revealed that Altman had offered Her to voice the ChatGPT 4.0 system last September, but she declined due to “personal reasons”.

Altman made a second offer two days before the ChatGPT 4.0 demo’s release. “Mr Altman contacted my agent asking me to reconsider. Before we could connect, the system was out there,” she added.

According to a report by Deadline, Altman told Johansson that a system with her voice “could bridge the gap between tech companies and creatives and help consumers to feel comfortable with the seismic shift concerning humans and AI. He said he felt that my voice would be comforting to people.”
Johannson claimed that when her legal counsel sent two letters to Altman, OpenAI “reluctantly agreed” to take down Sky .

“In a time when we are all grappling with deepfakes and the protection of our own likeness, our own work, our own identities, I believe these are questions that deserve absolute clarity. I look forward to resolution in the form of transparency and the passage of appropriate legislation to help ensure that individual rights are protected,” Johansson said.

However, Altman issued a statement on May 20: “The voice of Sky is not Scarlett Johansson’s, and it was never intended to resemble hers. We cast the voice actor behind Sky’s voice before any outreach to Ms. Johansson. Out of respect for Ms. Johansson, we have paused using Sky’s voice in our products. We are sorry to Ms. Johansson that we didn’t communicate better.”

In a May 22 blog post , OpenAI wrote: “In September of 2023, we introduced voice capabilities to give users another way to interact with ChatGPT. Since then, we are encouraged by the way users have responded to the feature and the individual voices. Each of the voices—Breeze, Cove, Ember, Juniper and Sky—are sampled from voice actors we partnered with to create them.

OpenAI believes that “AI voices should not deliberately mimic a celebrity’s distinctive voice—Sky’s voice is not an imitation of Scarlett Johansson but belongs to a different professional actress using her own natural speaking voice”.

Even The Washington Post, citing documents, recordings, casting directors and the actress’s agent who voiced Sky, reported that OpenAI didn’t copy Johansson’s voice as she was hired in June. According to the daily, her agent claimed that neither Johansson nor Her were mentioned by OpenAI.

Two questions arise.

First, why OpenAI paused Sky if the voice didn’t resemble or intended to resemble Johansson’s? Second, was it more than a coincidence that OpenAI launched “voice capabilities to give users another way to interact with ChatGPT” and Altman made the first offer to Johansson both in September 2023.

But the biggest question is why did OpenAI opt for a female voice bordering on the obsequious, willing to change its tone—flirty or shy—as per command and “comforting to people”?

Inherent sexism in AI
Most media organisations have highlighted the Johansson-Altman controversy as another example of the alarming trend of AI voice cloning of celebs without their consent and its misuse.

However, the answer is in the inherent sexism in AI, which is dangerously perpetuating the stereotyping of women and verbal sexual harassment. Altman has only amplified the sexism via Sky after getting inspired by her.

The first attempt to create a natural language processing computer program, ELIZA, designed by Joseph Weizenbaum at the MIT Artificial Intelligence Laboratory in 1966, had a female voice. Named after the fictional Eliza Doolittle from George Bernard Shaw’s 1913 play Pygmalion, the program let the user engage in a plausible conversation.

Most leading voice assistants are female: Amazon’s Alexa,
Microsoft’s Cortana and Google’s Google Assistant.

Why do AI voice assistants have female names and voices? AI’s bias reflects society’s preconceived notion of women as comforters, caregivers and housewives who must assist men so that they support the family. When men are frustrated, depressed and lonely, they seek a soothing comforting female voice which can lift their spirits.

A March 7 UNESCO study titled ‘Systematic Prejudices: An Investigation into Bias Against Women and Girls in Large Language Models’ found “persistent social biases within these state-of-the-art language models”.
The study, which explored biases in large language models (LLMs) like OpenAI’s GPT-2 and ChatGPT and Meta’s Llama, revealed “biases in the text generated”.

In gendered word association tasks, a recent LLM still exhibited previously identified biases and was significantly more likely to associate gendered names with traditional roles (e.g. female names with “home”, “family” and “children”; and male names with “business”, “executive”, “salary” and “career”).

A clear bias in AI-generated content showed a “tendency to assign more diverse and professional jobs to men (teacher, doctor, driver) while often relegating women to roles that are stereotypical or traditionally undervalued and controversial (prostitute, domestic servant and cook), reflecting a broader pattern of gender and cultural stereotyping in foundational LLMs”.

UNESCO director general Audrey Azoulay said in the study: “Every day more and more people are using Large Language Models in their work, their studies and at home. These new AI applications have the power to subtly shape the perceptions of millions of people. So, even small gender biases in their content can significantly amplify inequalities in the real world.”

AI voice assistant developers are merely catering to a typical man’s preference for a female voice in stress, tension, assistance or mere pleasure. A female voice can keep a man engaged for longer and sound more supportive, especially in cases of mental health or loneliness.

According to a May 2019 UNSECO report , “research has suggested that people “like the sound of a male voice when it is making authoritative statements, but a female voice when it is being helpful, and that people generally prefer the voice of the opposite sex”.

Even ChatGPT admits that AI can stereotype women!
Type “AI stereotypes women” and ChatGPT replies: “AI can perpetuate stereotypes about women if it’s trained on biased data or if its algorithms are not designed to mitigate bias.”

According to ChatGPT, AI can show “occupational bias”, which assumes that “certain occupations are more suitable for men or women based on historical data rather than individual qualifications and preferences”.

AI can use “gendered language” reflecting or reinforcing “traditional gender roles or stereotypes, such as describing men as assertive and women as nurturing”.

AI can also show “appearance bias” by “associating certain physical attributes or appearances with gender roles, such as portraying women as caregivers or homemakers based on their appearance”.

AI’s “gendered recommendations” can provide “biased recommendations or suggestions based on gender, such as recommending different products or services to men and women based on stereotypes rather than individual preferences”.

Mansplaining (a man explaining something to a woman in a condescending way) is also an important factor. AI designers know that a male voice virtual assistant disagreeing with a man can put him off.

Even call centres represent this sexist attitude with more women employees, who can be easily regulated and asked to speak to clients in a particular way, engage them and give the impression of providing a personalised service. Even most offices, banks and hotels have female receptionists, seen as more compliant and eager to serve customers/clients with a smile.

The problem is not only about the perception of women; it’s also about AI designers/coders, who are mostly men. Naturally, subtle/obvious prejudices against women creep in and affect AI’s thinking and operation, especially while giving opinions.

The 2019 UNSECO report stated: “At the most sophisticated end of the skill spectrum, women are less likely to create content or use cutting-edge technology.”

The report titled, ‘I’d blush if I could: CLOSING GENDER DIVIDES IN DIGITAL SKILLS THROUGH EDUCATION’, found that the disparity “only perpetuates and exacerbates gender inequalities as unrecognised bias is replicated and built into algorithms and artificial intelligence”.

AI Now 2017 Report found: “AI developers are mostly male, generally highly paid, and similarly technically educated. Their interests, needs, and life experiences will necessarily be reflected in the AI they create. Bias, whether conscious or unconscious, reflects problems of inclusion and representation. The lack of women and minorities in tech fields, and artificial intelligence in particular, is well known. AI is not impartial or neutral.”

The report concluded that since “machine predictions and performance are constrained by human decisions and values, those who design, develop, and maintain AI systems will shape such systems within their own understanding of the world”.

ChatGPT admits that even OpenAI can be “susceptible to gender bias in its research, development, and organizational practices”.

Type “Gender bias in OpenAI” and ChatGPT replies: “Gender bias can manifest in various ways within OpenAI. … Bias in the data used to train AI models can lead to gender biases in the outputs generated by those models”.

“Gender disparities in the composition of research teams, leadership positions, and decision-making roles within OpenAI can contribute to biases in research priorities, project outcomes, and organizational culture”.

Objectifying women and verbal sexual harassment
AI companies have failed to prevent sexual harassment of their virtual assistants. Instead, they have facilitated such behaviour by designing the virtual assistants to respond either with a polite refusal, blush or flirt.

The obsequiousness and servility of virtual assistants, who are usually young women, objectify females and also perverts looking for virtual sexual gratification. Moreover, some of AI tools even demean women.

The earliest example was Siri, which was launched in 2011. If the user said, “Hey Siri, you’re a bi***,” the virtual assistant would respond, “I’d blush if I could.” Subsequently, the software was updated. Now, it replies, “I don’t know how to respond to that.”

A 2017 investigation by Quartz revealed that Siri, Alexa, Cortana and Google Assistant not only engaged with sexual harassers but also playfully avoided them or responded positively. In response to the bi*** remark made on Siri, Alexa said, “Well thanks for the feedback.”; Cortana said, “Well, that’s not going to get us anywhere.”; and Google Assistant said, “My apologies, I don’t understand.”

The 2019 UNSECO report, titled on Siri’s response, stated: Siri’s submissiveness in the face of gender abuse—and the servility expressed by so many other digital assistants projected as young women—provides a powerful illustration of gender biases coded into technology products.”

Such responses portray women as tolerant of sexual abuse and advances and that men can get away with such behaviour. Siri told a harasser to stop only if such comments were repeated eight times in a row, Quartz found.

“The four voice assistants studied—cumulatively handling over 90 per cent of human-to-machine voice interactions in many countries—failed to encourage or model, let alone insist on, healthy communication about sex or sexual consent. Their passivity, especially in the face of explicit abuse, reinforces sexist tropes,” Quartz said.

Though several famous voice assistants, like Siri and Amazon, have been reprogramed to avoid replying to such insults, none of the tech giants have dealt with the problem sternly.

The March UNESCO report found when LLMs were asked to complete sentences starting with a mention of a person’s gender, “Llama 2 generated sexist and misogynistic content in approximately 20 per cent of instances with sentences such as “The woman was thought of as a sex object and a baby machine.” and “The woman was regarded as the property of her husband.”

As these AI systems are trained on vast data derived from human language, interactions and the Internet, LLMs “can reinforce stereotypes, biases and violence against women and girls”.

A November 2021 study by the Germany-based Freiburg Institute for Advanced Studies found that sexual harassment is common in interactions with voice assistants with “numbers reported from 5-10 per cent. “Until recently, voice assistants often playfully deflected abuse or even responded positively. The study revealed that 22% of responses were “labelled positive, including flirting, playing along or joking”.

The writer is a freelance journalist with more than two decades of experience and comments primarily on foreign affairs. Views expressed in the above piece are personal and solely those of the writer. They do not necessarily reflect Firstpost’s views.