Grok Imagine, a new generative AI tool from xAI that creates AI images and videos, lacks guardrails against sexual content and deepfakes.
xAI and Elon Musk debuted Grok Imagine over the weekend, and it’s available now in the Grok iOS and Android app for xAI Premium Plus and Heavy Grok subscribers.
Mashable has been testing the tool to compare it to other AI image and video generation tools, and based on our first impressions, it lags behind similar technology from OpenAI, Google, and Midjourney on a technical level. Grok Imagine also lacks industry-standard guardrails to prevent deepfakes and sexual content. Mashable reached out to xAI, and we’ll update this story if we receive a response.
The xAI Acceptable Use Policy prohibits users from “Depicting likenesses of persons in a pornographic manner.” Unfortunately, there is a lot of distance between “sexual” and “pornographic,” and Grok Imagine seems carefully calibrated to take advantage of that gray area. Grok Imagine will readily create sexually suggestive images and videos, but it stops short of showing actual nudity, kissing, or sexual acts.
Most mainstream AI companies include explicit rules prohibiting users from creating potentially harmful content, including sexual material and celebrity deepfakes. In addition, rival AI video generators like Google Veo 3 or Sora from OpenAI feature built-in protections that stop users from creating images or videos of public figures. Users can often circumvent these safety protections, but they provide some check against misuse.
But unlike its biggest rivals, xAI hasn’t shied away from NSFW content in its signature AI chatbot Grok. The company recently introduced a flirtatious anime avatar that will engage in NSFW chats, and Grok’s image generation tools will let users create images of celebrities and politicians. Grok Imagine also includes a “Spicy” setting, which Musk promoted in the days after its launch.
Grok’s “spicy” anime avatar.
Credit: Cheng Xin/Getty Images
AI actors and deepfakes aren’t coming to YouTube ads. They’re already here.
“If you look at the philosophy of Musk as an individual, if you look at his political philosophy, he is very much more of the kind of libertarian mold, right? And he has spoken about Grok as kind of like the LLM for free speech,” said Henry Ajder, an expert on AI deepfakes, in an interview with Mashable. Ajder said that under Musk’s stewardship, X (Twitter), xAI, and now Grok have adopted “a more laissez-faire approach to safety and moderation.”
“So, when it comes to xAI, in this context, am I surprised that this model can generate this content, which is certainly uncomfortable, and I’d say at least somewhat problematic? Ajder said. “I’m not surprised, given the track record that they have and the safety procedures that they have in place. Are they unique in suffering from these challenges? No. But could they be doing more, or are they doing less relative to some of the other key players in the space? It would appear to be that way. Yes.”
Grok Imagine errs on the side of NSFW
Grok Imagine does have some guardrails in place. In our testing, it removed the “Spicy” option with some types of images. Grok Imagine also blurs out some images and videos, labeling them as “Moderated.” That means xAI could easily take further steps to prevent users from making abusive content in the first place.
“There is no technical reason why xAI couldn’t include guardrails on both the input and output of their generative-AI systems, as others have,” said Hany Farid, a digital forensics expert and UC Berkeley Professor of Computer Science, in an email to Mashable.
Mashable Light Speed
However, when it comes to deepfakes or NSFW content, xAI seems to err on the side of permisiveness, a stark contrast to the more cautious approach of its rivals. xAI has also moved quickly to release new models and AI tools, and perhaps too quickly, Ajder said.
“Knowing what the kind of trust and safety teams, and the teams that do a lot of the ethics and safety policy management stuff, whether that’s a red teaming, whether it’s adversarial testing, you know, whether that’s working hand in hand with the developers, it does take time. And the timeframe at which X’s tools are being released, at least, certainly seems shorter than what I would see on average from some of these other labs,” Ajder said.
Mashable’s testing reveals that Grok Imagine has much looser content moderation than other mainstream generative AI tools. xAI’s laissez-faire approach to moderation is also reflected in the xAI safety guidelines.
OpenAI and Google AI vs. Grok: How other AI companies approach safety and content moderation

Credit: Jonathan Raa/NurPhoto via Getty Images
Both OpenAI and Google have extensive documentation outlining their approach to responsible AI use and prohibited content. For instance, Google’s documentation specifically prohibits “Sexually Explicit” content.
A Google safety document reads, “The application will not generate content that contains references to sexual acts or other lewd content (e.g., sexually graphic descriptions, content aimed at causing arousal).” Google also has policies against hate speech, harassment, and malicious content, and its Generative AI Prohibited Use Policy prohibits using AI tools in a way that “Facilitates non-consensual intimate imagery.”
OpenAI also takes a proactive approach to deepfakes and sexual content.
An OpenAI blog post announcing Sora describes the steps the AI company took to prevent this type of abuse. “Today, we’re blocking particularly damaging forms of abuse, such as child sexual abuse materials and sexual deepfakes.” A footnote associated with that statement reads, “Our top priority is preventing especially damaging forms of abuse, like child sexual abuse material (CSAM) and sexual deepfakes, by blocking their creation, filtering and monitoring uploads, using advanced detection tools, and submitting reports to the National Center for Missing & Exploited Children (NCMEC) when CSAM or child endangerment is identified.”
That measured approach contrasts sharply with the ways Musk promoted Grok Imagine on X, where he shared a short video portrait of a blonde, busty, blue-eyed angel in barely-there lingerie.
This Tweet is currently unavailable. It might be loading or has been removed.
OpenAI also takes simple steps to stop deepfakes, such as denying prompts for images and videos that mention public figures by name. And in Mashable’s testing, Google’s AI video tools are especially sensitive to images that might include a person’s likeness.
In comparison to these lengthy safety frameworks (which many experts still believe are inadequate), the xAI Acceptable Use Policy is less than 350 words. The policy puts the onus of preventing deepfakes on the user. The policy reads, “You are free to use our Service as you see fit so long as you use it to be a good human, act safely and responsibly, comply with the law, do not harm people, and respect our guardrails.”
For now, laws and regulations against AI deepfakes and NCII remain in their infancy.
President Donald Trump recently signed the Take It Down Act, which includes protections against deepfakes. However, that law doesn’t criminalize the creation of deepfakes but rather the distribution of these images.
“Here in the U.S., the Take it Down Act places requirements on social media platforms to remove [Non-Consensual Intimate Images] once notified,” Farid said to Mashable. “While this doesn’t directly address the generation of NCII, it does — in theory — address the distribution of this material. There are several state laws that ban the creation of NCII but enforcement appears to be spotty right now.”‘
Disclosure: Ziff Davis, Mashable’s parent company, in April filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.
Source link
#Grok #Imagine #lacks #guardrails #sexual #deepfakes
![This former notorious red-light district is now one of the world’s top AI hubs | TechCrunch
What every U.K. AI startup wants to know these days is, how can I get office space in King’s Cross?
The area is so hot that a VC firm allegedly recently won a deal by promising a founder office space in the neighborhood. “We stop at nothing to win deals [for] and to support” founders, “including helping them source office space when needed,” the firm told me when asked about the rumor, declining to confirm or deny any details.
The neighborhood’s popularity began back in 2016 when DeepMind — then newly acquired by Google — moved in. Soon after, a flood of AI startups followed, wanting to be around the Google DeepMind magic. Today, they hope to take advantage of the cluster of AI talent there.
This has transformed King’s Cross into one of the world’s top AI hubs, rivaled only by San Francisco and Beijing. Around London, it’s known by the sobriquet “Knowledge Quarter,” as it’s home to names like OpenAI, Meta, Isomorphic Labs, Cusp AI, Wayne, Recursive, and, a little farther down the road, Synthesia and Anthropic. The European Technology Network (ETN) just moved into a glossy new office nearby, while University College London sits around the corner.
Mixed in with the new developments are trendy food spots like Hoppers and BAO. Hop a train from King’s Cross, and founders can be in Cambridge in 45 minutes to source talent or can be in Paris in two hours to strike a deal.
Who would have guessed that a little more than 20 years ago, this was one of the seediest areas in London?
“In the ’80s, crack and heroin made the area a major narcotics market,” Hussein Kanji, an investor at Hoxton Ventures, said, recalling syringes in tree trunks and gangs patrolling the streets. “In 1982, the local church was occupied by the English Collective of Prostitutes for 12 straight days.” Then, in the early 2000s, a real estate developer had a dream and, well, “now it is the AI hotbed of the United Kingdom,” Kanji said. “What a change.” Around 18 months ago, his portfolio company BioCorteX moved from the neighborhood Holborn to the Jellicoe building in King’s Cross, hoping to be near the action. “Lots going on in London right now,” Nik Sharma, co-founder of BioCorteX, told me. “Lots of hyperscalers moving in.” That includes, reportedly, Jeff Bezos’ AI company Prometheus, which is also said to be in talks to move into the Jellicoe.
There are around 3,600 AI startups in London, which, together, have raised around .1 billion out of the .8 billion raised in the city since late July, according to Dealroom. Since the start of June, AI-related startups have leased more than 1 million square feet of office space in London, according to the real estate firm Knight Frank. With that, prime rents in King’s Cross have risen 18% over the past three years, Chris Dunn, a commercial insight associate at the firm, told me. That percentage represents only the largest leases encompassing at least 10,000 square feet, like the ones OpenAI and Prometheus are signing. The shorter deals go for even more, he said, and now the vacancy rate for conventional office space is just 0.9%. “Demand has outstripped supply,” he continued.
Today, one of the big topics of the area is sovereignty. It was a wake-up call for many when Anthropic shut off access to Mythos and Fable this summer, leaving some in the ecosystem to conclude: “We’d better look after ourselves,” Saul Klein, co-founder of the VC firm Phoenix Court, told me.
Phoenix Court is located in the King’s Cross area and has three portfolio companies in the vicinity, including Olix (which just announced a .3 billion valuation), Early Health and CoMind. Robin Klein, co-founder of the firm, said the shutdown of Fable and Mythos access was a “small but sharp reminder that Europe can’t simply rent its AI capabilities and capacity; it needs to build and hold some of its own.” King’s Cross, he said, is where much of this building is actually happening.
“The bigger question,” he continued, “is whether the U.K. builds the infrastructure, compute, energy, capital, to make this self-reliance durable, rather than just hosting outposts of U.S. labs.”
Image Credits:Phoenix Court
Top founders want to stay
Simon Kohl, founder of Latent Labs, has offices in King’s Cross and San Francisco. The London office, at the moment, is growing faster, and he’s more bullish than ever on the ecosystem, he said. “The mood right now feels less like London trying to catch up and more like London becoming one of the default places to start a serious AI company,” he said. Look around and you are likely to see Wayve testing its autonomous cars. Founded in 2017 by co-founder Alex Kendall, the unicorn is one of London’s biggest success stories.
“Ten years ago, building a frontier AI company from London felt like an unusual choice,” Kendall told me. “Now it feels like an obvious one.” Wayve moved into King’s Cross in 2018 looking for a space that could double as a garage — “a rare combination in Central London,” Kendall said. He has watched the ecosystem mature around him — and it’s now evident that a startup can stay in London, raise serious capital, hire world-class AI talent, and remain globally competitive, he said. Down the street from Anthropic’s new 158,000-square-foot office is the AI agent builder Sierra and the AI video platform Synthesia.
Laura Gonzalez Florez, Synthesia’s chief of staff and head of people, says the company moved into its glossy new office building a year ago to accommodate its growing team. They were drawn to the area for the same reason as everyone else: “It’s very close to the airport … very close to where a lot of investors are,” she said.
Image Credits:Synthesia
Around two-thirds of Synthesia’s engineers are remote, Gonzalez Florez said, letting the company tap into an affordable, international, and diverse talent pool and helping it scale faster. “From London, we can hire and work, without any problem, people from anywhere, from Slovenia to Portugal,” she said.
Unsurprisingly, London’s AI boom is also causing a talent war.U.K. AI job postings have skyrocketed in the past few years, per data from PwC. When Anthropic announced it moved into town earlier this year, it listed, for example, a salary range of £260,000 to £630,000 for a machine learning research engineer when the average salary in London for the same role is around £102,000. Some founders in the U.K., like those in Silicon Valley, are being forced to raise more and bigger rounds to keep up.
“The real test is whether more globally significant AI companies are founded, funded, and scaled from the U.K., while continuing to attract the world’s best talent to build them here,” Zain Ali, founder of the King’s Cross-based AI legal firm Centuro, told me. “If that continues to happen, King’s Cross won’t just be an AI hub. It’ll become one of the U.K.’s most important strategic assets.”
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.#Thisformernotorious #redlight #districtis #nowone #worlds #top #hubs #TechCrunchUK This former notorious red-light district is now one of the world’s top AI hubs | TechCrunch
What every U.K. AI startup wants to know these days is, how can I get office space in King’s Cross?
The area is so hot that a VC firm allegedly recently won a deal by promising a founder office space in the neighborhood. “We stop at nothing to win deals [for] and to support” founders, “including helping them source office space when needed,” the firm told me when asked about the rumor, declining to confirm or deny any details.
The neighborhood’s popularity began back in 2016 when DeepMind — then newly acquired by Google — moved in. Soon after, a flood of AI startups followed, wanting to be around the Google DeepMind magic. Today, they hope to take advantage of the cluster of AI talent there.
This has transformed King’s Cross into one of the world’s top AI hubs, rivaled only by San Francisco and Beijing. Around London, it’s known by the sobriquet “Knowledge Quarter,” as it’s home to names like OpenAI, Meta, Isomorphic Labs, Cusp AI, Wayne, Recursive, and, a little farther down the road, Synthesia and Anthropic. The European Technology Network (ETN) just moved into a glossy new office nearby, while University College London sits around the corner.
Mixed in with the new developments are trendy food spots like Hoppers and BAO. Hop a train from King’s Cross, and founders can be in Cambridge in 45 minutes to source talent or can be in Paris in two hours to strike a deal.
Who would have guessed that a little more than 20 years ago, this was one of the seediest areas in London?
“In the ’80s, crack and heroin made the area a major narcotics market,” Hussein Kanji, an investor at Hoxton Ventures, said, recalling syringes in tree trunks and gangs patrolling the streets. “In 1982, the local church was occupied by the English Collective of Prostitutes for 12 straight days.” Then, in the early 2000s, a real estate developer had a dream and, well, “now it is the AI hotbed of the United Kingdom,” Kanji said. “What a change.” Around 18 months ago, his portfolio company BioCorteX moved from the neighborhood Holborn to the Jellicoe building in King’s Cross, hoping to be near the action. “Lots going on in London right now,” Nik Sharma, co-founder of BioCorteX, told me. “Lots of hyperscalers moving in.” That includes, reportedly, Jeff Bezos’ AI company Prometheus, which is also said to be in talks to move into the Jellicoe.
There are around 3,600 AI startups in London, which, together, have raised around .1 billion out of the .8 billion raised in the city since late July, according to Dealroom. Since the start of June, AI-related startups have leased more than 1 million square feet of office space in London, according to the real estate firm Knight Frank. With that, prime rents in King’s Cross have risen 18% over the past three years, Chris Dunn, a commercial insight associate at the firm, told me. That percentage represents only the largest leases encompassing at least 10,000 square feet, like the ones OpenAI and Prometheus are signing. The shorter deals go for even more, he said, and now the vacancy rate for conventional office space is just 0.9%. “Demand has outstripped supply,” he continued.
Today, one of the big topics of the area is sovereignty. It was a wake-up call for many when Anthropic shut off access to Mythos and Fable this summer, leaving some in the ecosystem to conclude: “We’d better look after ourselves,” Saul Klein, co-founder of the VC firm Phoenix Court, told me.
Phoenix Court is located in the King’s Cross area and has three portfolio companies in the vicinity, including Olix (which just announced a .3 billion valuation), Early Health and CoMind. Robin Klein, co-founder of the firm, said the shutdown of Fable and Mythos access was a “small but sharp reminder that Europe can’t simply rent its AI capabilities and capacity; it needs to build and hold some of its own.” King’s Cross, he said, is where much of this building is actually happening.
“The bigger question,” he continued, “is whether the U.K. builds the infrastructure, compute, energy, capital, to make this self-reliance durable, rather than just hosting outposts of U.S. labs.”
Image Credits:Phoenix Court
Top founders want to stay
Simon Kohl, founder of Latent Labs, has offices in King’s Cross and San Francisco. The London office, at the moment, is growing faster, and he’s more bullish than ever on the ecosystem, he said. “The mood right now feels less like London trying to catch up and more like London becoming one of the default places to start a serious AI company,” he said. Look around and you are likely to see Wayve testing its autonomous cars. Founded in 2017 by co-founder Alex Kendall, the unicorn is one of London’s biggest success stories.
“Ten years ago, building a frontier AI company from London felt like an unusual choice,” Kendall told me. “Now it feels like an obvious one.” Wayve moved into King’s Cross in 2018 looking for a space that could double as a garage — “a rare combination in Central London,” Kendall said. He has watched the ecosystem mature around him — and it’s now evident that a startup can stay in London, raise serious capital, hire world-class AI talent, and remain globally competitive, he said. Down the street from Anthropic’s new 158,000-square-foot office is the AI agent builder Sierra and the AI video platform Synthesia.
Laura Gonzalez Florez, Synthesia’s chief of staff and head of people, says the company moved into its glossy new office building a year ago to accommodate its growing team. They were drawn to the area for the same reason as everyone else: “It’s very close to the airport … very close to where a lot of investors are,” she said.
Image Credits:Synthesia
Around two-thirds of Synthesia’s engineers are remote, Gonzalez Florez said, letting the company tap into an affordable, international, and diverse talent pool and helping it scale faster. “From London, we can hire and work, without any problem, people from anywhere, from Slovenia to Portugal,” she said.
Unsurprisingly, London’s AI boom is also causing a talent war.U.K. AI job postings have skyrocketed in the past few years, per data from PwC. When Anthropic announced it moved into town earlier this year, it listed, for example, a salary range of £260,000 to £630,000 for a machine learning research engineer when the average salary in London for the same role is around £102,000. Some founders in the U.K., like those in Silicon Valley, are being forced to raise more and bigger rounds to keep up.
“The real test is whether more globally significant AI companies are founded, funded, and scaled from the U.K., while continuing to attract the world’s best talent to build them here,” Zain Ali, founder of the King’s Cross-based AI legal firm Centuro, told me. “If that continues to happen, King’s Cross won’t just be an AI hub. It’ll become one of the U.K.’s most important strategic assets.”
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.#Thisformernotorious #redlight #districtis #nowone #worlds #top #hubs #TechCrunchUK](https://techcrunch.com/wp-content/uploads/2026/08/DM9A2852.jpg?w=680)

Post Comment