The internet is being rebuilt for machines | TechCrunch
Cloud infrastructure has long been designed around humans who search, click, scroll, and stream in a steady and predictable fashion. AI agents behave differently. They can unleash a swell of activity, spinning up multiple sub-agents that query hundreds of databases, search documents, and call APIs in seconds and then disappear as quickly as they arrived.
Under that premise, Amazon is redesigning a core piece of its cloud infrastructure. On Thursday, AWS launched its next generation of OpenSearch Serverless, a fully managed search and vector database — essentially a system for storing and retrieving information at scale — that’s designed specifically for agentic workloads. AWS says the new system can instantly scale up when agents trigger tasks and scale back down to zero when idle.
The launch reflects a growing realization across the tech industry: Infrastructure originally designed for a human-driven internet doesn’t work as well in a world increasingly populated by agents.
While AI agents still represent a relatively small portion of internet activity, machine-generated traffic is already significant, and poised to grow. Cloudflare says bots accounted for 31% of overall HTTP traffic over the last six months. AI crawlers, search engines, and assistants made up roughly a quarter of all bot requests during that period.
“Non-human traffic will exceed human traffic sometime in the first half of 2027,” said Lai Yi Ohlsen, senior product manager at Cloudflare, to TechCrunch.
At Google’s I/O developer conference last week, the company said users will be able to start delegating tasks to AI systems, like researching purchases, booking travel, browsing the web, and interacting with apps. But the buck doesn’t stop at consumer-focused AI agents. Enterprises are increasingly deploying agents internally and for their customers, creating new kinds of machine-generated traffic behind the scenes.
As a result, cloud providers and infrastructure companies have been reckoning with how to adapt systems built for humans to a world of agents that are constantly and autonomously retrieving information, invoking tools, and generating machine-to-machine traffic.
That’s where AWS’s new OpenSearch Serverless comes in.
“The timing is straightforward. Agents are moving from experimentation into production, and they create traffic patterns that previous infrastructure simply wasn’t designed for,” Tia White, general manager for Amazon OpenSearch Service, told TechCrunch. “They spike without warning, they go idle without notice, and enterprise needs search that keeps up without paying for empty or idle compute.”
The key technical change with this new generation is that it decouples compute from storage, allowing compute to scale up in seconds to accommodate agent traffic bursts and to scale down to zero, so customers pay $0 when agents are idle.
“Previously, even in our prior Serverless version, you had to have at least one instance operational and running because storage and compute were coupled,” White said. “You couldn’t just automatically spin up [compute] at the rate you needed to, so you always had idle compute reserved for your workload, whether you were using it or not.”
Think of it like always paying for a parking space, even when you’re not using it. With AWS’s upgraded Serverless, it’s more like paying for a metered parking spot.
At launch, OpenSearch Serverless will integrate natively with AI development platforms like Vercel and Kiro, so developers can deploy production-ready search and vector backends for agents without managing infrastructure.
The shift is emerging across the cloud industry. Databricks and Snowflake are repositioning themselves as AI memory and retrieval systems for enterprise data. Microsoft has rolled out updates to Azure designed to handle AI agent bursts and share memory between agents. Cloudflare, in a similar vein to Amazon, last month introduced infrastructure aimed at giving agents persistent environments and instant scalability.
The more companies deploy AI agents, the more pressure there will be to redesign infrastructure around machine-generated workloads, which in turn could make agents cheaper and easier to deploy at larger scales.
Cloud infrastructure has long been designed around humans who search, click, scroll, and stream in a steady and predictable fashion. AI agents behave differently. They can unleash a swell of activity, spinning up multiple sub-agents that query hundreds of databases, search documents, and call APIs in seconds and then disappear as quickly as they arrived.
Under that premise, Amazon is redesigning a core piece of its cloud infrastructure. On Thursday, AWS launched its next generation of OpenSearch Serverless, a fully managed search and vector database — essentially a system for storing and retrieving information at scale — that’s designed specifically for agentic workloads. AWS says the new system can instantly scale up when agents trigger tasks and scale back down to zero when idle.
The launch reflects a growing realization across the tech industry: Infrastructure originally designed for a human-driven internet doesn’t work as well in a world increasingly populated by agents.
While AI agents still represent a relatively small portion of internet activity, machine-generated traffic is already significant, and poised to grow. Cloudflare says bots accounted for 31% of overall HTTP traffic over the last six months. AI crawlers, search engines, and assistants made up roughly a quarter of all bot requests during that period.
“Non-human traffic will exceed human traffic sometime in the first half of 2027,” said Lai Yi Ohlsen, senior product manager at Cloudflare, to TechCrunch.
At Google’s I/O developer conference last week, the company said users will be able to start delegating tasks to AI systems, like researching purchases, booking travel, browsing the web, and interacting with apps. But the buck doesn’t stop at consumer-focused AI agents. Enterprises are increasingly deploying agents internally and for their customers, creating new kinds of machine-generated traffic behind the scenes.
As a result, cloud providers and infrastructure companies have been reckoning with how to adapt systems built for humans to a world of agents that are constantly and autonomously retrieving information, invoking tools, and generating machine-to-machine traffic.
That’s where AWS’s new OpenSearch Serverless comes in.
“The timing is straightforward. Agents are moving from experimentation into production, and they create traffic patterns that previous infrastructure simply wasn’t designed for,” Tia White, general manager for Amazon OpenSearch Service, told TechCrunch. “They spike without warning, they go idle without notice, and enterprise needs search that keeps up without paying for empty or idle compute.”
The key technical change with this new generation is that it decouples compute from storage, allowing compute to scale up in seconds to accommodate agent traffic bursts and to scale down to zero, so customers pay $0 when agents are idle.
“Previously, even in our prior Serverless version, you had to have at least one instance operational and running because storage and compute were coupled,” White said. “You couldn’t just automatically spin up [compute] at the rate you needed to, so you always had idle compute reserved for your workload, whether you were using it or not.”
Think of it like always paying for a parking space, even when you’re not using it. With AWS’s upgraded Serverless, it’s more like paying for a metered parking spot.
At launch, OpenSearch Serverless will integrate natively with AI development platforms like Vercel and Kiro, so developers can deploy production-ready search and vector backends for agents without managing infrastructure.
The shift is emerging across the cloud industry. Databricks and Snowflake are repositioning themselves as AI memory and retrieval systems for enterprise data. Microsoft has rolled out updates to Azure designed to handle AI agent bursts and share memory between agents. Cloudflare, in a similar vein to Amazon, last month introduced infrastructure aimed at giving agents persistent environments and instant scalability.
The more companies deploy AI agents, the more pressure there will be to redesign infrastructure around machine-generated workloads, which in turn could make agents cheaper and easier to deploy at larger scales.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
#internet #rebuilt #machines #TechCrunchagentic search,AI agents,AWS,aws opensearch serverless
![This former notorious red-light district is now one of the world’s top AI hubs | TechCrunch
What every U.K. AI startup wants to know these days is, how can I get office space in King’s Cross?
The area is so hot that a VC firm allegedly recently won a deal by promising a founder office space in the neighborhood. “We stop at nothing to win deals [for] and to support” founders, “including helping them source office space when needed,” the firm told me when asked about the rumor, declining to confirm or deny any details.
The neighborhood’s popularity began back in 2016 when DeepMind — then newly acquired by Google — moved in. Soon after, a flood of AI startups followed, wanting to be around the Google DeepMind magic. Today, they hope to take advantage of the cluster of AI talent there.
This has transformed King’s Cross into one of the world’s top AI hubs, rivaled only by San Francisco and Beijing. Around London, it’s known by the sobriquet “Knowledge Quarter,” as it’s home to names like OpenAI, Meta, Isomorphic Labs, Cusp AI, Wayne, Recursive, and, a little farther down the road, Synthesia and Anthropic. The European Technology Network (ETN) just moved into a glossy new office nearby, while University College London sits around the corner.
Mixed in with the new developments are trendy food spots like Hoppers and BAO. Hop a train from King’s Cross, and founders can be in Cambridge in 45 minutes to source talent or can be in Paris in two hours to strike a deal.
Who would have guessed that a little more than 20 years ago, this was one of the seediest areas in London?
“In the ’80s, crack and heroin made the area a major narcotics market,” Hussein Kanji, an investor at Hoxton Ventures, said, recalling syringes in tree trunks and gangs patrolling the streets. “In 1982, the local church was occupied by the English Collective of Prostitutes for 12 straight days.” Then, in the early 2000s, a real estate developer had a dream and, well, “now it is the AI hotbed of the United Kingdom,” Kanji said. “What a change.” Around 18 months ago, his portfolio company BioCorteX moved from the neighborhood Holborn to the Jellicoe building in King’s Cross, hoping to be near the action. “Lots going on in London right now,” Nik Sharma, co-founder of BioCorteX, told me. “Lots of hyperscalers moving in.” That includes, reportedly, Jeff Bezos’ AI company Prometheus, which is also said to be in talks to move into the Jellicoe.
There are around 3,600 AI startups in London, which, together, have raised around .1 billion out of the .8 billion raised in the city since late July, according to Dealroom. Since the start of June, AI-related startups have leased more than 1 million square feet of office space in London, according to the real estate firm Knight Frank. With that, prime rents in King’s Cross have risen 18% over the past three years, Chris Dunn, a commercial insight associate at the firm, told me. That percentage represents only the largest leases encompassing at least 10,000 square feet, like the ones OpenAI and Prometheus are signing. The shorter deals go for even more, he said, and now the vacancy rate for conventional office space is just 0.9%. “Demand has outstripped supply,” he continued.
Today, one of the big topics of the area is sovereignty. It was a wake-up call for many when Anthropic shut off access to Mythos and Fable this summer, leaving some in the ecosystem to conclude: “We’d better look after ourselves,” Saul Klein, co-founder of the VC firm Phoenix Court, told me.
Phoenix Court is located in the King’s Cross area and has three portfolio companies in the vicinity, including Olix (which just announced a .3 billion valuation), Early Health and CoMind. Robin Klein, co-founder of the firm, said the shutdown of Fable and Mythos access was a “small but sharp reminder that Europe can’t simply rent its AI capabilities and capacity; it needs to build and hold some of its own.” King’s Cross, he said, is where much of this building is actually happening.
“The bigger question,” he continued, “is whether the U.K. builds the infrastructure, compute, energy, capital, to make this self-reliance durable, rather than just hosting outposts of U.S. labs.”
Image Credits:Phoenix Court
Top founders want to stay
Simon Kohl, founder of Latent Labs, has offices in King’s Cross and San Francisco. The London office, at the moment, is growing faster, and he’s more bullish than ever on the ecosystem, he said. “The mood right now feels less like London trying to catch up and more like London becoming one of the default places to start a serious AI company,” he said. Look around and you are likely to see Wayve testing its autonomous cars. Founded in 2017 by co-founder Alex Kendall, the unicorn is one of London’s biggest success stories.
“Ten years ago, building a frontier AI company from London felt like an unusual choice,” Kendall told me. “Now it feels like an obvious one.” Wayve moved into King’s Cross in 2018 looking for a space that could double as a garage — “a rare combination in Central London,” Kendall said. He has watched the ecosystem mature around him — and it’s now evident that a startup can stay in London, raise serious capital, hire world-class AI talent, and remain globally competitive, he said. Down the street from Anthropic’s new 158,000-square-foot office is the AI agent builder Sierra and the AI video platform Synthesia.
Laura Gonzalez Florez, Synthesia’s chief of staff and head of people, says the company moved into its glossy new office building a year ago to accommodate its growing team. They were drawn to the area for the same reason as everyone else: “It’s very close to the airport … very close to where a lot of investors are,” she said.
Image Credits:Synthesia
Around two-thirds of Synthesia’s engineers are remote, Gonzalez Florez said, letting the company tap into an affordable, international, and diverse talent pool and helping it scale faster. “From London, we can hire and work, without any problem, people from anywhere, from Slovenia to Portugal,” she said.
Unsurprisingly, London’s AI boom is also causing a talent war.U.K. AI job postings have skyrocketed in the past few years, per data from PwC. When Anthropic announced it moved into town earlier this year, it listed, for example, a salary range of £260,000 to £630,000 for a machine learning research engineer when the average salary in London for the same role is around £102,000. Some founders in the U.K., like those in Silicon Valley, are being forced to raise more and bigger rounds to keep up.
“The real test is whether more globally significant AI companies are founded, funded, and scaled from the U.K., while continuing to attract the world’s best talent to build them here,” Zain Ali, founder of the King’s Cross-based AI legal firm Centuro, told me. “If that continues to happen, King’s Cross won’t just be an AI hub. It’ll become one of the U.K.’s most important strategic assets.”
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.#Thisformernotorious #redlight #districtis #nowone #worlds #top #hubs #TechCrunchUK This former notorious red-light district is now one of the world’s top AI hubs | TechCrunch
What every U.K. AI startup wants to know these days is, how can I get office space in King’s Cross?
The area is so hot that a VC firm allegedly recently won a deal by promising a founder office space in the neighborhood. “We stop at nothing to win deals [for] and to support” founders, “including helping them source office space when needed,” the firm told me when asked about the rumor, declining to confirm or deny any details.
The neighborhood’s popularity began back in 2016 when DeepMind — then newly acquired by Google — moved in. Soon after, a flood of AI startups followed, wanting to be around the Google DeepMind magic. Today, they hope to take advantage of the cluster of AI talent there.
This has transformed King’s Cross into one of the world’s top AI hubs, rivaled only by San Francisco and Beijing. Around London, it’s known by the sobriquet “Knowledge Quarter,” as it’s home to names like OpenAI, Meta, Isomorphic Labs, Cusp AI, Wayne, Recursive, and, a little farther down the road, Synthesia and Anthropic. The European Technology Network (ETN) just moved into a glossy new office nearby, while University College London sits around the corner.
Mixed in with the new developments are trendy food spots like Hoppers and BAO. Hop a train from King’s Cross, and founders can be in Cambridge in 45 minutes to source talent or can be in Paris in two hours to strike a deal.
Who would have guessed that a little more than 20 years ago, this was one of the seediest areas in London?
“In the ’80s, crack and heroin made the area a major narcotics market,” Hussein Kanji, an investor at Hoxton Ventures, said, recalling syringes in tree trunks and gangs patrolling the streets. “In 1982, the local church was occupied by the English Collective of Prostitutes for 12 straight days.” Then, in the early 2000s, a real estate developer had a dream and, well, “now it is the AI hotbed of the United Kingdom,” Kanji said. “What a change.” Around 18 months ago, his portfolio company BioCorteX moved from the neighborhood Holborn to the Jellicoe building in King’s Cross, hoping to be near the action. “Lots going on in London right now,” Nik Sharma, co-founder of BioCorteX, told me. “Lots of hyperscalers moving in.” That includes, reportedly, Jeff Bezos’ AI company Prometheus, which is also said to be in talks to move into the Jellicoe.
There are around 3,600 AI startups in London, which, together, have raised around .1 billion out of the .8 billion raised in the city since late July, according to Dealroom. Since the start of June, AI-related startups have leased more than 1 million square feet of office space in London, according to the real estate firm Knight Frank. With that, prime rents in King’s Cross have risen 18% over the past three years, Chris Dunn, a commercial insight associate at the firm, told me. That percentage represents only the largest leases encompassing at least 10,000 square feet, like the ones OpenAI and Prometheus are signing. The shorter deals go for even more, he said, and now the vacancy rate for conventional office space is just 0.9%. “Demand has outstripped supply,” he continued.
Today, one of the big topics of the area is sovereignty. It was a wake-up call for many when Anthropic shut off access to Mythos and Fable this summer, leaving some in the ecosystem to conclude: “We’d better look after ourselves,” Saul Klein, co-founder of the VC firm Phoenix Court, told me.
Phoenix Court is located in the King’s Cross area and has three portfolio companies in the vicinity, including Olix (which just announced a .3 billion valuation), Early Health and CoMind. Robin Klein, co-founder of the firm, said the shutdown of Fable and Mythos access was a “small but sharp reminder that Europe can’t simply rent its AI capabilities and capacity; it needs to build and hold some of its own.” King’s Cross, he said, is where much of this building is actually happening.
“The bigger question,” he continued, “is whether the U.K. builds the infrastructure, compute, energy, capital, to make this self-reliance durable, rather than just hosting outposts of U.S. labs.”
Image Credits:Phoenix Court
Top founders want to stay
Simon Kohl, founder of Latent Labs, has offices in King’s Cross and San Francisco. The London office, at the moment, is growing faster, and he’s more bullish than ever on the ecosystem, he said. “The mood right now feels less like London trying to catch up and more like London becoming one of the default places to start a serious AI company,” he said. Look around and you are likely to see Wayve testing its autonomous cars. Founded in 2017 by co-founder Alex Kendall, the unicorn is one of London’s biggest success stories.
“Ten years ago, building a frontier AI company from London felt like an unusual choice,” Kendall told me. “Now it feels like an obvious one.” Wayve moved into King’s Cross in 2018 looking for a space that could double as a garage — “a rare combination in Central London,” Kendall said. He has watched the ecosystem mature around him — and it’s now evident that a startup can stay in London, raise serious capital, hire world-class AI talent, and remain globally competitive, he said. Down the street from Anthropic’s new 158,000-square-foot office is the AI agent builder Sierra and the AI video platform Synthesia.
Laura Gonzalez Florez, Synthesia’s chief of staff and head of people, says the company moved into its glossy new office building a year ago to accommodate its growing team. They were drawn to the area for the same reason as everyone else: “It’s very close to the airport … very close to where a lot of investors are,” she said.
Image Credits:Synthesia
Around two-thirds of Synthesia’s engineers are remote, Gonzalez Florez said, letting the company tap into an affordable, international, and diverse talent pool and helping it scale faster. “From London, we can hire and work, without any problem, people from anywhere, from Slovenia to Portugal,” she said.
Unsurprisingly, London’s AI boom is also causing a talent war.U.K. AI job postings have skyrocketed in the past few years, per data from PwC. When Anthropic announced it moved into town earlier this year, it listed, for example, a salary range of £260,000 to £630,000 for a machine learning research engineer when the average salary in London for the same role is around £102,000. Some founders in the U.K., like those in Silicon Valley, are being forced to raise more and bigger rounds to keep up.
“The real test is whether more globally significant AI companies are founded, funded, and scaled from the U.K., while continuing to attract the world’s best talent to build them here,” Zain Ali, founder of the King’s Cross-based AI legal firm Centuro, told me. “If that continues to happen, King’s Cross won’t just be an AI hub. It’ll become one of the U.K.’s most important strategic assets.”
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.#Thisformernotorious #redlight #districtis #nowone #worlds #top #hubs #TechCrunchUK](https://techcrunch.com/wp-content/uploads/2026/08/DM9A2852.jpg?w=680)

Post Comment