The web is entering a new and significantly more complex phase beyond traditional crawlers. It’s no longer just about bots indexing pages or automatic tools gathering data, but about AI Agents capable of browsing, querying information, interacting with applications, and even executing actions on behalf of users or businesses. In this context, Cloudflare and GoDaddy have announced a strategic partnership with a clear goal: to give website owners more control over how their content is used by AI systems and, at the same time, promote open standards to identify these agents in a verifiable way.
The announcement, made public on April 7, stems from a shared idea: if the so-called web agent ecosystem grows without mechanisms for identity, trust, and control, many website owners — especially small businesses, creators, and independent sites — might end up receiving automated traffic that is hard to identify, manage, or potentially abusive. The partnership aims precisely to preempt this scenario with tools to see who is accessing their content, under what identity, and with which permissions.
Greater Control for Website Owners Against AI Crawlers
The first practical component of the agreement is the integration of AI Crawl Control from Cloudflare into GoDaddy’s hosting platform. This feature is designed so that website owners can better decide which AI crawlers can access their content, which should be blocked, and in certain cases, whether that access should be contingent on economic conditions. Cloudflare presents this tool as a way to protect intellectual property, improve site performance, and prevent repetitive bot scraping from distorting analytics or overloading origin servers.
The relevance here is not just the technology itself, but who it reaches. GoDaddy remains one of the largest players in the domain, hosting, and small business tools market, so this integration could bring advanced controls over AI crawlers to users who generally don’t deploy complex infrastructure or manage sophisticated web security rules. Put differently: Cloudflare provides the technical layer, and GoDaddy handles distribution across millions of small and medium-sized sites.
This point connects to an increasingly broad debate about the future economic model of the web. As AI-generated answers replace traditional search and click-based interactions, many publishers, creators, and digital businesses wonder how they will continue capturing value if their content gets consumed without enough reciprocal traffic. Cloudflare and GoDaddy see their partnership as a possible technical piece for this evolving balance: more visibility, greater decision-making capacity, and an infrastructure that shifts from implicit access to a permissions- and conditions-based model.
The Big Bet: Identifying Agents by Name, Verification, and Signature
The second, more ambitious aspect of the partnership is Agent Name Service (ANS), an open standard designed to give AI agents a verifiable identity based on established Internet technologies like DNS and PKI. The idea is that an agent won’t just be “another bot,” but an entity with a consistent name, a checkable identity, and the capability to be discovered and validated by other systems.
Cloudflare, meanwhile, isn’t just supporting ANS broadly. It also recalls that in 2025 it introduced Web Bot Auth as a cryptographic mechanism to verify bot and agent traffic, and Signature Agent Card as a way for developers to transparently communicate who operates an agent and its purpose. Under the alliance, the company argues that the future agent ecosystem should be built on open, verifiable standards rather than opaque identities.
This matters because the next phase of the internet won’t resemble the previous exactly. If an agent is going to query data, book services, perform specialized searches, or even carry out autonomous transactions, it will no longer be enough to distinguish just between “human” and “bot.” It will be necessary to know what agent it is, who operates it, what permissions it has, and whether its identity is genuine or simply impersonated. This is where Cloudflare and GoDaddy are trying to get ahead with a trust architecture that doesn’t rely solely on heuristics or reputation.
A Partnership with Heavy Political and Economic Implications for the Web’s Future
Beyond the official statement, what emerges is a fight to define the rules of the web in the coming years. GoDaddy had already advanced in November 2025 the development of ANS as a reliable identity system for AI agents, opening access to its API and publishing developer documentation on the standard. The deal with Cloudflare accelerates this effort by adding a significant layer of network infrastructure, security, and access control that has enormous influence on the internet.
The core issue is that AI is forcing a redesign not only of products but also of digital coexistence norms. Who can crawl, who can act, how to identify users, how to audit traffic, and how to safeguard content value are questions that are beginning to directly impact the economic model of the open web. Cloudflare and GoDaddy aren’t resolving these problems alone, but they are attempting to establish an initial technical layer so that this ecosystem isn’t dominated by opaque actors or an unequal power dynamic between large platforms and small site owners.
This partnership deserves attention. It’s not just about product integration or a press release about bot management. It’s, above all, a signal that the transition to an agent-based web has already started moving infrastructure, identity, and governance pieces. When companies like Cloudflare and GoDaddy begin building these layers, it’s generally because they expect that change won’t stay theoretical or distant — it’s imminent.
Frequently Asked Questions
What is Cloudflare’s AI Crawl Control?
It’s a tool that allows site owners to decide how AI crawlers interact with their content, including options to permit, block, or condition access. Cloudflare presents it as a feature to protect content, improve performance, and clean analytics affected by automated scraping.
What is Agent Name Service (ANS)?
ANS is an open standard promoted by GoDaddy to give AI agents a verifiable and discoverable identity based on DNS and PKI. The company describes it as a naming and identity system for agents at internet scale.
What would this mean for a small business involved in this partnership?
In theory, it provides easier tools to see and control how AI systems are crawling their sites, and later, to distinguish legitimate agents from opaque or malicious automated traffic without deploying complex infrastructure.
Are Cloudflare and GoDaddy creating a closed standard for AI agents on the web?
Not exactly. Both companies speak of open standards and a broad ecosystem of verifiable identity methods, with ANS as a central component supported by mechanisms like Web Bot Auth and Signature Agent Card.
via: cloudflare