Standards for Agentic Systems

Building Trust in the Age of AI Agents: A Call for Common Standards

Audio overview of this article (courtesy NotebookLM).

This article was co-authored with Duane Valz.

At the 2024 TED Conference, Vinod Khosla predicted that most Internet activity in the future will stem from AI agents doing work on our behalf. While today’s generative AI platforms create useful content and provide knowledge and guidance on a broad array of topics, they typically do so in a direct, prompt-by-prompt relationship with each of their users. Agentic systems promise additional capabilities, namely, autonomously fulfilling a variety of tasks based on high level user requests, and interacting with a range of services, applications and objects in order to do so, including other AI agents. Duane Valz examined the current and anticipated developments around AI agents in a recent piece. There seems to be little doubt that agentic systems, if the vision for them is fulfilled, will have a tremendous impact on our productivity and that they will generate commercial value. 

AI agents heighten the case for having common protocols and standards-driven technologies to support interoperability, efficient product development, a common set of expectations among users of AI agents, and consensus-based approaches to safety and legal/regulatory compliance. Drawing parallels from the early days of the Internet where content on different Internet browsers rendered differently in the absence of standards, maximizing the value of agentic systems will require interoperability through standards. 

The topic of developing common industry-wide standards for deployment of AI products and services to the public has received high level attention since ChatGPT was introduced in November 2022. Yet, beyond acknowledging the utility of such standards and how they might address known issues in AI model performance and safety, little has been done to either propose specific technology areas for standards development or to actually develop such standards.

This article examines specific AI features and functionality that lend themselves to standards-based technology, providing examples of opportunities that might well be addressed using industry consensus approaches. The article additionally looks at existing standards bodies and de novo AI standardization efforts, ultimately identifying opportunity gaps for supporting the successful development of agentic AI systems.  

Agentic systems

Broadly speaking, agents take a self-determined, input-dependent sequence of steps before returning a user-facing output. AI agents are controlled by an LLM that chooses which action to take at each step. Such planning and execution (reasoning) follows from an LLM’s natural language comprehension and generation capabilities, allowing agents to interact with each other and the world. Xi et al., 2023 provides a good review of the agentic system landscape.

Mustafa Suleyman (Deepmind co-founder; founder and former CEO of Inflection.AI) describes how an agent might generate an input-dependent sequence of steps:

Well, an action is no different, really, to predicting a sequence of words. So when you ask a model to complete a sequence of actions: Let's say it's like three things, basically, to book a restaurant that you and I can go to on a certain day. The first action would be to check the availability in both of our calendars. So that's a correct function, call reconcile the correct moment. So that's the second action. Make sure that it's a restaurant that has availability, so that check is another one. And then go and sign so that you can basically use the correct tool to book the right restaurant at the right time, put your credit card details down. Having obviously also checked that it's a restaurant that we both like, etcetera. So it's like four or five or six different steps just to produce that one “action”, sub-components.

Source

Tools such as calculators allow agents to expand their capabilities to areas outside which the base LLM was trained. But before an agent can effectively use such a tool, the tool has to be sufficiently described so that the agent understands when to use the tool. Additionally, the agent needs to know how to use the tool, e.g., the inputs needed to navigate the tool correctly. 

LangChain is an existing platform that supports some of the capabilities required for agentic systems.  For instance, LangChain allows users to create a chain of components such as prompts, language models, tools, or other chains, that are executed in sequence to get something done for the user. It has numerous use cases, including constructing agents that can invoke tools. Here is a visual representation of an agent constructed using LangChain:

Source

LangChain agents can interact with the external world through APIs. For example, Klarna provides a YAML file describing its API that allows an LLM to interact with it.  To date, however, LangChain’s platform has not become a de facto way in which to create AI agents. Developers have pointed out that it complicates code and that Langchain abstractions hinder customization.

Where standardization can help

While agentic systems are capable of striking results, the agentic ecosystem could be turbocharged by the adoption of enabling standards. We currently lack standards for data privacy, security is still a concern, and there are lots of places where we still have unreliable or inconsistent interaction that undermines agents' cooperation or access to useful tools. Moreover, while existing standards bodies are developing some AI-related standards (discussed below), none of them are specifically looking at enabling agentic systems. 

OpenAI has released its "swarm" framework for developing teams of agents, and Anthropic has released a version of Sonnet that can control a computer (in agent fashion), but both represent proprietary approaches without a standards-based foundation.

Standardization can be expected to enable agentic systems of a single provider to work better, and may be essential toward enabling interoperability of agentic systems from different providers.

Privacy 

Privacy safeguards related to model training are a patchwork of individual company policies. 

Meta claims to use data responsibly in training models:

“For publicly available online information, we filtered the dataset to exclude certain websites that commonly share personal information. Publicly shared posts from Instagram and Facebook – including photos and text – were part of the data used to train the generative AI models underlying the features we announced at Connect. We didn’t train these models using people’s private posts. We also do not use the content of your private messages with friends and family to train our AIs.”

Google uses publicly available user data, such as publicly shared Google documents, to train its AI models: 

“For example, we use publicly available information to help train Google’s AI models and build products and features like Google Translate, Gemini Apps, and Cloud AI capabilities.”

Microsoft published a “Responsible AI Standard,” stating that “Microsoft AI systems are designed to protect privacy in accordance with the Microsoft Privacy Standard.” This has two components: (i) transparency about the collection, use, and storage of data, and (ii) appropriate controls allowing consumers to choose how their data is used.

Each of these company’s model training policies is different despite complying with applicable data privacy laws. Therefore, to the extent data privacy in model training is not regulated by privacy laws such as the GDPR, industry privacy standards need to be developed such that agentic systems ingest – and, critically, share – user data responsibly. 

New industry standards could also address the scenarios below.

Existing privacy laws don’t read cleanly onto AI technology given that LLM architecture is different from traditional database-based storage and retrieval. The Hamburg Commissioner for Data Protection recently published a Discussion Paper highlighting these issues. For example, LLMs tokenize input text and tokens are processed into embeddings that have vectorial relationships. Therefore, any personal data included in training data is transformed into abstract mathematical representations. The Hamburg DPC suggests that because of this: (i) LLMs lack the necessary direct association to individuals that characterizes personal data in CJEU jurisprudence, and (ii) LLMs don't "process" or "store" personal data within the meaning of the GDPR. 

Still, agents will invariably act on behalf of particular users. Use cases such as asking an agent to make a restaurant reservation with a colleague may raise new privacy concerns because now the agent needs access to personal data such as credit card information, food preferences, and calendars. The personal agent may need to communicate credit card information to a reservation agent to achieve the end result. For complex tasks, there may also be human agents in the loop, as Valz points out. How should that personal data be safeguarded? Absent revised privacy laws, standards related to the treatment of personal information in model training, inference, and agentic systems will help develop public trust in AI applications.  

There are other areas of concern. Privacy attacks are targeted attempts to get an LLM to reproduce personal data present in training data. Jailbreaking models may allow a malicious attacker to bypass model controls and extract training data. These attacks could be mitigated through standards regulating inclusion of personal data in training. For example, standards that limit the types of PII available to a model during training would help. This would have the effect that a successful jailbreak would have nothing meaningful to extract. Conversely, standards for abstracting and encrypting PII that are available during training should reduce the risk of an “insecure” agent compromising the entire agentic system.

Security

The behavior of agentic systems should be bounded to limit harm/misuse such as fraud, disinformation, or national security (e.g., assisting in cyberattacks or controlling weapons). 

The National Institute of Standards and Technology (NIST) coordinates federal AI standards. Executive Order (EO) 13859 directed federal agencies to ensure that technical standards minimize vulnerability to malicious attacks in systems that use AI technologies and to develop supporting international standards. EO 14110 directed NIST to develop global technical standards for AI safety and security. In December 2023, NIST issued a public Request for Information (RFI) to assist it in carrying out this directive. No results have been issued yet. NIST is additionally working to encourage the adoption of the AI Risk Management Framework (AI RMF 1.0) in international standards such as the ISO/IEC 5338. None of these standardization efforts specifically target agentic systems.

NIST defines secure communication as having three pillars - confidentiality, authentication, and content-integrity. Secure communication is necessary (but not sufficient) to prevent harms such as fraud, disinformation, and compromised national security on any platform, including agentic systems. Agentic systems should be required to adopt secure communication protocols such as VPN, SSL, HTTPS and potentially more advanced protocols to the extent they are not doing so. 

When it comes to security-related standards for agentic systems, the following areas such as permissioning, identity verification, explainability, and watermarking are particularly relevant:

Permissioning

Agentic activities on the Internet should be identifiable for compliance with access control mechanisms. A recent Atlantic article offers good ideas for this. It proposes that communication packets corresponding to agentic activity should include a packet header identifying those packets as such. 

Second, taking inspiration from the time to live (TTL) framework for communication packets, it proposes setting an upper bound on the number of actions agentic systems can take in order to prevent agentic actions from continuing forever. Agentic systems designed to run forever should declare such operation, and should be subject to additional scrutiny and regulation.

Other suggestions propose limiting agentic actions to narrow veins of autonomy. For example, while agents may be allowed to read data freely from APIs, write access to third party APIs may be limited and controlled to prevent unintentional harm. Along similar lines, it may be desirable to regulate the types of goals agentic systems can set for themselves. For example, can they spawn sub-agents, what type and quantity of resources can they acquire to achieve their goals, what type of goals require human approval? 

Absent standards, human approval may be required for high stakes actions. Researchers at OpenAI are thinking about these issues. Standardizing the set of actions that are out of bounds for agentic systems would increase trustworthiness, security, and promote interoperability within agentic systems.

Furthermore, access to financial credentials should have special considerations (discussed further below).

Identify verification

In some scenarios, it will be necessary to know whether you’re interacting with an agent and to verify the agent’s identity. Self-identification as an agentic system could also be important for attributability purposes. For example, this could be important in financial transactions. On the other hand, this may not matter so much for automated rule-based customer-service systems (vs human customer-service representatives). 

Identify verification is an established field and mechanisms such as cryptographic authentication (e.g., digital keys or PKIs) and credential systems (e.g., certificates issued by a trusted authority) may offer solutions. Examples of standards in this space include NIST’s Digital Signature Standard (DSS), FIDO (Fast Identity Online) Alliance, Microsoft Entra Verified ID, and NIST’s Personal Identity Verification (PIV).

It will also be important to verify the identity of, and authorization for agent action from, the individual upon whose behalf the agent is acting. Such dual verification (agent and principal) is important and may require distinct protocols or, at minimum, the ability to distinguish the agent from the principal. 

Explainable AI (XAI)

Explainable AI is important for creating trustworthy AI systems. See, e.g., four principles of explainable AI by NIST.

The International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC) are developing standards in this area. ISO/IEC 22989 establishes terminology and describes concepts related to AI systems, such as transparency, explainability, controllability, bias, dataset, test data, validation data, and trained model. ISO/IEC CD TS 6254 (under development) relates to approaches for explainability and interpretability of ML models and AI systems. 

Similarly, IEEE has established P2976 XAI - an eXplainable AI Working Group that is working on a standard that defines requirements and constraints that need to be satisfied for an AI method, algorithm, application or system to be recognized as explainable. IEEE 2894-2024 is a draft guide for an architectural framework for explainable AI. And IEEE P7001 aims to develop measurable, testable levels of transparency, so that autonomous systems can be objectively assessed.

While the standards under development by ISO/IEC and IEEE apply don’t specifically cater to agentic systems, if each agent in the chain complies with explainability standards, the system as a whole should comply as well.

Nonetheless, more is required.  For trustworthy agentic systems, the ability to recreate agentic reasoning is important. One needs to understand exactly what an agent did in order to then determine the reasoning used to perform each step of an activity. This presupposes that there is activity traceability. This could potentially create a heavy data storage burden. So lightweight ways of logging and making auditable the actions of agents, alongside techniques for recreating the reasoning used by an agent, are critical pieces for a standard or standards in this area.

Existing standards may aid data collection in AI pipelines. For example, the W3C PROV standard provides a framework for describing provenance (entities, activities, agents) information and is extensible to capture AI-specific metadata in AI pipelines. MLflow is an open-source platform that logs hyperparameters, metrics, and artifacts, and can be extended to capture additional explainability-related metadata. 

Watermarking

EO 14110, which directed NIST to develop global technical standards for AI safety and security, included reducing the risk of harm from synthetic content, such as deep fakes. 

Publishers, creators, and consumers should be able to distinguish AI generated content from human generated content. It may be similarly helpful for AI agents to be able to distinguish content generated by other AI agents. One approach for doing so is to watermark AI generated content using invisible markers that are built into the content files. There are no current standardized approaches. For example, Meta labels content generated using its own AI tools, and is developing a tool to do the same for content created by OpenAI, Microsoft, Adobe, Midjourney, Shutterstock, and Google. Google has developed SynthID, a digital watermark that is embedded directly into the pixels of an image.¹ Valz has criticized SynthID as being more oriented to protecting the IP in images than providing useful signals to the public regarding the source of an image, including whether it was created using AI or not. In a collaborative effort including Adobe, Arm, Intel, and Microsoft, the Coalition for Content Provenance and Authenticity (C2PA) is developing an open technical standard providing the ability to trace the origin of images and videos by attaching an encrypted and signed log to the content file identifying its origin and subsequent changes. Some people argue that one issue with C2PA is that it is trivial to remove or change the metadata it attaches.

Given multiple watermarking implementations, detecting AI generated content requires checking for the presence of multiple watermarking implementations. Coalescing on an industry standard for both the creation and deciphering of watermarks would result in more efficient agentic system implementations and broader adoption.

AB 3211, the California Provenance, Authenticity, and Watermarking Standards Act also merits discussion. The bill centers on standards for watermarking synthetic content. AB 3211 mandates “maximally indelible watermarks,” which it defines as “a watermark that is designed to be as difficult to remove as possible using state-of-the-art techniques and relevant industry standards.” This means that complying with the bill may be a constantly moving target as state-of-the-art techniques evolve. It has passed one chamber of the California legislature, and the bill applies to every single generative AI system distributed in California, regardless of size, purpose, or who created it.   

Payments 

Given authentication and traceability concerns (discussed above in the context of permissions, identity verification, and explainable AI), leveraging blockchain technology for payments standardization merits discussion. Blockchain has inherent characteristics of interest:

• Immutability and Transparency: Blockchain records are immutable and transparent, meaning transactions can't be altered once confirmed, and all nodes in the network have access to the same ledger. If two AI agents are transacting, they can trust the accuracy of the transaction history, minimizing disputes or errors. Additionally, a blockchain ledger provides a single source of truth for auditors and regulators and enables real-time monitoring of transactions for traceability and compliance.

• Security and Cryptography: Each blockchain transaction is verified and added to the ledger by consensus mechanisms (e.g., proof-of-work, proof-of-stake). Agentic systems could use blockchain to safely exchange value and sensitive data without the risk of fraud or unauthorized access, enhancing the security of automated transactions.

• Decentralization: Blockchain operates without a central authority, enabling agentic systems to interact and transact directly with one another. This eliminates the need for intermediaries, and would allow AI agents to execute transactions independently, without relying on banks or payment processors, making payments more efficient and scalable. Scalability may be crucial in unlocking an economy driven by AI agents. 

• Interoperability: Blockchain can provide a standardized infrastructure for various systems and platforms to communicate and transact using a common protocol, independent of local currencies or payment systems. AI agents in different geographic regions or sectors could transact without needing to integrate with multiple currency systems or payment methods.

In short, blockchain is decentralized, transparent, and openly auditable. Blockchain can help track exactly what data an algorithm was trained on, when, by whom, as well as what other steps were taken to vet and verify that data. So in addition to payments, blockchain could help with identity verification and explainability. For example, with respect to watermarking, blockchain could combat misinformation by using cryptographic digital signatures and timestamps to clearly indicate what is authentic and what has been manipulated. Because AI governance currently lacks established standards, blockchain has an opportunity to become a part of standards solutions. Indeed, the IEEE has developed and is pursuing many blockchain standards outside of the context of agentic systems.

While payment standards for agentic systems will minimize risk, things can still go wrong. Therefore, insurance for agentic transactions and recourse for fraudulent agentic systems should be considered. These are some approaches that could be standardized:

• Smart Contract-Based Insurance: Smart contracts can automate and standardize insurance coverage for agentic transactions. These contracts can define the terms for different types of transactions, conditions, and triggers for claims, providing automated compensation in case of malfunction or fraud. Consider AI agents operating in financial markets, executing algorithmic trades. Financial firms could purchase insurance policies that protect against algorithm errors, system malfunctions, or cyberattacks affecting their AI agents. The trading activity is continuously monitored, and all transactions are recorded on a blockchain. If a financial firm detects a significant loss due to an error by another party’s agent or a cyberattack, the insurance smart contract could be triggered to cover a portion of the financial losses.

• Blockchain-Based Registry for Agentic Systems: A standardized blockchain registry for agentic systems could be developed, requiring AI agents to register before conducting transactions. This would create accountability and allow insurers or governing bodies to track the history of each system. For example, an IoT agent providing automated agricultural services (e.g., drone irrigation, autonomous tractors) could register each agent on a blockchain. If a farmer experiences a malfunction or fraud, they can review the system's history and file a claim with an insurer that monitors the registry.

• Reputation Systems and Tokenized Recourse: Blockchain-based reputation systems could be implemented to track the reliability and integrity of AI agents over time. Fraudulent systems could be penalized by lowering their reputation scores or imposing token-based penalties. For example, a delivery network could use a tokenized reputation system. Each delivery transaction is logged, and if an agent fails to deliver a package or engages in fraudulent activities, it loses reputation points and tokens, reducing its ability to participate in the network.

In conclusion, while the promise of agentic systems opens up exciting possibilities, achieving their full potential will depend heavily on the establishment of common standards. These standards will serve as the backbone for ensuring interoperability, safety, and regulatory compliance, much like early Internet standards did for web development. Moving forward, industry leaders, policymakers, and standards bodies must collaborate more effectively to address the opportunity gaps. The path to scalable, safe, and efficient agentic systems lies in the commitment to shared standards that balance innovation with public interest.

1. https://medium.com/@duanevalz/to-what-end-watermarks-for-generative-ai-content-must-satisfy-multiple-objectives-6573fc135e2a