When AI agents see your data, who watches them?

A traditional application usually has relatively predictable paths through which data moves.

An agent is different. It can receive information, decide what it needs, call a tool, retrieve more information, pass that information to an LLM, call another tool and potentially trigger further actions.

So the security question changes from:

“Does this application have access to sensitive data?”

to:

“What did the agent actually access, why did it access it, and what did it do with the information afterwards?”

Organizations may know that AI is being used, but not necessarily understand where sensitive information enters the AI system and where it goes afterward.

There is a video in the YouTube channel of IBM technology (a great source for AI informational videos) that describes this as a visibility problem and emphasizes tracking the data flow through AI systems.

Imagine an AI security agent that has access to your server logs. It sounds harmless until you realize that those logs may contain usernames, IP addresses, email addresses, customer information or other personal data.

Now the agent isn’t just processing “security logs.” It is processing potentially sensitive information. And then the agent might send some of that information to an LLM, use a tool to investigate it, write something to a database, or pass information to another system.

Your agent should not be a black box!

You should have answer to these questions:

• What did the agent receive?
• What did it ask the LLM?
• Which tools did it use?
• What data did those tools return?
• How long did it take?
• How much did it cost?
• What did the agent ultimately do?

And potentially the most important question:

• Can I reconstruct what happened after the fact?

You should make your agent(s) observable and be able to trace all its actions. And no, telling your agent to write everything it does to a file is not a viable option here. You need external systems that watch your agents and present everything it does.

Such data (what the agent is doing) is called telemetry. OpenTelemetry for example is an open source framework that provide such information in a standardized way to move through the observability pipeline.

Then you need a platform that reads that telemetry and present it to you in an easy to read and navigate way. There can be alerts, AI embedded that monitor the agents data for any abnormalities and irregular patterns/behaviors etc.

I am currently experimenting with Langfuse to monitor my agents. The traces from the agents are collected with Strands Agents and OpenTelemetry which gives a precise information in a standardized format for Langfuse to consume.

This article is the first in a series called Securing and Observing AI Agents where I will be presenting you different techniques to hack, secure and observe AI systems.