Your unprotected API Keys support criminal activities

I bet you have seen stories of stolen API keys for LLM providers. Have you wondered what the “thieves” do with these keys? Perhaps you thought they are just using them for their own projects. Well, in a way they do, but not the way you probably think of.

How are stolen API keys used?

Most of these stolen API keys power dark web AI tools to connect to the respective LLM providers and power their agents. It is true that the LLM providers monitor their API gateway infrastructure for suspicious activities, but hackers also find new ways to cheat them.

Dark Web AI tools

There are dozens of dark web AI tools that let you use them however you like - no restrictions, no guardrails.

• You want to make weapons - here how you do it
• You want to make drugs - here is the recipe
• You want to print fake money - here is what you have to do
• You want to hack a company or a person - here is how you do it… and .. let me do that for you.. It’s scary.

Some of these tools you might have already heard: DarkGPT, WormGPT, SpamGPT, GhostGPT, FraudGPT… the list goes on. Most of them are accessible from the dark web only and in order to use them you have to pay with crypto through shady Telegram channels.

So one can say that by not protecting your API keys you increase the chances of your keys being used for criminal activities.

Protecting your keys

The good thing is that one can take measures to limit the chance of such a thing happening:

Set a limit on your API key

This is like the most basic and fundamental thing one MUST do. Every provider allows you to do that. Just set a limit on your key based on your expected usage - per day, per week, per model, however you want it.

Key per project

Use a different key for every project. If you are on a team you can create a subset of keys so each employee gets one.

Store keys outside of the project vault and configuration files

Keys should not be stored at any cost in configuration files nor project vaults that are being synced with version control platforms like GitHub. Even if you think that today only you have access (private repo), tomorrow you will have a partner or a team and you will give them access. One of them will make a mistake or leak the key through another channel and all goes south.

Check your source for leaked keys

A great way to check if there are keys stored in your source is to use a tool called https://github.com/gitleaks/gitleaks. It detects secrets like passwords, API keys, and tokens in git repos, files, or in the input stream of the app.

Change the permissions of the folder where your secrets live

If you store your keys in a separate folder that your app reads, make sure to change its permission so that only you and the necessary service accounts can read. Avoid giving your AI agents permissions to access this folder.

Use API gateway

API Gateways for LLM are a great way to manage internal and external keys to LLMs. They support authentication, limits, guardrails, monitoring, etc. Deploying such gateway internally will drastically decrease the chances of leaking API keys simply because there are less secrets leaving your organization’s environment. E.g. you have a project that a team of 5 people is working on.

• You create a key in your favourite LLM Provider (you can even pick several if you like)
• Set a limit on the key(s)
• Connect them to your gateway
• Create internal key for each team member (service accounts)
• Monitor how keys are used (traffic, models, tools, etc)
• You can also set limits internally on your gateways so that no employee nor agent spends a massive amount of tokens because of misconfiguration or a cyber attack

Two examples of API gateways are LiteLLM and Portkey

Summary

By protecting your API keys for LLM providers you are not only ensuring business continuity but also decreasing the power of the dark web AI tools used for criminal activities.