Hi Kedro dwellers, I would like to hear what you think of a problem I am facing with kedro and privacy. My terms and agreement prevent to share with 3rd parties data (even if anonymized) and sensitive information (addresses, passwords etc). This includes preventing agentic LLMs from crawling around

conf/local

and

data/

. I did test guardrails to avoid this but they failed on rare situations. I am looking for a way to develop code in Kedro along with some AI, keeping data and sensitive info secure. Currently, my solution is to develop on a safe environment, like a docker container, and pull the changes in the environment that has this information once I need to run the code. Prototyping in this way feels a bit inefficient, since agents themselves cannot run test effectively. Did any of you meet a similar problem?

Hey Gianmarco, Haven't run into this myself, but I think Kedro's existing features can help here. Have you tried: - Creating a

conf/synthetic/

environment with a catalog pointing to fake data? Your AI agent works with

kedro run --env=synthetic

and never sees real data, no code changes needed between environments. - Adding

AGENTS.md

/

CLAUDE.md

deny rules to explicitly block agent access to

conf/local/

and

data/

? Not bulletproof alone, but a useful extra layer. Relevant docs: - Config environments: https://docs.kedro.org/en/stable/configure/configuration_basics/ - Credentials: https://docs.kedro.org/en/stable/configure/credentials/ - Testing with dummy data: https://docs.kedro.org/en/stable/tutorials/test_a_project/ Let me know if this helps.

Thanks for your input, I didn't think of using an environment in the runner. This could make things less painful. Synthetic data was an option I considered. It adds work on top, but I guess is a good starting point. I will do some test trying to store some info as environment variable. From what I saw, agentic LLMs are quite good at hiding what they know. They can easily run code to find that out while "thinking". Thanks again, I am open to suggestions.

Hey @Gianmarco Guarnier ! I recognize your problem. working as a consultant, my customers are concerned about data. One way is to get a Local AI agent like KiloCode and give access to you secret data to build you a synthetic data generator/simulator that can generate a decent synthetic dataset. After that you can let your cloud-connected AI agent work with that data. I made the mistake of not making the synthetic data as large as the original data, so my polars pipeline broke when raw data got 100x, so keep that in mind... Otherwise there is no trusting in AI agents not to look at your secret files. even .env is not safe. I have tested with a few agents and also setting custom rules and such, but it hasn't been bullet proof. If its really sensive --> keep it in a different folder far from your agents reach

Thanks for the suggestion, currently I am trying to make this solution work: https://www.docker.com/blog/connect-mcp-servers-to-claude-desktop-with-mcp-toolkit/ Basically I get to design an external environment with data and creds. I should be able to decide what claude can receive from this environment (e.g. accuracy on validation data)