https://kedro.org/ logo
Join Slack
Powered by
Hey folks. I am trying to pass mlflow credentials ...
# questions
s
Hey folks. I am trying to pass mlflow credentials to streamlit page via oc.env [via kedro]. Any idea what could be causing the credentials to not be passed? These exist in the ECS task definition but not being passed to the streamlit page.
Copy code
credentials.yml

mlflow:
  tracking_uri: ${oc.env:MLFLOW_TRACKING_URI}
  username: ${oc.env:MLFLOW_TRACKING_USERNAME}
  password: ${oc.env:MLFLOW_TRACKING_PASSWORD}
ERROR:
Copy code
Exception: MLflow tracking URI not found in credentials.
This works perfectly fine when testing in local env with manually passing the credentials to 
envars
, something like 
export MLFLOW_TRACKING_USERNAME=username
. more info on kedro way
h
Someone will reply to you shortly. In the meantime, we've found some posts that could help answer your question.
l
Have you registered the resolver correctly?
s
The omegaconfig under settings.py? Yes I think so. If you mean something else, kindly elaborate
l
correct
how are these variables that are not being picked up defined? through a .env file?
s
yeah, so that is generated in aws ecs cpnsole, through a terraform file.
Copy code
container_definitions = jsonencode([
    {
      name      = local.image_name,
      image     = "${aws_ecr_repository.main.repository_url}:${var.artifactory_source_tag}",
      cpu       = var.cpu
      memory    = var.memory
      essential = true
      environment = [
        { name : "MLFLOW_TRACKING_URI", value : "<https://www.genericdomain.com>" }
      ]
      secrets = [
        {
          name      = "MLFLOW_TRACKING_USERNAME",
          valueFrom = data.aws_ssm_parameter.mlflow_tracking_server_username.arn
        },
        {
          name      = "MLFLOW_TRACKING_PASSWORD",
          valueFrom = data.aws_ssm_parameter.mlflow_tracking_server_password.arn
        }
      ]
l
can you check that on these machine the said env variables are set?
👍 1
i.e., when you log in the machine and you print the env
s
yes they are set and other credentials are getting loaded. The streamlit page also relies on some of these envars and wouldn't load otherwise.
meanwhile I was also testing your previous point on resolver.
Copy code
import omegaconf
    from omegaconf import OmegaConf

    st.write(omegaconf.__version__)
    # Set a test environment variable
    os.environ["TEST_VARIABLE"] = "test_value"

    cfg = OmegaConf.create({"value": "${oc.env:TEST_VARIABLE}"})

    # Try to access
    try:
        st.write(f"Resolved value: {cfg.value}")
    except Exception as e:
        st.write(f"Error: {e}")
I got the desired output confirming these are working properly:
Copy code
2.3.0

Resolved value: test_value
okay i may have been able to identify the problem. The omegaconfig is not able to resolve the envars in prod environment. Local :
Copy code
2.3.0

Resolved value: test_value
prod env:
Copy code
2.3.0

Error: Unsupported interpolation type oc.env full_key: value object_type=dict
The test variable defined and called in the script is not being recognized in the prod environment while the directory structure, settings.py, etc are all configured in same manner. Any idea what do I need to change @Laurens Vijnck? Thanks for your help so far
4 Views
Open in Slack
PreviousNext
Powered by