https://kedro.org/ logo
Join Slack
Powered by
Hello, team! I am trying to setup a custom mlflow ...
# questions
c
Hello, team! I am trying to setup a custom mlflow server with docker-compose:
Copy code
services:
  minio:
    image: minio/minio
    container_name: minio
    ports:
      - "6060:6060"
      - "6001:6001"
    volumes:
      - minio-data:/data
    env_file:
      - .env
    command: server --address 0.0.0.0:6060 --console-address ":6001" /data
    restart: always
    logging:
      driver: "local"
      options:
        max-size: 10m
    healthcheck:
      test: ["CMD", "curl", "-f", "<http://localhost:6060/minio/health/live>"]
      interval: 30s
      timeout: 20s
      retries: 3

  createbuckets:
    image: minio/mc
    depends_on:
      - minio
    env_file:
      - .env
    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc alias set myminio http://${EXTERNAL_IP}:6060 ${MINIO_ROOT_USER} ${MINIO_ROOT_PASSWORD};
      /usr/bin/mc mb myminio/ml-artifacts;
      /usr/bin/mc admin policy attach myminio readwrite --user ${MINIO_ROOT_USER};
      exit 0;
      "

  # Postgres
  db:
    image: postgres:13
    container_name: mlflow_db
    expose:
      - "5433"
    ports:
      - "5433:5433"
    environment:
      - POSTGRES_USER=mlflow
      - POSTGRES_PASSWORD=mlflow
      - POSTGRES_DATABASE=mlflow
    command: -p 5433
    restart: always


  # MLflow

  mlflow:
    build:
      context: mlflow
      dockerfile: mlflow.Dockerfile
    container_name: mlflow
    restart: always
    env_file:
      - .env
    ports:
      - "5000:5000"
    environment:
      - MLFLOW_S3_ENDPOINT_URL=<http://minio:6000>
      - AWS_ACCESS_KEY_ID=${MINIO_ROOT_USER}
      - AWS_SECRET_ACCESS_KEY=${MINIO_ROOT_PASSWORD}
      - MLFLOW_S3_IGNORE_TLS=true
    command: >
      mlflow server
      --backend-store-uri <postgresql://mlflow:mlflow@db:5433/mlflow>
      --host 0.0.0.0
      --default-artifact-root <s3://ml-artifacts/>

volumes:
  minio-data:
The problem is that I'm not sure how should I modify the mlflow.yml config file to point to this mlflow server. Any insights?
h
Someone will reply to you shortly. In the meantime, this might help:
j
hmmm reminds me of what I did for my "from zero to mlops" workshop, and I think it mostly worked out of the box
does
Copy code
$ cat conf/base/mlflow.yml
server:
  mlflow_tracking_uri: <http://0.0.0.0:5000>
work @Camilo Pi帽贸n?
馃憤 2
鉂わ笍 1
l
Copy code
mlflow:
    container_name: mlflow
    build:  "services/mlflow"
    ports:
      - "5001:5000"
    command: mlflow server --host 0.0.0.0 --serve-artifacts
    healthcheck:
      test: wget <http://localhost:5000/health> || exit 1
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 30s
Copy code
matrix-pipeline:
    image: ${IMG}
    container_name: matrix
    entrypoint:
      - "/bin/sh"
      - -ecx
      - |
          kedro run --runner ThreadRunner -e test -p test
    environment:
      MLFLOW_ENDPOINT: <http://mlflow:5000>
馃憤 2
Quickly dropping copy paste from my setup, but this should do the trick
c
Yeah this worked perfectly, sorry for the late response! Thank you!!
Open in Slack
PreviousNext
Powered by