in globals yaml i try to use something like ```split folder Kedro #questions

in globals.yaml i try to use something like: ```s...

Sergei Benkovich

01/29/2023, 9:12 AM

in globals.yaml i try to use something like:

Copy code

split_folder: "split_1"

folders:
  raw: "{split_folder}/01_raw"

but it doesnt work and i just get a new folder called, {split_folder}/01_raw is there anyway to accomplish this? i’m running several versions one after the other, i want each one in different folder, but don’t want to have to change paths for all the subdirs i defined...

Merel

01/30/2023, 12:14 PM

You’ll need to add

before your templated value like:

Copy code

split_folder: "split_1"

folders:
  raw: "${split_folder}/01_raw"

Sergei Benkovich

01/31/2023, 7:46 PM

it was a typo here, i’ll verify i’m running correctly

Sergei Benkovich

01/31/2023, 7:54 PM

not sure why this is the output…any suggestions? @Merel thanks in advance 🙂

Sergei Benkovich

01/31/2023, 8:25 PM

and this is in catalog

Merel

02/01/2023, 8:49 AM

That looks alright to me.. and can you confirm you’ve set the config loader settings like:

Copy code

from kedro.config import TemplatedConfigLoader  # new import

CONFIG_LOADER_CLASS = TemplatedConfigLoader
CONFIG_LOADER_ARGS = {"globals_pattern": "*globals.yml"}

Merel

02/01/2023, 8:56 AM

aaaahh I see the issue now! You’re doing templating within your

globals

file as well, which isn’t possible. Your globals file should only hold the values and then you can use template placeholders in your parameters and catalog files.

Merel

02/01/2023, 8:57 AM

So your setup should be something like: globals.yml:

Copy code

base_folder: data

folders:
  raw: /01_raw
  int: /intermediate

catalog.yml:

Copy code

example_iris_data:
  type: pandas.CSVDataSet
  filepath: ${base_folder}/${folders.raw}/iris.csv

Sergei Benkovich

02/01/2023, 9:19 AM

oh…YES! so trivial and yet i missed it 🙂 awesome thanks you!

👍 1

Sergei Benkovich

02/01/2023, 9:24 AM

oh..noooo i was quick to party, took time for the new folders to update on the ide hmmm i’m missing something?

Merel

02/01/2023, 9:25 AM

I’m not sure if this will make a difference, but I notice you’re using strings for everything. I’m not and it seems to all be working fine.

Merel

02/01/2023, 9:26 AM

specifically the entry in the catalog with the templating

Sergei Benkovich

02/01/2023, 9:28 AM

tried removing , still get same behavior 😞

Merel

02/01/2023, 9:29 AM

And you’re definitely using the

TemplatedConfigLoader

Sergei Benkovich

02/01/2023, 9:29 AM

yes

Merel

02/01/2023, 9:35 AM

Very odd.. your setup looks good to me. I’ll try creating a new project with the same settings.

Sergei Benkovich

02/01/2023, 9:36 AM

appreciate the help… i guess i’m missing something 🙏

Merel

02/01/2023, 10:48 AM

I’m really not quite sure what’s going on in your project.. I’ve copied the exact same setup and all is running fine 😕 Which Kedro version are you using? And could you perhaps create a new project, maybe based on one of the starters and see if templating works there?

Sergei Benkovich

02/01/2023, 10:57 AM

ok i see the problem 😞 i defined the TemplatedConfigLoader in the settings file but i was using a different version in my main…. sorry for the time waste 🫣 thank you very much for the help!

👍 1

Merel

02/01/2023, 12:20 PM

aahh no worries! I’m glad you found the issue 🙂

6 Views

Open in Slack

Previous Next