https://kedro.org/ logo
Powered by
#questions
Title
# questions
s

Sergei Benkovich

01/29/2023, 9:12 AM
in globals.yaml i try to use something like:
Copy code
split_folder: "split_1"

folders:
  raw: "{split_folder}/01_raw"
but it doesnt work and i just get a new folder called, {split_folder}/01_raw is there anyway to accomplish this? i’m running several versions one after the other, i want each one in different folder, but don’t want to have to change paths for all the subdirs i defined...
m

Merel

01/30/2023, 12:14 PM
You’ll need to add 
$
before your templated value like:
Copy code
split_folder: "split_1"

folders:
  raw: "${split_folder}/01_raw"
s

Sergei Benkovich

01/31/2023, 7:46 PM
it was a typo here, i’ll verify i’m running correctly
not sure why this is the output…any suggestions? @Merel thanks in advance 🙂
and this is in catalog
m

Merel

02/01/2023, 8:49 AM
That looks alright to me.. and can you confirm you’ve set the config loader settings like:
Copy code
from kedro.config import TemplatedConfigLoader  # new import

CONFIG_LOADER_CLASS = TemplatedConfigLoader
CONFIG_LOADER_ARGS = {"globals_pattern": "*globals.yml"}
?
aaaahh I see the issue now! You’re doing templating within your 
globals
file as well, which isn’t possible. Your globals file should only hold the values and then you can use template placeholders in your parameters and catalog files.
So your setup should be something like: globals.yml:
Copy code
base_folder: data

folders:
  raw: /01_raw
  int: /intermediate
catalog.yml:
Copy code
example_iris_data:
  type: pandas.CSVDataSet
  filepath: ${base_folder}/${folders.raw}/iris.csv
s

Sergei Benkovich

02/01/2023, 9:19 AM
oh…YES! so trivial and yet i missed it 🙂 awesome thanks you!
👍 1
oh..noooo i was quick to party, took time for the new folders to update on the ide hmmm i’m missing something?
m

Merel

02/01/2023, 9:25 AM
I’m not sure if this will make a difference, but I notice you’re using strings for everything. I’m not and it seems to all be working fine.
specifically the entry in the catalog with the templating
s

Sergei Benkovich

02/01/2023, 9:28 AM
tried removing , still get same behavior 😞
m

Merel

02/01/2023, 9:29 AM
And you’re definitely using the 
TemplatedConfigLoader
?
s

Sergei Benkovich

02/01/2023, 9:29 AM
yes
m

Merel

02/01/2023, 9:35 AM
Very odd.. your setup looks good to me. I’ll try creating a new project with the same settings.
s

Sergei Benkovich

02/01/2023, 9:36 AM
appreciate the help… i guess i’m missing something 🙏
m

Merel

02/01/2023, 10:48 AM
I’m really not quite sure what’s going on in your project.. I’ve copied the exact same setup and all is running fine 😕 Which Kedro version are you using? And could you perhaps create a new project, maybe based on one of the starters and see if templating works there?
s

Sergei Benkovich

02/01/2023, 10:57 AM
ok i see the problem 😞 i defined the TemplatedConfigLoader in the settings file but i was using a different version in my main…. sorry for the time waste 🫣 thank you very much for the help!
👍 1
m

Merel

02/01/2023, 12:20 PM
aahh no worries! I’m glad you found the issue 🙂
5 Views
Powered by