Rickard Ström
12/14/2022, 6:06 PMJordan
12/15/2022, 10:41 AMdatajoely
12/15/2022, 10:50 AMRickard Ström
12/15/2022, 11:02 AMJordan
12/15/2022, 11:18 AMrun.py
file in the project root:
import re
import yaml
from yaml.loader import SafeLoader
import subprocess
with open("./conf/base/parameters.yml") as f:
datasets = yaml.load(f, Loader=SafeLoader)["datasets"]
for dataset in datasets:
start_date = re.findall("(\d+)", dataset)[0]
subprocess.run(
f"kedro run --pipeline {start_date}_execution", shell=True, check=True
)
It ain’t pretty, but it gets the job done.Rickard Ström
12/15/2022, 11:20 AMJordan
12/15/2022, 4:12 PM