Max S
03/07/2025, 4:29 PMproject_path = Path(__file__).parent.parent
print(project_path)
conf_path = str(project_path / settings.CONF_SOURCE)
conf_loader = OmegaConfigLoader(conf_source=conf_path)
# This line shows how to access the catalog configuration. You can access other configuration in the same way.
conf_catalog = conf_loader["catalog"]
catalog = DataCatalog.from_config(catalog=conf_catalog)
kedro_data = []
for dataset_name, dataset_loader in kedro_integration.datasets_from_catalog(
catalog
).items():
kedro_data.append(dataset_loader)
data_manager[dataset_name] = dataset_loader
companies = catalog.load("companies")
print(companies)
For me the above code throws the following error:
DatasetError: Failed while loading data from dataset CSVDataset(filepath=data/01_raw/companies.csv, load_args={}, protocol=file, save_args={'index': False}).
[Errno 2] No such file or directory: 'data/01_raw/companies.csv'
Isn’t that what the config loader and DataCatalog are supposed to solve? that there is no issue with file paths etc. The project_path
is definitely correct…Hall
03/07/2025, 4:29 PMNok Lam Chan
03/10/2025, 10:09 AM