https://kedro.org/ logo
#questions
Title
# questions
d

Debanjan Banerjee

08/26/2023, 4:17 PM
Hey Everyone! A question for Kedro Users. Is there a way you can get schema/ metadata for all datasets in an efficient way from the catalog. I want to use the metadata especially datatypes downstream but is there a way you can get that without worrying about the
DataSet
type.
👀 1
j

Juan Luis

08/26/2023, 5:18 PM
hey @Debanjan Banerjee! "metadata" as in
Copy code
dataset_name:
  type: x
  metadata:
    ...  # this
?
and schemas, this is assuming datasets are tabular (
pandas.CSVDataSet
,
polars.GenericDataSet
,
pandas.SQLQueryDataSet
...)?
d

Debanjan Banerjee

08/26/2023, 5:23 PM
Yes like datatypes , column names ,etc
j

Juan Luis

08/26/2023, 5:24 PM
it's an excellent question and I don't think we have a good answer for it at the moment. I contend building a
kedro-openlineage
plugin should be the right answer to this question https://github.com/kedro-org/kedro-viz/issues/1459#issuecomment-1662165936 but I don't know if others have already tried it