Is there a way to pass dataclasses to nodes? To keep the parameter count low, I've been making my functions accept a dictionary of parameters. The choice of a dictionary is natural because it's a perfect correspondence with nested yaml elements.

Hi Roger, could you please clarify what exactly you want to do and give an example? Answering to your questions: 1. Yes, you can pass dataclasses to nodes, for that you can use PickleDataset or MemoryDataset but kedro doesn’t do that automatically for you 2. Parameters are stored as dictionaries using MemoryDatasets in the catalog by default

To be more specific, can nodes directly accept non-primitive types, so given

from pydantic_settings import BaseSettings
from vertexai.generative_models import SafetySetting

class ProcessingConfig(BaseSettings):
    param1: str
    param2: list[str] | None
    param3: dict[str, str]
    param4: str
    param5: bool = True
    param6: list[SafetySetting] | None

Instead of:

node(
    name="process_data",
    func=process_func,
    inputs=[
        "params:param1",
        "params:param2",
        "params:param3",
        "params:param4",
        "params:param5",
        "params:param6",
    ],
    outputs="some_output",
)

I want:

node(
    name="process_data",
    func=process_func,
    inputs=[
        "params:processing_config",
    ],
    outputs="some_output",
)

Where

type(processing_config)

is

ProcessingConfig

. Is this addressed by (1) above? Can you send me the link to the docs for this?

Thank you for sharing extra details. Answering to your questions: 1. Nodes can accept all the types 2. But you cannot pass your dataclass as a parameter straightaway 3. What you can do is input your parameters as usual to some node which returns your dataclass, so you can use it later on

Number 3 is interesting! I could create a "layer" of nodes that just parses the raw parameters and outputs more complex objects. I'll give this a try.