Hello everyone Using `OmegaConfigLoader` and passing paramet Kedro #questions

Hello everyone, Using `OmegaConfigLoader` and pass...

Elaine Resende

11/29/2023, 9:40 PM

Hello everyone, Using

OmegaConfigLoader

and passing parameters through

extra_params

inside

KedroSession.create

gives me an error:

Copy code

InterpolationKeyError: Interpolation key 'test_ids' not found                                                                                                                                                              
                                 full_key: query_test_recommender_to_filter.sql                                                                                                                                                                         
                                 object_type=dict

The given

extra_params

are a set of strings, which I use in the

catalog.yml

to parameterize a SQL query. Does anyone know how to help me? Thanks in advance

Ankita Katiyar

11/29/2023, 10:34 PM

Hey, what’s your catalog entry look like? Are you using the

runtime_params:

resolver?

Elaine Resende

11/30/2023, 12:09 PM

Hi, my catalog is like this:

Copy code

query_job_recommender_to_filter:
  type: SQLQueryDataset
  credentials: cred
  sql: " 
      SELECT 
           *
      FROM job
      WHERE id IN (${test_ids})"

Elaine Resende

11/30/2023, 12:12 PM

I have a method for kedro like below, as I am using FastAPI to request from kedro:

Copy code

def kedro(
        pipeline='',
        tag=[],
        node_names=[],
        from_nodes=[],
        to_nodes=[],
        from_inputs=[],
        to_outputs=[],
        params={}
    ):
    path_name = str(f'{Path(__file__).resolve().parent.parent}/kedro_proj')
    package_name = 'kedro_proj'
    bootstrap_project(path_name)
    os.chdir(path_name)
    

    with KedroSession.create(package_name, extra_params=params) as session:
        return session.run(
                from_nodes=from_nodes,
                to_nodes=to_nodes,
                from_inputs=from_inputs,
                to_outputs=to_outputs,
                pipeline_name=pipeline,
            )

and I call it passing the pipeline name and the params to extra_params.

Ankita Katiyar

12/01/2023, 11:51 AM

Try using the

runtime_params:

resolver - https://docs.kedro.org/en/stable/configuration/advanced_configuration.html#how-to-override-configuration-with-[…]rameters-with-the-omegaconfigloader

Copy code

query_job_recommender_to_filter:
  type: SQLQueryDataset
  credentials: cred
  sql: " 
      SELECT 
           *
      FROM job
      WHERE id IN (${runtime_params:test_id})"

Elaine Resende

12/01/2023, 12:51 PM

It worked, thank you

🙌 1

🙌🏽 1

Open in Slack

Previous Next