Hi all, Can parameters be passed dynamically to th...
# questions
u
Hi all, Can parameters be passed dynamically to the SQL query in nodes.py used by a SQLQueryDataset? SELECT * FROM sys.tables where SCHEMA_NAME(schema_id) = @Parameter Catalog Definition raw_SchemaType_sql: type: pandas.SQLQueryDataset sql: >- SELECT * FROM sys.tables where SCHEMA_NAME(schema_id) =@Parameter credentials: SqlServerCredentials load_args: chunksize: 100 layer: raw Node definition node( get_cohort_from_db, name="raw_SchemaType_sql", inputs="raw_SchemaType_sql", outputs=None, tags=[sqlserver_currentDB"], ), Thanks in advance for your answers !!
d
Are you looking to use a Kedro parameter (e.g. from
parameters.yml
), or just configure a value for
@Parameter
based on the Kedro environment? The latter is easily doable, the latter is more complicated. I'm not the best person to answer questions on the newer config loaders (let me see if I can pull in some help!), but in past Kedro versions you could seemingly extend the config loader to provide such functionality (see https://stackoverflow.com/a/75409579). This is a bit hacky, as you're using protected attributes in the Kedro API; maybe there's a more straightforward way? For the first case, you could also consider using
ibis.TableDataset
instead, and then you can pass a regular Kedro parameter to the node and do the filtering within the node. This has the benefit of being simpler/less hacky, and you could always execute
.to_pandas()
within the node if you need it in that format.
👍 1