Hello everyone! Is it possible to use a jinja generated catalog in order to generate all the nodes in a pipeline. I haven’t found a way yet to generate every node that would save every table to GCP with an identity function. The catalog contains around 150 tables to backup and I would like to avoid declaring every node.

What is the source of your 150 table list? Where are they specified?

Within my jinja templating

Can you use PartitionedDataSet instead?

My source is a bunch of tables from a SQL Server (multiple databases (not all) and multiple tables within those databases(not all))