https://kedro.org/ logo
#questions
Title
# questions
c

Christianne Rio Ortega

04/11/2023, 3:38 AM
Hi All, Another noob question... So we're currently using databricks as the compute engine, and we would like to use azure secrets to grab data from our snowflake instance. Anyone here have done this? would really appreciate the help. I've seen codes but not sure if its a node or a function. TIA!
d

datajoely

04/11/2023, 5:15 AM
the best way to do this is a
before_pipeline_run
or
after_context_created
hook, pick up the credentials and then inject them into the catalog object
c

Christianne Rio Ortega

04/12/2023, 10:09 PM
@datajoely I was able to use credentials, but I'm just wondering how will I grab data from Snowflake from the catalog? I was thinking of having a SQL file within a folder, then the path will be stored in a catalog. but not sure about how the mechanism of calling the SQL file will work...
d

datajoely

04/13/2023, 7:58 AM
So for this you should use the new SnowparkTableDataSet which uses snowflake’s new dataframe aPI
2 Views