https://kedro.org/ logo
#questions
Title
# questions
s

Si Yan

04/21/2023, 8:11 PM
Hi All, I am new to Kedro. I need to load data from Snowflake in Kedro. I searched some previous posts and found that Snowflake dataset is now available in Kedro 0.18.7. But I can’t find any documentation showing how to use it. Can I write a sql query like sqlQueryDataset? How to define the credentials? Could someone give an example? Thanks!
m

marrrcin

04/21/2023, 9:43 PM
Right now there is a SnowparkTableDataSet https://github.com/kedro-org/kedro-plugins/blob/a203f53cff7cb51b81b0feb2cd3c9c877e88df32/kedro-datasets/kedro_datasets/snowflake/snowpark_dataset.py (docstrings are really descriptive there). The dataset will give you access to a lazy Snowpark DataFrame (no eager data loading), so you can use Snowpark's APIs to process it as you like. Keep in mind that it only works with Python 3.8
s

Si Yan

04/21/2023, 11:56 PM
Thanks a lot, marrrcin!