Hi All, I am new to Kedro. I need to load data fro...
# questions
Hi All, I am new to Kedro. I need to load data from Snowflake in Kedro. I searched some previous posts and found that Snowflake dataset is now available in Kedro 0.18.7. But I can’t find any documentation showing how to use it. Can I write a sql query like sqlQueryDataset? How to define the credentials? Could someone give an example? Thanks!
Right now there is a SnowparkTableDataSet https://github.com/kedro-org/kedro-plugins/blob/a203f53cff7cb51b81b0feb2cd3c9c877e88df32/kedro-datasets/kedro_datasets/snowflake/snowpark_dataset.py (docstrings are really descriptive there). The dataset will give you access to a lazy Snowpark DataFrame (no eager data loading), so you can use Snowpark's APIs to process it as you like. Keep in mind that it only works with Python 3.8
Thanks a lot, marrrcin!