Hi, guys. Maybe somebody knows what it is the patt...
# questions
o
Hi, guys. Maybe somebody knows what it is the pattern when you have very long queries? I'm getting data from BigQuery trought pandas.GBQQueryDataSet but the queries are so long, making the catalog.yaml looks dirty. Is that the right way or should I store the queries in files and call from them, or store the queries as views in Bigquery? What do you use to do? Any suggestion is appreciated. Thanks in advance.
1
n
For SQLDataSet you can specify a file path with the sql, not sure why this is not included for the GBQDataset. I will have a closer look later. https://kedro.readthedocs.io/en/stable/kedro.extras.datasets.pandas.SQLQueryDataSet.html
K 1
o
Thank you so much, @Nok Lam Chan. My bad: I checked again the pandas.GBQQueryDataSet documentation and I found the filepath parameter that you mention. So, I created a file.sql with the query and then passed in its filepath as parameter: it works smoothly
n
Awesome 🙂 Great to hear it works well
👍 1