Where in the kedro project structure do most folks...
# questions
a
Where in the kedro project structure do most folks manage their sql files? • data seems like it could be appropriate • maybe src? • maybe a separate sql dir?
d
Kedro's project structure is built for Python code, so there's no recommended place for SQL files, per se? Are you running SQL files in nodes (e.g. using
pyspark.sql
, or is it for something outside of what Kedro manages? As a fun fact, early versions of Kedro (before it was open source, and even before it was called Kedro) did support R and SQL, too (I could remember wrong, but I think it was something like
src/python
,
src/sql
,
src/R
), but Kedro chose to focus on handling Python source well.
a
Both actually: • sql for pyspark • but also sql used by some Catalog datasets For the second case, I'm thinking /data might not be a bad location.
👍 1
b
I’ve tried a few different ways, but I’ve sort fallen into putting my sql inside node functions that can except params to be templated. I have some custom datasets that except a sql string that gets executed in the save method.