Hi all, I am an economics researcher interested in incorporating Kedro into our lab workflow. Our research projects typically involve ingesting and processing a large number of disparate datasets, so Kedro will help keep track of our pipelines. We are currently using SCons to create our task DAGs, which does not work well for large datasets. However, before introducing Kedro, I have a few questions. 1. We ingest, process, join, and aggregate all our data in Python. However, often, our analysis relies on R and/or STATA packages to produce tables and figures. Is it possible to integrate R / STATA scripts into Kedro pipelines? If so, how? 2. We currently have a set of scripts that validate data quality and produce log files / data summaries every time we write data to disk. Is there a way to somehow override the wrapper that Kedro uses to save data such that it will run these scripts? Julien

Hi @Julien Berman, Thank you for showing interest in Kedro.

We ingest, process, join, and aggregate all our data in Python. However, often, our analysis relies on R and/or STATA packages to produce tables and figures. Is it possible to integrate R / STATA scripts into Kedro pipelines? If so, how?

Though I haven't tried running R scripts but I feel this is possible, considering a kedro node.func expects a python callable and within the function, you can have a subprocess which runs R scripts.

We currently have a set of scripts that validate data quality and produce log files / data summaries every time we write data to disk. Is there a way to somehow override the wrapper that Kedro uses to save data such that it will run these scripts?

I can think of 2 ways doing this - 1. Implement a custom dataset which has validations within the save method 2. Implement custom hook implementations using our DatasetSpecs . Hooks are very well suited for these use cases. Thank you