https://kedro.org/ logo
#questions
Title
# questions
f

Flavien

03/04/2024, 11:30 AM
Hi fellows, is there a way to avoid to write the
stats.json
file when running pipelines? I couldn't find the information in the documentation.
Apparently like this
DISABLE_HOOKS_FOR_PLUGINS = ("kedro-viz",)
n

Nok Lam Chan

03/04/2024, 1:16 PM
It is from viz, we have a pending ticket to move that into a .viz folder so it’s less intrusive. Would that be sufficient?
@Ravi Kumar Pilla
👀 1
f

Flavien

03/04/2024, 1:49 PM
To give a bit context, I am fighting Terraform, Airflow and AWS at the same time which leads to struggles with read/write permissions. My pipeline fails because it can not write
stats.json
which is why I wanted to remove it. I can also manage two virtual environments, the production one not including
kedro-viz
which should solve the problem too, I guess.
n

Nok Lam Chan

03/04/2024, 1:52 PM
Make total sense, in this case
DISABLE_HOOKS_FOR_PLUGINS = ("kedro-viz",)
should helps. | I did a quick search in docs.kedro.org and found nothing, so I think we should definitely add a mention somewhere in the docs. Unfortunately we can't have comments in the JSON file itself, so probably somewhere in the docs?
f

Flavien

03/04/2024, 1:54 PM
Could we add an example in the section above? As an example on how to use it.
n

Nok Lam Chan

03/04/2024, 1:59 PM
I think so, do you want to start a PR for this? I think the best thing to do is add an example in kedro-viz documentation, then link the example in the section above.
f

Flavien

03/04/2024, 2:00 PM
I can, sure.
❤️ 1
n

Nok Lam Chan

03/04/2024, 2:00 PM
Thanks!
r

Ravi Kumar Pilla

03/04/2024, 6:16 PM
Hi @Flavien, I am not sure why the pipeline run fails as it should ideally spit out a warning as below instead of an error. However, as you and Nok pointed out -
DISABLE_HOOKS_FOR_PLUGINS = ("kedro-viz",)
should help resolve the issue. Thanks @Nok Lam Chan
Copy code
Logs for kedro run with only read permissions on project dir
INFO   Completed 20 out of 20 tasks                                                                                          sequential_runner.py:90
[03/04/24 11:59:02] INFO   Pipeline execution completed successfully.                                                                                        runner.py:119
          INFO   Loading data from ingestion.prm_spine_table_clone (MemoryDataset)...                                                                        data_catalog.py:483
          WARNING Unable to write dataset statistics for the pipeline: [Errno 13] Permission denied: 'stats.json'
Thank you
f

Flavien

03/04/2024, 6:20 PM
@Ravi Kumar Pilla, my apologies, my message was not clear. The pipeline does not fail and is similar to your example.
👍 1
thankyou 1