hi team, I had this issue and thought it would be ...
# questions
d
hi team, I had this issue and thought it would be good to share and looking for advice. I intended to set a parameter "quoting" for saving csv value in catalog.yml (image 2). In the normal to_csv() function, you would use quoting=csv.QUOTE_NONNUMERIC as a parameter but this won't work in catalog.yml as it doesn't know modual 'csv'. one way is manually set the desired integer (image 3) but found that value was actually changed from 3 to 2 (image1, 3 used to stands for csv_QUOTE_NONNUMERIC but now it is 2 ) in the latest version 'csv'. is there any way we could fetch this dynamically (like how quoting = .csv_QUOTE_NONNUMERIC works in normal to_csv() in catalog?
m
One thing that comes to my mind, that does not involve custom datasets (which would be the obvious implementation here), is that you could use
TemplatedConfigLoader
. In your
settings.py
use:
Copy code
from kedro.config import TemplatedConfigLoader
import csv

CONFIG_LOADER_CLASS = TemplatedConfigLoader
CONFIG_LOADER_ARGS = {
    "globals_dict": {"csv_quoting": csv.QUOTE_NONNUMERIC},
}
and in the
catalog.yml
Copy code
your_data:
  type: pandas.CSVDataSet
  filepath: data/test.csv
  save_args:
    index: false
    quoting: ${csv_quoting}
🎉 1
d
Cool, thanks Marrrcin, I will try out