kedro-plugins

requirements.txt

MemoryDataSet

copy_mode

assign

DataSetError: Failed while saving data to data set ParquetDataSet(filepath=/Users/jose_darnott/PycharmProjects/planta-litio/data/01_raw/data_sql.parquet, load_args={'engine': pyarrow}, protocol=file, save_args={'engine': pyarrow}). Duplicate column names found:  ['timestamp', 'lims_BFIL CO3', 'lims_BFIL Ca %', ...]

data_sql:
  type: pandas.ParquetDataSet
  filepath: data/01_raw/data_sql.parquet
  load_args:
    engine: pyarrow
  save_args:
    engine: pyarrow
  layer: III

MemoryDataSet

main

ValueError: Duplicate keys found in 
<project repo>/conf/base/parameters/pr
epare.yml and:
- 
<project repo>/conf/base/parameters/in
gest.yml: train_pipeline

train_pipeline

# The following is a list of parameters for ingest pipeline for each namespace (train, inference)


# Parameters for train namespace
train_pipeline:
  ingestion_options:
    #Portfolio to use
    portfolio_name: has_meds_portfolio.HasMedsPortfolio
    # Feature store sub-pipes, only one for now.
    feature_store_subpipe_name: BasicFeaturePipeline
    # Expected output columns
    expected_columns:
      datetime: datetime64[ns]
      patient_id: int64
      age_days: int64
      Male: int64
      binary_smoking_status: object
      overall_censorship_time: datetime64[ns]
      months_until_overall_censorship: int64
      death_date: datetime64[ns]

# Parameters for inference namespace
# currently same as train but this will change
# first updated to Nightly Porrtfolio then to
# an api call to the valuation queue.
inference_pipeline:
  ingestion_options:
    #Portfolio to use
    portfolio_name: has_meds_portfolio.HasMedsPortfolio
    # Feature store sub-pipes, only one for now.
    feature_store_subpipe_name: BasicFeaturePipeline
    # Expected output columns
    expected_columns:
      datetime: datetime64[ns]
      patient_id: int64
      age_days: int64
      Male: int64
      binary_smoking_status: object
      overall_censorship_time: datetime64[ns]
      months_until_overall_censorship: int64
      death_date: datetime64[ns]

# all parameters for prepare pipeline are in train_pipeline namespace
train_pipeline:
  preparation_options:
    # target params
    target_death_buffer_months: 2
    
    # split params 
    splitter: TimeSeriesSplit
    holdout_size: 0.3

kedro-datasets 1.4.0 does not provide the extra 'pickle.pickledataset'

kedro.extras

PartitionedDataSet

PartionedDataSet

kedro run --env=dev

ValueError: Failed to format pattern '${s3_root_path}': no config value found, no default provided

gto

semver >= 3

kedro-viz

kedro-viz 6.3.0

semver < 3

kedro-viz

semver < 3

semver

3.0.1

type: kedro.extras.datasets.pandas.HTMLDataSet

kedro.io.core.DataSetError: An exception occurred when parsing config for DataSet 'boxplot_figures_cfa':
Class 'kedro.extras.datasets.pandas.HTMLDataSet' not found or one of its dependencies has not been installed.

ONNX

Pickle

pipeline

ONNX

publish

Picke

ONNX

convert_sklearn

.onnx

kedro.extras.datasets.matplotlib.MatplotlibWriter

'Figure' object has no attribute 'save'

sav_fig

save

auto-sklearn

3.9

python 3.9.16

pyenv

Pipfile

3.9

kedro viz

plots:
  type: PartitionedDataSet
  path: data/08_reporting/plots
  dataset:
    type: plotly.JSONDataSet
  filename_suffix: '.json'

fig = plotly.io.read_json(file); fig.show()

CircularDependencyError: Circular dependencies exist among these items: [node1 ...., node2]

PartitionedDataSet

PartitionedDataSet

scenario_x/
├── iter_1/
│   ├── run_1.csv
│   ├── run_2.csv
│   └── run_3.csv
└── iter_2/
    ├── run_1.csv
    ├── run_2.csv
    └── run_3.csv
scenario_y/
├── iter_1/
│   ├── run_1.csv
│   ├── run_2.csv
│   └── run_3.csv
└── iter_2/
    ├── run_1.csv
    ├── run_2.csv
    └── run_3.csv

_partitioned_csvs: &_partitioned_csvs
  type: PartitionedDataSet
  dataset:
    type: pandas.CSVDataSet
    load_args:
      index_col: 0
    save_args:
      index: true
  overwrite: true
  filename_suffix: ".csv"

_partitioned_jsons: &_partitioned_jsons
  type: PartitionedDataSet
  dataset:
    type: json.JSONDataSet
  filename_suffix: ".json"

my_csv_part_ds:
  path: data/07_model_output/my_csv_part_ds
  <<: *_partitioned_csvs

my_json_part_ds:
  path: data/07_model_output/my_json_part_ds
  <<: *_partitioned_jsons

def node_that_generates_part_ds(scenario, **kwargs):
  res = {'scenario_x/iter_1/run_1': df1, 'scenario_x/iter_1/run_2': df2,  .... and so on}}
  return res

res

bool

str

{%- if <http://cookiecutter.my|cookiecutter.my>_flag != "False" %}

127.0.0.1

localhost

[etape 1] > [etape 2] > [if score_etape2 < X ] > [etape4]
				      > [if score_etape2 >= X ] > [etape5]

@singledispatchmethod

functools

fit()

predict()

ValueError: Invalid Node definition: first argument must be a function, not 'singledispatchmethod'.

if not callable(func):
            raise ValueError(
                _node_error_message(
                    f"first argument must be a function, not '{type(func).__name__}'."
                )
            )

random_state: 42

random_state: ${random_state}

kedro run 
>>> [...] 
TypeError: Cannot cast scalar from dtype('<U15') to dtype('int64') according to the rule 'safe'

random_state

"42"

42

int(params['random_state'])