Ofir
05/05/2023, 2:22 PMJuan Luis
05/05/2023, 2:25 PMOfir
05/05/2023, 2:27 PMOfir
05/05/2023, 2:29 PMOfir
05/05/2023, 2:30 PMOfir
05/05/2023, 2:34 PMOfir
05/05/2023, 2:35 PMNok Lam Chan
05/05/2023, 2:35 PMNok Lam Chan
05/05/2023, 2:36 PMfsspec
, so it works on most object store i.e. s3, gcfs, etcOfir
05/05/2023, 2:36 PMNok Lam Chan
05/05/2023, 2:37 PMParquetDataSet
alreadyOfir
05/05/2023, 2:38 PMOfir
05/05/2023, 2:38 PMOfir
05/05/2023, 2:39 PMLazyParquetDataSet
class (possibly regenerate and then load from disk).Nok Lam Chan
05/05/2023, 2:43 PMNok Lam Chan
05/05/2023, 2:44 PMOfir
05/05/2023, 2:44 PMOfir
05/05/2023, 2:45 PMNok Lam Chan
05/05/2023, 2:45 PMOfir
05/05/2023, 2:45 PMNok Lam Chan
05/05/2023, 2:48 PMkedro-dvc
or build a plugin to integrate kedro with it.Ofir
05/05/2023, 2:49 PMNok Lam Chan
05/05/2023, 2:53 PMNok Lam Chan
05/05/2023, 2:53 PMOfir
05/05/2023, 2:56 PMOfir
05/05/2023, 2:56 PMfile://
- the local file system is default in the absence of any protocol, it also permits relative paths.Ofir
05/05/2023, 2:58 PMNok Lam Chan
05/05/2023, 5:06 PMNok Lam Chan
05/05/2023, 5:06 PMNok Lam Chan
05/05/2023, 5:07 PMNok Lam Chan
05/05/2023, 5:08 PM