Jon Cohen
07/24/2023, 6:17 PMdatajoely
07/24/2023, 6:18 PMJon Cohen
07/24/2023, 6:24 PMdatajoely
07/24/2023, 6:28 PMDeepyaman Datta
07/24/2023, 7:26 PMfind_pipelines()
), we wanted people to be able to use it by default. However, lot of users may have work-in-progress pipelines, and we didn't want to prevent them from using autodiscovery/running other pipelines just because they're still trying to get some pipeline working.Jon Cohen
07/24/2023, 7:27 PMfind_pipelines
to error on warnings? Or is the answer to just not use pipeline autodiscoverydatajoely
07/24/2023, 7:52 PMDeepyaman Datta
07/25/2023, 6:48 AMIs there a way to mark a pipeline as a work in progress and then tellNo, there's no such built-in functionality. If you think it makes sense to haveto error on warnings? Or is the answer to just not use pipeline autodiscoveryfind_pipelines
find_pipelines
raise an error if a discovered pipeline has issues, I think that could be considered--feel free to raise an issue. I don't know how many people would use it, but IMO there's little cost to adding a flag to find_pipelines
, like find_pipelines(raise_errors=False)
.
If you think it makes sense to have a mechanism to mark a WIP pipeline and for find_pipelines
to raise an error on a non-WIP pipeline... I mean, feel free to raise an issue, too. However, I think this is a broader change (how do you mark a pipeline as WIP? should a WIP pipeline be excluded from autodiscovery, or only excluded if it fails?), and I'm not sure how useful it would be unless see a lot of demand for it.
For now, you can easily use a modified find_pipelines
if your organization/use case needs it; find_pipelines
is a function defined here: https://github.com/kedro-org/kedro/blob/0.18.11/kedro/framework/project/__init__.py#L302-L369 so you can create a modified function for now if there's significant benefit to doing soNok Lam Chan
07/26/2023, 9:08 PMraise_error=True
a default?Deepyaman Datta
07/26/2023, 9:09 PMNok Lam Chan
07/26/2023, 9:11 PM