Marc Gris08/22/2023, 10:39 AM
kedro run -p data_processing --tags=users
I’m wondering why it is trying to load the filters params for the filtering pipeline while I’m running only the data_processing pipeline ? Those
[08/21/23 16:39:45] WARNING /home/kedro_docker/.local/lib/python3.9/site-packages/kedro/framework/project/__init__.py:359: UserWarning: warnings.py:109 An error occurred while importing the 'dodo_kedro.pipelines.filtering' module. Nothing defined therein will be returned by 'find_pipelines'. Traceback (most recent call last): File "/home/kedro_docker/.local/lib/python3.9/site-packages/kedro/framework/project/__init__.py", line 357, in find_pipelines pipeline_module = importlib.import_module(pipeline_module_name) File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 680, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 850, in exec_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/workspaces/dodo-kedro/src/dodo_kedro/pipelines/filtering/__init__.py", line 6, in <module> from .pipeline import create_pipeline File "/workspaces/dodo-kedro/src/dodo_kedro/pipelines/filtering/pipeline.py", line 7, in <module> from . import nodes File "/workspaces/dodo-kedro/src/dodo_kedro/pipelines/filtering/nodes.py", line 41, in <module> for _filter in catalog.load("params:filters")] File "/home/kedro_docker/.local/lib/python3.9/site-packages/kedro/io/data_catalog.py", line 473, in load dataset = self._get_dataset(name, version=load_version) File "/home/kedro_docker/.local/lib/python3.9/site-packages/kedro/io/data_catalog.py", line 406, in _get_dataset raise DatasetNotFoundError(error_msg) kedro.io.core.DatasetNotFoundError: Dataset 'params:filters' not found in the catalog - did you mean one of these instead: parameters, params:models, params:columns.users warnings.warn(
are indeed not defined, since I had “commented-out” those in order to debug something… Granted: This is not a “drama” since it only raises a warning and still allows to run the data_processing pipeline. Yet, I must say it’s a little confusing to see a warning for a pipeline that was not called This seems to me to related to the lazy loading of datasets for which I have opened a feature request on github. Looking forwards to reading your thoughts / comments. Regards M
marrrcin08/22/2023, 11:18 AM
Aleksander Jaworski08/22/2023, 11:24 AM
Marc Gris08/22/2023, 11:57 AM
Juan Luis08/23/2023, 8:11 AM
Marc Gris08/23/2023, 8:18 AM
Thanks for your comment 🙏🏼 M
datajoely08/23/2023, 8:19 AM
and register them yourself for now
Marc Gris08/23/2023, 8:44 AM
I end up with
kedro run -p filtering
This is a node in another pipeline… I’m therefore re-posting what I’ve just posted previously: My initial feature request was indeed originally for datasets. But couldn’t, or even shouldn’t, it be broadened to pipelines as well ? The spirit being: Why load something that is not needed / called for ? & Why should one be blocked from working on pipeline a because things are broken in pipeline b ? Many thanks in advance for you comments / suggestions. Cheers M.
TypeError: Inputs of 'recommend' function expected ['scores', 'k'], but got ['filtered_scores', 'params:k', 'params:user_id_col', 'params:item_id_col', 'params:score_col']
Juan Luis08/25/2023, 9:32 AM
function? about the historical context on why things work this way, we lack that insight at the moment, but we haven't received similar reports in the past. so maybe users were finding ways of working around this without telling us.
Marc Gris08/25/2023, 10:40 AM
does the job of course 🙏🏼 My posts are not so much about “complaining” than they are about “giving feedbacks” that hopefully / humbly could be helpful to you guys 🙂 What I described above, seems to be to be a reasonable expectation from a first time kedro user… Hence my sharing this here. Thanks again & have a nice day, M.
Juan Luis08/25/2023, 11:58 AM
Marc Gris08/25/2023, 12:10 PM