Mate Scharnitzky
07/28/2023, 11:21 AM/notebook
directory. I’m trying to load some nodes that I created locally but it doesn’t find path. Two questions:
• How can load kedro context to this notebook?
• How can I load python modules developed as part of the kedro project?
Thank you!
I looked into this but somehow I can’t make it work:
https://docs.kedro.org/en/0.18.11/notebooks_and_ipython/kedro_and_notebooks.htmldatajoely
07/28/2023, 11:25 AMJuan Luis
07/28/2023, 11:37 AM%load_ext kedro.ipython
give?Mate Scharnitzky
07/28/2023, 12:05 PMfrom gci_pricing.pipelines.feature_engineering.treasury import (
create_treasury_total_features,
create_treasury_trx_type_features
)
I’m getting ModuleNotFoundError: No module named 'gci_pricing'
.Juan Luis
07/28/2023, 12:07 PMsys.path
thing. you're trying that inside a notebook right?Mate Scharnitzky
07/28/2023, 12:07 PM%load_ext kedro.ipython
, I’m getting ModuleNotFoundError: No module named 'kedro'
.Juan Luis
07/28/2023, 12:08 PMimport sys
print(sys.prefix, sys.executable)
and see if it matches what you expected?Mate Scharnitzky
07/28/2023, 12:09 PM/home/ec2-user/SageMaker/<my project name>/notebooks/data_engineering
/home/ec2-user/SageMaker/pyspark /home/ec2-user/SageMaker/pyspark/bin/python
Juan Luis
07/28/2023, 12:18 PMkedro info
, which kedro
, and which python
@Mate Scharnitzky?Mate Scharnitzky
07/28/2023, 12:19 PMwhich kedro
/anaconda3/bin/kedro
Juan Luis
07/28/2023, 12:21 PMkedro
is installed in the base
environment of Anaconda, which is not the same as /home/ec2-user/SageMaker/pyspark
Mate Scharnitzky
07/28/2023, 12:21 PMJuan Luis
07/28/2023, 12:21 PM/home/ec2-user/SageMaker/pyspark/bin/python -m pip install kedro
and then in your notebook you should be able to do %load_ext kedro.ipython
Mate Scharnitzky
07/28/2023, 12:21 PMpyspark
envJuan Luis
07/28/2023, 12:22 PMMate Scharnitzky
07/28/2023, 12:24 PMJuan Luis
07/28/2023, 12:25 PMMate Scharnitzky
07/28/2023, 12:26 PMdatajoely
07/28/2023, 12:26 PMMate Scharnitzky
07/28/2023, 12:27 PMpyspark
wasn’t persistent in SageMakerdatajoely
07/28/2023, 12:30 PMMate Scharnitzky
07/28/2023, 12:31 PMNok Lam Chan
07/28/2023, 12:44 PM