https://kedro.org/ logo
Powered by
#questions
Title
# questions
g

Giulio Morina

08/17/2023, 11:33 AM
Hi there! I have a jupyter notebook on a managed service not located within a kedro project folder. I load the Kedro ipython extension with no issues via 
%load_ext kedro.ipython
but then when I run 
%reload_kedro <folder_root>
I get “UsageError: Line magic function 
%reload_kedro
not found.” Feels like I am missing something obvious, any tips?
d

datajoely

08/17/2023, 11:55 AM
It’s pretty likely the notebook environment/kernel selected doesn’t have kedro installed
can you do 
import kedro
okay?
g

Giulio Morina

08/17/2023, 12:04 PM
Yeah 
import kedro
works fine
d

datajoely

08/17/2023, 12:35 PM
hmm and do you know which version of Kedro you’re using
g

Giulio Morina

08/17/2023, 12:36 PM
0.18.4
d

datajoely

08/17/2023, 12:38 PM
after doing your first load of 
%load_ext kedro.ipython
can you access the catalog fine?
g

Giulio Morina

08/17/2023, 12:39 PM
No it’s not defined. I do get this after loading:
Copy code
WARNING  Kedro extension was registered but couldn't find a Kedro project. Make  
sure you run '%reload_kedro <project_root>'.
d

datajoely

08/17/2023, 12:40 PM
Oh! gotcha
so it’s a working directory issue
I think - the path you’re finding here isn’t right 
%reload_kedro path/to/proj
g

Giulio Morina

08/17/2023, 12:42 PM
But the problem is that I cannot set the right path as when I run the 
%reload_kedro
line magic it says that the line magic 
%reload_kedro
is not found.
d

datajoely

08/17/2023, 12:43 PM
the first one needs to be initialised properly - you’re not actually loading the extension in a way it can expose 
%reload_kedro
let me escalate this
n

Nok Lam Chan

08/17/2023, 1:00 PM
Hey @Giulio Morina, quick question. • 
%load_ext kedro.ipython
does this work? If not then 
%load_ext kedro.extras.extensions.ipython
Can you do 
%reload_kedro
after you run the above command?
g

Giulio Morina

08/17/2023, 1:04 PM
Both 
%load_ext kedro.ipython
and 
%load_ext kedro.extras.extensions.ipython
work, but 
%reload_kedro
does not work after running either of the commands above
n

Nok Lam Chan

08/17/2023, 1:08 PM
So 
reload_kedro
doesn’t exist after you do 
%load_ext
?
g

Giulio Morina

08/17/2023, 1:08 PM
exactly
n

Nok Lam Chan

08/17/2023, 1:09 PM
hm, let me quickly test it with 18.4
If it’s easier I can do a quick call to solve this by 3pm. Otherwise it may be helpful to do a 
tree
in the root directory. I need to know these • where is the 
pyproject.toml
• where is the notebook
Copy code
def load_ipython_extension(ipython):
    """
    Main entry point when %load_ext kedro.ipython is executed, either manually or
    automatically through `kedro ipython` or `kedro jupyter lab/notebook`.
    IPython will look for this function specifically.
    See <https://ipython.readthedocs.io/en/stable/config/extensions/index.html>
    """
    ipython.register_magic_function(magic_reload_kedro, magic_name="reload_kedro")

    if _find_kedro_project(Path.cwd()) is None:
        logger.warning(
            "Kedro extension was registered but couldn't find a Kedro project. "
            "Make sure you run '%reload_kedro <project_root>'."
        )
        return
This is weird because from the log you show, the log is emitted after 
%reload_kedro
registered.
For the mean time, if you are in a rush. 
from kedro.ipython import reload_kedro
reload_kedro(path)
This should also work.
@Giulio Morina does this works?
g

Giulio Morina

08/18/2023, 1:37 PM
Hey @Nok Lam Chan! Sorry I got carried away with other work 😅 that gives me this error
Copy code
ValueError: Given configuration path either does not exist or is not a valid directory: src
n

Nok Lam Chan

08/18/2023, 2:48 PM
Can you do a 
tree
command to show how is it structured? • where is your 
pyproject.toml
• where is your notebook file
g

Giulio Morina

08/18/2023, 2:56 PM
I’ve pruned the tree a bit, but this is how it looks:
Copy code
.
├── analysis <- KEDRO PROJECT FOLDER
│   ├── README.md
│   ├── conf
│   │   ├── base
│   │   │   ├── catalog
│   │   │   │   └── main.yml
│   │   │   ├── globals.yml
│   │   │   ├── parameters.yml
│   ├── pyproject.toml
│   ├── setup.cfg
│   ├── src
│   │   ├── analysis
│   │   │   ├── extract_datasets.py
│   │   │   ├── hooks.py
│   │   │   ├── pipeline_registry.py
│   │   │   ├── settings.py
│   │   ├── <http://dev-requirements.in|dev-requirements.in>
│   │   ├── dev-requirements.txt
│   │   ├── <http://requirements.in|requirements.in>
│   │   ├── requirements.txt
│   │   ├── setup.py
├── notebooks
│   ├── test.ipynb <- NOTEBOOK 
├── parameters.yaml
├── pyproject.toml
✅ 1
Note that if I move the notebook test.ipynb inside the analysis folder then it all works fine (I can just do `%reload_kedro and it works). But ideally I’d like to keep it outside
n

Nok Lam Chan

08/18/2023, 2:58 PM
Ah
Can you try 
reload_kedro("../")
?
It need the 
pyproject.toml
to locate the root, not the src file
g

Giulio Morina

08/18/2023, 3:03 PM
If I do 
reload_kedro("../")
I get `
Copy code
RuntimeError: There's no '[tool.kedro]' section in the 'pyproject.toml'. Please add '[tool.kedro]' section to the 
file with appropriate configuration parameters.
as I believe it picks up the pyproject.toml in the root folder which is not the one in Kedro project. If I do 
reload_kedro("../analysis")
then I get the previous error:
Copy code
ValueError: Given configuration path either does not exist or is not a valid directory: src
n

Nok Lam Chan

08/18/2023, 3:08 PM
I see, this is a monorepo with project nested inside projects. It may cause the backtracking of 
pyproject.toml
indeed. This is a very good example 🙂 Let me run a few tests and get back to you.
g

Giulio Morina

08/18/2023, 3:09 PM
Thanks Nok! It’s not critical, so please do not prioritise if you have things 🙂
n

Nok Lam Chan

08/18/2023, 3:21 PM
I have better understanding of the problem now. 
reload_kedro
have a weak assumption that it is a Kedro Project. Right now you have a Python project outside, and within it there is a kedro project. So when it looks up to find 
pyproject.toml
it finds 
roots/pyproject.toml
rather than `analysis/pyproject.toml`(the kedro project)
Can I ask one last thing, did you change anything in 
settings.py
or the 
conf_source
? I try to reproduce your error but I didn’t get a ValueError when I do 
reload_kedro("../analysis")
g

Giulio Morina

08/18/2023, 3:25 PM
I didn’t set it all the project(s) up so I’m not 100% sure 😅 But quickly checking it seems it has been changed. There is a custom context and custom config loader so it may mess up things
✅ 1
4 Views
Powered by