https://kedro.org/ logo
#questions
Title
# questions
g

Gilad Rubin

02/29/2024, 7:19 AM
I'm encountering issues when running the %load_ext kedro.ipython cell in VSCode built-in Jupyter editor on a remote compute. Many times it gets stuck running the cell and renders the remote compute unworkable, eventually requiring a reboot. I haven't figured out in which scenarios this tends to happen, but I encounter it a lot. Can anyone help or direct me to how to provide more information? Thanks, Gilad
d

datajoely

02/29/2024, 8:03 AM
So this feature has been released for ~2 days at this point! We tested it on local jupyter, vscode and databricks but not remote jupyter IIRC. Do you get a stack trace at all?
j

Juan Luis

02/29/2024, 8:29 AM
@Gilad Rubin is using load_ext, didn’t mention load_node I think
d

datajoely

02/29/2024, 8:29 AM
🙈 I should read slowly
😄 1
j

Juan Luis

02/29/2024, 8:30 AM
but echoing @datajoely question, @Gilad Rubin when it does get stuck, does it ever print any logs?
g

Gilad Rubin

02/29/2024, 11:53 AM
This is the only "log" I'm getting (I couldn't find any other logs in VSCode):
Copy code
[02/29/24 11:50:29] INFO     Registered line magic '%reload_kedro'                                   __init__.py:51
                    INFO     Registered line magic '%load_node'                                      __init__.py:53
[02/29/24 11:50:30] INFO     Resolved project path as:                                              __init__.py:164
                             /mnt/batch/tasks/shared/LS_root/mounts/clusters/.....                                                          
                             To set a different path, run '%reload_kedro <project_root>'                           
[02/29/24 11:50:44] INFO     Registering new custom resolver: 'km.random_name'                    mlflow_hook.py:65
[02/29/24 11:50:47] INFO     Kedro project CF Estimation                                            __init__.py:134
                    INFO     Defined global variable 'context', 'session', 'catalog' and            __init__.py:135
                             'pipelines'                                                                           
[02/29/24 11:50:51] INFO     Registered line magic 'run_viz'                                        __init__.py:141
Perhaps it has to do with the mlflow hook? maybe it freezes when it can't establish a connection to my Azure mlflow workspace...? What is the meaning of "registering new custom resolver"? I'll try to notice that next time I run it and see where it gets stuck.
d

datajoely

02/29/2024, 11:57 AM
can you change your log level to DEBUG?
j

Juan Luis

02/29/2024, 12:17 PM
okay so just to clarify, you got those logs and then it froze? if so, that's useful. it means that it's making some progress but then getting stuck somewhere
this is a Linux machine you're connecting to with SSH, I assume?
g

Gilad Rubin

02/29/2024, 2:11 PM
@datajoely How do I change the log level? @Juan Luis This is the output when it's working. I'll take a look where it freezes next time it happens and let you know. And yes, it's a linux machine. I'm connecting through the Azure extension in VSCode (I don't know if it's SSH based or something else)
d

datajoely

02/29/2024, 2:15 PM
If you have a logging.yml in your project change it where it says INFO
K 1
j

Juan Luis

02/29/2024, 2:33 PM
This is the output when it's working. I'll take a look where it freezes next time it happens and let you know.
👍🏼 thanks!
K 1
i

Iñigo Hidalgo

02/29/2024, 3:36 PM
FYI when running notebooks on VSCode I sometimes see additional logging in the tab Output for the extension Jupyter so if the notebook interface freezes it could be worth checking the logs there too
this 2
K 1
g

Gilad Rubin

03/05/2024, 10:02 AM
Ok, I'm in this state now. Trying to run the simple "kedro" command on my terminal within the project folder and nothing happens. Running the kedro command outside of the project folder and it works. Trying to run commands from vscode jupyter notebook is stuck. I've attached some screenshots.
👀 1
Edit: I tried to run "htop" on the terminal and I'm getting segmentation fault. I'm trying to understand if it's cloud related or something with kedro
j

Juan Luis

03/05/2024, 10:22 AM
👀 hmmm if
htop
is segfaulting... something very weird is going on
the fact that
kedro
outside the directory immediately fails, but inside blocks, means that there's something going on possibly with one of the hooks
honestly
kedro
doesn't have enough logging or diagnostics to tell us what's going on. for example, I'd expect a call to
kedro
to fail immediately (since you're not providing any command)
g

Gilad Rubin

03/05/2024, 10:25 AM
kedro outside of the project folder doesn't fail. and the fact that it provides options is a feature not a bug imo :)
j

Juan Luis

03/05/2024, 10:28 AM
yes it should give options - and fail 🙂 same as
Copy code
$ cp
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file target_file
       cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file ... target_directory
$ echo $?  # nonzero
64
@Gilad Rubin in the terminal that's stuck (if it's the case still) can you cancel and try
kedro --version
, followed by
kedro info
?
g

Gilad Rubin

03/05/2024, 12:20 PM
The workstation completely died and now I can't even restart it 😄 I usually try to update my kedro version frequently. I think it's the latest version
j

Juan Luis

03/05/2024, 12:45 PM
ugh 😅
good to know. the
kedro --version
is more to see if it works, not so much for the output 😄
g

Gilad Rubin

03/05/2024, 1:07 PM
I'll try these commands next time and update 🙂 I can't seem to find a pattern regarding when this happens.
4 Views