I'm encountering issues when running the %load_ext kedro.ipython cell in VSCode built-in Jupyter editor on a remote compute. Many times it gets stuck running the cell and renders the remote compute unworkable, eventually requiring a reboot. I haven't figured out in which scenarios this tends to happen, but I encounter it a lot. Can anyone help or direct me to how to provide more information? Thanks, Gilad

So this feature has been released for ~2 days at this point! We tested it on local jupyter, vscode and databricks but not remote jupyter IIRC. Do you get a stack trace at all?

@Gilad Rubin is using load_ext, didn’t mention load_node I think

🙈 I should read slowly

but echoing @datajoely question, @Gilad Rubin when it does get stuck, does it ever print any logs?

This is the only "log" I'm getting (I couldn't find any other logs in VSCode):

[02/29/24 11:50:29] INFO     Registered line magic '%reload_kedro'                                   __init__.py:51
                    INFO     Registered line magic '%load_node'                                      __init__.py:53
[02/29/24 11:50:30] INFO     Resolved project path as:                                              __init__.py:164
                             /mnt/batch/tasks/shared/LS_root/mounts/clusters/.....                                                          
                             To set a different path, run '%reload_kedro <project_root>'                           
[02/29/24 11:50:44] INFO     Registering new custom resolver: 'km.random_name'                    mlflow_hook.py:65
[02/29/24 11:50:47] INFO     Kedro project CF Estimation                                            __init__.py:134
                    INFO     Defined global variable 'context', 'session', 'catalog' and            __init__.py:135
                             'pipelines'                                                                           
[02/29/24 11:50:51] INFO     Registered line magic 'run_viz'                                        __init__.py:141

Perhaps it has to do with the mlflow hook? maybe it freezes when it can't establish a connection to my Azure mlflow workspace...? What is the meaning of "registering new custom resolver"? I'll try to notice that next time I run it and see where it gets stuck.

can you change your log level to DEBUG?

okay so just to clarify, you got those logs and then it froze? if so, that's useful. it means that it's making some progress but then getting stuck somewhere

@datajoely How do I change the log level? @Juan Luis This is the output when it's working. I'll take a look where it freezes next time it happens and let you know. And yes, it's a linux machine. I'm connecting through the Azure extension in VSCode (I don't know if it's SSH based or something else)

If you have a logging.yml in your project change it where it says INFO

This is the output when it's working. I'll take a look where it freezes next time it happens and let you know.

👍🏼 thanks!

FYI when running notebooks on VSCode I sometimes see additional logging in the tab Output for the extension Jupyter so if the notebook interface freezes it could be worth checking the logs there too

Ok, I'm in this state now. Trying to run the simple "kedro" command on my terminal within the project folder and nothing happens. Running the kedro command outside of the project folder and it works. Trying to run commands from vscode jupyter notebook is stuck. I've attached some screenshots.

👀 hmmm if

htop

is segfaulting... something very weird is going on

kedro outside of the project folder doesn't fail. and the fact that it provides options is a feature not a bug imo :)

yes it should give options - and fail 🙂 same as

$ cp
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file target_file
       cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file ... target_directory
$ echo $?  # nonzero
64

The workstation completely died and now I can't even restart it 😄 I usually try to update my kedro version frequently. I think it's the latest version

ugh 😅

I'll try these commands next time and update 🙂 I can't seem to find a pattern regarding when this happens.