Hi all, Is there any guidance about... • How to run kedro in HPC? Should we use HOOK?

Someone will reply to you shortly. In the meantime, this might help:

I am not familiar with HPC, what's different from running any other Python program? Hooks are used for extending Kedro, i.e. logging, integrate with in-house system, profiling, setting up Spark connection etc, which are not part of the execution logic.

@Nok Lam Chan HPC is high performance computing. So we would like to use GPU in cluster. Can I defined what node I would like to run in Bash script, then submit to GPU and running the selected nodes?

Ya that's definitely doable, you just need to slice the pipeline with the cli arguments to specify pipeline or nodes. You can also group these nodes with tags

what's cli arguments?

I mean the Command Line Interface arguments https://docs.kedro.org/en/stable/development/commands_reference.html#modifying-a-kedro-run