https://kedro.org/ logo
Join Slack
Powered by
what if you could see the progress of the pipeline...
# user-research
j
what if you could see the progress of the pipeline visually, and also immediately spot failing nodes? any early thoughts?
❤️ 9
frog wow scroll 6
party wizard 4
K 10
d
Would be very nice. I've literally wanted this for years. Can it not only be in the VSCode plugin? 🙏 Just thinking: how does this position Kedro with orchestrators that support local runs? I think its fine--this is more for the DE/DS for interactive development, while you still need an orchestrator for deployment--but just worth clarifying given Kedro is often confused with orchestrators.
👀 2
c
Looks great! Cool seeing that 
reviews
is the slowest to read. If this would also work when the pipeline is executed from the command line that would be nice.
💡 2
p
Amazing feature !
Can you tell me please how to open kedro viz on a side bar of vscode ?
as it is shown in the animation
j
p
amazing, thanks ! It seems I cannot filter out some nodes (based on tags for instance), do you know if this is an expected feature ?
j
yes indeed , stay tuned :)
Can it not only be in the VSCode plugin? 🙏
If this would also work when the pipeline is executed from the command line that would be nice.
how do you folks imagine this happening @Deepyaman Datta @Chris Schopp?
> how does this position Kedro with orchestrators that support local runs? I think its fine--this is more for the DE/DS for interactive development, while you still need an orchestrator for deployment--but just worth clarifying given Kedro is often confused with orchestrators. I think orchestrators like Airflow, Prefect, Dagster etc have some elements of autonomy (schedules, retries, logs etc) that aren't present here. given that (for now, see question above) this is about improving the developer experience, I don't think this would move us away from the "micro-orchestrator" category.
c
If the feature only works when clicking on the button I would only use it when I intentionally want to see it (as opposed to it being a bonus to my current workflow of frequently running 
kedro run --async -t customers
) I always have Kedro Viz (filtered to tag 
customers
, e.g.,) on a side monitor. I feel like this feature would help me notice when the timing of nodes/datasets is different than I'm expecting.
💡 1
d
how do you folks imagine this happening
On my end, I just don't use VSCode, so at least if it could be available through vanilla Kedro Viz would be nice. Is it tied to VSCode in any way?
j
it doesn't necessarily have to be tied. to me it's easier to imagine in a bound or controlled context (like everything happening within your IDE), but we could explore how feasible it would be to do it elsewhere. I don't think it would be easy but I could be wrong, maybe it's actually the same effort to achieve it inside and outside VS Code
d
Also, just thinking—unless Kedro-Viz is hosted, having to launch from within the UI is limiting for any "realistic" use case, where a pipeline may run for a long time. Feel like a common scenario may be: • Wonder, "Oh, I started my run, is it going to finish soon?" • Launch 
kedro viz
• See the current state • "Oh, not close, let me check back later" and close 
kedro viz
But I suppose this is possible by parsing the Kedro execution log or something?
j
conversation to be continued in https://github.com/kedro-org/kedro-viz/issues/2310 !
G 1
2 Views
Open in Slack
PreviousNext
Powered by