I'm trying to run `kedro new` in non-interactive w...
# questions
j
I'm trying to run
kedro new
in non-interactive ways so it's compatible with Jupyter shell commands (
!kedro new ...
). I see two ways: •
yes "Project Name" | kedro new --starter=xxx
: works, but it's UNIX-only (don't think this will work on Windows), assumes there is only one question, and looks a bit arcane. • `vim kedro.yaml ... && kedro new --starter=xxx --config=kedro.yaml`: works, but I'm creating a file that I will only use once, plus it's not very easy to discover what structure should the file have (one has to navigate to the source code of the starter in question, locate the
prompts.yml
, and mimic those keys) I see that this has been unchanged since basically "forever" but I'm wondering what are folks opinions on having a way to pass these configs to the CLI? something like
kedro new --starter=xxx --project_name=yyy
d
this has been the recommended way to answer these questions
kedro new --starter=xxx --config=kedro.yaml
I’m actually in favour of improving this like you suggest
✔️ 1
I’d also like to include the telemetry opt in/out decision in that workflow too
☝🏼 1
a
Personally I think the current flow is usually fine: 1. It’s not very common to make a kedro project in a non-interactive way (in addition to Jupyter, which hopefully will be solved so it can be interactive, CI is the only one that comes to mind). Certainly this isn’t something that gets asked a lot - people are used to just doing it interactively. AFAIK the whole
config-file
option was created to support this CI use case but I imagine it’s not used much at all 2.
yes
works for answering multiple questions (since it can respond to multiple prompts, and you can always just press enter in
kedro new
to use the defaults). Agreed that it’s rather arcane and doesn’t work on Windows, but if you’re doing something on CI then doing this programmatically or using a config file is not such a weird thing to do (similarly for just creating the
.telemetry
file) Would passing flags directly through
kedro new
be better? I think maybe. On the one hand, it would be a nice simplification for the cases that a kedro starter just requires
project_name
(which at the moment I think is all of them? And probably 99% of any ones that people have made outside kedro too tbh). And then we could also add the telemetry flag in there too. But I don’t think it would work well with the full generality of kedro starters that might have multiple fields you need to enter (how do you document and pass through arbitrary fields from click?). More generally I think we should reconsider the new project flow (like
kedro starters list
is disjoint from
kedro new
) and try and delegate as much to cookiecutter as possible (see https://github.com/kedro-org/kedro/issues/1692).
👀 1
j
thanks for the clarification on the
yes
command @Antony Milne (I only knew

this other Yes

). about the rest and gh-1692, will have a proper look tomorrow
a
I’m a big fan of the other Yes 😄
🎸 1
btw from what I remember there are a lot of features that would be nice for kedro to have in cookiecutter that have been sort of on hold forever. Maybe some of them have actually been released now, but I think not unfortunately.
j
yeah, last time I checked cookiecutter is in low or no maintenance
d
I’ve been tracking the copier library - but also Cookiecutter doesn’t need a lot of maintenance these days right?
j
TIL: cookiecutter allows non-interactive runs:
Copy code
cookiecutter --no-input gh:msabramo/cookiecutter-supervisor program_name=foobar startsecs=10
(from the docs)
👍 1
whatever we do here, let's tackle it after https://github.com/kedro-org/kedro/issues/1692 😄
👍 1