fmfreeze
05/02/2023, 5:22 PMDeepyaman Datta
05/02/2023, 5:27 PMsrc
folder under there I think~.Juan Luis
05/02/2023, 5:29 PM[tool.kedro]
in pyproject.toml
would remain untouched, and as long as there's a settings.py
, everything should be fine? but I might be missing thingssrc
in the ProjectMetadata
. but maybe it shouldn't have to be that way š¤source_dir
in pyproject.toml
, it turns out!fmfreeze
05/02/2023, 5:38 PMDeepyaman Datta
05/02/2023, 5:46 PMYou can configure the source directory location, but it is still going to require aI misspoke on this/misread the code; it will default tofolder under there I think.src
src
is source_dir
isn't provided, but you should be able to override it.
But when it comes to kedro-viz, things might get even trickier?!I don't think Kedro-Viz will be any worse; it basically also delegates the context-loading to
kedro.framework.startup.bootstrap_project
, so as long as that works, you could be fine.Juan Luis
05/02/2023, 5:48 PM--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,7 @@
[tool.kedro]
package_name = "spaceflights"
project_name = "Spaceflights"
+source_dir = "spaceflights"
kedro_init_version = "0.18.8"
[tool.isort]
this moves you from a src
layout to a "flat" layout (instead of src/spaceflights/settings.py
, you'd have spaceflights/settings.py
)fmfreeze
05/03/2023, 2:38 PMJuan Luis
05/03/2023, 2:40 PMmv src/spaceflights/ .
, hence having a spaceflights
directory at the root of the project. is that what you were attempting?fmfreeze
05/03/2023, 2:52 PMModuleNotFoundError: No module named spaceflights
).Juan Luis
05/03/2023, 3:22 PMtree
output? here's mine at the moment:
.
āāā README.md
āāā conf
ā āāā ...
āāā data
ā āāā ...
āāā pyproject.toml
āāā setup.cfg
āāā spaceflights
āāā __init__.py
āāā __main__.py
āāā __pycache__
āāā pipeline_registry.py
āāā pipelines
āāā settings.py
fmfreeze
05/03/2023, 3:43 PMJuan Luis
05/03/2023, 3:44 PMsource_dir
for my tree above:
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,7 @@
[tool.kedro]
package_name = "spaceflights"
project_name = "Spaceflights"
+source_dir = "."
kedro_init_version = "0.18.8"
(by using source_dir = "spaceflights"
Kedro was looking in the wrong place - I'm not sure how I got it working yesterday)An ideal solution would be even more flexible to achieve a more granular structure.
E.g. having settings.py & pipeline_registry.py in it's own kedro subfolder.I agree it would be nice indeed. at the moment it's hardcoded: https://github.com/kedro-org/kedro/blob/2e70dec396567d2ba38456179aad0d4ae8e83b31/kedro/framework/project/__init__.py#L256-L259 would you want to open an issue about it @fmfreeze? (no guarantees about it, but at least might serve to spark broader discussion)
fmfreeze
05/03/2023, 6:05 PMJuan Luis
05/03/2023, 6:18 PMfmfreeze
05/03/2023, 9:26 PMJuan Luis
05/03/2023, 9:48 PMsettings.py
and pipeline_registry.py
- so, to allow for the customization you proposed @fmfreeze, Kedro would need to make those lines more generic and allow for some configuration, similar to what source_dir
does. does this make sense?fmfreeze
05/16/2023, 8:35 AMsource_dir
works also nice. šJuan Luis
05/16/2023, 11:57 AM