Hi, everyone. I'm new to Kedro and trying to run a...
# questions
o
Hi, everyone. I'm new to Kedro and trying to run a Kedro pipeline inside (or from) a Google App Engine. Google App Engine is a container-based solution, that mounts the container file system as read-only mode. I created the datasets in remote storages for a decoupled design and also to avoid errors because of the read-only file's mode. Even if the pipeline runs, creating the datasets and artifacts (I'm running the spaceflight tutorial pipeline), and the App Engine log doesn't show errors nor warnings, the Kedro log do arise one single error: "ERROR Failed to read the file: plugin.py:111 /workspace/pyproject.toml. [Errno 30] Read-only file system: '/workspace/pyproject.toml' " This error doesn't appear when running the pipeline in local nor in the local development server (dev_appserver.py). The error is not locking me, but I would like to understand its implications and a posible way to solve it. Any hint is very appreciated. Thanks in advance.
m
1. Don't use app engine, it's an old technology. 2. Use cloud run and use fsspec in your datasets to connect to GCS. Most of the built in datasets support this out of the box.
👍 1
o
Thank you. As you mentioned, ffspec is working smoothly, I have no issues with datasets or artifacts. Regarding not using App Engine, it's the component enabled or disposed of so far. I'll try to get Cloud Run enabled for the solution.
Thanks a lot👍
👍 1