https://kedro.org/ logo
#questions
Title
# questions
m

Mark Druffel

03/11/2024, 9:06 PM
Sorry posted a plugins question here again, I'll do this right at some point
d

Deepyaman Datta

03/11/2024, 9:09 PM
It seems it's imported at the top of the module where
SparkDataset
is defined, so technically yes. 😞 I can raise a PR (unless you want to) to relax that.
❀️ 1
Looking at the code, it's also annoying that
hdfs
dependencies are required even if you're not using it.
m

Mark Druffel

03/11/2024, 11:33 PM
Thanks for the info @Deepyaman Datta, I'll take a shot at a PR appreciate it!
πŸš€ 2
Sorry this probably isn't the right forum, but I'm following the contribution guidelines and hit the below error when I ran
make test
. Not sure if I need to do something to address those hook errors or if I can just ignore...
d

Deepyaman Datta

03/11/2024, 11:46 PM
I'm not familiar with these tests off the top of my head, but it's probably not a big deal if just they're failing, unless you're specifically working on session extension hooks. I'm guessing they're pretty niche/self-contained. On a side note (since we're in this thread), are you trying to make a PR to the
SparkDataset
? In that case, you probably want to be running the tests for that repo.
gratitude thank you 1
m

Mark Druffel

03/11/2024, 11:49 PM
^^I was just realizing that as I kept going through the guide, I always forget kedro-datasets is a separate repo πŸ€¦β€β™‚οΈ
d

Deepyaman Datta

03/11/2024, 11:51 PM
I'm seeing that the contribution guidelines should probably be updated, seeing as they refer to the old
core
and
extras
distinction (from when datasets were part of the same repo). In case you haven't found it yet, https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/CONTRIBUTING.md are the datasets contribution docs.
n

Nok Lam Chan

03/12/2024, 1:22 PM
@Mark Druffel It's normal to see these error, this i what @Elena Khaustova experienced too. The reason for it is the test is asserting the number of hooks. If you follow strictly the guide, it should work. i.e.
pip install ".[test"]
. The reason why it fails is that you are most likely not working in a "fresh" environment. For example, you already have
kedro-telemetry
installed, thus this affect the number of hooks
kedro
detect and thus the test is failed. If you try to create a draft PR, you would see those tests passing on CI.
πŸ‘ 1
Maybe there's some better ways to handle these tests?
If you find anything we should change in the guide and make this contribution process more obivous, please go ahead to open an issue or even a PR if you want. This is, after all written for the open source contributors.
e

Elena Khaustova

03/12/2024, 2:17 PM
@Nok Lam Chan Just checked that these tests are failing for me even when using a clear environment and installing just test requirements
Copy code
make install-test-requirements
make test
So we can maybe think of how to fix that cause it’s a bit confusing
n

Nok Lam Chan

03/12/2024, 2:25 PM
πŸ‘€
Let me try this, https://github.com/kedro-org/kedro/blob/main/.github/workflows/unit-tests.yml This is what get run in our CI and it's basically
make install-test-requirements
make test
Can you show the output you get from
kedro info
?
I forgot to reply this yesterday, I can confirm a fresh install will still fail the tests. I am not sure about why but I will dig into it today. It can be tracked in this issue. Test fails after following contribution guide and install dependencies in a fresh environment.
@Mark Druffel May I ask what OS are you using? Is it Windows or Mac?
m

Mark Druffel

03/18/2024, 5:09 PM
Sorry @Nok Lam Chan I went on vacation last week so I'm just catching up. I was on a Mac Sonoma 14.3.1
n

Nok Lam Chan

03/18/2024, 5:15 PM
Thanks @Mark Druffel, we fixed this on
kedro
anyway, I think that's the same issue that you encountered initially :) See: https://github.com/kedro-org/kedro/issues/3702
❀️ 1