Lorenzo Castellino
10/31/2022, 8:48 AMnp.random.set_state()
before the nodes that require it but I would like to hear what your solution to "reproducible randomness" looks like 🙂Nok Lam Chan
10/31/2022, 9:08 AMLorenzo Castellino
10/31/2022, 9:17 AMpipeline_registry.py
file, the appropriate node.py
file or in another import location better suited for it?Nok Lam Chan
10/31/2022, 9:58 AMnode.py
may not be the best since there could be multiple nodes file using numpy.
pipeline_registry.py
is fine, or if you really want to make it explicit then a before_pipeline_run
hook would do the job.Lorenzo Castellino
10/31/2022, 9:59 AMbefore_pipeline_run
method. Thanks for the discussion and your view on the matter! 🙂
Maybe it might be something that I could add to the "common use cases" section of the docs? There is already a before_pipeline_run
example but it's quite generic. Maybe a more pragmatic example could be a nice addition.Nok Lam Chan
10/31/2022, 3:46 PMBen Levy
10/31/2022, 5:47 PMsettings.py
Deepyaman Datta
11/01/2022, 2:35 AMsettings.py
may work, but I think before_pipeline_run
is a lot more idiomatic.Lorenzo Castellino
11/03/2022, 11:16 AMnp.random.RandomState()
that can be passed down to the various classes that accept the random_state
kwarg.
The reasoning behind it is explained in this SO discussion which is also linked in the aforementioned documentation articles.
My idea at this point is to actually pickle the rng variable and load it where needed. This feels a bit tedious due to the fact that all the corresponding functions and pipelines needs to be updated accordingly.
Is there a more elegant way to do so? I still think that Hooks might be the key but I can't see how...Ben Levy
11/03/2022, 1:10 PMrng.py
) and then just import it into each node function if that's the strategy you want to employ(from rng import RNG
)Lorenzo Castellino
11/03/2022, 1:48 PM