https://kedro.org/ logo
#questions
Title
# questions
f

Francis Duval

03/21/2024, 1:08 PM
Hi! My question is probably basic, but I can't find any documentation about it. I simply want to pass a function as an argument in a Kedro node. For instance:
Copy code
import numpy as np
def softmax(values):
    exp_values = np.exp(values)
    softmax_values = exp_values / np.sum(exp_values )

    return softmax_values


def compute_function(func, values):
    return func(values)


compute_function(softmax, [1, 2, 3, 4, 5])
But I want it in a node:
Copy code
node(
  func='compute_function',
  inputs=[softmax, 'result_from_other_node'], # This will not work!
  outputs='result',
  name='name'
)
Thanks!
n

Nok Lam Chan

03/21/2024, 1:13 PM
Where do you get this
func
from? Why can't you simply use
softmax(value)
?
f

Francis Duval

03/21/2024, 1:15 PM
softmax
and
compute_function
are defined in
nodes.py
, and imported in
pipeline.py
.
n

Nok Lam Chan

03/21/2024, 1:16 PM
Copy code
node(
  func='compute_function',
  inputs=[softmax, 'result_from_other_node'], 
  outputs='result',
  name='name'
)
My question is, how is this different from this? Why do you need another indirection function?
Copy code
node(
  func=softmax,
  inputs=[ 'result_from_other_node'],
  outputs='result',
  name='name'
)
f

Francis Duval

03/21/2024, 1:18 PM
Maybe the example I gave simplified my problem too much
n

Nok Lam Chan

03/21/2024, 1:19 PM
inputs
and
ouptuts
expected datasets or parameters. If you have a higher level function, you should look for
functools
or something like a decorator. Whatever it is, that should be done prior setting up the node and you pass that function directiont to
func
👍🏽 1
f

Francis Duval

03/21/2024, 1:20 PM
Thanks! I'll look at functools, I feel like this is what I need!
👍🏼 1
I managed to get it working with
partial
and
update_wrapper
from functools, thanks!
👍🏼 1
So for instance here, my function
compute_accuracies_nn_weight
takes as arguments a weight function
weight_func
as well as a grid of parameters of this weight function
**args_weight_func_grid
. So I did this:
Copy code
def modified_softmax(values, alpha):
    amplified_values = values * alpha
    exp_amplified_values = np.exp(amplified_values)
    softmax_values = exp_amplified_values / np.sum(exp_amplified_values)

    return softmax_values


node(
    func=update_wrapper(
        wrapper=partial(compute_accuracies_nn_weight, weight_func=modified_softmax, alpha=[1, 2, 5, 10, 15]),
        wrapped=compute_accuracies_nn_weight
    ),
    inputs={
        'df_inference': 'df_inference',
        'df_search': 'df_search',
        'nn_idx_matrix': 'nn_idx_matrix',
        'nn_sim_matrix': 'nn_sim_matrix',
        'k_grid': 'params:k_grid',
    },
    outputs='accuracies_knn_softmax_weights',
    name='compute_accuracies_knn_softmax_weights'
),
n

Nok Lam Chan

03/21/2024, 2:01 PM
Copy code
func=update_wrapper(
        wrapper=partial(compute_accuracies_nn_weight, weight_func=modified_softmax, alpha=[1, 2, 5, 10, 15]),
        wrapped=compute_accuracies_nn_weight
    )
This should move to
node.py
most likely. There's nothing wrong to do that in
pipeline.py
, purely for organising code only
f

Francis Duval

03/21/2024, 2:01 PM
Thanks for the tip!
2 Views