Dear team, what is the best practice to pass a AWS role in kedro? I need to read/write data in AWS. I'll have access to the role name/arn. I'm thinking that a few options:
1. Assume role in a hook e.g., `after_context_created`/
before_dataset_loaded
/
before_dataset_saved
but not sure if this gets propagated. From here, save the temporary credentials as env variables.
2. Create a custom data set and assume the role there.
PS - I'm using
pandas.ParquetDataset
and
partitions.PartitionedDataset