meharji arumilli10/11/2023, 5:22 AM
is copied to
. And now there is a second project and its .whl is also installed at AWS Linux instance, and when its corresponding DAG is run, it again looks at
. Here the parameters inside
are different for project1 and project2. How can it be done to have
specific to each DAG?
Ankita Katiyar10/11/2023, 3:53 PM
class KedroOperator(BaseOperator): @apply_defaults def __init__( self, package_name: str, pipeline_name: str, node_name: str, project_path: str, env: str, *args, **kwargs ) -> None: super().__init__(*args, **kwargs) self.package_name = package_name self.pipeline_name = pipeline_name self.node_name = node_name self.project_path = project_path self.env = env def execute(self, context): configure_project(self.package_name) with KedroSession.create(self.package_name, self.project_path, env=self.env, conf_source=<path to conf_source>) as session: #ADD CONF SOURCE here session.run(self.pipeline_name, node_names=[self.node_name])