https://kedro.org/ logo
#questions
Title
# questions
a

Antonio Perelló Moragues

09/06/2023, 10:33 AM
Hi Kedro community! Could anybody provide me with an example on how to connect to AWS Athena using the
pandas.SQLQueryDataSet
?
d

datajoely

09/06/2023, 10:37 AM
So all Kedro does is create the connection object using SQLAlchemy and pass it to
pd.read_sql_query
so part 1 of this tutorial should be everything you need https://medium.com/codex/connecting-to-aws-athena-databases-using-python-4a9194427638
a

Antonio Perelló Moragues

09/06/2023, 10:45 AM
Thanks. I've tried, but I get this error:
Copy code
Failed while loading data from data set SQLQueryDataSet(...), 'OptionEngine' has no attribute 'execute'
I'm using:
Copy code
aws_credentials:
  con: awsathena+pandas://{aws_access_key_id}:{aws_secret_access_key}@athena.{region_name}.<http://amazonaws.com/{schema_name}?s3_staging_dir={s3_staging_dir}|amazonaws.com/{schema_name}?s3_staging_dir={s3_staging_dir}>
  connect_args:
    aws_secret_access_key: ...
    aws_access_key_id: ...
    region_name: ...
    schema_name: ...
    s3_staging_dir: ...
and this in the catalog:
Copy code
test_df:
  type: pandas.SQLQueryDataSet
  sql: "select * from test.test_table"
  credentials: aws_credentials
d

datajoely

09/06/2023, 10:46 AM
hmm
even with the pyathena extension installed?
could you try downgrading sqlalchemy?
and seeing as you’re doing
select *
can you try
pandas.SQLTableDataSet
👍 1