Spark job execution on EKS using IRSA

pi_485
Level 2
Spark job execution on EKS using IRSA

Hi,

We have Dataiku on EC2 instance and EKS is present in another account. We have configured Dataiku to access EKS using assume roles.

We had a discussion with AWS and they suggested to use service account and annotations for accessing AWS services (like, S3).

Is there a way that we can implement service accounts and annotations in Dataiku to execute spark jobs in pods?

0 Kudos
1 Reply
pi_485
Level 2
Author

IRSA implementation requires Spark docker image rebuilt.

0 Kudos