We have Dataiku on EC2 instance and EKS is present in another account. We have configured Dataiku to access EKS using assume roles.
We had a discussion with AWS and they suggested to use service account and annotations for accessing AWS services (like, S3).
Is there a way that we can implement service accounts and annotations in Dataiku to execute spark jobs in pods?
IRSA implementation requires Spark docker image rebuilt.