Community Conundrum 28: News Engagement is live! Read More

Number of spark executors

Level 1
Number of spark executors

Hello,

I am using Spark-on-k8s to run my spark program. However, even though I am setting spark.executor.instances to 20(as indicated by the attached image), only 2 pods/executors are launched in the cluster. What could have gone wrong?

Capture.PNG 

0 Kudos
0 Replies
A banner prompting to get Dataiku DSS