Survey banner
Switching to Dataiku - a new area to help users who are transitioning from other tools and diving into Dataiku! CHECK IT OUT

Dataiku Spark Remote Connection

sangkim
Level 2
Dataiku Spark Remote Connection

Hello, I am using Dataiku 12.5.2 and currently running Spark 2.4. Dataiku is installed on a server named A, while Spark is installed on a server named B, configured as a standalone Spark installation without Hadoop. Both server A and server B are capable of TCP communication and allow SSH access. How can I use the Spark on server B from Dataiku on server A?

※ I have confirmed that the Spark installed on server A, where Dataiku is also installed, is well integrated.
@spark


Operating system used: linux

0 Kudos
2 Replies
Turribeach

Please review the different Spark integration options in the documentation:

https://doc.dataiku.com/dss/latest/spark/installation.html#setting-up-spark-integration

 

0 Kudos
sangkim
Level 2
Author

I am using Dataiku and Spark in an on-premise environment. Spark is configured as a standalone installation without Hadoop. Due to the current project configuration, I cannot use Kubernetes (k8s) or Docker. I would like to integrate Dataiku on server A with Spark on server B.

0 Kudos