Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Configuring Spark and Hadoop with Dataiku DSS
Added on March 21, 2024 8:20PM
Likes: 0
Replies: 3
moustapha00
Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 3 ✭
Hi! I'm encountering difficulties configuring Spark and Hadoop with Dataiku DSS on my local Windows machine. I've successfully installed Dataiku DSS (Free Edition with advanced features trial), Spark (version 3.5.1), and Hadoop (version 3.3.0) on my system. However, I'm unable to configure Spark for use within Dataiku DSS, and I also aim to configure Hadoop to allow dataset uploads from HDFS.
The issue occurs within the setup and configuration process of Dataiku DSS on my local Windows machine.
What version are you using?
Dataiku DSS: Free Edition (with advanced features trial)
Spark: 3.5.1
Hadoop: 3.3.0
I've searched online for configuration guides and tried various configurations, but I haven't been successful in getting Spark and Hadoop properly configured within Dataiku DSS.
Operating system used: Windows
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,384 NeuronHi, as per the Dataiku Windows page the Dataiku Windows download is provided for experimental purposes and it's unsupported. If you want to fully test all the features you will need to be in a supported configuration. The VM solution is probably best for you if you are in Windows.
-
Thanks, I'm now on the VM. How can I configure Spark and Hadoop?
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,384 Neuron