Spark integration problem on k8s
Hi all ,
I have problem with deploying. the spark on k8s and it shows me the below error .
dataiku@dataiku---design:~$ /data/design/bin/dssadmin install-spark-integration -standaloneArchive spark-3.3.4-bin-hadoop3.tgz
[+] Saving installation log to /data/design/run/install.log
[+] Standalone mode selected
+ Using SPARK_HOME=/home/dataiku/dataiku-dss-12.5.1/spark-standalone-home
*** Error: Spark version not supported: 3.5.0
Answers
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptionsHi,
The standalineArchive version you are using doesn't seem to match the DSS version.
PLease try using this one instead :
https://downloads.dataiku.com/public/studio/12.5.1/dataiku-dss-spark-standalone-12.5.1-3.4.1-generic-hadoop3.tar.gz
Which will use the expected version -
Thanks for your response AextT
Now I'm facing this error below :dataiku@dataiku---design:~$ /data/design/bin/dssadmin install-spark-integration -standaloneArchive dataiku-dss-spark-standalone-12.5.1-3.4.1-generic-hadoop3.tar.gz\?_gl\=1\*1lvtflo\*_ga\*MjE0NzM3Njg5Ni4xNjc4MzUwNjMw\*_ga_B3YXRYMY48\*MTcwNzQ0MDkwNy41OC4xLjE3MDc0NDEwNTYuNjAuMC4w [+] Saving installation log to /data/design/run/install.log [+] Standalone mode selected + Using SPARK_HOME=/home/dataiku/dataiku-dss-12.5.1/spark-standalone-home *** Error: could not find Spark version
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptions
Hi,
there seems to be junk characters at the end of the filename.
Please try to wget https://downloads.dataiku.com/public/studio/12.5.1/dataiku-dss-spark-standalone-12.5.1-3.4.1-generic-hadoop3.tar.gz
then re-run with just the filename
dataiku-dss-spark-standalone-12.5.1-3.4.1-generic-hadoop3.tar.gz
-
I'm not sure where is the problem
can you check it for me please ?dataiku@dataiku---design:~$ /data/design/bin/dssadmin install-spark-integration -standaloneArchive dataiku-dss-spark-standalone-12.5.1-3.4.1-generic-hadoop3.tar.gz [+] Saving installation log to /data/design/run/install.log [+] Standalone mode selected + Using SPARK_HOME=/home/dataiku/dataiku-dss-12.5.1/spark-standalone-home *** Error: could not find Spark version
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptions
Hi,
Can you please open a support ticket with instance diagnostics?
https://doc.dataiku.com/dss/latest/troubleshooting/diagnosing.html
https://doc.dataiku.com/dss/latest/troubleshooting/obtaining-support.htmlThanks