Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I am trying to connect databrick's data using SCP/SFTP connection however I am getting following error:
- HTTP Code:500
- Error type: java.io.IOException.
Please advice how I can add databrick's tables to dataiku.
If it's a Table Object in Databricks, use the JDBC URL of the Databricks Cluster and use that to create a connection in Dataiku. Once you created the connection you would be able to pull that as a dataset.
If you want to load the underlying file of the table, then it depends on what storage you are using. Let's assume you are using AWS S3, for that you need to build an S3 connection and load the files.
If this doesn't help, please share more details about your use case and how you are connecting.
Thanks for the direction on this one. Would you have a link to an example that does this? I am trying to get my team moving in this direction but they lack experience in this method. Thanks!
Its a table object in databricks, and I am using JDBC URL of the databricks cluster.
Additional technical details
Please let me know if I am doing some step in a wrong way?
You need to use "Other SQL Databases" under the SQL Databases connection category and provide the JDBC URL connection details.
SCP/SFTP is for File systems not for Databases