Connect databrick's data to dataiku

mm

I am trying to connect databrick's data using SCP/SFTP connection however I am getting following error:

- HTTP Code:500

- Error type: java.io.IOException.

Please advice how I can add databrick's tables to dataiku.

Shashank

If it's a Table Object in Databricks, use the JDBC URL of the Databricks Cluster and use that to create a connection in Dataiku. Once you created the connection you would be able to pull that as a dataset.

If you want to load the underlying file of the table, then it depends on what storage you are using. Let's assume you are using AWS S3, for that you need to build an S3 connection and load the files.

If this doesn't help, please share more details about your use case and how you are connecting.

mm

Its a table object in databricks, and I am using JDBC URL of the databricks cluster.

1. I created "SCP/SFTP connection" in "DSS Settings" in "Connections". Is this the right way or shall I create some other type of connection?
2. When I try to import data using "New SFTP Dataset ", it gives following error:

   Additional technical details

   • HTTP code: 500
   • Error type:java.io.IOException

Please let me know if I am doing some step in a wrong way?

Shashank

You need to use "Other SQL Databases" under the SQL Databases connection category and provide the JDBC URL connection details.

SCP/SFTP is for File systems not for Databases

jbader

Thanks for the direction on this one. Would you have a link to an example that does this? I am trying to get my team moving in this direction but they lack experience in this method. Thanks!