Has anyone connected to an external database using the API Connect Plugin using API Keys?

[JT1]

I'm trying to connect to an external database using an API Connect Plug in but keep getting an error: Test failed: Failed to read from python dataset : <class 'rest_api_client.RestAPIClientError'> : Error 401: b'{\n "message" : "Missing authentication header.",\n "status" : 401\n}'

---

Operating system used: Mac

[tgb417]

@JT1 

Welcome to the Dataiku community. We are so glad to have you join us here.

I connect to data sources via the API connect plugin multiple times an hour.

It does take a bit of work to figure out how to use.  At some point a tutorial should exist on using this tool.

That said a few tips that may help.

1. I use a Rest API tool like postman to make sure that I know exactly how the REST API I am working with actually works.  If  I can’t make the API work with Postman, I don’t know enough yet to get it to work from API Connect
2. Under setting for the project there is a place to store your API key at the project level.  I also tend to store some constant variable on that setting screen.  Things like the Max Page size and starting skip value for paginated APIs.
3. Make sure that you are using the latest version of the plugin. Over the past few years there have been a number of good reliability and performance fixes to the API Connect plugin.  (Thanks Alex)

Hope one of these may help a bit.  

[JT1]

Hi Tom @tgb417 ! Thank you for your reply! On Postman, it looks like everything is working, however, I keep getting errors. Do you know if the API Connect can only take JSON files or is CSV ok? Also I was provided with an API Key and Secret number, would I choose the API Key as the login type or Basic Auth?  

[tgb417]

This question is not clear to me.

Do you know if the API Connect can only take 
JSON files or is CSV ok?

You are saying "take JSON files".  The API Connect plugin is designed to take as input only dataiku datasets.  They may be in any to the typical forms supported by Dataiku.  If by you "take JSON files" you mean that the API sending data to Dataiku would some how send a CSV file formatted result.  I have not used the API connect to ingest a CSV file from a web server URL. I've only received JSON formated data which is very rich but definitly a bit harder to process.  On the other hand I have used the standard HTTP Network connection to get .CSV datafiles.  However, in those cases I have not had to use authentication. 

There is also an ODATA plugin that may be helpful for you depending on the details of your use case.  There is also the ability to get data over SFTP that can be helpful to get HTML data.  Finally if you have to you can of course write a bit of Python to do what you are trying to do.

Although I use API Connect every day, I do not use all of it's features.  It is my understanding that Basic Auth is for a typical User Name and Password.  I have never had to use the API Key and Secret number type of authentication with the API connect plugin.

You may want to reach out to your organizations Dataiku Customer Success folks, or the author of the plugin regarding your particular use case.

[Turribeach]

What database are you trying to connect to? What type of API does the database provide? What kind of authentication does it support? Can you provide the API documentation showing how to connect to it? 

[JT1 · Accepted solution]

Hi, someone from Dataiku was able to help me, I had to change the header parameters to match the key and value pairing. Thank you!!