[BUG] API Service endpoint with enrichment through SQL connection

daniel_adornes

Hi everyone!

So, this is a solved issue that might be helpful for someone else.

I'm using DSS 9.0 and this was my experience with endpoint enrichment through SQL connection.

When configuring a prediction endpoint to enrich the incoming request with new features, there are two options:

1) Bundle the enrichment dataset together with the model (basically copying it fully - not optimal) or

2) Configuring it to execute an on-line query to a SQL database while serving the model (probably best option in most scenarios).

When trying to use the second option, I first forgot to set the corresponding connection in the infrastructure configuration of my Kubernetes cluster in the DSS Deployer. The container failed to start up on Kubernetes accusing connection not found (easy one). Then, I set the correct connection by using the dropdown list, which automatically copied the corresponding json configuration to my clipboard. Then I just needed to add it as a connection below (see attachment).

The buggy part is that DSS stores the password in an encrypted format. So, this json copied to the clipboard (to make your life easier) will actually fail when serving the requests. The SQL query will try the encrypted password as if it was the real password for the user of your database. The solution is to replace the encrypted password by the plain password in the json configuration. Don't forget that you need to redeploy your API Service.

One other minor issue (maybe a bug, maybe not), is that when configuring a SQL connection in the global administration settings, there is the option of setting the full URL connection. If you set that, even partially (not right) and change your mind and set then the distinct fields (for host, user, password, database, etc.), DSS will still store that "dirty" value (under ...DATA_DIR/configs/connections.json). Then, when you are doing what I described before (setting a connection for an endpoint to enrich request features), the retrieved json (that one aimed at making your life easier) will bring the dirty value, which may cause confusion, though it has absolutely no effect as soon as useURL is set to false.

That's it! Enjoying the ride and wishing Dataiku gets better and better! Also, hopefully it is helpful to someone else.

tgb417

@daniel_adornes
,

Welcome to the Dataiku Community.

And thank you for this wonderful post. I love all of the detail you have shared here.

cc: @CoreyS

daniel_adornes

Thks @tgb417
!! Appreciate that! Have a great week!