Use a 3rd party DLL as a recipe

vtnghia

Hi all,

We have a situation where the customer has his own in-house developed tool in which the calculation back-end is capsuled as a DLL. The question is if we could import that DLL and use it as a recipe somehow.

With thanks and best regards,

Nghia.

tgb417

@vtnghia
,

Welcome to the Dataiku Community. We are glad to have you join us.

I suspect the answer is that you could hack something together. I know that folks have wraped a .DLL and use it from Python. You can also call some containerized process from DSS.

However the core challenge in your scenario is that DSS runs under Linux in production. And a .DLL file is by it's nature a Windows Code Object. There are some folks talking about using WINE under linux to be able to use .DLL code on Linux. For example I found this post. https://stackoverflow.com/questions/13065982/can-i-wrap-windows-dll-to-use-it-in-python-under-linux However, I've got to imagine that the performance of the code is decreased due to several layers of overhead.

If the .DLL was developed in house for performance reasons I'd consider re-coding. I know that that approach is sometimes not always possible for 100 crazy reasons.

If it is monolythic and you don't have the code, the other question is could you create a REST API on windows that would wrap around this code. And then you could use that API from withing DSS.

Good luck with your project. Let us know how you get along. Other please jump in here if you have any good ideas.

vtnghia

Dear @tgb417
Tom,

Many thanks for the detailed suggestion.

The DLL is really developed in-house and is currently up and running as a standard tool (having an independent interface by itself) where we're trying to bring the calculations into Dataiku to utilize all data sources. So, convincing the customer to re-code is not that easy.

I have a discussion with my team regarding building a REST API to wrap the DLL, they said it's doable.

My next question is do we have any minimum requirement that the REST API needs to follow so that it could be used in the Dataiku environment?

With thanks and best regards,

Nghia.

tgb417

@vtnghia
,

I've queried REST APIs from DSS Process using both the PYTHON Requests library and the API Connect Plugin. Although I find the API Connect Plugin a bit finicky about the JSON that it will easily take in. And the REST API that I typically work with a bit lax about the JSON it puts out. (Here is a discussion of some of the challenges I've had. ) The plugin is not that bad. I've had some useful results. Given that you will be coding the web interface to your underlying .DLL file you can probably control the quality of the JSON you are putting out so it will work well with the API Connect Plugin.

Have a play around with DSS and the Requests Library from a Jupyter Notebook or Python Recipe, and / or the API Connect Plugin. See how that goes.

My next question is how performant this whole thing will end up being and needing to be. If you like what you are seeing then hack a quick prototype to see if you can make this all connect. I would definitely not do this for a high volume production environment. However, as part of a proof of concept or low volume light use use case this might get you through.

Let us know how you are getting along with your project.