How do I create a backend APIs for the various transformations and visualizations in a flow?
I am trying to create an ML application which can display the various types of transformations that happen in a dataset, like the count of certain rows, their min, max, etc. This app will take data from the flow and display whatever information is needed for the particular run.
I need a way to pass the information to this app. I checked out the API Designer and could see that it is available for model results.
Is the same available for other queries too. For example, if I want to know the max of a particular column, or if I want to group by some columns and then find the sum of a particular column. These information should be fetched from the flow and passed in the API.
Best Answer
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,160 Neuron
The API Designer is really aimed at designing custom APIs that can be used for prediction or real time scoring, etc. In your case you should really be looking at the built-in and very powerful Dataiku API and what you can do with datasets directly. This will save you a lot of development time since these APIs can interact with Dataiku objects directly. In particular I think all of what mentioned in your post could be achieved using Dataset Metrics which can also be retrieved using the Dataiku API.
Finally you don't say where are you bulding this "ML Application" but Dataiku can also be a great place to develop custom Web Applications which can easily interact with the Dataiku datasets.