Structured JSON output by LLM via Snowflake Cortex

felix_s

Hi, my company recently added access to LLMs via Snowflake Cortex through Dataiku's LLM Mesh feature. When trying to generate structured JSON output (either in the UI via Prompt Studio or via the Python API according to the official documentation), I run into the following error:

The Snowflake Cortex (SQL) completion API does not support the following setting: responseFormat (type=json)

According to the Snowflake documentation, structured output is in principle supported via the AI_COMPLETE function. So to me it seems that what is missing is a mapping to this function within Dataiku/Dataiku's Python API.

Is this a feature that you are currently working on? It would be great if structured JSON output via Snowflake Cortex LLMs would be available soon.

Thank you and kind regards

Dataiku version used: 14.4.4

MaximeA

Hello,

Thank you for the report and for sharing the Snowflake documentation reference.

What you are seeing is expected with the current Snowflake Cortex integration in Dataiku. Today, Dataiku does not use Snowflake’s AI_COMPLETE SQL function for this integration. For the SQL-based Snowflake Cortex client, Dataiku currently relies on Snowflake’s legacy COMPLETE function, and structured JSON output is not supported through that SQL client.

More generally, we are not planning significant additional investment in the current SQL-based Cortex client. For Snowflake Cortex models, Dataiku also (and mainly) supports Snowflake’s legacy Cortex REST completion API, and structured output support for that REST-based path was added in Dataiku 14.5.2.

So, if structured JSON output is required, the recommended path is to use a Dataiku version where structured output is supported for the Snowflake Cortex REST client.

We are also working on adding support for Snowflake Cortex’s newer OpenAI-compatible chat completions API. This is the direction we expect to rely on going forward, and structured output support is planned as part of that integration as well.

Hope it helps.

Kind regards

felix_s

https://community.dataiku.com/discussion/comment/47522#Comment_47522

Thank you very much @MaximeA. This helps!