Support for native PDF input to LLMs via Dataiku LLM Mesh
Hi,
I'm interested in the timeline of developments in the Dataiku LLM Mesh Python API regarding native document input, particularly PDFs.
Many of the LLM providers that Dataiku interfaces with (e.g. Snowflake Cortex, Google Cloud) already provide the ability to natively handle PDF inputs in prompts (see e.g. here for Snowflake or here for Google Cloud). At the same time, Dataiku LLM Mesh currently does not support this (as far as I can see from the documentation). The closest workaround I found is the possibility to first render PDF pages as images (e.g. via the OCR plugin), which can then be processed via the multimodal input capabilities.
Is there another way to process PDFs natively via the LLM Mesh that I am missing? If not, I would be very interested a rough estimate on when users can expect this to be available. I think it would be great for Dataiku to support this feature of the LLM providers it interfaces.
Thank you and kind regards
Felix
Dataiku version used: 14.4.4
Dataiku version used: 14.4.4
Answers
-
Hi Felix, thanks for sharing this and for the detailed examples.
You are right PDF files cannot currently be sent directly to models through the LLM Mesh, the workaround you mentioned is the available option for now.
We will log a feature request for native PDF support. We cannot commit to a timeline but the context you provided is very helpful.
Best, Stéphane
