Data Type
Hi Guys, I just use embedding dataset recipe in Dataiku. My dataset containing two columns: a code column and a description column. The code values contain leading zeros (for example, 012345).
After creating embeddings and performing search in knowledge bank Dataiku, the retrieved results display the code without the leading zeros (for example, 123456 instead of 0123456).
This appears to be similar to the issue discussed here:
The code column should be treated as a string identifier, not as a numeric value. Losing the leading zeros causes problems because the code no longer matches the original source data.
Any recommendations would be appreciated.
Comments
-
Hello,
Thank you for reporting the issue. We have had the issue with data formatted as a double with trailing zeros. The origin of the issue seems to be the same: pandas inference when loading the dataset before indexing documents and metadata.
We are actively working on a fix to change this behavior.
Kind regards
