Submit your use case or success story to the 2023 edition of the Dataiku Frontrunner Awards ENTER YOUR SUBMISSION

Writing dataset to Elasticsearch

Kok
Level 1
Writing dataset to Elasticsearch

Hi,

We setup a connection in dataiku to Elasticsearch and need to write a dataset with dense_vector fields to Elastic. How can we define a dense_vector field in the dataset?

Thanks!

0 Kudos
2 Replies
AlexT
Dataiker

Hi,

dense_vector type is not support in DSS as type. I've tested to manually define the mapping on ES dataset, but it also fails Mapping update failed  (HTTP code 400) {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"No handler for type [dense_vector] declared on field [new_column]"}].

As a workaround you can write it as a string initially and then via the ES API, create a new index and then use ES reindex API to copy the data. 

0 Kudos
Kok
Level 1
Author

Hi Alex,

Thank you for your reply. In that case, I believe I have to use ES API to do the import.

 

0 Kudos

Setup info

?
Tags (1)
A banner prompting to get Dataiku