Writing dataset to Elasticsearch

Kok

Hi,

We setup a connection in dataiku to Elasticsearch and need to write a dataset with dense_vector fields to Elastic. How can we define a dense_vector field in the dataset?

Thanks!

Alexandru

Hi,

dense_vector type is not support in DSS as type. I've tested to manually define the mapping on ES dataset, but it also fails Mapping update failed (HTTP code 400) {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"No handler for type [dense_vector] declared on field [new_column]"}].

As a workaround you can write it as a string initially and then via the ES API, create a new index and then use ES reindex API to copy the data.

Kok

Hi Alex,

Thank you for your reply. In that case, I believe I have to use ES API to do the import.

Kok

Hi,

I am using Elasticsearch python api and try to import data from dataiku python recipe to Elasticsearch. But I kept getting "AuthenticationException(401, 'security_exception', 'missing authentication credentials for REST request " error.

I use the same python code running on my local machine and it working fine. But the same code running in dataiku kept getting the error.

Is there any sample code for importing data to Elasticsearch with elastic python api?

Thanks!