How to create an Elasticsearch DSS using an index pattern

Highlighted
utilum
Level 1
How to create an Elasticsearch DSS using an index pattern

Due to the volume of our data, we regularly switch to writing to a new index, with a simple incremental: my_index_1, my_index_2, etc... And we have many of them. Elasticsearch API allows use of a pattern for querying, i.e. my_index_*.



Dataiku is happy connecting to a specific index, But when I try this on a working Dataiku connection the index pattren like this Dataiku refuses to save it...



What to do?

0 Kudos
1 Reply
Alex_Combessie Dataiker
Dataiker
Re: How to create an Elasticsearch DSS using an index pattern

Hello,



You can achieve this by creating an index alias which uses the wildcard magic to catch the indices you want. This follows the official ElasticSearch doc: https://www.elastic.co/guide/en/elasticsearch/reference/5.2/indices-aliases.html




POST /_aliases
{
"actions" : [
{ "add" : { "index" : "test*", "alias" : "all_test_indices" } }
]
}


Then you will reference this index alias from the settings of the dataset within DSS.



Cheers,



Alex

0 Kudos
Labels (2)