Join us on July 16th as we explore real-world Reinforcement Learning Learn more

How to create an Elasticsearch DSS using an index pattern

Level 1
How to create an Elasticsearch DSS using an index pattern

Due to the volume of our data, we regularly switch to writing to a new index, with a simple incremental: my_index_1, my_index_2, etc... And we have many of them. Elasticsearch API allows use of a pattern for querying, i.e. my_index_*.



Dataiku is happy connecting to a specific index, But when I try this on a working Dataiku connection the index pattren like this Dataiku refuses to save it...



What to do?

0 Kudos
1 Reply
Dataiker
Dataiker

Hello,



You can achieve this by creating an index alias which uses the wildcard magic to catch the indices you want. This follows the official ElasticSearch doc: https://www.elastic.co/guide/en/elasticsearch/reference/5.2/indices-aliases.html




POST /_aliases
{
"actions" : [
{ "add" : { "index" : "test*", "alias" : "all_test_indices" } }
]
}


Then you will reference this index alias from the settings of the dataset within DSS.



Cheers,



Alex

0 Kudos
Labels (2)