Multivariate LSTM in Dataiku

Darius679

I've completed this tutorial on how to implement an LSTM in dataiku.

https://knowledge.dataiku.com/latest/kb/analytics-ml/time-series/ts-forecast/time-series-code/deep-learning-ts.html

Basically I want to understand how to adapt this for multiple variables.

I have a data set that has many 'runs' of a chemical process over a 10 day period, and different observations of the process over those 10 days like pH, temperature, etc. I want to use an LSTM that will predict the observations of the next day given a a few of the previous days.

I can't find anything like this or any examples of any multivariate LSTMs implemented in dataiku. Do you know of any tutorials that could help?

Darius679

I have figured it out after much effort. The gist of the changes are

• Making the windows -- they should have the format [observations for time 1, observations for time 2 ...] for however long you want your windows to be.
• the windowprocesser.py must change to accommodate a longer array of length window_size * num_features. You should also remove the scaling because it doesn't respect the ranges of different variables, only taking the max/min
• When building the model, you should change the 1 to num_features in the reshape layer. This makes the array into the right shape for training the LSTM

CoreyS

Thank you for sharing your solution with the rest of the community @Darius679
!

NIDHS

Thanks for sharing this. Could you maybe share the snippets of the changes that needs to be made to windowprocessor.py and Architecture to run multivariate LSTM?