Multivariate LSTM in Dataiku

Darius679
Darius679 Registered Posts: 6 ✭✭✭✭

I've completed this tutorial on how to implement an LSTM in dataiku.

https://knowledge.dataiku.com/latest/kb/analytics-ml/time-series/ts-forecast/time-series-code/deep-learning-ts.html

Basically I want to understand how to adapt this for multiple variables.

I have a data set that has many 'runs' of a chemical process over a 10 day period, and different observations of the process over those 10 days like pH, temperature, etc. I want to use an LSTM that will predict the observations of the next day given a a few of the previous days.

I can't find anything like this or any examples of any multivariate LSTMs implemented in dataiku. Do you know of any tutorials that could help?

Best Answer

  • Darius679
    Darius679 Registered Posts: 6 ✭✭✭✭
    Answer ✓

    I have figured it out after much effort. The gist of the changes are

    • Making the windows -- they should have the format [observations for time 1, observations for time 2 ...] for however long you want your windows to be.
    • the windowprocesser.py must change to accommodate a longer array of length window_size * num_features. You should also remove the scaling because it doesn't respect the ranges of different variables, only taking the max/min
    • When building the model, you should change the 1 to num_features in the reshape layer. This makes the array into the right shape for training the LSTM

Answers

Setup Info
    Tags
      Help me…