Multivariate LSTM in Dataiku

Darius679 Registered Posts: 6 ✭✭✭✭

I've completed this tutorial on how to implement an LSTM in dataiku.

Basically I want to understand how to adapt this for multiple variables.

I have a data set that has many 'runs' of a chemical process over a 10 day period, and different observations of the process over those 10 days like pH, temperature, etc. I want to use an LSTM that will predict the observations of the next day given a a few of the previous days.

I can't find anything like this or any examples of any multivariate LSTMs implemented in dataiku. Do you know of any tutorials that could help?

Best Answer

  • Darius679
    Darius679 Registered Posts: 6 ✭✭✭✭
    Answer ✓

    I have figured it out after much effort. The gist of the changes are

    • Making the windows -- they should have the format [observations for time 1, observations for time 2 ...] for however long you want your windows to be.
    • the must change to accommodate a longer array of length window_size * num_features. You should also remove the scaling because it doesn't respect the ranges of different variables, only taking the max/min
    • When building the model, you should change the 1 to num_features in the reshape layer. This makes the array into the right shape for training the LSTM


  • CoreyS
    CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭

    Thank you for sharing your solution with the rest of the community @Darius679

    NIDHS Registered Posts: 1

    Thanks for sharing this. Could you maybe share the snippets of the changes that needs to be made to and Architecture to run multivariate LSTM?

Setup Info
      Help me…