Multivariate LSTM in Dataiku

Solved!
Darius679
Level 2
Multivariate LSTM in Dataiku

I've completed this tutorial on how to implement an LSTM in dataiku.

https://knowledge.dataiku.com/latest/kb/analytics-ml/time-series/ts-forecast/time-series-code/deep-l...

Basically I want to understand how to adapt this for multiple variables.

I have a data set that has many 'runs' of a chemical process over a 10 day period, and different observations of the process over those 10 days like pH, temperature, etc. I want to use an LSTM that will predict the observations of the next day given a a few of the previous days.

I can't find anything like this or any examples of any multivariate LSTMs implemented in dataiku. Do you know of any tutorials that could help?

0 Kudos
1 Solution
Darius679
Level 2
Author

I have figured it out after much effort. The gist of the changes are

  • Making the windows -- they should have the format [observations for time 1, observations for time 2 ...] for however long you want your windows to be.
  • the windowprocesser.py must change to accommodate a longer array of length window_size * num_features. You should also remove the scaling because it doesn't respect the ranges of different variables, only taking the max/min
  • When building the model, you should change the 1 to num_features in the reshape layer. This makes the array into the right shape for training the LSTM

View solution in original post

0 Kudos
2 Replies
Darius679
Level 2
Author

I have figured it out after much effort. The gist of the changes are

  • Making the windows -- they should have the format [observations for time 1, observations for time 2 ...] for however long you want your windows to be.
  • the windowprocesser.py must change to accommodate a longer array of length window_size * num_features. You should also remove the scaling because it doesn't respect the ranges of different variables, only taking the max/min
  • When building the model, you should change the 1 to num_features in the reshape layer. This makes the array into the right shape for training the LSTM
0 Kudos
CoreyS
Dataiker Alumni

Thank you for sharing your solution with the rest of the community @Darius679

Looking for more resources to help you use Dataiku effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as โ€˜Accepted Solutionโ€™ to help others like you!
0 Kudos