Native support for Markov models

Options
Antal
Antal Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 85 Neuron

(VisualML) support for building different types of Markov chain models.

Very relevant models in marketing attribution, process analysis and customer journey analysis.

2
2 votes

New · Last Updated

Comments

  • tgb417
    tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,595 Neuron
    Options

    Would you share a bit more about how you envision it being included in dataiku dss. With my limited knowledge of markov chains models it is my understanding that one needs to look at current and possibly prior records to get the prediction. (Sort of like time series forecasting.) In most cases dataiku models look at one record at a time (forecasting being an exception.)

    One other thought is does sci-kit learn support markov models? If so you might be able to add a custom model to dataiku’s visual ml. That said it appears that markov models have been removed from sci-kit learn.

    https://scikit-learn.org/stable/related_projects.html

    So some more detail might help the team at dataiku understand a bit better what you are looking for.

  • Antal
    Antal Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 85 Neuron
    Options

    I would expect it to work indeed a bit like with Time Series Forecasting at the data level.

    Markov needs an ordered series of events (states) per unique identifier (ie customer).

    These states are structured into paths and the model calculates probabilities of going from one state to a given next state, taking into account one or more previous states.

    These models can be used for example for channel attribution (how much did each state contribute to the desired end result) or calculating what the most probable path is to go from state A to state B.

    Highly useful (still!) in all sorts of marketing contexts.

    Markov was removed from sklearn, but it wouldn't have worked as a custom model in a regular prediction VisualML anyway due to the ordered series requirement of the underlying data.

Setup Info
    Tags
      Help me…