Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
how we can achieve Data Lineage in DataIku
Added on March 7, 2024 9:52AM
Likes: 0
Replies: 6
preetigupta15
Registered Posts: 1 ✭
How we can achieve table and column decencies in recipes.
Answers
-
Elie Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered, Product Ideas Manager Posts: 32 DataikerHello @preetigupta15
,It depends on your use case and what you're trying to do but a first level of answer could be :
Currently, here are the different ways to do data lineage in Dataiku :
- Dataset/table level lineage: directly from the flow
- Column-level lineage: Dataiku 12.5.0 released in January 2024 added a new Flow view called “Column usage”, to see where some given columns appear in your flow
We're currently working to improve the lineage capabilities. If you're interested in being part of the user test do not hesitate to come back to me.
Have a nice day
-
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,618 Neuron@ElieA
,Can you point to any knowledge base articles, tutorials or the like that would help us learn more about these features? That would be helpful to me to learn more.
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,406 Neuron
Column Usage is very cool.
-
Can't wait to try it! (our teams are working on upgrading so in a few days)
-
Elie Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered, Product Ideas Manager Posts: 32 Dataiker
-
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,618 Neuron
@Elie ,
Thanks for the update. I've been experimenting with this feature the past week. I've been going back and looking at some older projects with cross project dependencies. This has been very helpful.