how we can achieve Data Lineage in DataIku

preetigupta15

How we can achieve table and column decencies in recipes.

Elie

Hello @preetigupta15
,

It depends on your use case and what you're trying to do but a first level of answer could be :

Currently, here are the different ways to do data lineage in Dataiku :

- Dataset/table level lineage: directly from the flow

- Column-level lineage: Dataiku 12.5.0 released in January 2024 added a new Flow view called “Column usage”, to see where some given columns appear in your flow

We're currently working to improve the lineage capabilities. If you're interested in being part of the user test do not hesitate to come back to me.

Have a nice day

tgb417

@ElieA
,

Can you point to any knowledge base articles, tutorials or the like that would help us learn more about these features? That would be helpful to me to learn more.

Turribeach

Column Usage is very cool.

me2

Can't wait to try it! (our teams are working on upgrading so in a few days)

Elie

Hello @me2 , Hello @tgb417 ,

Hope you're doing well.

Coming back to you on the data lineage topic: the feature has been released in 13.2!

Please see here a video and all the explanations on the feature and here the ref doc.

Have a nice day

tgb417

@Elie ,

Thanks for the update. I've been experimenting with this feature the past week. I've been going back and looking at some older projects with cross project dependencies. This has been very helpful.