Submit your innovative use case or inspiring success story to the 2023 Dataiku Frontrunner Awards! LET'S GO

Are user-defined meanings strictly global?

Are user-defined meanings strictly global?
As far as I can tell, user-defined meanings are global i.e. if a meaning is created for use in one project useful, but I also could see this being cumbersome if projects have a lot of custom meanings.

Our use case is we are reading primarily from an external system via a REST API. I have been using a custom plugin to read data from the other system. I like that I can set the schema via a plugin because the other system allows metadata to be queried. That metadata includes data value mappings (e.g. 1 - yes, 0 - no, 2 - not sure, -1 - N/A). I have been trying to come up with a good way of importing those mappings into a project when a dataset is imported. The values-mapping type of user-defined meanings are a very useful concept because they separate storage type from the labels (usually ints to string) and allow someone viewing the data to easily see what number means what.

I imagine us having many projects with different users having access to each project. I could keep using meanings and use prefixes to distinguish them (although I could see the odd case where having the meanings visible to the wrong users might be a problem). I could also import the value mappings as datasets and use joins to fill in the values but this is not very automated and will make the projects quite messy.

Any tips? Anyone else looking into a similar use-case?
0 Kudos
3 Replies

At the moment, UDMs are indeed only global. We are thinking about the possibility to add per-project-UDMs but do not have a timeframe for that yet.

From your description, couldn't you perform the replacement of values directly in your plugin ? Read the metadata mapping and then read the data and transform it before returning from the plugin code ?
Level 2
I think that is a reasonable work-around. Ideally, I would like to use functionality that is meant for what I am trying to accomplish. UDMs are appealing because they explicitly represent the idea of a key-value mapping for a datatype.
0 Kudos
Level 5

bumping this oldie to just say project level or even user level meanings would be wonderful. they have proven fantastic for validation of upload files and our user base is all over them - global meanings can get out of hand quickly!

0 Kudos


Labels (1)
A banner prompting to get Dataiku