Setting a default output location for any visual recipes in Dataiku
We would like to be able to set a default location for output (Ex: S3_Prefix)
Would like to be able to set this at a team level.
Have not seen any documentation that would indicate this is possible.
If this is not available, I would like to request a feature enhancement for this capability.
As the admin, I have encountered this many times when assisting our end users where they just forgot to change the output location where DSS auto selects, which is not usually the required location.
Any recommendation on how to handled these issue will be greatly appreciated
Operating system used: AL2
Best Answer
-
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,595 NeuronOptionsI am not an S3 expert. However, as I was reading your note I was wondering if you setup multiple S3 Connections out of DSS for different teams.
https://doc.dataiku.com/dss/latest/connecting/s3.html
Each connection would have the appropriate configuration for that team, and from a security point of view you may be able to expose just the right connection to S3 to each group.
After that I'd suggest a support ticket. The support team is great and they may be able to help you find a successful configuration. Just a thought in case it is helpful.
Good luck and let us know how it goes.
If you find that the feature does not exist. I'd encourage you to add this as a product idea on this page.https://community.dataiku.com/t5/Product-Ideas/idb-p/Product_Idea
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,757 NeuronOptionsIt’s not possible to set a default connection for dataset outputs. If that is what you are asking. However if the user only has access to a single connection then they won’t be able to select an incorrect one. You should make sure you remove permissions to write dataset outputs from any connections where the users shouldn’t write data to. Failing that you could write some Python API calls to identify projects that use incorrect connections, assuming the criteria you use to determine that can be translated into code.
-
jrmathieu63 Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Registered Posts: 26 ✭✭✭✭✭Options
Thanks Tom,
Have place a product idea request since this would help end users when there are 100s of connections defined for an enterprise platform.
Our S3 team connection are defined using AWS STS Assume roles, which has been recommended by Dataiku support and is working properly.
We communicate the S3 prefix to the end user must uses for outputs when they are initially assigned to the platform but would be better if this the default since they must navigate the list of outputs to locate this location.
John