-
Is there a list of all the builtin variables like ${projectKey}, ${dssUserLogin}, etc
I noticed the user of some builtin variables in the documentation like: * projectKey jobProjectKey * dssUserLogin * jobProjectKey * jobId * activityId but it's there a list somewhere with all the possibilities, and where they can be used? For example is not clear to me if I can use ${jobProjectKey} in the Administration >…
-
propagate schema fails at partitioning sync recipe
I added a column to the start of an SQL pipeline and needed to ensure that the schema change was propagated to subsequent tables. Thankflully, DSS "propage schema" functionality allowed me to automate this task. dataiku's convenient functionality to propagate upstream changes in schema However, at some point I encountered…
-
Exporting documentation to wiki for Confluence
Hi all, I'm looking for a way to programmatically fill the Wiki of a project. For example, I'd like to create an article that contains the Data Catalogue and another that contains the Flow documentation. Both should be updated every week through a scenario. I know there is a native data catalogue and an export available…
-
Change Project image through code
This might be a bit of a silly question, but is there a possibility to change the image used for a project through code (or another automated way)? The reason for this is that, after duplicating projects from Development to Acceptance, we would like to distinguish between them through the image but want to get rid of the…
-
Auditing projects by tags and statuses
Once we properly tag projects (using global tags), what options do we have as an admin to audit which projects have certain tags? Certain project statuses? As a user, we can search by these in the project-list page, but is there any way to export/share/archive the results? Thanks!
-
Model Scorning Process - Metadata
Hello, Within the Model training process, we have the option of adding additional Metadata labels to add to a version of a saved model on training. When the model is being scored, there is an option to include the model metadata in the model output. The types of Metadata included in the output are the ID, Version, Fullid…
-
Efficient way to write massive dataset to output in Dataiku
Hi Team, I'm using pyspark with Dataiku after processing the data, I'm facing an issue with writing the data to the output. Could you please suggest an efficient way to write the data to the output? Dataset size: 40million(approx.) Getting Error at line 15 while writing(as the data is massive) #Recipe 1 import dataiku 2…
-
write_with_schema exists, how can I write only if the data is new?
Hi, I have a Python recipe that creates datasets I want to write to two separate Snowflake tables. I would like to only write to the snowflake tables if the data is new. So, I need to be able to check if new data is in the table and then write if it is not. thx Operating system used: Windows 10
-
Is it possible (supported) to change a PROJECT NAME ?
Hello, I see in the interface that we can rename a project. Is this something supported ? (ie: will we be able to run/build anything after renaming the project without any problem ?) If I try, I can already see that the project name changed, but the web address to this project still shows the old name... for ex: - my…
-
Documenting global variables
I use global variables often and in typical project may have a half dozen or more variables. While there are many advantages to using variables over hard coding key parameters, I do struggle keeping track of what the variables mean and how they are used. Especially when returning to a project after some time. Without any…