Using Dataiku
- When I attempt to use the Sync visual recipe to put a Redshift table into S3, using the "Redshift to S3" recipe engine, I get this rather mysterious error: Job failed: [Amazon](500310) Invalid operati…
Last answer by pillsyWe are running Dataiku version 8.05. The column names may be sensitive, but it appears the problem is in the date handling for one of them, as this is the only part of the query that has a ':' in it:
TO_CHAR(cast("redacted" as TIMESTAMPTZ),\'YYYY-MM-DDThh24:MI:SS.MSZ\')I will check with my contact in IT to see how they want the ticket filed.
Last answer by pillsyWe are running Dataiku version 8.05. The column names may be sensitive, but it appears the problem is in the date handling for one of them, as this is the only part of the query that has a ':' in it:
TO_CHAR(cast("redacted" as TIMESTAMPTZ),\'YYYY-MM-DDThh24:MI:SS.MSZ\')I will check with my contact in IT to see how they want the ticket filed.
- Hi, Im using global variable to get the list of country as below: Global Variable { "var_country_dummy": [ "malaysia", "singapore" ]} Python call the variable countryArray = dataiku.get_custom_variabl…Last answer by Marija
Hi, is there any update for this? Thanks , Marija
Last answer by MarijaHi, is there any update for this? Thanks , Marija
- Hi guys, Is there a possibility to lower all the column name in the analysis module ? I have 30 columns in my file , i synced it into a postgresql table and after that, i would lower all the column na…Last answer byLast answer by tassosv
Hi,
If you would like to do it using a recipe and not code scripts. You can change:
1) from table view to column view
2) Select all columns or those you would like to rename
3) Select actions and from their menu, rename
4) From rename menu choose convert to lowercase
- I am using a formula, but if I want to use a column that has spaces in its name, I just get an error. Is there a way to make it work?Solution by
Solution by cperdigouYou can use
numval("column with spaces")to retrieve the values!
See:
https://doc.dataiku.com/dss/latest/advanced/formula.html#reading-column-values
- Hey everybody, I've got a large project, which is updated as needed a couple of times per week with a comprehensive scenario. (Actually it's two projects and two scenarios, drawing a line between raw …Solution by
Solution by MarlanHi @cuezumo
,What is the Build mode setting for the relevant Build step in your scenario? The default mode of "Build required datasets" doesn't rebuild SQL datasets if those datasets are at the beginning of a flow (or the entire flow). The solution is to set the Build mode to "Force-rebuild dataset and dependencies" which will rebuild all SQL datasets. This may be not at all what is going on in your situation but we have certainly experienced data refreshes not happening when expected due to the wrong Build mode setting. More on this situation in this product idea.
Marlan
- Is it possible to export/import recipes? I've done export/import of whole projects, and all the recipes were preserved with fidelity. Can one do it with an individual recipe?Last answer by
Last answer by MarkPundursThat article says, "Of course, it is possible to copy the entire Prepare recipe, as with any visual recipe."
That's exactly what I don't know how to do - not between projects, anyway.
- I try to join two data sets, but returns the following error: "Column name can not contain comma, quotation mark, leading or trailing white spaces with DSS engine" Is there any way to remove multiple …Solution by
Solution by Alan_FustéHi Farid,
You can do it with a Python Recipe using:
df.columns = df.columns.str.replace(',', '').replace(' ','')Another option is to use a Preparation Recipe and add step "Rename Columns" but you have to change each column manually.
- Hi, I am trying to remove duplicates in a Prepare recipe, but as far as I can tell that is not possible, even though I would think its a pretty basic feature. I have an expression in the Prepare recip…Last answer by
Last answer by tgb417Welcome to the Dataiku community.
You might want to take a look at Spark pipelines. Or if you are using a SQL server you can use SQL pipelines described further in How to enable SQL pipelines in the Flow. These are apparently implemented as views that get added to the SQL database so they don't get created as tables taking up more space.
You will find that not all features are available when working in this way. Here is a little more about Where ... it all happens that might be helpful to you.
- Are you aware of a step we could add to our visual prep script to translate languages?Last answer by
Last answer by Alex_CombessieHi @UserBird
You can now use the Dataiku plugin for Google Cloud Translation for that
Hope it helps,
Alex
- Hi I want to calculate Date difference between 2 columns by excluding the weekends. I was wondering is there any formula by which I can calculate the same in the Visual Recipe. I however managed to so…Last answer by
Last answer by fchataigner2If you have Python code able to perform the computation, then you can use it in a Prepare recipe on a row-by-row basis. You can also probably replicate your computations with the Extract date component processor (to get the dayOfWeek of the start and end dates), the Compute difference processor (to compute the number of days between the 2 dates) and a bit of Formula step.
For a purely visual approach, you'll need to have a dataset of the dates of the non-weekend days (possibly the non-holiday dates too) covering the time range present in your dataset, use a join recipe from you dataset on the non-weekend days to retrieve the days where start <= non-weekend-day <= end, then a Grouping recipe to count the number of non-empty non-weekend-day joined.