-
DtypeWarning. Columns have mixed types
Hi, I have checked whole flow for the column '0' and is set to 'string' and 'Text' everywhere. Sadly when this data is dump to SQL Table values are converted to "int" Any idea?
-
CHAR(1) columns turning into lengths of 2 with spaces in Exports
We have to use data preps from many database tables on different platforms that have columns defined as CHAR(1) to keep them as length's of 1. Otherwise the exports change them to lengths of 2 with spaces added on. So, an indicator column with only "Y" or "N" becomes "Y " (added space) or "N " (added space). Using a data…
-
Best way to clean data for country meaning
I have a dataset which includes ISO 3166-1 alpha-3 country codes which are being detected properly, and full country names where about 14% are being flagged as not passing validation. Is there some place I can get the canonical list of country names dataiku uses for meaning? And if I can get that list, what's the most time…
-
Change Auto-Typing to an off or on option with default “Off”
Would like to have the Auto-Typing setup as an option that can be turned off and on with the default being “Off”. This feature is changing my unit serial numbers (230836735F) to a Float (2.30836735E8) which causes me to lose records when joining on the unit serial numbers field in a following step. This will cause my…
-
Using date in DataIKU
Hi, Despite going through documentation multiple times, I still don't really understand how dates work in DSS. I'm importing dataset from a connection. Without turning on any of the options in Date & Time handling, this is how data looks like: It says that the data type is string, while in the database itself it is, in…
-
Looking to replicate a SUM(COUNTIF) formula in Dataiku
I am working on a scorecard in Dataiku and I would like to calculate the percentage of completion in a set number of columns. Basically, I would like to replicate this formula in excel: =SUM(COUNTIF(ColumnX:ColumnXX,"*")/Total Number of Columns) and am having issues. The columns are a mix of strings, integers, and text,…
-
Prevent Y / N column from being Auto-Detected as Boolean column
For my dataset I have over 200 columns with many of them Y/N flag columns. I would like to keep them as text and not infer the type. There is a checkbox that I am leaving unchecked: Normalize booleans Normalize all possible boolean values (0, 1, yes, no, …) to 'true' and 'false'
-
How do i create a categorisation model for a reviews dataset
Hi there - new to dataiku, Lets say i have an excel sheet of 2 columns where one has app reviews and the other has dates they were posted. Is there a video tutorial anywhere or example where i can create a model to categorise the app reviews into categories eg) ux/ui problem or customer service problem as well as include…
-
How to correctly do time conversions
I have a column that has been parsed and is in UTC, when I try to format the date to be in eastern / New York time I get a new column that is -5 hours, but isn't the current the current difference -4 hours? I'm sure this has something to do with daylight savings time vs normal time, but I just want to ensure that my…
-
How to stop keep formatting using prepare / formula recipe
I have string value called hours, formatted as "00" and a value called minutes formatted as "07". Im trying to use the concat function to create an HH:mm value = 00:07 but for some reason when using concat, it gets rid of trailing 0s and results in 0:7. is there a way around this? if not is there any other way I can create…