Dataiku recipes don't keep the data types in output datasets

Options
acuesta
acuesta Registered Posts: 5 ✭✭✭✭

DB: MySQL Server 5.6



Dataiku: 4.1



I have been trying to avoid the varchar(500) and bigint types that Dataiku generates (in the output dataset) as a consecuence of applying a recipe on an input dataset.



My objective is: if in my input dataset a column name is of type varchar(25), after applying a recipe the output dataset should still have the same type varchar(25) instead of the varchar(500) that it generates.



I have not been able to figure out how to do such thing.



I tried to set manually the storage type in the recipe for that column but doesn't seem to work, even if I edit the JSON maxlength manually.



I know that I can edit the CREATE TABLE statement from the advanced tab in the Output dataset, but that is not a thing that I would like to mantain.

The main problem I found is the following:





Thanks and regards,



Adrián

Tagged:

Best Answer

  • matthias_funke
    matthias_funke Dataiker Alumni Posts: 12 ✭✭✭✭✭
    Answer ✓
    Options
    Adrian, I can really only think of work-arounds, what are your thoughts?

    1. You already mentioned you can edit the "create table" statement. You could potentially do that in python code using the DSS API

    2. Could you use DSS managed dataset instead of MySQL as the back-end? You don't have the same limitations

    3. Could you use PostgreSQL instead of MySQL?

Answers

Setup Info
    Tags
      Help me…