Using Dataiku

Hi @PaulBavaz
,

There isn't a way to pull if a specific prepare recipe processor is compatible with the "in-database" engine in the same way that get_engines_details() can provide engine compatibility information at the recipe-level.

For an existing prepare recipe, get_selected_engine_details() will return a statusMessage field that will return a single error message if the prepare recipe has one:

recipe = project.get_recipe('recipe_name')
status = recipe.get_status()
status.get_selected_engine_details()['statusMessage']

However, if multiple processors within a prepare recipe are returning an error, there isn't a way to pull each processor and the corresponding error message. In the example above, note that multiple processors are "Not translatable to SQL", but you will only get one message returned:

So, it might makes the most sense to refer to the list of in-database fully supported processors and partially supported processors, and even hardcode these if that worked for your use case.

Let me know if you have any further questions about this.

Thanks,
Sarina

Hi @PaulBavaz
,

There isn't a way to pull if a specific prepare recipe processor is compatible with the "in-database" engine in the same way that get_engines_details() can provide engine compatibility information at the recipe-level.

For an existing prepare recipe, get_selected_engine_details() will return a statusMessage field that will return a single error message if the prepare recipe has one:

recipe = project.get_recipe('recipe_name')
status = recipe.get_status()
status.get_selected_engine_details()['statusMessage']

However, if multiple processors within a prepare recipe are returning an error, there isn't a way to pull each processor and the corresponding error message. In the example above, note that multiple processors are "Not translatable to SQL", but you will only get one message returned:

So, it might makes the most sense to refer to the list of in-database fully supported processors and partially supported processors, and even hardcode these if that worked for your use case.

Let me know if you have any further questions about this.

Thanks,
Sarina

Python .egg files are deprecated and are replaced by Wheel format. Since DSS uses pip which does not support installing egg format you won't be able to add egg packages directly from DSS.

You should be able to convert your .egg file to wheel: https://wheel.readthedocs.io/en/stable/reference/wheel_convert.html

Once converted you should be to use pip to install.

Python .egg files are deprecated and are replaced by Wheel format. Since DSS uses pip which does not support installing egg format you won't be able to add egg packages directly from DSS.

You should be able to convert your .egg file to wheel: https://wheel.readthedocs.io/en/stable/reference/wheel_convert.html

Once converted you should be to use pip to install.

Hi @mgirard
,

easiest way is to download chromedriver from google and do the setup with your package manager. What's the operating system of your dss instance?

Best regards,

Andreas

Hi @Hazou
,

OK, I think I'm understanding what you are doing better...

Check out this solution - the problem posted appears to be similar to yours: https://community.dataiku.com/t5/Using-Dataiku-DSS/Refresh-read-schema-on-dataset-via-API/m-p/8730/highlight/true#M4405

Does this help?

Marlan

The error message says that if you're passing scalar values, you have to pass an index. Pandas unfortunately always needs an index when created a DatFrame from Dictionary. What this is essentially asking for is a column number for each dictionary to correspond to each dictionary. You can either set it yourself, or use an object with the following structure so pandas can determine the index itself:

df = pd.DataFrame({'A': [a], 'B': [b]})

or use scalar values and pass an index:

pd.DataFrame({'A': a, 'B': b}, index=[0])

In most cases, it might be an issue with:

• the delimiters in your data.
• confused by the headers/column of the file.

The error tokenizing data may arise when you're using separator (for eg. comma ',') as a delimiter and you have more separator than expected (more fields in the error row than defined in the header). So you need to either remove the additional field or remove the extra separator if it's there by mistake. The better solution is to investigate the offending file and to fix it manually so you don't need to skip the error lines.

I also added html5lib and lxml to my python code env and verified it with

import html5lib
import lxml