-
The recipe execution is taking long time due to handling a large volume of data in dataiku
We are experiencing long execution times for a recipe in Dataiku due to handing large datasets, while we have implemented partitioning using a filter on a specific column, it still takes 1.5-2 hours to partitioning 30M records. Is there a more efficient way to handle and process this data quickly and effectively because…
-
Influx DB to Dataiku
Hi all! Do anyone have solution to connect Influx DB to Dataiku? Operating system used: Linux
-
Validate at SQL recipe using the public API (python)
How do you validate a SQL recipe using the public API?
-
SQLExecutor2 exec_recipe_fragment multiple statements and
I'm following the tutorial found here https://www.dataiku.com/learn/guide/code/python/use-python-sql.html but had some additional questions. 1. Is it possible to submit multiple statements with SQLExecutor2.exec_recipe_fragment()? The script creates multiple volatile tables before constructing the final table. I want to…
-
How do you use SQL Script in recipe as application
I have a flow where I use SQL scripts to create datasets (see image). I follow the instructions as described here: https://knowledge.dataiku.com/latest/courses/o16n/dataiku-applications/create-app-as-recipe.html The issue is that when the flow is converted to a recipe as an application, while the log reports that the names…
-
Error: Invalid number of columns in query
Hi, After create my connection with sucess to my postgresql database. I've the following error when I import my table in my project : An invalid argument has been encountered : Invalid number of columns in query (2, expected 0) SELECT * FROM "public"."ape_ref" LIMIT 10000. Please check dataset schema Could you explain me…
-
An error occurred ERROR: permission denied for schema dku_tshirt Position: 67
I get the following An error occurred ERROR: permission denied for schema dku_tshirt Position: 67 when trying to run SELECT campaign, AVG(total), SUM(total), COUNT(campaign) FROM "dku_tshirt"."DKU_TUTORIAL_SQL_customers_enriched" WHERE birthdate >= '1980-01-01' AND birthdate < '1990-01-01' GROUP BY campaign in the Dataiku…
-
How to write from pandas Dataframe to PostgreSQL with a geometry type
Hello, In my python receipt, I would like to write in my PostreSQL database (with PostGis extension) data with a geometry type. Therefore, I declared a schema and created my table with this specific schema schema = [{"name": 'id', "type": 'int'}, {"name": 'value', "type": 'string'}, {"name": 'the_geom', "type":…
-
Plugin doesn't isn't shown anymore but still exists
Hi, I was creating a plugin in order to correct a weird behavior of one of our engine : while syncing a Teradata table to HDFS using the TDCH engine, dates (in a "%Y-%m-%D" format) are imported as dates in the schema even though they are strings. I created a plugin recipe that copy the synced HDFS dataset to another using…
-
Google Big Query in DataIku
Is there a feature to connect with Google Big Query like the amazon S3 connector in "cloud storage"?