-
Force substring to integer
Here is the sample table: IDColumn A1AA20012BB2002 I want to add a Column B, in which the values are forced to be integer. I know I can do Column B = substring (Column A, -4), and Dataiku will automatically convert the values to integer, but the conversion process is a black box to me, and I don't know what's the…
-
torch.cuda.is_available() is False
I am trying to load a pickle file of a pre-trained model to my code recipe but get the following error message: I have already selected the "ai-exec-t4-gpu" in the "Containerized Execution" tab of the environment. I do not understand exactly what could be going wrong here. Appreciate your help!
-
Trouble creating custom Python code environments
I am trying to create a new Python Code Env in my locally installed instance of Dataiku. The only non-conda based option is Python 3.7. All the other options (Python 3.5, 3.6, 3.8, 3.9, 3.10, 3.11) are listed as "Not available" in the dropdown. I have Python 3.10.11 locally installed, but somehow Dataiku is not recognizing…
-
Python py_310_sample_llm environment not working in Jupyter Notebooks
I've downloaded the LLM Starter Kit from the Dataiku gallery (Flow | Dataiku) and am trying to use it on my local installation of Dataiku. When doing the exercises in the "Basic use - code" flow zone, I can get the code to run in the code window, but when I try to edit it in a notebook, the kernel won't work; it just stays…
-
AttributeError: DataFrame object has no attribute _session
I got the error while running the following piece of code. Can anyone tell me whats wrong ? # get output dataset LOANS_FE = dataiku.Dataset("LOANS_FE") # write input dataframe to output dataset dku_snowpark.write_with_schema(LOANS_FE,snowdf) AttributeError Traceback (most recent call last)<ipython-input-34-bdd96a80a1c7> in…
-
Record unmatched rows from two datasets (without code recipe)
I would like to compare Dataset 1 and Dataset 2 (see the sample below), and record all the differences in Dataset 3. I have been exploring the functions in the Join recipe but no luck so far. Note that in Dataset 1 the ID field (join key) is unsorted, I could manually sort it in the prepare recipe but I would like to…
-
Visual recipe switch?
In visual recipe is there an existing function that acts as a switch to turn on/off of a recipe? For example I can create table 2 from table 1 using a prepare recipe by changing the values in column B based on a condition in column C, and table 2 will be used in a downstream process for validation. Now I would like to…
-
Does Schema propagation work on existing columns in a prepare recipe
Hi All, I had a question on whether schema propagation works in the prepare recipe ? For if there is a column in the input dataset which has a datatype of BIGINIT then during the first run the output dataset will also have the datatype as BIGINT But during a future run if the datatype of the column in the input dataset…
-
Connect Dataiku to Postgres
Hello, Dataiku Team I made a connection between a Postgres RDS and Dataiku. It worked ok. I decided to make a backup in a new virtual machine (ubuntu) in a EC2 of the RDS. I restored it on the installed Postgres in this virtual machine. Before restoring, I created a new user and bd to restore it. I made the new connection…
-
Not In Function when matching values of 2 datasets
Hello, im a very new user of DataIku, but experienced in writing SQL. I have a use case where i need to match the IDs of Table A to the IDs of Table B, and exclude the records in Table A which are matching with Table B. In SQL, i would usually just do a Where Table A.ID NOT IN Table B.ID, but in DataIku, i dont see any…