You now have until September 15th to submit your use case or success story to the 2022 Dataiku Frontrunner Awards!ENTER YOUR SUBMISSION

How to Get data from dataset to use it as Variables

Solved!
Richard_CDC
Level 2
How to Get data from dataset to use it as Variables

Hi,

I m using a scenario to update "Global variables" in my projet variables

I have a dataset With:

DIRAge
AURA35
PAUL48
.......

 

I want to put the column "DIR" in Variables to be able to used the data as filter in my dtaiku project. But it doesn t work because the result in my "Global Variables" have "[" and i can't use it. If I remove List() i have an error 

TypeError: DIR    AURA
Name: 0, dtype: object is not JSON serializable

 

My code

import dataiku

from dataiku.scenario import Scenario
scenario = Scenario()

df_data=dataiku.Dataset("Test_DIR").get_dataframe(infer_with_pandas=False)

df_data = df_data.iloc[0]

Scenario().set_project_variables(Var=list(df_data))

 

Result in "Project Variables"

Global variables

{
"Var": [
"AURA"
]
}

What can i use to replace list() ?


Operating system used: Windows


Operating system used: Windows

0 Kudos
1 Solution
arnaudde
Dataiker
Dataiker

If you just want the first element of the list you can get it by doing 

Scenario().set_project_variables(Var=list(df_data)[0])

View solution in original post

0 Kudos
5 Replies
arnaudde
Dataiker
Dataiker

Hello,

Project variables should be valid json objects. In your example the variable contains an array so you need to escape the brackets ("[" should be "\["). You can use the python json library to encode python objects in json with the json.dumps function. 

Scenario().set_project_variables(Var=json.dumps(list(df_data)))


I hope this helps

0 Kudos
Richard_CDC
Level 2
Author

Thanks for the help, but now i have:

{
"Var": "[\"AURA\"]"
}

 

And i really don't know how to remove "[\ I would like to have as result: 

{
"Var": "AURA"
}

Any idea  ?

 

0 Kudos
arnaudde
Dataiker
Dataiker

If you just want the first element of the list you can get it by doing 

Scenario().set_project_variables(Var=list(df_data)[0])
0 Kudos
Richard_CDC
Level 2
Author

Thank you it s working.

 

Now i want to build a dataset, but when i built de dataset 'Test_envoie_boucle', the filter just before doesn't apply.

2022-01-17_17h07_41.png

 I don't understand because i RECURSIVE_FORCED_BUILD the dataset. And when i do it manualy

that's working.

 Do i do something wrong in my code ?

 

import dataiku
import json
from dataiku.scenario import Scenario

scenario = Scenario()

df_data=dataiku.Dataset("Contact").get_dataframe(infer_with_pandas=False)
count = df_data.count()

df_data = df_data.iloc[3]

Scenario().set_project_variables(Var=list(df_data)[0])

Scenario().build_dataset('Test_envoie_boucle', buildt="RECURSIVE_FORCED_BUILD")

0 Kudos
arnaudde
Dataiker
Dataiker

The project variable `Var` probably does not hold the value you expect it to hold
Can you share the activity log from the jobs page so that we can investigate ? 

0 Kudos