How to Get data from dataset to use it as Variables

Richard_CDC

Hi,

I m using a scenario to update "Global variables" in my projet variables

I have a dataset With:

DIR	Age
AURA	35
PAUL	48
...	....

I want to put the column "DIR" in Variables to be able to used the data as filter in my dtaiku project. But it doesn t work because the result in my "Global Variables" have "[" and i can't use it. If I remove List() i have an error

TypeError: DIR    AURA
Name: 0, dtype: object is not JSON serializable

My code

import dataiku

from dataiku.scenario import Scenario
scenario = Scenario()

df_data=dataiku.Dataset("Test_DIR").get_dataframe(infer_with_pandas=False)

df_data = df_data.iloc[0]

Scenario().set_project_variables(Var=list(df_data))

Result in "Project Variables"

Global variables

{
"Var": [
"AURA"
]
}

What can i use to replace list() ?

---

Operating system used: Windows

---

Operating system used: Windows

arnaudde

If you just want the first element of the list you can get it by doing

Scenario().set_project_variables(Var=list(df_data)[0])

arnaudde

Hello,

Project variables should be valid json objects. In your example the variable contains an array so you need to escape the brackets ("[" should be "\["). You can use the python json library to encode python objects in json with the json.dumps function.

Scenario().set_project_variables(Var=json.dumps(list(df_data)))

I hope this helps

Richard_CDC

Thanks for the help, but now i have:

{
"Var": "[\"AURA\"]"
}

And i really don't know how to remove "[\ I would like to have as result:

{
"Var": "AURA"
}

Any idea ?

Richard_CDC

Thank you it s working.

Now i want to build a dataset, but when i built de dataset 'Test_envoie_boucle', the filter just before doesn't apply.

 I don't understand because i RECURSIVE_FORCED_BUILD the dataset. And when i do it manualy

that's working.

Do i do something wrong in my code ?

import dataiku
import json
from dataiku.scenario import Scenario

scenario = Scenario()

df_data=dataiku.Dataset("Contact").get_dataframe(infer_with_pandas=False)
count = df_data.count()

df_data = df_data.iloc[3]

Scenario().set_project_variables(Var=list(df_data)[0])

Scenario().build_dataset('Test_envoie_boucle', buildt="RECURSIVE_FORCED_BUILD")

arnaudde

The project variable `Var` probably does not hold the value you expect it to hold
Can you share the activity log from the jobs page so that we can investigate ?

akember182

Hello I have a similar requirement.

I have a dataset (my_dataset) that looks like this

Code. Year

122. 2021

I want to extract the 2021 figure and save it as a variable.

I know I can use code in a scenario along these lines

import dataiku

from dataiku.scenario import Scenario
scenario = Scenario()

df_data=dataiku.Dataset("my_dataset").get_dataframe(infer_with_pandas=False)

df_data = df_data['Year']

Scenario().set_project_variables("year_variable"=list(df_data)[0])

But I can't quite seem to get it working, any advice?

Turribeach

removed...