How to Get data from dataset to use it as Variables
Hi,
I m using a scenario to update "Global variables" in my projet variables
I have a dataset With:
| DIR | Age |
| AURA | 35 |
| PAUL | 48 |
| ... | .... |
I want to put the column "DIR" in Variables to be able to used the data as filter in my dtaiku project. But it doesn t work because the result in my "Global Variables" have "[" and i can't use it. If I remove List() i have an error
TypeError: DIR AURA
Name: 0, dtype: object is not JSON serializable
My code
import dataiku
from dataiku.scenario import Scenario
scenario = Scenario()
df_data=dataiku.Dataset("Test_DIR").get_dataframe(infer_with_pandas=False)
df_data = df_data.iloc[0]
Scenario().set_project_variables(Var=list(df_data))
Result in "Project Variables"
Global variables
{
"Var": [
"AURA"
]
}
What can i use to replace list() ?
Operating system used: Windows
Operating system used: Windows
Best Answer
-
If you just want the first element of the list you can get it by doing
Scenario().set_project_variables(Var=list(df_data)[0])
Answers
-
Hello,
Project variables should be valid json objects. In your example the variable contains an array so you need to escape the brackets ("[" should be "\["). You can use the python json library to encode python objects in json with the json.dumps function.
Scenario().set_project_variables(Var=json.dumps(list(df_data)))
I hope this helps -
Thanks for the help, but now i have:
{
"Var": "[\"AURA\"]"
}And i really don't know how to remove "[\ I would like to have as result:
{
"Var": "AURA"
}Any idea ?
-
Thank you it s working.
Now i want to build a dataset, but when i built de dataset 'Test_envoie_boucle', the filter just before doesn't apply.
I don't understand because i RECURSIVE_FORCED_BUILD the dataset. And when i do it manualy
that's working.
Do i do something wrong in my code ?
import dataiku
import json
from dataiku.scenario import Scenarioscenario = Scenario()
df_data=dataiku.Dataset("Contact").get_dataframe(infer_with_pandas=False)
count = df_data.count()df_data = df_data.iloc[3]
Scenario().set_project_variables(Var=list(df_data)[0])
Scenario().build_dataset('Test_envoie_boucle', buildt="RECURSIVE_FORCED_BUILD")
-
The project variable `Var` probably does not hold the value you expect it to hold
Can you share the activity log from the jobs page so that we can investigate ? -
akember182 Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 2 ✭Hello I have a similar requirement.
I have a dataset (my_dataset) that looks like this
Code. Year
122. 2021
I want to extract the 2021 figure and save it as a variable.
I know I can use code in a scenario along these lines
import dataiku
from dataiku.scenario import Scenario
scenario = Scenario()df_data=dataiku.Dataset("my_dataset").get_dataframe(infer_with_pandas=False)
df_data = df_data['Year']
Scenario().set_project_variables("year_variable"=list(df_data)[0])
But I can't quite seem to get it working, any advice?
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,971 Neuronremoved...