Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
How to Get data from dataset to use it as Variables
Added on January 17, 2022 9:59AM
Likes: 0
Replies: 7
Richard_CDC
Registered Posts: 6 ✭✭✭
Hi,
I m using a scenario to update "Global variables" in my projet variables
I have a dataset With:
| DIR | Age |
| AURA | 35 |
| PAUL | 48 |
| ... | .... |
I want to put the column "DIR" in Variables to be able to used the data as filter in my dtaiku project. But it doesn t work because the result in my "Global Variables" have "[" and i can't use it. If I remove List() i have an error
TypeError: DIR AURA
Name: 0, dtype: object is not JSON serializable
My code
import dataiku
from dataiku.scenario import Scenario
scenario = Scenario()
df_data=dataiku.Dataset("Test_DIR").get_dataframe(infer_with_pandas=False)
df_data = df_data.iloc[0]
Scenario().set_project_variables(Var=list(df_data))
Result in "Project Variables"
Global variables
{
"Var": [
"AURA"
]
}
What can i use to replace list() ?
Operating system used: Windows
Operating system used: Windows
Best Answer
-
If you just want the first element of the list you can get it by doing
Scenario().set_project_variables(Var=list(df_data)[0])
Answers
-
Hello,
Project variables should be valid json objects. In your example the variable contains an array so you need to escape the brackets ("[" should be "\["). You can use the python json library to encode python objects in json with the json.dumps function.
Scenario().set_project_variables(Var=json.dumps(list(df_data)))
I hope this helps -
Thanks for the help, but now i have:
{
"Var": "[\"AURA\"]"
}And i really don't know how to remove "[\ I would like to have as result:
{
"Var": "AURA"
}Any idea ?
-
Thank you it s working.
Now i want to build a dataset, but when i built de dataset 'Test_envoie_boucle', the filter just before doesn't apply.
I don't understand because i RECURSIVE_FORCED_BUILD the dataset. And when i do it manualy
that's working.
Do i do something wrong in my code ?
import dataiku
import json
from dataiku.scenario import Scenarioscenario = Scenario()
df_data=dataiku.Dataset("Contact").get_dataframe(infer_with_pandas=False)
count = df_data.count()df_data = df_data.iloc[3]
Scenario().set_project_variables(Var=list(df_data)[0])
Scenario().build_dataset('Test_envoie_boucle', buildt="RECURSIVE_FORCED_BUILD")
-
The project variable `Var` probably does not hold the value you expect it to hold
Can you share the activity log from the jobs page so that we can investigate ? -
akember182 Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 2 ✭Hello I have a similar requirement.
I have a dataset (my_dataset) that looks like this
Code. Year
122. 2021
I want to extract the 2021 figure and save it as a variable.
I know I can use code in a scenario along these lines
import dataiku
from dataiku.scenario import Scenario
scenario = Scenario()df_data=dataiku.Dataset("my_dataset").get_dataframe(infer_with_pandas=False)
df_data = df_data['Year']
Scenario().set_project_variables("year_variable"=list(df_data)[0])
But I can't quite seem to get it working, any advice?
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,384 Neuronremoved...