Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I'm have storeed my JSON configuration on a S3 bucket and i wanted load it into dataiku global variables. But i'm facing challenges with the code to do it.
Could you please help me with it?
First you will need to load the JSON data from your file stored in S3 into a Python dictionary. It is possible by using the get_object() method from the boto3 package (assuming that the code is executed with the relevant AWS IAM permissions).
From there, you can update the global variables using the Python client of Dataiku's API.
Hope this helps!
Hello @HarizoR ,
i'm trying to do this with the below code and im getting error at the last line while setting the project variable.
import dataiku import json import pandas as pd configuration = dataiku.Folder("U44wokeq") configuration.get_info() with configuration.get_download_stream("NGE_ODH_R3_DEV_CONFIG.json") as f: data = f.read() my_json = data.decode('utf8').replace("'", '"') # Load the JSON to a Python list & dump it back out as formatted JSON data = json.loads(my_json) s = json.dumps(data, indent=4, sort_keys=True) client = dataiku.api_client() project = client.get_project(dataiku.get_custom_variables()['projectKey']) project_variables = project.get_variables() project_variables['standard'] = s project.set_variables(project_variables)
def _perform_empty(self, method, path, paraDataikuException: com.dataiku.common.server.DKUControllerBase$MalformedRequestException: Could not parse a ProjectVariables from request body
What you describe here differs quite a bit from your original question:
- You are connecting to your S3 data through a managed folder instead of direct access.
- You are setting up a project variable and not a global variable.
That being said, the fix for your problem should be simple: you should directly pass the data dict object to project_variables["standard"] and not the s string:
project_variables["standard"] = data