set_definition_and_payload() error on sync recipe
![Meirkhan](https://us.v-cdn.net/6038231/uploads/Dataiku/nAvatar8.png)
Hi Dataikers,
The same code works for SQL recipe. But I am having a hard time applying it to the sync recipe
builder_gcs = None builder_gcs = SyncRecipeCreator(f"compute_{dataset_name}_gcs", project=dataiku_project) builder_gcs = builder_gcs.with_input(dataset_name, dataiku_project_name) builder_gcs = builder_gcs.with_new_output(f"{dataset_name}_gcs", GCS_CONNECTION) recipe_gcs = builder_gcs.build() recipe_def = recipe_gcs.get_definition_and_payload() recipe_payload = recipe_def.get_payload() # Modifications on definition - but does not work without modifs too recipe_def.set_payload(recipe_payload) recipe_gcs.set_definition_and_payload(recipe_def) Error: DataikuException: java.lang.IllegalArgumentException: Cannot set unparseable payload on recipe recipe_name of type sync
Even though I did not do any changes on payload (and payload is empty string for sync anyway).
Any ideas?
Answers
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,211 Dataiker
Hi,
Can you please confirm your exact DSS version? Since get_definition_and_payload() was deprecated.
You can instead try using get_settings()
Regardless. on DSS 9.0.5 . I wasn't able to replicate the issue with similar code :
As you can see the Update recipe worked without any exception only warnings are related to the deprecations.
-
I have DSS version 7.0.2
I am starting to think that it is because DSS cannot find the recipe.
Cause among the errors I have:
HTTPError: 400 Client Error: Bad Request for url: http://127.0.0.1:10001/dip/publicapi/projects/MEIR_SANDBOX/recipes/compute_parcel_minus_building_refacto_gcs
I tested by giving complete random name of recipe, which for sure does not exists and I am receiving the same error.
But it is the case only for the SYNC recipe and code works fine with other recipe types.
In this case another question raises - what can be the reason for DSS for not finding sync recipes?
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,211 Dataiker
Hi,
Does the following work for an existing Sync recipe?
recipe = project.get_recipe("myrecipe") status = recipe.get_status() print(status)
Thanks,
-
-
I have a test instance of DSS v 9.0.4
I confirm that it works there too
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,211 Dataiker
Can you try adding :
recipe_gcs = dataiku_project.get_recipe(f"compute_{dataset_name}_gcs")
Before :
recipe_def = recipe_gcs.get_definition_and_payload()
recipe_payload = recipe_def.get_payload()
recipe_def.set_payload(recipe_payload)
recipe_gcs.set_definition_and_payload(recipe_def)In your original code and see if this still fails on DSS7 does your original code fail on DSS 9.0.4 as well?
Thanks
-
Yes I did.
Whataver option of accessing sync recipe I try, it works on DSS 9.0.4 and fails on DSS 7.0.2
Full error if it helps:
2021-10-28 17:56:14 http://127.0.0.1:10001 "GET /dip/publicapi/projects/MEIR_SANDBOX/recipes/compute_parcel_minus_building_refacto_gcs HTTP/1.1" 200 1500 2021-10-28 17:56:14 http://127.0.0.1:10001 "PUT /dip/publicapi/projects/MEIR_SANDBOX/recipes/compute_parcel_minus_building_refacto_gcs HTTP/1.1" 400 None
--------------------------------------------------------------------------- HTTPError Traceback (most recent call last) /opt/dataiku-dss-7.0.2/python/dataikuapi/dssclient.py in _perform_http(self, method, path, params, body, stream, files, raw_body) 903 stream = stream) --> 904 http_res.raise_for_status() 905 return http_res /data/DSS/5.1/code-envs/python/Python36_2/lib/python3.6/site-packages/requests/models.py in raise_for_status(self) 939 if http_error_msg: --> 940 raise HTTPError(http_error_msg, response=self) 941 HTTPError: 400 Client Error: Bad Request for url: http://127.0.0.1:10001/dip/publicapi/projects/MEIR_SANDBOX/recipes/compute_parcel_minus_building_refacto_gcs During handling of the above exception, another exception occurred: DataikuException Traceback (most recent call last) <ipython-input-70-d85988868863> in <module> 3 recipe_payload = recipe_def.get_payload() 4 recipe_def.set_payload(recipe_payload) ----> 5 recipe_gcs.set_definition_and_payload(recipe_def) /opt/dataiku-dss-7.0.2/python/dataikuapi/dss/recipe.py in set_definition_and_payload(self, definition) 47 return self.client._perform_json( 48 "PUT", "/projects/%s/recipes/%s" % (self.project_key, self.recipe_name), ---> 49 body=definition.data) 50 51 ######################################################## /opt/dataiku-dss-7.0.2/python/dataikuapi/dssclient.py in _perform_json(self, method, path, params, body, files, raw_body) 918 919 def _perform_json(self, method, path, params=None, body=None,files=None, raw_body=None): --> 920 return self._perform_http(method, path, params=params, body=body, files=files, stream=False, raw_body=raw_body).json() 921 922 def _perform_raw(self, method, path, params=None, body=None,files=None, raw_body=None): /opt/dataiku-dss-7.0.2/python/dataikuapi/dssclient.py in _perform_http(self, method, path, params, body, stream, files, raw_body) 909 except ValueError: 910 ex = {"message": http_res.text} --> 911 raise DataikuException("%s: %s" % (ex.get("errorType", "Unknown error"), ex.get("message", "No message"))) 912 913 def _perform_empty(self, method, path, params=None, body=None, files = None, raw_body=None): DataikuException: java.lang.IllegalArgumentException: Cannot set unparseable payload on recipe MEIR_SANDBOX.compute_parcel_minus_building_refacto_gcs of type sync