Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hi Dataikers,
The same code works for SQL recipe. But I am having a hard time applying it to the sync recipe
builder_gcs = None
builder_gcs = SyncRecipeCreator(f"compute_{dataset_name}_gcs", project=dataiku_project)
builder_gcs = builder_gcs.with_input(dataset_name, dataiku_project_name)
builder_gcs = builder_gcs.with_new_output(f"{dataset_name}_gcs", GCS_CONNECTION)
recipe_gcs = builder_gcs.build()
recipe_def = recipe_gcs.get_definition_and_payload()
recipe_payload = recipe_def.get_payload()
# Modifications on definition - but does not work without modifs too
recipe_def.set_payload(recipe_payload)
recipe_gcs.set_definition_and_payload(recipe_def)
Error: DataikuException: java.lang.IllegalArgumentException: Cannot set unparseable payload on recipe recipe_name of type sync
Even though I did not do any changes on payload (and payload is empty string for sync anyway).
Any ideas?
Hi,
Can you please confirm your exact DSS version? Since get_definition_and_payload() was deprecated.
You can instead try using get_settings()
Regardless. on DSS 9.0.5 . I wasn't able to replicate the issue with similar code :
As you can see the Update recipe worked without any exception only warnings are related to the deprecations.
I have DSS version 7.0.2
I am starting to think that it is because DSS cannot find the recipe.
Cause among the errors I have:
HTTPError: 400 Client Error: Bad Request for url: http://127.0.0.1:10001/dip/publicapi/projects/MEIR_SANDBOX/recipes/compute_parcel_minus_building_ref...
I tested by giving complete random name of recipe, which for sure does not exists and I am receiving the same error.
But it is the case only for the SYNC recipe and code works fine with other recipe types.
In this case another question raises - what can be the reason for DSS for not finding sync recipes?
Hi,
Does the following work for an existing Sync recipe?
recipe = project.get_recipe("myrecipe")
status = recipe.get_status()
print(status)
Thanks,
I have a test instance of DSS v 9.0.4
I confirm that it works there too
Can you try adding :
recipe_gcs = dataiku_project.get_recipe(f"compute_{dataset_name}_gcs")
Before :
recipe_def = recipe_gcs.get_definition_and_payload()
recipe_payload = recipe_def.get_payload()
recipe_def.set_payload(recipe_payload)
recipe_gcs.set_definition_and_payload(recipe_def)
In your original code and see if this still fails on DSS7 does your original code fail on DSS 9.0.4 as well?
Thanks
Yes I did.
Whataver option of accessing sync recipe I try, it works on DSS 9.0.4 and fails on DSS 7.0.2
Full error if it helps:
2021-10-28 17:56:14 http://127.0.0.1:10001 "GET /dip/publicapi/projects/MEIR_SANDBOX/recipes/compute_parcel_minus_building_refacto_gcs HTTP/1.1" 200 1500 2021-10-28 17:56:14 http://127.0.0.1:10001 "PUT /dip/publicapi/projects/MEIR_SANDBOX/recipes/compute_parcel_minus_building_refacto_gcs HTTP/1.1" 400 None
--------------------------------------------------------------------------- HTTPError Traceback (most recent call last) /opt/dataiku-dss-7.0.2/python/dataikuapi/dssclient.py in _perform_http(self, method, path, params, body, stream, files, raw_body) 903 stream = stream) --> 904 http_res.raise_for_status() 905 return http_res /data/DSS/5.1/code-envs/python/Python36_2/lib/python3.6/site-packages/requests/models.py in raise_for_status(self) 939 if http_error_msg: --> 940 raise HTTPError(http_error_msg, response=self) 941 HTTPError: 400 Client Error: Bad Request for url: http://127.0.0.1:10001/dip/publicapi/projects/MEIR_SANDBOX/recipes/compute_parcel_minus_building_ref... During handling of the above exception, another exception occurred: DataikuException Traceback (most recent call last) <ipython-input-70-d85988868863> in <module> 3 recipe_payload = recipe_def.get_payload() 4 recipe_def.set_payload(recipe_payload) ----> 5 recipe_gcs.set_definition_and_payload(recipe_def) /opt/dataiku-dss-7.0.2/python/dataikuapi/dss/recipe.py in set_definition_and_payload(self, definition) 47 return self.client._perform_json( 48 "PUT", "/projects/%s/recipes/%s" % (self.project_key, self.recipe_name), ---> 49 body=definition.data) 50 51 ######################################################## /opt/dataiku-dss-7.0.2/python/dataikuapi/dssclient.py in _perform_json(self, method, path, params, body, files, raw_body) 918 919 def _perform_json(self, method, path, params=None, body=None,files=None, raw_body=None): --> 920 return self._perform_http(method, path, params=params, body=body, files=files, stream=False, raw_body=raw_body).json() 921 922 def _perform_raw(self, method, path, params=None, body=None,files=None, raw_body=None): /opt/dataiku-dss-7.0.2/python/dataikuapi/dssclient.py in _perform_http(self, method, path, params, body, stream, files, raw_body) 909 except ValueError: 910 ex = {"message": http_res.text} --> 911 raise DataikuException("%s: %s" % (ex.get("errorType", "Unknown error"), ex.get("message", "No message"))) 912 913 def _perform_empty(self, method, path, params=None, body=None, files = None, raw_body=None): DataikuException: java.lang.IllegalArgumentException: Cannot set unparseable payload on recipe MEIR_SANDBOX.compute_parcel_minus_building_refacto_gcs of type sync