Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hello!
Please excuse my long post but I recommend you read it fully to see what I am trying to accomplish!
TLDR:
My Goals:
Here is a taste of how I imagine the library for goal 2 to work. This could be changed I am up for any suggestions Dataiku has to offer!
# this calls the delete columns shaker in the prepare visual recipe.
dataset_obj = project_obj.getDataset("dataset_name")
dataset_obj.prepare.delete_columns(columns=["col2","col3"])
# keys are cell values and values in the dictionary are the replacement value
find_and_replace_params = {"sale":"sales", "datku":"dataiku"}
dataset_obj.prepare.find_and_replace(col_name="col1", find_and_replace_params)
My Dream:
My research and questions on how to accomplish my goal 1:
Hi.
So as you have commented, REST API just simply modify the JSON files.
We do not have detailed documentation on all the parameters of the steps, I think the best way would be to just create a prepare recipe with a bunch of steps and manually look through the JSON.
I believe you have already played with the Dataiku API, but here is a code snippet to retrieve the steps and how to modify the steps manually.
client = dataiku.api_client()
project = client.get_project(PROJECT_KEY)
recipe = project.get_recipe(recipe_name=RECIPE_NAME)
payload = recipe.get_definition_and_payload()
payload_json = payload.get_json_payload()
#modify payload_json["steps"][n_step]
payload.set_json_payload(payload_json)
YSL
Thank you!
I think there are a lot of comparisons with this idea to the Optimus library:
https://github.com/ironmussa/Optimus/
the only difference with my idea would be that we would wrap around the DSS internal library code visual recipes.