Using Dataiku
- Hi, I am trying to set up dataiku API to send email to custom users with custom message once endpint gets triggered. because scenarios can not run in parallel i would not be able to use the following …Last answer by Sarina
Hi @NiloofarDadras
,
If you want to send an email from an API endpoint, it's probably easiest to do this simply through a Python function in the API endpoint, and use Python to send the email. You can likely use the Python email package or any similar package.
Here is an example:Here is the example Python function endpoint code where my SMTP server is running on 0.0.0.0:1025:
import smtplib # Import the email modules we'll need from email.message import EmailMessage # Insert here initialization code # This function is the entry point for your Python function API # You can call this function with passing "param1", "param2", # "param3" as arguments to the API call # (see REST API documentation) def api_py_function(): print('do some stuff') print('sending email') msg = EmailMessage() msg.set_content('Your endpoint executed with the following results...') msg['Subject'] = 'Endpoint email' msg['From'] = 'sarina@dataiku.com' msg['To'] = 'sarina@dataiku.com' # Send the message via our own SMTP server. s = smtplib.SMTP('0.0.0.0:1025') s.send_message(msg) s.quit() return {}
Thanks,
SarinaLast answer by SarinaHi @NiloofarDadras
,
If you want to send an email from an API endpoint, it's probably easiest to do this simply through a Python function in the API endpoint, and use Python to send the email. You can likely use the Python email package or any similar package.
Here is an example:Here is the example Python function endpoint code where my SMTP server is running on 0.0.0.0:1025:
import smtplib # Import the email modules we'll need from email.message import EmailMessage # Insert here initialization code # This function is the entry point for your Python function API # You can call this function with passing "param1", "param2", # "param3" as arguments to the API call # (see REST API documentation) def api_py_function(): print('do some stuff') print('sending email') msg = EmailMessage() msg.set_content('Your endpoint executed with the following results...') msg['Subject'] = 'Endpoint email' msg['From'] = 'sarina@dataiku.com' msg['To'] = 'sarina@dataiku.com' # Send the message via our own SMTP server. s = smtplib.SMTP('0.0.0.0:1025') s.send_message(msg) s.quit() return {}
Thanks,
Sarina - Hi, I am creating a Spark recipe. I have connected the relevant tables to the input and output of recipe. But when I run the recipe (complex), it failed while writing the output to one table (the reci…Last answer by Catalina
Hi @Dawood154
,The error indicates Caused by: io.netty.channel.StacklessClosedChannelException. This may occur due to broken connection or other network issues.
Last answer by CatalinaHi @Dawood154
,The error indicates Caused by: io.netty.channel.StacklessClosedChannelException. This may occur due to broken connection or other network issues.
- Hi All, I am trying to understand how virtualization in DSS works. In the following example, SQL pipelines are enabled and virtualization is allowed for 'split_1' and 'split_2'. When building 'stacked…Solution bySolution by Sarina
Hi @yashpuranik
,
Thank you you for query and examples! Indeed this is expected behavior.
When using SQL pipelines, the initial "input" datasets and the final "output" datasets cannot be virtualized. So if a dataset is not intermediate (i.e. it doesn't exist in the flow between two datasets), then it's not possible to virtualize the dataset. Virtualization usually only makes sense in the context that the data will eventually be used in an ultimate written dataset. So the assumption is that all output datasets do need to be created. Rebuliding the flow will rebuild the split recipe, so any output datasets of the split recipe will be created. If you truly weren't going to use the data in "split_2", then selecting the split option "drop data" instead of creating the dataset "split_2" would probably be the best option.
Let me know if that doesn't make sense or if you have any additional questions.
Thanks,
Sarina - Hi, Thanks for help I need to create 20 logistic models from one dataset itself using the same process. Please help me with thisLast answer byLast answer by Zach
Hi @jaga
,Documentation on how to create, train, and deploy models via Python is available here: https://doc.dataiku.com/dss/latest/python-api/ml.html
If you have any questions about the process, feel free to ask. Please provide your code when doing so.
Thanks,
Zach
- I am working on the project and I have many recipes containing the same steps. How can I make a "template" of this recipe so as to use it across the flow? Operating system used: Mac OSLast answer byLast answer by Miguel Angel
Hi,
There are a number of methodologies available for reusing code within DSS. In the context of the Prepare recipe, the most straightforward are:
1) Copy/paste the Prepare steps into other Prepare recipes
2) Copy the entire Prepare recipe
3) Copy the subflow
Select the Prepare recipe > Flow Actions > Copy Subflow > Copy To
- * The x-axis is date and it is coming in jumbled order * The x-axis is string but in different charts the same values are coming in different order How can this be fixed? Operating system used: Window…Solution bySolution by Zach
Hi @msr
,You can change the sort order of the X-axis by clicking the ▾ arrow next to the column name in the chart editor. You can sort by natural (alphabetical) order, or sort based on the record count.
Also, for your chart that's using a date, I could be wrong, but it looks like the type of the "Metric date" column might not actually be a date. Could you please verify that the type is correct in the dataset?
If it's not a date, you might need to parse the date first. See this article for more information: Managing dates.
Thanks,
Zach
- Is it possible to apply a preparation script directly from the dataiku API? inside a notebook i have this (inputdataset and ModelID has been defined in prior cell): ... df=inputdataset.get_dataframe()…Solution bySolution by Sarina
Hi @Echternacht
,
There are two options that I see here.
When you train a model using Visual ML, the preparation script will automatically be applied to the input dataset. If you want to simply run a training, then indeed you could do this from the API:import dataiku client = dataiku.api_client() project = client.get_default_project() model = project.get_saved_model('MODEL_ID') # to get the list of ML tasks, to pick for the next step analysis.list_ml_tasks() ml_task = analysis.get_ml_task('PREVIOUS_ML_TASK_RESULT') train_ml_task = ml_task.train()
This will simply perform a training, which will also encompass running the preparation script set in the model analysis screen.
The other option would be to deploy your script to the flow as a recipe:
Then, you can simply run the recipe from the API, and use the output dataset of the recipe as your input to your predictor.predict() function. For example, I've deployed my script as the recipe "compute_training_prepared_final" here:In my Python script I can then run:
import dataiku from dataiku import pandasutils as pdu import pandas as pd client = dataiku.api_client() project = client.get_default_project() # get recipe recipe = project.get_recipe('compute_training_prepared_modified') # get model model = dataiku.Model('MODEL') predictor = model.get_predictor() # get the output dataset of the deployed script single_recipe_output = recipe.get_settings().get_recipe_outputs()['main']['items'][0]['ref'] # run the deployed script recipe.run() # get output df output_dataset = dataiku.Dataset(single_recipe_output) output_df = output_dataset.get_dataframe() # now you can run predictor.predict() on the output dataset of the deployed script predictor.predict(output_df)
Thanks,
Sarina - Hi all, If I convert numbers into Italian format the data with the , decimal are displayed as strings and misaligned. How can I align the resulting strings right. Thanks for your help Operating system…Last answer byLast answer by Zach
Hi @rbonazzoBfw
,If I'm understanding correctly, the numbers render correctly like shown in my screenshots when you first upload them, but the format breaks after you convert them to Italian format?
Could you please provide more information about how exactly you're converting the numbers? For example, if you're using a Prepare recipe, could you please post a screenshot that shows the step that converts the numbers and also shows your column headers?
Thanks,
Zach
- I have created a scenario, which I call from within the Application Designer. I then create an instance of the project, and execute the scenario from within the instance. Although I can see the scenar…Last answer byLast answer by Alexandru
Hi @Erlebacher
,Did you create an App-as-recipe? If you did you can keep the temporary project instance and its log for investigation by selecting "keep Instance for investigating failures"
Let me know if that helps. If not and you already have the scenario logs you can view the job logs from a specific job by clicking into the particular step and going to the actual job log by clicking on "sched_build.."
Kind Regards,
- Hi, I want to attach a .pptx file to a Wiki article using the Python API. To do this, I added the file to a managed folder and then tried the following (example found in the Documentation, Wikis — Dat…Solution by