O365 connectors within Teams will be deprecated and notifications from this service
We setup a script to send out scenarios checking using get_message_sender()
& send()
APIs to team channel. Starting today, this message popup:
"Action Required: O365 connectors within Teams will be deprecated and notifications from this service will stop. Learn more about the timing and how the Workflows app provides a more flexible and secure experience. If you want to continue receiving these types of messages, you can use a workflow to post messages from a webhook request. Set up workflow"
It seems that Microsoft will deprecate O365 connectors starting from Aug, 2024 and retire this at Oct, 2024. Is there an easy solution to solve this in v12?
Operating system used: windows 11
Answers
-
Yes, I'm seeing the same thing, since we also use
get_message_sender()
andsend()
and I would like to understand how the integration between these APIs and Teams workflows should be configured? -
We are having the same scenario. Are there any plans to update Dataiku to allow integration with workflows instead of O365 connectors?
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,063 Neuron
I have raised this with Dataiku Support and they have confirmed the integration works when using Power Automate Workflows. The steps are pretty simple, I did myself on my organisation although I can't test it since we do not have permissions to run Power Automate Workflows. One thing is that even the integration works with Power Automate Workflows the reporters end in error due to the Power Automate Workflows returning HTTP code 202 rather than 200. Dataiku said they will fix this in an upcoming v13 patch release.
-
Hi @Turribeach ,
Any chance you could share the steps? I understand we still need to wait for the v13 patch release before it will work properly. -
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,063 Neuron
So in my organisation Teams you go to a channel, select the 3 dots, select Workflows, then search for webhook and select the "Post to a channel when a webhook request is received" template. Give it a name, click on Next abd then click on Add workflow. You should now get the "Your workflow has been created" message and the webhook URL you can use in Dataiku. To see the Workflows you created you can go to Workflows on the Teams left side bar.
-
Thanks @Turribeach !
I tried this and used the new webhook in thesend()
function, eg:message_sender.send(message=html_message, sendAsHTML=True, webhookUrl=<new webhook>)
but I guess it didn't work because of this issue where it returns HTTP code 202 rather than 200 that you mentioned. Will wait for the patch release 🤞 -
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,063 Neuron
What error do you get? Can you post the full stack trace? Are you able to trap the error? Did the message get delivered?
-
[2024/07/10-10:14:10.926] [qtp579447973-449766] [DEBUG] [dku.tracing] - [ct: 513] Done call: /api/tintercom/scenarios/send-message [POST] time=512ms user=stanislava.schaefer@olamagri.com auth=<AC:user:stanislava.schaefer@olamagri.com via: ticket:scenario_python_step:IRONOREFMRETURNSPREDICTION.CAPE_FM_DAILY_RUN,via: scenario-run: scenario=IRONOREFMRETURNSPREDICTION.CAPE_FM_DAILY_RUN runId=2024-07-10-10-14-06-390,via: stanislava.schaefer@olamagri.com> [variablesJson={} projectKey=IRONOREFMRETURNSPREDICTION] thrown=IOException [2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process] - Traceback (most recent call last): [2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process] - File "/data/dataiku/datadir/scenarios/IRONOREFMRETURNSPREDICTION/CAPE_FM_DAILY_RUN/2024-07-10-10-14-06-390/custom-step-Send Teams Message/script.py", line 143, in <module> [2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process] - message_sender.send(message=html_message, sendAsHTML=True, [2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process] - File "/home/dataiku/dataiku-dss-12.6.1/python/dataiku/scenario/messaging.py", line 35, in send [2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process] - return backend_json_call("scenarios/send-message", data) [2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process] - File "/home/dataiku/dataiku-dss-12.6.1/python/dataiku/core/intercom.py", line 550, in backend_json_call [2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process] - return _handle_json_resp(backend_api_post_call(path, data, **kwargs), err_msg = err_msg) [2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process] - File "/home/dataiku/dataiku-dss-12.6.1/python/dataiku/core/intercom.py", line 649, in _handle_json_resp [2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process] - raise Exception("%s: %s" % (err_msg, _get_error_message(err_data).encode("utf8"))) [2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process] - Exception: None: b'API call to post message on Teams failed 202: Accepted'
-
The message did not get delivered.
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,063 Neuron
Ummm that's not good. OK I just checked the instructions I got from Dataiku Support and they are slightly different. Rather than using an existing Workflow template. Can you test with these instructions instead?
Step 1 - Create a workflow in Microsoft Teams that posts a card to a channel when a
- In Microsoft Teams, open the Workflows apps from the left panel.
- Click "+ New flow"
- Click "+ Create from blank"
- Add a trigger When a Teams webhook request is received
- Who can trigger the flow: Anyone
- Click "+ New step"
- Add an action of type Control named Apply to each
- Select an output from previous steps: dynamic content "attachments"
- Click "Add an action"
- Select Post a card in a chat or channel operation
- Post as: Flow bot
- Post in: Channel
- Team: select a team
- Chanel: select the destination channel
- Adaptive Card: dynamic content "content"
- Save
- Expand the trigger When a Teams webhook request is received and copy the HTTP POST URL
Step 2 - Configure a Scenario reporter- In DSS, open a scenario and add a Microsoft Teams reporter
- Webhook URL: Paste the URL you just copied
- Message: Copy/paste the following snippet
{ "attachments":[ { "contentType":"application/json", "content":{ "$schema":"http://adaptivecards.io/schemas/adaptive-card.json", "type":"AdaptiveCard", "version":"1.0", "body":[ { "type":"Container", "items":[ { "type":"TextBlock", "text":"${scenarioName} run report", "weight":"bolder", "size":"medium", "style":"heading", "wrap":true, "color":"attention" }, { "type":"TextBlock", "wrap":true, "text":"[${scenarioName}](${scenarioRunURL}): **${outcome}**", "color":"good" }, { "type":"FactSet", "facts":[ { "title":"Project", "value":"${scenarioProjectKey}" }, { "title":"Triggered by", "value":"${triggerName}" } ] } ] } ] } } ] }
-
I followed the instructions and tested and I'm seeing the same issue:
2024/07/10-10:32:52.642] [FT-ScenarioThread-WMRZQVQZ-453044] [WARN] [dip.scenario.run_thread] scenario IRONOREFMRETURNSPREDICTION.CAPE_FM_DAILY_RUN#2024-07-10-10-31-27-126 - Failure during report on end of scenario java.io.IOException: API call to post message on Teams failed 202: Accepted at com.dataiku.dip.integrations.im.TeamsIntegrationHandler.sendNotification(TeamsIntegrationHandler.java:210) at com.dataiku.dip.integrations.im.TeamsIntegrationHandler.handleEvent(TeamsIntegrationHandler.java:162) at com.dataiku.dip.scheduler.reports.ScenarioReporterRunner.sendMessage(ScenarioReporterRunner.java:165) at com.dataiku.dip.scheduler.reports.ScenarioReporterRunner.sendOnEnd(ScenarioReporterRunner.java:98) at com.dataiku.dip.scheduler.ScenarioThread.execute(ScenarioThread.java:181) at com.dataiku.dip.futures.FutureThreadBase.run(FutureThreadBase.java:136)
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,063 Neuron
Hi, you are doing in code using the send() method right? Do you mind trying a Reporter in the Scenario settings? I believe this is what Dataiku Support tested and even though the Scenario Reporter ended in error it should deliver the message.
-
Hi, no in the above I tried exactly as outlined and used a Reporter in the Scenario settings.
-
Dear Microsoft Teams users,
I confirm that the new webhook generated by Workflows/PowerAutomate is not fully compatible with the DSS/Teams integration in the existing DSS releases. The reason is that they changed the payload of the JSON expected by the Webhook.
What works (kind of):
- Scenario reporter with custom JSON message (see the example posted above: { "attachments": […] }). The reporter will be marked as failed because DSS does not recognize HTTP 202 as a success but the message is correctly delivered on the Teams channel
- Send a message scenario step with custom JSON message (same as above, except if will mark the scenario as failed, even though message will be delivered on the Teams channel)
What does not work:
- Scenario reporter with non-JSON message
- "Send a message" step with non-JSON message
- Python API "message_sender.send()"
The good news is that this will be fixed in the upcoming DSS 13.0.2. Stay tuned.
-
Thanks @apichery
Looking forward to the fix in DSS 13.0.2 - will this be available for cloud platform users as well?Regarding:
- Scenario reporter with custom JSON message (see the example posted above:
{ "attachments": […] }
). The reporter will be marked as failed because DSS does not recognize HTTP 202 as a success but the message is correctly delivered on the Teams channel
» This did not work for me. I don't know if there are some limitations to posting in private vs public Teams channels however? I tested it for a private channel.
- Scenario reporter with custom JSON message (see the example posted above:
-
Thank you. This saved me a lot of time 😀
-
Small update… the support for Workflows webhooks will actually be added to DSS 12.6.5 and 13.0.1.
-
For the bad news…Workflows in Microsoft Teams are currently unable to send message to private channels.
So DSS - as any other tool - will be unable to send message to private channels until Microsoft solve this.(Search for "private" in the comments on the thread with the retirement announce:
) -
Starting with DSS 12.6.5 and 13.0.1, you'll have to write the following to send a message through a webhook based on Workflows.
message_sender.send(message=…, webhookUrl=<new webhook>, webhookType="WORKFLOWS")
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,063 Neuron
Smashing news, many thanks for backporting this fix. We will try to put pressure on Microsoft to fix the private channel issue.
-
Yeah this is a bummer, and given I doubt Microsoft will change this, I'll need to move to a chat rather than a public channel
-
… and 12.6.5 has been released. Enjoy your new Microsoft Teams webhooks (except for those posting to private channels)
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,063 Neuron
For anyone on this thread it might be worth up voting this feedback suggestion to prevent the retirement of Teams Webhooks:
https://feedbackportal.microsoft.com/feedback/idea/80ed6877-b642-ef11-b4ad-000d3a7aba8b
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,063 Neuron
Microsoft posted an update on their blog post:
Basically they now deprecate this functionality at the end of 2025 but you will need to update the Webhooks URL before the end of 2024 and the new URL will be provided 90 days before that. But they will still stop allowing new Teams Webhooks being created from August 15th.
There are also updates to this post in that all the alternatives provided are not good replacements:https://feedbackportal.microsoft.com/feedback/idea/80ed6877-b642-ef11-b4ad-000d3a7aba8b
-
After some struggles I've found that the new Workflows capability is just barely fit for purpose - as long as you're setting the webhook up on a non-private channel, you can configure the Flow to post to the channel as Flow Bot. For my user group and organisation that is sufficient, but I guess in principle some people will need the ability to restrict visibility of the notifications.
-
Thanks @apichery
I'm able to usemessage_sender.send()
as before. Unfortunately however my HTML formatting is not showing up correctly in the Teams message. Do you have any advice for how to fix this part? Thanks again for the upgrades. -
Nevermind. I just discovered that MS Teams does not render HTML in adaptive cards…ugh. I guess the solution is to use Markdown formatting: Text formatting in cards - Teams | Microsoft Learn