O365 connectors within Teams will be deprecated and notifications from this service

yfzh · July 5

We setup a script to send out scenarios checking using get_message_sender() & send() APIs to team channel. Starting today, this message popup:

"Action Required: O365 connectors within Teams will be deprecated and notifications from this service will stop. Learn more about the timing and how the Workflows app provides a more flexible and secure experience. If you want to continue receiving these types of messages, you can use a workflow to post messages from a webhook request. Set up workflow"

It seems that Microsoft will deprecate O365 connectors starting from Aug, 2024 and retire this at Oct, 2024. Is there an easy solution to solve this in v12?

Operating system used: windows 11

Stanislava · July 8

Yes, I'm seeing the same thing, since we also useget_message_sender() and send()and I would like to understand how the integration between these APIs and Teams workflows should be configured?

Jorge · July 9

We are having the same scenario. Are there any plans to update Dataiku to allow integration with workflows instead of O365 connectors?

Turribeach · July 10

I have raised this with Dataiku Support and they have confirmed the integration works when using Power Automate Workflows. The steps are pretty simple, I did myself on my organisation although I can't test it since we do not have permissions to run Power Automate Workflows. One thing is that even the integration works with Power Automate Workflows the reporters end in error due to the Power Automate Workflows returning HTTP code 202 rather than 200. Dataiku said they will fix this in an upcoming v13 patch release.

Stanislava · July 10

Hi @Turribeach ,
Any chance you could share the steps? I understand we still need to wait for the v13 patch release before it will work properly.

Turribeach · July 10

https://community.dataiku.com/discussion/comment/43960#Comment_43960

So in my organisation Teams you go to a channel, select the 3 dots, select Workflows, then search for webhook and select the "Post to a channel when a webhook request is received" template. Give it a name, click on Next abd then click on Add workflow. You should now get the "Your workflow has been created" message and the webhook URL you can use in Dataiku. To see the Workflows you created you can go to Workflows on the Teams left side bar.

Stanislava · July 10

https://community.dataiku.com/discussion/comment/43962#Comment_43962

Thanks @Turribeach !
I tried this and used the new webhook in the send() function, eg:
message_sender.send(message=html_message, sendAsHTML=True, webhookUrl=<new webhook>)
but I guess it didn't work because of this issue where it returns HTTP code 202 rather than 200 that you mentioned. Will wait for the patch release 🤞

Turribeach · July 10

https://community.dataiku.com/discussion/comment/43963#Comment_43963

What error do you get? Can you post the full stack trace? Are you able to trap the error? Did the message get delivered?

Stanislava · July 10

https://community.dataiku.com/discussion/comment/43964#Comment_43964

[2024/07/10-10:14:10.926] [qtp579447973-449766] [DEBUG] [dku.tracing]  - [ct: 513] Done call: /api/tintercom/scenarios/send-message [POST] time=512ms user=stanislava.schaefer@olamagri.com auth=<AC:user:stanislava.schaefer@olamagri.com via: ticket:scenario_python_step:IRONOREFMRETURNSPREDICTION.CAPE_FM_DAILY_RUN,via: scenario-run: scenario=IRONOREFMRETURNSPREDICTION.CAPE_FM_DAILY_RUN runId=2024-07-10-10-14-06-390,via: stanislava.schaefer@olamagri.com> [variablesJson={} projectKey=IRONOREFMRETURNSPREDICTION] thrown=IOException
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  - Traceback (most recent call last):
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -   File "/data/dataiku/datadir/scenarios/IRONOREFMRETURNSPREDICTION/CAPE_FM_DAILY_RUN/2024-07-10-10-14-06-390/custom-step-Send Teams Message/script.py", line 143, in <module>
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -     message_sender.send(message=html_message, sendAsHTML=True,
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -   File "/home/dataiku/dataiku-dss-12.6.1/python/dataiku/scenario/messaging.py", line 35, in send
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -     return backend_json_call("scenarios/send-message", data)
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -   File "/home/dataiku/dataiku-dss-12.6.1/python/dataiku/core/intercom.py", line 550, in backend_json_call
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -     return _handle_json_resp(backend_api_post_call(path, data, **kwargs), err_msg = err_msg)
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -   File "/home/dataiku/dataiku-dss-12.6.1/python/dataiku/core/intercom.py", line 649, in _handle_json_resp
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -     raise Exception("%s: %s" % (err_msg, _get_error_message(err_data).encode("utf8")))
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  - Exception: None: b'API call to post message on Teams failed 202: Accepted'

Stanislava · July 10

https://community.dataiku.com/discussion/comment/43965#Comment_43965

The message did not get delivered.

Turribeach · July 10

https://community.dataiku.com/discussion/comment/43966#Comment_43966

Ummm that's not good. OK I just checked the instructions I got from Dataiku Support and they are slightly different. Rather than using an existing Workflow template. Can you test with these instructions instead?

Step 1 - Create a workflow in Microsoft Teams that posts a card to a channel when a

In Microsoft Teams, open the Workflows apps from the left panel.
Click "+ New flow"
Click "+ Create from blank"
Add a trigger When a Teams webhook request is received
- Who can trigger the flow: Anyone
Click "+ New step"
Add an action of type Control named Apply to each
- Select an output from previous steps: dynamic content "attachments"
Click "Add an action"
Select Post a card in a chat or channel operation
- Post as: Flow bot
- Post in: Channel
- Team: select a team
- Chanel: select the destination channel
- Adaptive Card: dynamic content "content"
Save
Expand the trigger When a Teams webhook request is received and copy the HTTP POST URL

Step 2 - Configure a Scenario reporter

In DSS, open a scenario and add a Microsoft Teams reporter
- Webhook URL: Paste the URL you just copied
- Message: Copy/paste the following snippet

{
"attachments":[
{
"contentType":"application/json",
"content":{
"$schema":"http://adaptivecards.io/schemas/adaptive-card.json",
"type":"AdaptiveCard",
"version":"1.0",
"body":[
{
"type":"Container",
"items":[
{
"type":"TextBlock",
"text":"${scenarioName} run report",
"weight":"bolder",
"size":"medium",
"style":"heading",
"wrap":true,
"color":"attention"
},
{
"type":"TextBlock",
"wrap":true,
"text":"[${scenarioName}](${scenarioRunURL}): **${outcome}**",
"color":"good"
},
{
"type":"FactSet",
"facts":[
{
"title":"Project",
"value":"${scenarioProjectKey}"
},
{
"title":"Triggered by",
"value":"${triggerName}"
}
]
}
]
}
]
}
}
]
}

Stanislava · July 10

https://community.dataiku.com/discussion/comment/43967#Comment_43967

I followed the instructions and tested and I'm seeing the same issue:

2024/07/10-10:32:52.642] [FT-ScenarioThread-WMRZQVQZ-453044] [WARN] [dip.scenario.run_thread] scenario IRONOREFMRETURNSPREDICTION.CAPE_FM_DAILY_RUN#2024-07-10-10-31-27-126 - Failure during report on end of scenario
java.io.IOException: API call to post message on Teams failed 202: Accepted
	at com.dataiku.dip.integrations.im.TeamsIntegrationHandler.sendNotification(TeamsIntegrationHandler.java:210)
	at com.dataiku.dip.integrations.im.TeamsIntegrationHandler.handleEvent(TeamsIntegrationHandler.java:162)
	at com.dataiku.dip.scheduler.reports.ScenarioReporterRunner.sendMessage(ScenarioReporterRunner.java:165)
	at com.dataiku.dip.scheduler.reports.ScenarioReporterRunner.sendOnEnd(ScenarioReporterRunner.java:98)
	at com.dataiku.dip.scheduler.ScenarioThread.execute(ScenarioThread.java:181)
	at com.dataiku.dip.futures.FutureThreadBase.run(FutureThreadBase.java:136)

Turribeach · July 10

https://community.dataiku.com/discussion/comment/43968#Comment_43968

Hi, you are doing in code using the send() method right? Do you mind trying a Reporter in the Scenario settings? I believe this is what Dataiku Support tested and even though the Scenario Reporter ended in error it should deliver the message.

Stanislava · July 10

https://community.dataiku.com/discussion/comment/43969#Comment_43969

Hi, no in the above I tried exactly as outlined and used a Reporter in the Scenario settings.

apichery · July 10

Dear Microsoft Teams users,

I confirm that the new webhook generated by Workflows/PowerAutomate is not fully compatible with the DSS/Teams integration in the existing DSS releases. The reason is that they changed the payload of the JSON expected by the Webhook.

What works (kind of):

Scenario reporter with custom JSON message (see the example posted above: { "attachments": […] }). The reporter will be marked as failed because DSS does not recognize HTTP 202 as a success but the message is correctly delivered on the Teams channel
Send a message scenario step with custom JSON message (same as above, except if will mark the scenario as failed, even though message will be delivered on the Teams channel)

What does not work:

Scenario reporter with non-JSON message
"Send a message" step with non-JSON message
Python API "message_sender.send()"

The good news is that this will be fixed in the upcoming DSS 13.0.2. Stay tuned.

Stanislava · July 10

Thanks @apichery
Looking forward to the fix in DSS 13.0.2 - will this be available for cloud platform users as well?

Regarding:

Scenario reporter with custom JSON message (see the example posted above: { "attachments": […] }). The reporter will be marked as failed because DSS does not recognize HTTP 202 as a success but the message is correctly delivered on the Teams channel
» This did not work for me. I don't know if there are some limitations to posting in private vs public Teams channels however? I tested it for a private channel.

me2 · July 10

https://community.dataiku.com/discussion/comment/43976#Comment_43976

Thank you. This saved me a lot of time 😀

apichery · July 10

https://community.dataiku.com/discussion/comment/43977#Comment_43977

https://community.dataiku.com/discussion/comment/43976#Comment_43976

Small update… the support for Workflows webhooks will actually be added to DSS 12.6.5 and 13.0.1.

apichery · July 10

https://community.dataiku.com/discussion/comment/43978#Comment_43978

For the bad news…Workflows in Microsoft Teams are currently unable to send message to private channels.
So DSS - as any other tool - will be unable to send message to private channels until Microsoft solve this.

(Search for "private" in the comments on the thread with the retirement announce: https://devblogs.microsoft.com/microsoft365dev/retirement-of-office-365-connectors-within-microsoft-teams/ )

apichery · July 10

https://community.dataiku.com/discussion/comment/43963#Comment_43963

Starting with DSS 12.6.5 and 13.0.1, you'll have to write the following to send a message through a webhook based on Workflows.

message_sender.send(message=…, webhookUrl=<new webhook>, webhookType="WORKFLOWS")

Turribeach · July 10

https://community.dataiku.com/discussion/comment/43983#Comment_43983

Smashing news, many thanks for backporting this fix. We will try to put pressure on Microsoft to fix the private channel issue.

Stanislava · July 11

https://community.dataiku.com/discussion/comment/43982#Comment_43982

Yeah this is a bummer, and given I doubt Microsoft will change this, I'll need to move to a chat rather than a public channel

apichery · July 11

… and 12.6.5 has been released. Enjoy your new Microsoft Teams webhooks (except for those posting to private channels)

Turribeach · July 16

For anyone on this thread it might be worth up voting this feedback suggestion to prevent the retirement of Teams Webhooks:

https://feedbackportal.microsoft.com/feedback/idea/80ed6877-b642-ef11-b4ad-000d3a7aba8b

Turribeach · July 25

Microsoft posted an update on their blog post:

https://devblogs.microsoft.com/microsoft365dev/retirement-of-office-365-connectors-within-microsoft-teams/

Basically they now deprecate this functionality at the end of 2025 but you will need to update the Webhooks URL before the end of 2024 and the new URL will be provided 90 days before that. But they will still stop allowing new Teams Webhooks being created from August 15th.

There are also updates to this post in that all the alternatives provided are not good replacements:

https://feedbackportal.microsoft.com/feedback/idea/80ed6877-b642-ef11-b4ad-000d3a7aba8b

bdforbes · September 26

After some struggles I've found that the new Workflows capability is just barely fit for purpose - as long as you're setting the webhook up on a non-private channel, you can configure the Flow to post to the channel as Flow Bot. For my user group and organisation that is sufficient, but I guess in principle some people will need the ability to restrict visibility of the notifications.

Stanislava · September 27

https://community.dataiku.com/discussion/comment/43983#Comment_43983

Thanks @apichery
I'm able to use message_sender.send()as before. Unfortunately however my HTML formatting is not showing up correctly in the Teams message. Do you have any advice for how to fix this part? Thanks again for the upgrades.

Stanislava · September 27

https://community.dataiku.com/discussion/comment/44567#Comment_44567

Nevermind. I just discovered that MS Teams does not render HTML in adaptive cards…ugh. I guess the solution is to use Markdown formatting: Text formatting in cards - Teams | Microsoft Learn

O365 connectors within Teams will be deprecated and notifications from this service

Answers

Categories

Setup Info

Tags