O365 connectors within Teams will be deprecated and notifications from this service

yfzh

We setup a script to send out scenarios checking using get_message_sender() & send() APIs to team channel. Starting today, this message popup:

"Action Required: O365 connectors within Teams will be deprecated and notifications from this service will stop. Learn more about the timing and how the Workflows app provides a more flexible and secure experience. If you want to continue receiving these types of messages, you can use a workflow to post messages from a webhook request. Set up workflow"

It seems that Microsoft will deprecate O365 connectors starting from Aug, 2024 and retire this at Oct, 2024. Is there an easy solution to solve this in v12?

Operating system used: windows 11

Stanislava

Yes, I'm seeing the same thing, since we also useget_message_sender() and send()and I would like to understand how the integration between these APIs and Teams workflows should be configured?

Jorge

We are having the same scenario. Are there any plans to update Dataiku to allow integration with workflows instead of O365 connectors?

Turribeach

I have raised this with Dataiku Support and they have confirmed the integration works when using Power Automate Workflows. The steps are pretty simple, I did myself on my organisation although I can't test it since we do not have permissions to run Power Automate Workflows. One thing is that even the integration works with Power Automate Workflows the reporters end in error due to the Power Automate Workflows returning HTTP code 202 rather than 200. Dataiku said they will fix this in an upcoming v13 patch release.

Stanislava

Hi @Turribeach ,
Any chance you could share the steps? I understand we still need to wait for the v13 patch release before it will work properly.

Turribeach

https://community.dataiku.com/discussion/comment/43960#Comment_43960

So in my organisation Teams you go to a channel, select the 3 dots, select Workflows, then search for webhook and select the "Post to a channel when a webhook request is received" template. Give it a name, click on Next abd then click on Add workflow. You should now get the "Your workflow has been created" message and the webhook URL you can use in Dataiku. To see the Workflows you created you can go to Workflows on the Teams left side bar.

Stanislava

https://community.dataiku.com/discussion/comment/43962#Comment_43962

Thanks @Turribeach !
I tried this and used the new webhook in the send() function, eg:
message_sender.send(message=html_message, sendAsHTML=True, webhookUrl=<new webhook>)
but I guess it didn't work because of this issue where it returns HTTP code 202 rather than 200 that you mentioned. Will wait for the patch release 🤞

Turribeach

https://community.dataiku.com/discussion/comment/43963#Comment_43963

What error do you get? Can you post the full stack trace? Are you able to trap the error? Did the message get delivered?

Stanislava

https://community.dataiku.com/discussion/comment/43964#Comment_43964

[2024/07/10-10:14:10.926] [qtp579447973-449766] [DEBUG] [dku.tracing]  - [ct: 513] Done call: /api/tintercom/scenarios/send-message [POST] time=512ms user=stanislava.schaefer@olamagri.com auth=<AC:user:stanislava.schaefer@olamagri.com via: ticket:scenario_python_step:IRONOREFMRETURNSPREDICTION.CAPE_FM_DAILY_RUN,via: scenario-run: scenario=IRONOREFMRETURNSPREDICTION.CAPE_FM_DAILY_RUN runId=2024-07-10-10-14-06-390,via: stanislava.schaefer@olamagri.com> [variablesJson={} projectKey=IRONOREFMRETURNSPREDICTION] thrown=IOException
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  - Traceback (most recent call last):
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -   File "/data/dataiku/datadir/scenarios/IRONOREFMRETURNSPREDICTION/CAPE_FM_DAILY_RUN/2024-07-10-10-14-06-390/custom-step-Send Teams Message/script.py", line 143, in <module>
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -     message_sender.send(message=html_message, sendAsHTML=True,
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -   File "/home/dataiku/dataiku-dss-12.6.1/python/dataiku/scenario/messaging.py", line 35, in send
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -     return backend_json_call("scenarios/send-message", data)
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -   File "/home/dataiku/dataiku-dss-12.6.1/python/dataiku/core/intercom.py", line 550, in backend_json_call
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -     return _handle_json_resp(backend_api_post_call(path, data, **kwargs), err_msg = err_msg)
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -   File "/home/dataiku/dataiku-dss-12.6.1/python/dataiku/core/intercom.py", line 649, in _handle_json_resp
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  -     raise Exception("%s: %s" % (err_msg, _get_error_message(err_data).encode("utf8")))
[2024/07/10-10:14:10.929] [Exec-450965] [INFO] [process]  - Exception: None: b'API call to post message on Teams failed 202: Accepted'

Stanislava

https://community.dataiku.com/discussion/comment/43965#Comment_43965

The message did not get delivered.

Turribeach

https://community.dataiku.com/discussion/comment/43966#Comment_43966

Ummm that's not good. OK I just checked the instructions I got from Dataiku Support and they are slightly different. Rather than using an existing Workflow template. Can you test with these instructions instead?

Step 1 - Create a workflow in Microsoft Teams that posts a card to a channel when a 

• In Microsoft Teams, open the Workflows apps from the left panel.
• Click "+ New flow"
• Click "+ Create from blank"
• Add a trigger When a Teams webhook request is received
  • Who can trigger the flow: Anyone
• Click "+ New step"
• Add an action of type Control named Apply to each
  • Select an output from previous steps: dynamic content "attachments"
• Click "Add an action"
• Select Post a card in a chat or channel operation
  • Post as: Flow bot
  • Post in: Channel
  • Team: select a team
  • Chanel: select the destination channel
  • Adaptive Card: dynamic content "content"
• Save
• Expand the trigger When a Teams webhook request is received and copy the HTTP POST URL

Step 2 - Configure a Scenario reporter

• In DSS, open a scenario and add a Microsoft Teams reporter
  • Webhook URL: Paste the URL you just copied
  • Message: Copy/paste the following snippet 

{
"attachments":[
{
"contentType":"application/json",
"content":{
"$schema":"http://adaptivecards.io/schemas/adaptive-card.json",
"type":"AdaptiveCard",
"version":"1.0",
"body":[
{
"type":"Container",
"items":[
{
"type":"TextBlock",
"text":"${scenarioName} run report",
"weight":"bolder",
"size":"medium",
"style":"heading",
"wrap":true,
"color":"attention"
},
{
"type":"TextBlock",
"wrap":true,
"text":"[${scenarioName}](${scenarioRunURL}): **${outcome}**",
"color":"good"
},
{
"type":"FactSet",
"facts":[
{
"title":"Project",
"value":"${scenarioProjectKey}"
},
{
"title":"Triggered by",
"value":"${triggerName}"
}
]
}
]
}
]
}
}
]
}

Stanislava

https://community.dataiku.com/discussion/comment/43967#Comment_43967

I followed the instructions and tested and I'm seeing the same issue:

2024/07/10-10:32:52.642] [FT-ScenarioThread-WMRZQVQZ-453044] [WARN] [dip.scenario.run_thread] scenario IRONOREFMRETURNSPREDICTION.CAPE_FM_DAILY_RUN#2024-07-10-10-31-27-126 - Failure during report on end of scenario
java.io.IOException: API call to post message on Teams failed 202: Accepted
	at com.dataiku.dip.integrations.im.TeamsIntegrationHandler.sendNotification(TeamsIntegrationHandler.java:210)
	at com.dataiku.dip.integrations.im.TeamsIntegrationHandler.handleEvent(TeamsIntegrationHandler.java:162)
	at com.dataiku.dip.scheduler.reports.ScenarioReporterRunner.sendMessage(ScenarioReporterRunner.java:165)
	at com.dataiku.dip.scheduler.reports.ScenarioReporterRunner.sendOnEnd(ScenarioReporterRunner.java:98)
	at com.dataiku.dip.scheduler.ScenarioThread.execute(ScenarioThread.java:181)
	at com.dataiku.dip.futures.FutureThreadBase.run(FutureThreadBase.java:136)

Turribeach

https://community.dataiku.com/discussion/comment/43968#Comment_43968

Hi, you are doing in code using the send() method right? Do you mind trying a Reporter in the Scenario settings? I believe this is what Dataiku Support tested and even though the Scenario Reporter ended in error it should deliver the message.

Stanislava

https://community.dataiku.com/discussion/comment/43969#Comment_43969

Hi, no in the above I tried exactly as outlined and used a Reporter in the Scenario settings.

apichery

Dear Microsoft Teams users,

I confirm that the new webhook generated by Workflows/PowerAutomate is not fully compatible with the DSS/Teams integration in the existing DSS releases. The reason is that they changed the payload of the JSON expected by the Webhook.

What works (kind of):

• Scenario reporter with custom JSON message (see the example posted above: { "attachments": […] }). The reporter will be marked as failed because DSS does not recognize HTTP 202 as a success but the message is correctly delivered on the Teams channel
• Send a message scenario step with custom JSON message (same as above, except if will mark the scenario as failed, even though message will be delivered on the Teams channel)

What does not work:

• Scenario reporter with non-JSON message
• "Send a message" step with non-JSON message
• Python API "message_sender.send()"

The good news is that this will be fixed in the upcoming DSS 13.0.2. Stay tuned.

Stanislava

Thanks @apichery
Looking forward to the fix in DSS 13.0.2 - will this be available for cloud platform users as well?

Regarding:

• Scenario reporter with custom JSON message (see the example posted above: { "attachments": […] }). The reporter will be marked as failed because DSS does not recognize HTTP 202 as a success but the message is correctly delivered on the Teams channel
  » This did not work for me. I don't know if there are some limitations to posting in private vs public Teams channels however? I tested it for a private channel.

me2

https://community.dataiku.com/discussion/comment/43976#Comment_43976

Thank you. This saved me a lot of time 😀

apichery

https://community.dataiku.com/discussion/comment/43977#Comment_43977

https://community.dataiku.com/discussion/comment/43976#Comment_43976

Small update… the support for Workflows webhooks will actually be added to DSS 12.6.5 and 13.0.1.

apichery

https://community.dataiku.com/discussion/comment/43978#Comment_43978

For the bad news…Workflows in Microsoft Teams are currently unable to send message to private channels.
So DSS - as any other tool - will be unable to send message to private channels until Microsoft solve this.

(Search for "private" in the comments on the thread with the retirement announce: https://devblogs.microsoft.com/microsoft365dev/retirement-of-office-365-connectors-within-microsoft-teams/ )

apichery

https://community.dataiku.com/discussion/comment/43963#Comment_43963

Starting with DSS 12.6.5 and 13.0.1, you'll have to write the following to send a message through a webhook based on Workflows.

message_sender.send(message=…, webhookUrl=<new webhook>, webhookType="WORKFLOWS")

Turribeach

https://community.dataiku.com/discussion/comment/43983#Comment_43983

Smashing news, many thanks for backporting this fix. We will try to put pressure on Microsoft to fix the private channel issue.

Stanislava

https://community.dataiku.com/discussion/comment/43982#Comment_43982

Yeah this is a bummer, and given I doubt Microsoft will change this, I'll need to move to a chat rather than a public channel

apichery

… and 12.6.5 has been released. Enjoy your new Microsoft Teams webhooks (except for those posting to private channels)

Turribeach

For anyone on this thread it might be worth up voting this feedback suggestion to prevent the retirement of Teams Webhooks:

https://feedbackportal.microsoft.com/feedback/idea/80ed6877-b642-ef11-b4ad-000d3a7aba8b

Turribeach

Microsoft posted an update on their blog post:

https://devblogs.microsoft.com/microsoft365dev/retirement-of-office-365-connectors-within-microsoft-teams/

Basically they now deprecate this functionality at the end of 2025 but you will need to update the Webhooks URL before the end of 2024 and the new URL will be provided 90 days before that. But they will still stop allowing new Teams Webhooks being created from August 15th.

There are also updates to this post in that all the alternatives provided are not good replacements:

https://feedbackportal.microsoft.com/feedback/idea/80ed6877-b642-ef11-b4ad-000d3a7aba8b