-
Re: does DSS support switch IAM role?
Hi, DSS now supports switching IAM roles for access to S3 connections. The best practice here is to separate out your s3 access policy into a separate role and then set up assumeRole permissions on y…3 · -
Re: Reading a file with 40M+ records using PySpark
Hi, It's important to recognize here that Dataiku will simply execute whatever Spark code you ask it to run on the Spark cluster you have configured. In the Spark code you've described, the…1 · -
Re: Scenario triggered by an external change
Hi, If I understand your goal to be: I want my scenario to run when rows are added or removed from a sql dataset. You can just write the following query in the sql query trigger: Select count(*) from…2 · -
Re: Scenario custom trigger with a projet variable
Hi, If we assume that we have a project-level variable called rmse, then we could create a trigger like this to fire off a scenario when that variable is above a number (in this case 4): import datai…1 · -
Re: How to sync a partitioned table to a single table efficiently
Hi, Your statement is definitely correct. I'm not sure I understand the issue fully, but I'll try to talk about some concepts around partitioning and hopefully that helps. Lets say I have a…2 ·
