Using Dataiku

Hi @sridarvenkatesa
,

Your thought is to export the project to a zip file, unzip it, make the changes (using a search and replace tool presumably), and then zip and reimport it?

I don't know for sure but I'd think that could work.

You could also use the Python API (dataikuapi) to make the changes in place. Let me know if this is of interest as I have some code you could modify to do this. We are using this approach to partially convert SQL recipes in our projects from Netezza SQL to Snowflake SQL.

Marlan

Hi @sridarvenkatesa
,

Your thought is to export the project to a zip file, unzip it, make the changes (using a search and replace tool presumably), and then zip and reimport it?

I don't know for sure but I'd think that could work.

You could also use the Python API (dataikuapi) to make the changes in place. Let me know if this is of interest as I have some code you could modify to do this. We are using this approach to partially convert SQL recipes in our projects from Netezza SQL to Snowflake SQL.

Marlan

Hi,

DSS apparently can't use the column type of that date for partitioning. The simplest is to make a view out of the table and use the view instead of the original table in your DSS dataset. Something like

create view xxxxx as select cast(date_col as NVARCHAR(50)), ... from the_table

(or some TO_CHAR(...) with an appropriate format)

Hi @MClark05

Yes, you can create an FS connection pointing to this directory and create a managed folder based on it. When done, use python APIs listed below the mentioned doc page to read the data from it.

Hi @tinaresh
please feel free to utilize the Product Ideas board. The Product Ideas board is here to let you share and exchange your ideas on how to improve Dataiku. Here are some resources to help get you started: Suggest an idea

I hope this helps!

Hello @shoareau
,

Thank you for posting the question on Community.

> How can i get the absolute path of the file inside DDS ?

If your managed folder is stored on the local filesystem of the DSS host, you can use the Folder.file_path(filename) function to extract the filesystem path for a given file within a folder. Here is sample code.

import dataiku

FOLDER_ID = 'VdrA1ZMC'

REPO = dataiku.Folder(FOLDER_ID)
list_files = REPO .list_paths_in_partition()
REPO.file_path(list_files[0])

Please note that this Folder.file_path(filename) function is available only if your managed folder is stored on the local filesystem of the DSS host. If your managed folder is stored on other places such as HDFS and S3, this function cannot be used and you will need to call the Folder.get_download_stream(path) function to retrieve the file contents from the folder as follows.

import dataiku

FOLDER_ID = 'RoQHOCas'

REPO = dataiku.Folder(FOLDER_ID)
list_files = REPO .list_paths_in_partition()
with REPO.get_download_stream(list_files[0]) as stream:
    data = stream.read()
    # You can handle / manipulate the file contents (data) here.

I hope this would help. Please let us know if you have any further questions.

Sincerely,
Keiji, Dataiku Technical Support

Hi,

You can use a Group recipe with those two columns as keys.

If there are more columns, you need to decide what to do with them and use the aggregation section of the recipe for that.

I hope this helps.

Best regards

Hi @dave
,

Ahh thank you for the example So I'm assuming when you say "Business" you're referring to a project? Sounds like what you need to use is a "Trigger after Scenario" trigger.

In your example, in the settings of Business Y Scenario A1, create a "Trigger after Scenario" trigger where the "Scenario" to trigger after each run is Business X Scenario A.

Scenario A1 can run when Scenario A either finishes, runs successful, produces warnings, fails, or is aborted.

So just to clarify a bit more, in the "Scenario" field of Scenario A1 in Business Y, you would select "Scenario A (BUSINESSX)" and select "runs successfully" in the "when the other scenario..." field.