Export dataset to CSV with variable filename

Filip_Pejsa
Level 2
Export dataset to CSV with variable filename

Hi,

Is there a simple, non-coding way to export a dataset to a folder (via export recipe) under a file name reflecting a project variable value ?

E.g. the source dataset name is "output", the project variable is {"filedate": "30May2023"} , the exported file in the folder should be output_${filedate}, i.e. "output_30May2023.csv".

Thanks for advice.


Operating system used: Windows 10

0 Kudos
2 Replies
SarinaS
Dataiker

Hi @Filip_Pejsa,

The Export to Folder recipe does use the input dataset name as the exported filename for any files exported to the folder. 

Screenshot 2023-06-02 at 4.05.43 PM.png

If you need to specify the export file name based on a project variable, you would have to use code to do this. However, you could create a quite simple recipe that does this and turn it into a plugin, which would allow for reuse of the plugin in the form a visual recipe for all other users.

Here is an example 

# -*- coding: utf-8 -*-
import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu

# Read recipe inputs
Input_data = dataiku.Dataset("<INPUT_DATASET>")
Input_data_df = Input_data.get_dataframe()

client = dataiku.api_client()
project = client.get_project('<PROJECT_ID>')
variables = project.get_variables()
filedate = variables['standard']['filedate']
filename = 'output_' + filedate + '.csv'

# Write recipe outputs
outputfolder = dataiku.Folder("<FOLDERID>")
outputfolder_info = outputfolder.get_info()

with outputfolder.get_writer(filename) as w:
    w.write(Input_data_df.to_csv().encode())


You can then convert this recipe to a plugin with the following option from a Python recipe:

Screenshot 2023-06-02 at 4.21.05 PM.png

You will need to modify the plugin parameters and input and output roles, and it might help to review the documentation on this. Once you have a plugin, users can select your plugin recipe from the flow.

Thank you,
Sarina 

tgb417

@SarinaS ,

I’m looking forward to seeing how this works out. 

 

--Tom
0 Kudos

Labels

?
Labels (1)
A banner prompting to get Dataiku