Export dataset to CSV with variable filename

Filip_Pejsa

Hi,

Is there a simple, non-coding way to export a dataset to a folder (via export recipe) under a file name reflecting a project variable value ?

E.g. the source dataset name is "output", the project variable is {"filedate": "30May2023"} , the exported file in the folder should be output_${filedate}, i.e. "output_30May2023.csv".

Thanks for advice.

---

Operating system used: Windows 10

Sarina

Hi @Filip_Pejsa
,

The Export to Folder recipe does use the input dataset name as the exported filename for any files exported to the folder.

 

If you need to specify the export file name based on a project variable, you would have to use code to do this. However, you could create a quite simple recipe that does this and turn it into a plugin, which would allow for reuse of the plugin in the form a visual recipe for all other users.

Here is an example

# -*- coding: utf-8 -*-
import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu

# Read recipe inputs
Input_data = dataiku.Dataset("<INPUT_DATASET>")
Input_data_df = Input_data.get_dataframe()

client = dataiku.api_client()
project = client.get_project('<PROJECT_ID>')
variables = project.get_variables()
filedate = variables['standard']['filedate']
filename = 'output_' + filedate + '.csv'

# Write recipe outputs
outputfolder = dataiku.Folder("<FOLDERID>")
outputfolder_info = outputfolder.get_info()

with outputfolder.get_writer(filename) as w:
    w.write(Input_data_df.to_csv().encode())

You can then convert this recipe to a plugin with the following option from a Python recipe:

You will need to modify the plugin parameters and input and output roles, and it might help to review the documentation on this. Once you have a plugin, users can select your plugin recipe from the flow.

Thank you,
Sarina 

tgb417

@SarinaS
,

I’m looking forward to seeing how this works out.