Want to Stop Rebuilding "Expensive" Parts of your Flow? Explicit Builds are the Answer!READ MORE

How to change a file name while writing file in S3 or exporting in managed folder using dataiku

Solved!
pritam003
Level 2
How to change a file name while writing file in S3 or exporting in managed folder using dataiku

Hi Team 

can you please let us know how to change file name

 

  • while writing in S3 
  • exporting it to a managed folder.

 

regards,

pritam

 

0 Kudos
1 Solution
AlexT
Dataiker
Dataiker

Hi @pritam003,

Changing the file name when using the Visual Export to Folder recipe is not possible.

You can use Python recipe and read/write APIs. Create a python with managed folder ( backed on S3) as the output.

import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu

managed_folder_id = "URKU7Oqb"

# Read dataset convert df
my_dataset = dataiku.Dataset("customers_labeled_prepared")
df = my_dataset.get_dataframe()


# Write recipe outputs
output_folder = dataiku.Folder(managed_folder_id)
output_folder.upload_stream("some_name.csv", df.to_csv(index=False).encode("utf-8"))

 

View solution in original post

0 Kudos
3 Replies
AlexT
Dataiker
Dataiker

Hi @pritam003,

Changing the file name when using the Visual Export to Folder recipe is not possible.

You can use Python recipe and read/write APIs. Create a python with managed folder ( backed on S3) as the output.

import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu

managed_folder_id = "URKU7Oqb"

# Read dataset convert df
my_dataset = dataiku.Dataset("customers_labeled_prepared")
df = my_dataset.get_dataframe()


# Write recipe outputs
output_folder = dataiku.Folder(managed_folder_id)
output_folder.upload_stream("some_name.csv", df.to_csv(index=False).encode("utf-8"))

 

0 Kudos
pritam003
Level 2
Author

Thanks @AlexT  this helped a lot.

 

can you please let us know how we can create a managed folder using python api and get the managed folder id through it.

 

TIA 

pritam

0 Kudos
AlexT
Dataiker
Dataiker
0 Kudos