Create a file in sftp

Mingo

Hello,

I would like to create a flat file and drop it to a SFTP server. I did set up the connection.

First I created a managed folder and used the recipe 'Export to folder'. It worked , but I have a limited choice of format. And obviously ".txt" is not on the list.

Then I used a python recipe to export the file in the managed folder :

# -*- coding: utf-8 -*-
import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu

# Read recipe inputs
test = dataiku.Dataset("TEST")
test_df = test.get_dataframe()

folder_name=dataiku.Folder("v2")
folder_name_path=folder_name.get_path()

test_df.to_csv(path_or_buf=folder_name_path+"/project.txt",index=False,sep=";")

But I have the following Error:

Job failed: Error in Python process: At line 11: <type 'exceptions.Exception'>: Folder is not on the local filesystem (uses SFTP), cannot perform direct filesystem access. Use the read/write API instead.

This code work for a managed folder on my local server.

So I'm wondering how I can create a txt files on a ftp folder.

Thanks in vance

Liev

Hi @Mingo

You need to declare your output folder as well like this

out_folder = dataiku.Folder("some_sftp_folder")

Then you can generate the csv of your dataframe and upload it to the folder

filename = "some_file_name.csv"
data = test_df.to_csv(index=False)

out_folder.upload_data(filename, data)

Hope this helps!

Mingo

Great, It works.

Thanks Liev

I was obviously missing the upload step.