Create a file in sftp

Options
Mingo
Mingo Registered Posts: 7 ✭✭✭
edited July 16 in Using Dataiku

Hello,

I would like to create a flat file and drop it to a SFTP server. I did set up the connection.

First I created a managed folder and used the recipe 'Export to folder'. It worked , but I have a limited choice of format. And obviously ".txt" is not on the list.

Then I used a python recipe to export the file in the managed folder : 

# -*- coding: utf-8 -*-
import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu

# Read recipe inputs
test = dataiku.Dataset("TEST")
test_df = test.get_dataframe()

folder_name=dataiku.Folder("v2")
folder_name_path=folder_name.get_path()

test_df.to_csv(path_or_buf=folder_name_path+"/project.txt",index=False,sep=";")

But I have the following Error:

Job failed: Error in Python process: At line 11: <type 'exceptions.Exception'>: Folder is not on the local filesystem (uses SFTP), cannot perform direct filesystem access. Use the read/write API instead.

This code work for a managed folder on my local server.

So I'm wondering how I can create a txt files on a ftp folder.

Thanks in vance

· Share on FacebookShare on Twitter

Best Answer

  • Liev
    Liev Dataiker Alumni Posts: 176 ✭✭✭✭✭✭✭✭
    edited July 17 Answer ✓
    Options

    Hi @Mingo

    You need to declare your output folder as well like this

    out_folder = dataiku.Folder("some_sftp_folder")

    Then you can generate the csv of your dataframe and upload it to the folder

    filename = "some_file_name.csv"
data = test_df.to_csv(index=False)

out_folder.upload_data(filename, data)

    Hope this helps!

    · Share on FacebookShare on Twitter

Answers

Bookmark this topic

Categories

Setup Info
    Tags
      Help me…