python

Options
tale
tale Registered Posts: 2 ✭✭✭

i am working on python language and I need to search specific file form sftp server , I try to use paramiko but it just list all file and directory that found on folder , so can any one help me to search specific file form sftp server using python


Operating system used: window

Answers

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 Dataiker
    Options

    Hi @tale
    ,

    In DSS you can use files on an SFTP connection without code. The steps would be:
    1) Create an SFTP connection using username/password or using the SSH key must be of the dss user (~/.ssh/id_rsa ) which needs to be allowed on the remote server

    2) Create managed folder pointing to the path you want to the SFTP connection

    3) Create "Files in Folder" pointing to the SFTP managed folder flow the(b + Dataset - Internal - Files in Folder)

    4) Configure the under Advanced options the regex or glob you want to use.

    Screenshot 2022-06-08 at 14.13.16.png

    Then you can use this dataset upstream in the recipe, if you need to move to filesystem managed or SQL dataset simply use a Sync recipe.

    If you must use code you can have a look at online examples using paramiko: : https://stackoverflow.com/questions/30626615/how-to-download-only-the-latest-file-from-sftp-server-with-paramiko

  • tale
    tale Registered Posts: 2 ✭✭✭
    Options

    I appreciate your quick answering , @AlexT

    i must use code and also i see the online solution provided on stack overflow , on that case it retrieve latest data but what i need is searching specific file from sftp server selected folder

Setup Info
    Tags
      Help me…