Submit your inspiring success story or innovative use case to the 2022 Dataiku Frontrunner Awards! ENTER YOUR SUBMISSION

python

tale
Level 1
python

i am working on  python language  and I need to search specific file form sftp server , I try to use paramiko but it just list all file and directory  that found on  folder , so can any one help me to search specific file form sftp server using python


Operating system used: window

0 Kudos
2 Replies
AlexT
Dataiker
Dataiker

Hi @tale ,

In DSS you can use files on an SFTP connection without code. The steps would be:
1) Create an SFTP connection using username/password or using the SSH key must be of the dss user (~/.ssh/id_rsa ) which needs to be allowed on the remote server

2) Create managed folder pointing to the path you want to the SFTP connection

3) Create "Files in Folder"  pointing to the SFTP managed folder flow the(b + Dataset - Internal - Files in Folder)

4) Configure the under Advanced options the regex or glob you want to use.

Screenshot 2022-06-08 at 14.13.16.png

Then you can use this dataset upstream in the recipe, if you need to move to filesystem managed or SQL dataset simply use a Sync recipe.

If you must use code you can have a look at online examples using paramiko: https://stackoverflow.com/questions/30626615/how-to-download-only-the-latest-file-from-sftp-server-w...

 

 

0 Kudos
tale
Level 1
Author

I appreciate your quick answering , @AlexT 
i must use code  and also i see the online solution provided on stack overflow , on that case it retrieve latest  data  but what i need is searching specific file from sftp server  selected folder

0 Kudos