You now have until September 15th to submit your use case or success story to the 2022 Dataiku Frontrunner Awards!ENTER YOUR SUBMISSION

Mount shared drive to DSS managed folder

Solved!
pnaik1
Level 3
Mount shared drive to DSS managed folder

Hi all,

I want to access multiple files located in a folder on a shared G drive. I read few posts which tells about mounting local/shared drives to Dataiku managed folder for accessing these files on Dataiku. Can someone please elaborate how to do this? 

Thanks in advance!

1 Solution
AlexT
Dataiker
Dataiker

Hi,

Just to clarify by G Drive you mean Google drive

if that’s  the case you use Google Drive plugin to read files from Google Drive : 

https://www.dataiku.com/product/plugins/googledrive/

 

If you a shared drive that happens to be G:// on your machine then you would need to setup a CIFS mount on the DSS server to mount that network share and then add it as local file system connection.

 

 

View solution in original post

0 Kudos
4 Replies
AlexT
Dataiker
Dataiker

Hi,

Just to clarify by G Drive you mean Google drive

if that’s  the case you use Google Drive plugin to read files from Google Drive : 

https://www.dataiku.com/product/plugins/googledrive/

 

If you a shared drive that happens to be G:// on your machine then you would need to setup a CIFS mount on the DSS server to mount that network share and then add it as local file system connection.

 

 

0 Kudos
pnaik1
Level 3
Author

Hi @AlexT Thanks for the reply. I am talking about shared drive G://, sorry for confusion. I have few question regarding same:

1. When I checked properties of my G drive, File System is shown as NTFS, so is it still possible to access it on Dataiku? or does this works only for specific FSs only. I don't have much experience with FSs so I hope question is making sense.

2. When searching regarding CIFS mounts, I found https://serverfault.com/questions/105633/mount-remote-cifs-smb-share-as-a-folder-not-a-drive-letter and after I ran:

net use \\Server\ShareName\Directory

 command, I was shown "The command completed successfully". Don't think it will be that simple but is the mount part done and I can move to creating Dataiku filesystem connection? 

Thanks in advance!

0 Kudos
AlexT
Dataiker
Dataiker

Hi,

The Steps will vary depending on the Authentication required in environment  , ideally you should ask Windows Admin for a service account which has access Share to use for the setup. 

Then mount the drive  in DSS steps will vary depending on the OS, here is an example: 

https://www.serverlab.ca/tutorials/linux/storage-file-systems-linux/mounting-smb-shares-centos-7/

 

 

0 Kudos
Turribeach
Level 5

We mount several network shares in our Dataiku servers using SMB and cifs-utils as suggested by Alex's post. As per Alex's link the steps will vary depending on the Linux OS you are using so Google the question accordingly (ie "how to mount a windows share in RHEL 7").

Here are a few tips from our experience using network shares in our Dataiku server:

1) Do not mount your network share in your DSS node directory. If you do that then the network share will be included in any backup you take of the DSS node directory. Instead mount your network shares in a directory outside DSS (ie /dataiku/shares/share1,/dataiku/shares/share2, etc). See next tip on how to expose it.

2) To expose the network share in Dataiku first create a file system folder in Dataiku, upload a dummy file to it via DSS (so that the folder gets created in the file system) and then go on the OS and create a symlink from the root of the file system folder to your network share, see sample below:

ln -s /dataiku/shares/share1 /dataiku/nodes/designer/managed_folders/SomeFileSystem/ProjectID/FolderID/remote_share

3) If you add the network share mounts to /etc/fstab so that they are mounted automatically on boot make sure you use these parameters to make sure the server boots even if these network shares are unavailable:

,nofail,x-systemd.device-timeout=9

Enjoy!