Mount shared drive to DSS managed folder
Hi all,
I want to access multiple files located in a folder on a shared G drive. I read few posts which tells about mounting local/shared drives to Dataiku managed folder for accessing these files on Dataiku. Can someone please elaborate how to do this?
Thanks in advance!
Best Answer
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,215 Dataiker
Hi,
Just to clarify by G Drive you mean Google drive
if that’s the case you use Google Drive plugin to read files from Google Drive :
https://www.dataiku.com/product/plugins/googledrive/
If you a shared drive that happens to be G:// on your machine then you would need to setup a CIFS mount on the DSS server to mount that network share and then add it as local file system connection.
Answers
-
Hi @AlexT
Thanks for the reply. I am talking about shared drive G://, sorry for confusion. I have few question regarding same:1. When I checked properties of my G drive, File System is shown as NTFS, so is it still possible to access it on Dataiku? or does this works only for specific FSs only. I don't have much experience with FSs so I hope question is making sense.
2. When searching regarding CIFS mounts, I found https://serverfault.com/questions/105633/mount-remote-cifs-smb-share-as-a-folder-not-a-drive-letter and after I ran:
net use \\Server\ShareName\Directory
command, I was shown "The command completed successfully". Don't think it will be that simple but is the mount part done and I can move to creating Dataiku filesystem connection?
Thanks in advance!
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,215 Dataiker
Hi,
The Steps will vary depending on the Authentication required in environment , ideally you should ask Windows Admin for a service account which has access Share to use for the setup.
Then mount the drive in DSS steps will vary depending on the OS, here is an example:
https://www.serverlab.ca/tutorials/linux/storage-file-systems-linux/mounting-smb-shares-centos-7/
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,987 Neuron
We mount several network shares in our Dataiku servers using SMB and cifs-utils as suggested by Alex's post. As per Alex's link the steps will vary depending on the Linux OS you are using so Google the question accordingly (ie "how to mount a windows share in RHEL 7").
Here are a few tips from our experience using network shares in our Dataiku server:
1) Do not mount your network share in your DSS node directory. If you do that then the network share will be included in any backup you take of the DSS node directory. Instead mount your network shares in a directory outside DSS (ie /dataiku/shares/share1,/dataiku/shares/share2, etc). See next tip on how to expose it.
2) To expose the network share in Dataiku first create a file system folder in Dataiku, upload a dummy file to it via DSS (so that the folder gets created in the file system) and then go on the OS and create a symlink from the root of the file system folder to your network share, see sample below:
ln -s /dataiku/shares/share1 /dataiku/nodes/designer/managed_folders/SomeFileSystem/ProjectID/FolderID/remote_share
3) If you add the network share mounts to /etc/fstab so that they are mounted automatically on boot make sure you use these parameters to make sure the server boots even if these network shares are unavailable:,nofail,x-systemd.device-timeout=9
Enjoy!