API to create a directory under a managed folder not local

gnaldi62

Hi again,

I've a managed folder under HDFS and want to create a subfolder under it but cannot find under the

API documentation the method or the function to do it. How can this be done ? It should be possible

because from the UI that is possible...

Thanks. Regards.

Giuseppe

fchataigner2

since the notion of folder does not exist for all connection types that you can put managed folders on (as in: it doesn't exist for cloud storages), the is no API to directly create directories inside managed folders. The directories are automatically created as needed when you write a file. For example calling https://doc.dataiku.com/dss/latest/python-api/managed_folders.html#dataiku.Folder.upload_data with `upload_data("/subdir1/subdir2/stuff", [])` will create the sub-directories subdir1 and subdir2 if they don't exist. You could write empty files to force directories to be created.

daida

Hi fchataigner2,

the upload_data() only exists in dataiku package. How can I do it outside of DSS? In the dataikuapi package there is no this method but put_file(). However put_file() doesn't create the subdirectory automatically like upload_data().

Damien13

Here is a function that addresses your need

def make_dir(managed_folder_id, subfolder_rel_path):
    """
    create a subfolder in a folder
    subfolder_path is relative path such as "subfolder/"
        or "my/subfolder/path/"
    the function will create all necessary subfolders if several are nested and not created yet
    if a folder already exists, the function will not create it, and not delete the content of it
    """
    # Get the folder
    client = dataiku.api_client()
    project = client.get_default_project()
    folder = project.get_managed_folder(managed_folder_id)

    # Create an empty file called "empty_file.txt" in the current working directory
    file_path = "empty_file.txt"
    open(file_path, "w").close()

    subfolder_rel_path_with_file = os.path.relpath(
        os.path.join(subfolder_rel_path, "empty_file.txt")
    )

    # Put the file into the DSS managed folder
    contents = folder.put_file(subfolder_rel_path_with_file, os.path.abspath(file_path))

    # delete the file that has been copied to the subfolder
    folder.delete_file(subfolder_rel_path_with_file)