Creating directories within python script, os.mkdir causing issues

Options
kman88
kman88 Registered Posts: 5 ✭✭✭✭

Hello, I want to do something very simpple. I would like to create directories within my output or input folders that currently do not exist. Normally this would easily be achievable through the python command os.mkdir. Here is my code:

import dataikuimport pandas as pd, numpy as npfrom dataiku import pandasutils as pduimport os# Read recipe inputstest = dataiku.Folder("SZxomfdY")test_info = test.get_info()# Write recipe outputsoutputfolder = dataiku.Folder("DBryoOH2")outputfolder_info = outputfolder.get_info()#for file in test.list_paths_in_partition():#print(file)outputdir = outputfolder_info['accessInfo']['root']newoutput = os.path.join(outputdir,'newdir')print(newoutput)os.mkdir(newoutput)

however I get the error

FileNotFoundError                         Traceback (most recent call last)<ipython-input-29-3b604d987ec9> in <module>()     19 newoutput = os.path.join(outputdir,'newdir')     20 print(newoutput)---> 21 os.mkdir(newoutput)

I don't understand though, it can't locate the folder however it shouldn't be locating the folder, it should be making the folder. Does anyone have any idea what's going on?

Answers

  • Ignacio_Toledo
    Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 411 Neuron
    Options

    Hi @kman88
    ,

    The FileNotFoundError also happens when you want to create a new folder within a folder that doesn't exist in the first place. However, your python code should work (I did test it just in case, and it was successful).

    Could you share the output from

    print(newoutput)

    ?

    If that doesn't give some idea of what might be happening, the next steps would be to check what kind of managed folder are you using. Is it a local filesystem managed folder?

  • Marine
    Marine Dataiker, Registered Posts: 29 Dataiker
    Options

    Hi @kman88
    ,

    To create directories in DSS, I'd recommend using the Dataiku's API instead of mkdir, it is more convenient to manage permissions and handle the user isolation framework.

    Could this thread solve your issue ?

Setup Info
    Tags
      Help me…