Reading compressed pickled files from managed folder using joblib

pkansal
pkansal Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Registered Posts: 23 ✭✭✭✭

I have saved compressed pickle files in a managed folder. Normally, I read it as follows:

with open(pickle_file,'''rb') as f:

load(f)

How do I read it from a managed folder?

Tagged:

Answers

  • Sarina
    Sarina Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 317 Dataiker
    edited July 17

    Hi @pkansal
    ,

    You can read objects from managed folders, including pickle files like so:

    import dataiku
    folder = dataiku.Folder("folder_name")
    with folder.get_download_stream(pickle_file) as f:
      model = f.read()
    

    Let us know if you have any questions about this.

    Thank you,
    Sarina

  • pkansal
    pkansal Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Registered Posts: 23 ✭✭✭✭

    The file is compressed with LZMA. How do I specify that compression here?

Setup Info
    Tags
      Help me…