How can I save a fine-tuned HuggingFace model to a Dataiku Managed Folder?

Bart
Bart Registered Posts: 4 ✭✭✭
edited August 19 in Using Dataiku

Hello,

I am having an issue with saving a HuggingFace model that I am fine-tuning.

The default implementation for saving HF models is to use:

model.save_pretrained("/path/to/directory")

However, as far as I can tell, this does not work in dataiku. Even after I use the following code to get the path:

folder = dataiku.Folder("xxxxxxx")
d = folder.get_info()
path = d['accessInfo']['root']

I get an error saying that my permission to the path is denied.

In the past I have used pickle objects and saved those, but the specific model I am using this time is not pickleable.

Any help would be much appreciated!

Operating system used: Windows

Tagged:

Answers

Setup Info
    Tags
      Help me…