Loading data from pickle files

Anders
Anders Registered Posts: 7 ✭✭✭
in Using Dataiku

Hi,

I have some pandas dataframes that are saved to files outside Dataiku. I have put the files into a managed folder, and would like to import them as pandas dataframes.

I have tried different variations like this:

folder = dataiku.Folder("folder_name")

with folder.get_download_stream('df_filename.pkl') as f:
model = f.read()

This just leaves me with a binary variable, "model", that I can't convert to a dataframe. I have tried pickle.load(f) and pickle.load(f.read), but that does not help. Is there anyone that have experience with this?

Thanks,

Anders

Tagged:
· Share on Facebook

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023, Circle Member Posts: 2,722 Neuron

    This post shows how to load a pickle file:

    https://community.dataiku.com/t5/Using-Dataiku/Importing-a-pickle-model-in-Dataiku/m-p/33528

    But I am not sure what you are trying to do. A pickle file is a binary model file. What exactly are you trying to achieve?

    · Share on Facebook
  • Anders
    Anders Registered Posts: 7 ✭✭✭

    Thanks,

    What I try to do is to load a saved dataframe and use it as a dataframe in Dataiku. In python I can do something like:

    df = joblib.load(nameOfSavedFile)

    This will give me a pandas dataframe (df) that I can use. But loading the file into Dataiku leaves me with the binary content of the file, which I haven't found out how to convert to a dataframe.

    · Share on Facebook
Bookmark this topic

Categories

Setup Info
    Tags
      Help me…