When i try to write a pandas dataframe to a dataset, i.e.
df = pd.DataFrame(...) ds = dataiku.Dataset(...) ds.write_with_schema(df, dropAndCreate=True)
i get the following error,
TypeError: 'ObjectBlock' object is not iterable
Have anyone tried something similar? And/or do you know what might be going wrong? Inspecting the dataset, I can see that the schema is written as intended, but no data is written.
EDIT: The error occurs only when I call the code from outside dataiku, if I create a recipe inside dataiku, it works as intended.
if it works in DSS, then maybe the cause is a discrepancy in package versions, notably of Pandas and/or Numpy. Can you get a `pip list` of the python environment where you get this error. Also, what's the full stacktrace of the error? (ie where is this error raised from)
The full stack trace is,
you're indeed using a very recent Pandas, and python 3.7. DSS' python code actually doesn't handle it yet, so you should revert your pandas to pandas>=1.0,<1.1, and possibly use a python3.6 if that doesn't solve the error