Write a dataset with more than 10k rows
![dharman](https://us.v-cdn.net/6038231/uploads/Dataiku/nAvatar6.png)
dharman
Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 4 ✭✭✭
From a Python recipe, I am trying to write a dataframe from a SQL query that has a little over 13k rows. Using the following code, DSS is only writing 10k rows. How do I get the remaining rows?
DataWithSample_df = SampleActiveMem_df # SQL output that I've reviewed # Write recipe outputs DataWithSample = dataiku.Dataset("DataWithSample") DataWithSample.write_schema_from_dataframe(DataWithSample_df) for df in DataWithSample.iter_dataframes(chunksize=15000): DataWithSample.write_from_dataframe(DataWithSample_df)
Best Answer
-
dharman Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 4 ✭✭✭
Solved: user error by misnaming a dataframe. My mistake