How to add data to a existing dataset with python?

Highlighted
UserBird Dataiker
Dataiker
How to add data to a existing dataset with python?
I have data set by name weather_data , i want to add data everyday to this dataset

How can i do this with python?
0 Kudos
2 Replies
larispardo
Level 3
Re: How to add data to a existing dataset with python?
Of course, now that I read ot after a good sleep I see what you mean, thanks dev. It would depend on how you have your new data, I guess the easiest way will be with the panda's append function. I believe this page will help you: https://stackoverflow.com/questions/14988480/pandas-version-of-rbind
0 Kudos
kenjil Dataiker
Dataiker
Re: How to add data to a existing dataset with python?
You may want to use

- the append mode on the output dataset. This setting is available in the input/output tab of the recipe => The data with be append in the output dataset. Note that this mode is only available on recipe with output dataset using an infrastructure allowing append (e.g. it is not possible with HDFS)

- the partition on the output dataset. Each day you will write your weather data in the partition of that day on the dataset. This mode works whatever the connection. See https://doc.dataiku.com/dss/latest/partitions/index.html for more details.
0 Kudos
Labels (4)