Discover this year's submissions to the Dataiku Frontrunner Awards and give kudos to your favorite use cases and success stories!READ MORE

R Recipe Streaming

rmoore
Neuron
Neuron
R Recipe Streaming

We're working on a project utilizing an R notebook with some very large datasets and are wondering what the recommended approach is for working with a dataset that does not fit into memory. 

We are big fans of the streaming API for Python - is there any equivalent for R?

Thanks!

3 Replies
Triveni
Dataiker
Dataiker

Hello,

The dataiku R library allows you to read your data in by chunks. More information can be found in the docs here : https://doc.dataiku.com/dss/api/8.0/R/dataiku/reference/dkuReadDataset.html

0 Kudos
rmoore
Neuron
Neuron
Author

Thanks @Triveni. The ability to read a random sample from the dataset is terrific, but can you tell me if it's possible to read records from a specific offset? For example, I want to iterate through the entire set but only take 10,000 records at a time.

0 Kudos
tanguy
Level 3

+1 for this feature in R

0 Kudos

Labels

?
Labels (3)
A banner prompting to get Dataiku