chunked download of get_download_stream
![tomas](https://us.v-cdn.net/6038231/uploads/lithium_attachments/n5261i79DA1BFFD65C0DD1.png)
tomas
Registered, Neuron 2022 Posts: 120 ✭✭✭✭✭
Hi, where can I find an example of a chunked download of the file from the Folder python object (dataiku.core.managed_folder.Folder)?
In this example I would like to write by a specified chunk size, for example print out something after every 1MB:
x=in_folder.get_download_stream( "/"+f )
with out_folder.get_writer(f) as writer:
parts = list( x.read_chunked(1024*1024) )
for part in parts:
print('Part')
writer.write( part )
print('File uploaded')
ResponseNotChunked: Response is not chunked. Header 'transfer-encoding: chunked' is missing.
Thanks
Best Answer
-
Hi,
The data is not transferred using chunked encoding, so you can't use read_chunked. However, you can just read by blocks using the regular Python IO API: read(size)