Survey banner
The Dataiku Community is moving to a new home! We are temporary in read only mode: LEARN MORE

Does Multiprocessing work on DSS ?

Level 1
Does Multiprocessing work on DSS ?

Hi !!

I'm using DSS (Version 5.1.4) and want to know if multiprocessing works with a python recipe.

I tried to process a pandas dataframe using packages like multiprocessing, swifter or pandarallel (2, 4 or 8 core) but it doesn't change the processing speed (DSS is installed on a 8 core machine).  

Is there a specific way (or setting) to apply parallel processing on pandas dataframe ?

Thank you !! 

0 Kudos
1 Reply
There is no specific way to call multiprocess in a Python recipe within DSS. Please note that python code being ran in DSS is the same as python being ran outside of DSS, we don't do anything special to it.
Here are the links that I would recommend looking into with regards to this matter:
0 Kudos