Want to Stop Rebuilding "Expensive" Parts of your Flow? Explicit Builds are the Answer!READ MORE

Does Multiprocessing work on DSS ?

Mass
Level 1
Does Multiprocessing work on DSS ?

Hi !!

I'm using DSS (Version 5.1.4) and want to know if multiprocessing works with a python recipe.

I tried to process a pandas dataframe using packages like multiprocessing, swifter or pandarallel (2, 4 or 8 core) but it doesn't change the processing speed (DSS is installed on a 8 core machine).  

Is there a specific way (or setting) to apply parallel processing on pandas dataframe ?

Thank you !! 

0 Kudos
1 Reply
AlexandreL
Dataiker
Dataiker
Hi,
 
There is no specific way to call multiprocess in a Python recipe within DSS. Please note that python code being ran in DSS is the same as python being ran outside of DSS, we don't do anything special to it.
 
Here are the links that I would recommend looking into with regards to this matter:
0 Kudos