Discover the winners & finalists of the 2022 Dataiku Frontrunner Awards!READ THEIR USE CASES

How can I utilize all my machine's cores for building a machine learning model?

Solved!
okiriza
Level 2
How can I utilize all my machine's cores for building a machine learning model?
This should correspond to n_jobs in sklearn models' fit method.
0 Kudos
1 Solution
Clément_Stenac
Dataiker
Hi,

At the moment, this is only implemented for the RandomForest and SGD algorithms (go to Settings > Algorithms and set the Parallelism option).

Note that if you ask for several algorithms, up to 2 algorithms are trained in parallel (we don't go above that for memory reasons since each algorithm has its own copy of the data).

We'll be soon adding more options for parallelism, including for Grid searching.

View solution in original post

2 Replies
PGutierrez
Dataiker Alumni
Hi, this can be set in the "Parallelism" parameter. However this is available only for some models, such as Random Forest.
Clément_Stenac
Dataiker
Hi,

At the moment, this is only implemented for the RandomForest and SGD algorithms (go to Settings > Algorithms and set the Parallelism option).

Note that if you ask for several algorithms, up to 2 algorithms are trained in parallel (we don't go above that for memory reasons since each algorithm has its own copy of the data).

We'll be soon adding more options for parallelism, including for Grid searching.