Help Shape the Future of Dataiku Join our User Research Program

How can I utilize all my machine's cores for building a machine learning model?

Solved!
okiriza
Level 2
How can I utilize all my machine's cores for building a machine learning model?
This should correspond to n_jobs in sklearn models' fit method.
0 Kudos
1 Solution
Clément_Stenac
Dataiker
Dataiker
Hi,

At the moment, this is only implemented for the RandomForest and SGD algorithms (go to Settings > Algorithms and set the Parallelism option).

Note that if you ask for several algorithms, up to 2 algorithms are trained in parallel (we don't go above that for memory reasons since each algorithm has its own copy of the data).

We'll be soon adding more options for parallelism, including for Grid searching.

View solution in original post

2 Replies
PGutierrez
Dataiker
Dataiker
Hi, this can be set in the "Parallelism" parameter. However this is available only for some models, such as Random Forest.
Clément_Stenac
Dataiker
Dataiker
Hi,

At the moment, this is only implemented for the RandomForest and SGD algorithms (go to Settings > Algorithms and set the Parallelism option).

Note that if you ask for several algorithms, up to 2 algorithms are trained in parallel (we don't go above that for memory reasons since each algorithm has its own copy of the data).

We'll be soon adding more options for parallelism, including for Grid searching.

View solution in original post

Labels (1)
A banner prompting to get Dataiku DSS