Community Conundrum 25:Feature Visualization is now live! Read More

Avoid computing p-values in logistic regression

Level 1
Avoid computing p-values in logistic regression

Hi, 



I am training a logistic regression in the doctor, however after training and scoring of the model (according to the logs) I have a memory error. I think this error is due to the computation of features to explain the model, is it possible to chose not to compute these features ?




[2018-05-29 15:15:21,674] [13200/MainThread] [INFO] [root] Output params are {'resolved': {'logit': {'penalty': 'l2', 'multi_class': u'ovr', 'C': 0.01}, 'algorithm': u'LOGISTIC_REGRESSION'}, 'other': {}}
[2018-05-29 15:15:21,674] [13200/MainThread] [INFO] [root] END - Fitting model
[2018-05-29 15:15:21,674] [13200/MainThread] [INFO] [root] START - Saving model
[2018-05-29 15:15:21,679] [13200/MainThread] [INFO] [root] PICKLING LogisticRegression(C=0.01, class_weight=None, dual=False, fit_intercept=True,
intercept_scaling=1, max_iter=100, multi_class=u'ovr', n_jobs=1,
penalty='l2', random_state=1337, solver='liblinear', tol=0.0001,
verbose=0, warm_start=False)
[2018-05-29 15:15:21,681] [13200/MainThread] [INFO] [root] END - Saving model
[2018-05-29 15:15:21,681] [13200/MainThread] [INFO] [root] START - Scoring model
[2018-05-29 15:15:21,682] [13200/MainThread] [INFO] [root] Intrinsic scoring
[2018-05-29 15:15:21,682] [13200/MainThread] [INFO] [root] Computing regression coeffs
[2018-05-29 15:15:21,682] [13200/MainThread] [INFO] [root] FEATURES 30 CLF COEF 30
[2018-05-29 15:15:21,682] [13200/MainThread] [INFO] [root] CLF Intercept: [-1.87283529]
[2018-05-29 15:15:21,812] [13200/MainThread] [INFO] [root] Coefs: [ -1.87283529e+00 -2.74150938e-01]
[2018/05/29-15:16:33.161] [FT-TrainWorkThread-G7UFDiiP-4863] [INFO] [dku.analysis.ml.python] T-4dKTnPjI - Processing thread joined ...
[2018/05/29-15:16:33.163] [FT-TrainWorkThread-G7UFDiiP-4863] [INFO] [dku.analysis.ml.python] T-4dKTnPjI - Joining processing thread ...
[2018/05/29-15:18:05.229] [KNL-python-single-command-kernel-monitor-4872] [INFO] [dku.kernels] - Process done with code 137
[2018/05/29-15:18:05.246] [KNL-python-single-command-kernel-monitor-4872] [INFO] [dip.tickets] - Destroying API ticket for analysis-ml-***-nnzrg12 on behalf of admin
[2018/05/29-15:18:05.247] [MRT-4865] [INFO] [dku.kernels] - Getting kernel tail
[2018/05/29-15:18:05.352] [MRT-4865] [INFO] [dku.kernels] - Getting kernel tail
[2018/05/29-15:18:05.354] [MRT-4865] [ERROR] [dku.analysis.prediction] - Processing failed
com.dataiku.dip.exceptions.ProcessDiedException: Process died (exit code: 137, killed - maybe out of memory ?)


Thanks for the help!



Max

0 Kudos
3 Replies
Dataiker
Dataiker
Hey Mouch Mouch,

Good to see you. Updating DSS to the latest version should solve the issue!

Cheers,

Alex
0 Kudos
Level 1
Author
Thanks Alex!
It worked like a charm!

Max
0 Kudos
Dataiker
Dataiker
thank Clément 🙂
0 Kudos
Labels (1)