[2022/08/31-15:31:34.669] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.prediction] - ****************************************** [2022/08/31-15:31:34.670] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.prediction] - ** Start train session s1 [2022/08/31-15:31:34.670] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.prediction] - ****************************************** [2022/08/31-15:31:34.674] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.shaker.data] T-6TiIkKTR - [ct: 5] Need to compute sampleId before checking memory cache [2022/08/31-15:31:34.675] [FT-TrainWorkThread-Fnjgqoah-1353] [DEBUG] [dip.shaker.runner] T-6TiIkKTR - [ct: 6] Script settings sampleMax=104857600 processedMax=-1 [2022/08/31-15:31:34.675] [FT-TrainWorkThread-Fnjgqoah-1353] [DEBUG] [dip.shaker.runner] T-6TiIkKTR - [ct: 6] Processing with sampleMax=104857600 processedMax=524288000 [2022/08/31-15:31:34.675] [FT-TrainWorkThread-Fnjgqoah-1353] [DEBUG] [dip.shaker.runner] T-6TiIkKTR - [ct: 6] Computed required sample id : 46a243505274b8ea74971fc53d15553a-NA-ac2e0aa81c1215a34ba9f85052ba5ff71661958446296--d751713988987e9331980363e24189ce [2022/08/31-15:31:34.675] [FT-TrainWorkThread-Fnjgqoah-1353] [DEBUG] [dku.shaker.cache] T-6TiIkKTR - Shaker MemoryCache get on dataset DEEPLEARNING.heart key=ds=287966266e4a6b8e0c7af80f7bce140b--scr=8e9b3a996957a453c3a63231e8e0bad1--samp=46a243505274b8ea74971fc53d15553a-NA-ac2e0aa81c1215a34ba9f85052ba5ff71661958446296--d751713988987e9331980363e24189ce: hit [2022/08/31-15:31:34.675] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.shaker.schema] T-6TiIkKTR - [ct: 6] Column Age meaning=LongMeaning fail=0 [2022/08/31-15:31:34.675] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.shaker.schema] T-6TiIkKTR - [ct: 6] Column Sex meaning=Gender fail=0 [2022/08/31-15:31:34.675] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.shaker.schema] T-6TiIkKTR - [ct: 6] Column ChestPainType meaning=Text fail=0 [2022/08/31-15:31:34.676] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.shaker.schema] T-6TiIkKTR - [ct: 7] Column RestingBP meaning=LongMeaning fail=0 [2022/08/31-15:31:34.676] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.shaker.schema] T-6TiIkKTR - [ct: 7] Column Cholesterol meaning=LongMeaning fail=0 [2022/08/31-15:31:34.676] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.shaker.schema] T-6TiIkKTR - [ct: 7] Column FastingBS meaning=LongMeaning fail=0 [2022/08/31-15:31:34.676] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.shaker.schema] T-6TiIkKTR - [ct: 7] Column RestingECG meaning=Text fail=0 [2022/08/31-15:31:34.676] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.shaker.schema] T-6TiIkKTR - [ct: 7] Column MaxHR meaning=LongMeaning fail=0 [2022/08/31-15:31:34.676] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.shaker.schema] T-6TiIkKTR - [ct: 7] Column ExerciseAngina meaning=Boolean fail=0 [2022/08/31-15:31:34.676] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.shaker.schema] T-6TiIkKTR - [ct: 7] Column Oldpeak meaning=DoubleMeaning fail=0 [2022/08/31-15:31:34.676] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.shaker.schema] T-6TiIkKTR - [ct: 7] Column ST_Slope meaning=Text fail=0 [2022/08/31-15:31:34.676] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.shaker.schema] T-6TiIkKTR - [ct: 7] Column HeartDisease meaning=Boolean fail=0 [2022/08/31-15:31:34.773] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.datasets.file] T-6TiIkKTR - [ct: 104] Building Filesystem handler config: {"path":"/home/dataiku/dss/uploads/DEEPLEARNING/datasets/heart","notReadyIfEmpty":false,"filesSelectionRules":{"mode":"ALL","excludeRules":[],"includeRules":[],"explicitFiles":[]}} [2022/08/31-15:31:34.773] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.datasets.ftplike] T-6TiIkKTR - Enumerating Filesystem dataset prefix= [2022/08/31-15:31:34.773] [FT-TrainWorkThread-Fnjgqoah-1353] [DEBUG] [dku.datasets.fsbased] T-6TiIkKTR - [ct: 104] Building FS provider for dataset handler: DEEPLEARNING.heart [2022/08/31-15:31:34.773] [FT-TrainWorkThread-Fnjgqoah-1353] [DEBUG] [dku.datasets.fsbased] T-6TiIkKTR - [ct: 104] FS Provider built [2022/08/31-15:31:34.774] [FT-TrainWorkThread-Fnjgqoah-1353] [DEBUG] [dku.fs.local] T-6TiIkKTR - [ct: 105] Enumerating local filesystem prefix=/ [2022/08/31-15:31:34.776] [FT-TrainWorkThread-Fnjgqoah-1353] [DEBUG] [dku.fs.local] T-6TiIkKTR - [ct: 107] Enumeration done nb_paths=1 size=35921 [2022/08/31-15:31:34.776] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.input.push] T-6TiIkKTR - USTP: push selection.method=HEAD_SEQUENTIAL records=100000 ratio=0.02 col=null [2022/08/31-15:31:34.776] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.format] T-6TiIkKTR - [ct: 107] Extractor run: limit={"maxBytes":-1,"maxRecords":100000,"ordering":{"enabled":false,"rules":[]}} totalRecords=0 [2022/08/31-15:31:34.776] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku] T-6TiIkKTR - getCompression filename=**heart.csv** [2022/08/31-15:31:34.776] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku] T-6TiIkKTR - getCompression filename=**heart.csv** [2022/08/31-15:31:34.777] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.format] T-6TiIkKTR - [ct: 108] Start uncompressed stream: /home/dataiku/dss/uploads/DEEPLEARNING/datasets/heart/heart.csv / totalRecsBefore=0 [2022/08/31-15:31:34.777] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku] T-6TiIkKTR - getCompression filename=**heart.csv** [2022/08/31-15:31:34.795] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.format] T-6TiIkKTR - [ct: 126] after stream totalComp=35921 totalUncomp=35921 totalRec=918 [2022/08/31-15:31:34.795] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.format] T-6TiIkKTR - [ct: 126] Extractor run done, totalCompressed=35921 totalRecords=918 [2022/08/31-15:31:34.802] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.splits] T-6TiIkKTR - [ct: 133] Checking if splits are up to date. Policy: type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=heart,sel=(method=head-s,records=100000),r=0.8,s=1337, instance id: 91fa259596d0773de3beb036211717d6-0 [2022/08/31-15:31:34.803] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.splits] T-6TiIkKTR - [ct: 134] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=heart,sel=(method=head-s,records=100000),r=0.8,s=1337 i=91fa259596d0773de3beb036211717d6-0 [2022/08/31-15:31:34.804] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.splits] T-6TiIkKTR - [ct: 135] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=heart,sel=(method=head-s,records=100000),r=0.8,s=1337 i=91fa259596d0773de3beb036211717d6-0 [2022/08/31-15:31:34.805] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.splits] T-6TiIkKTR - [ct: 136] Checking if splits are up to date. Policy: type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=heart,sel=(method=head-s,records=100000),r=0.8,s=1337, instance id: 91fa259596d0773de3beb036211717d6-0 [2022/08/31-15:31:34.805] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.splits] T-6TiIkKTR - [ct: 136] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=heart,sel=(method=head-s,records=100000),r=0.8,s=1337 i=91fa259596d0773de3beb036211717d6-0 [2022/08/31-15:31:34.805] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.splits] T-6TiIkKTR - [ct: 136] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=heart,sel=(method=head-s,records=100000),r=0.8,s=1337 i=91fa259596d0773de3beb036211717d6-0 [2022/08/31-15:31:34.807] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.ml.python] T-6TiIkKTR - [ct: 138] Joining processing thread ... [2022/08/31-15:31:34.807] [MRT-1354] [INFO] [dku.analysis.ml.python] - Running a preprocessing set: pp1 in /home/dataiku/dss/analysis-data/DEEPLEARNING/m0wnmYdk/6TiIkKTR/sessions/s1/pp1 [2022/08/31-15:31:34.807] [MRT-1354] [INFO] [dku.block.link] - Started a socket on port 44629 [2022/08/31-15:31:34.808] [MRT-1354] [INFO] [dku.ml.kernel] - Writing output of python-single-command-kernel to /home/dataiku/dss/analysis-data/DEEPLEARNING/m0wnmYdk/6TiIkKTR/sessions/s1/pp1/train.log [2022/08/31-15:31:34.808] [MRT-1354] [INFO] [dku.code.envs.resolution] - Executing Python activity in env: Deeplearning [2022/08/31-15:31:34.808] [MRT-1354] [INFO] [dku.code.projectLibs] - EXTERNAL LIBS FROM DEEPLEARNING is {"gitReferences":{},"pythonPath":["python"],"rsrcPath":["R"],"importLibrariesFromProjects":[]} [2022/08/31-15:31:34.808] [MRT-1354] [INFO] [dku.code.projectLibs] - chunkFolder is /home/dataiku/dss/config/projects/DEEPLEARNING/lib/R [2022/08/31-15:31:34.808] [MRT-1354] [INFO] [dku.python.single_command.kernel] - Starting Python process for kernel python-single-command-kernel [2022/08/31-15:31:34.808] [MRT-1354] [INFO] [dip.tickets] - Creating API ticket for analysis-ml-DEEPLEARNING-1spcFrH on behalf of admin id=analysis-ml-DEEPLEARNING-1spcFrH_UU5V7dgloWHM [2022/08/31-15:31:34.809] [MRT-1354] [INFO] [dku.security.process] - Starting process (regular) [2022/08/31-15:31:34.811] [MRT-1355] [INFO] [dku.analysis.ml.python] - TrainAdditionalThread done [2022/08/31-15:31:34.825] [MRT-1354] [INFO] [dku.security.process] - Process started with pid=10422 [2022/08/31-15:31:34.825] [MRT-1354] [INFO] [dku.processes.cgroups] - Will use cgroups [] [2022/08/31-15:31:34.825] [MRT-1354] [INFO] [dku.processes.cgroups] - Applying rules to used cgroups: [] [2022/08/31-15:31:34.846] [KNL-python-single-command-kernel-monitor-1358] [DEBUG] [dku.resourceusage] - Reporting start of CRU:{"context":{"type":"ANALYSIS_ML_TRAIN","authIdentifier":"admin","projectKey":"DEEPLEARNING","analysisId":"m0wnmYdk","mlTaskId":"6TiIkKTR","sessionId":"s1"},"type":"LOCAL_PROCESS","id":"pU3tu2HiJMXhMH9M","startTime":1661959894846,"localProcess":{"cpuCurrent":0.0}} [2022/08/31-15:31:34.890] [process-resource-monitor-10422-1359] [DEBUG] [dku.resource] - Process stats for pid 10422: {"pid":10422,"commandName":"/home/dataiku/dss/code-envs/python/Deeplearning/bin/python","cpuUserTimeMS":30,"cpuSystemTimeMS":30,"cpuChildrenUserTimeMS":0,"cpuChildrenSystemTimeMS":0,"cpuTotalMS":60,"cpuCurrent":0.0,"vmSizeMB":246,"vmRSSMB":16,"vmHWMMB":16,"vmRSSAnonMB":10,"vmDataMB":9,"vmSizePeakMB":246,"vmRSSPeakMB":16,"vmRSSTotalMBS":0,"majorFaults":0,"childrenMajorFaults":0} Installing debugging signal handler [2022/08/31-15:31:35.977] [MRT-1354] [INFO] [dku.link.secret_protected] - Connected to kernel [2022/08/31-15:31:35.978] [MRT-1354] [INFO] [dku.block.link.interaction] - Execute link command respClazz=true respTypeToken=false respIsString=false is=false asyncInputStream=false os=false [2022-08-31 15:31:35,977] [10422/MainThread] [INFO] [dataiku.base.socket_block_link] Connecting to localhost (127.0.0.1) at port 44629 [2022-08-31 15:31:35,977] [10422/MainThread] [INFO] [dataiku.base.socket_block_link] Connected to localhost (127.0.0.1) at port 44629 [2022-08-31 15:31:36,041] [10422/MainThread] [INFO] [dataiku.doctor.utils.dku_pickle] Setting cloudpickle as the pickling tool /home/dataiku/dataiku-dss-11.0.2/python/dataiku/doctor/dkuapi.py:16: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec() argspec = inspect.getargspec(api) [2022-08-31 15:31:36,072] [10422/MainThread] [INFO] [root] Running analysis command: train_prediction_keras [2022-08-31 15:31:36,072] [10422/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] disabling diagnostics [2022-08-31 15:31:36,073] [10422/MainThread] [INFO] [dataiku.doctor.commands] PPS is {'feature_selection_params': {'method': 'NONE', 'random_forest_params': {'n_trees': 30, 'depth': 10, 'n_features': 25}, 'lasso_params': {'alpha': [0.01, 0.1, 1.0, 10.0, 100.0], 'cross_validate': True}, 'pca_params': {'n_features': 25, 'variance_proportion': 0.9}, 'correlation_params': {'min_abs_correlation': 0.0, 'max_abs_correlation': 1.0, 'n_features': 25}, 'custom_params': {'code': '# type your code here'}}, 'preprocessingFitSampleRatio': 1.0, 'preprocessingFitSampleSeed': 1337, 'target_remapping': [{'sourceValue': '0', 'mappedValue': 0, 'sampleFreq': 410}, {'sourceValue': '1', 'mappedValue': 1, 'sampleFreq': 508}], 'skipPreprocessing': False, 'per_feature': {'RestingBP': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'ST_Slope': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'RestingECG': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'MaxHR': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'ExerciseAngina': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Boolean'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'FastingBS': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'Sex': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Gender'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'HeartDisease': {'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'TARGET', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Boolean'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'Oldpeak': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'ChestPainType': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'Age': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}, 'Cholesterol': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main'}}, 'reduce': {'enabled': False, 'kept_variance': 0.0}, 'feature_generation': {'pairwise_linear': {'behavior': 'DISABLED'}, 'polynomial_combinations': {'behavior': 'DISABLED'}, 'manual_interactions': {'interactions': []}, 'numericals_clustering': {'k': 0, 'all_features': False, 'input_features': [], 'behavior': 'DISABLED'}, 'categoricals_count_transformer': {'all_features': False, 'input_features': [], 'behavior': 'DISABLED'}}} [2022-08-31 15:31:36,073] [10422/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Loading train set [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [root] Reading with dtypes: None [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Age: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Sex: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for ChestPainType: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for RestingBP: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Cholesterol: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for FastingBS: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for RestingECG: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for MaxHR: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for ExerciseAngina: str (schema_type=boolean feature_type=CATEGORY feature_role=INPUT) [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Oldpeak: (schema_type=double feature_type=NUMERIC feature_role=INPUT) [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for ST_Slope: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for HeartDisease: (schema_type=boolean feature_type=CATEGORY feature_role=TARGET) [2022-08-31 15:31:36,074] [10422/MainThread] [INFO] [root] Reading with FIXED dtypes: {'Age': , 'Sex': 'str', 'ChestPainType': 'str', 'RestingBP': , 'Cholesterol': , 'FastingBS': , 'RestingECG': 'str', 'MaxHR': , 'ExerciseAngina': 'str', 'Oldpeak': , 'ST_Slope': 'str', 'HeartDisease': } [2022-08-31 15:31:36,079] [10422/MainThread] [INFO] [root] Loaded table [2022-08-31 15:31:36,079] [10422/MainThread] [INFO] [dataiku.doctor.utils] Coercion done [2022-08-31 15:31:36,079] [10422/MainThread] [INFO] [dataiku.doctor.utils.split] Loaded train df: shape=(738,12) [2022-08-31 15:31:36,079] [10422/MainThread] [INFO] [dataiku.doctor.utils.listener] END - Loading train set [2022-08-31 15:31:36,079] [10422/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Loading test set [2022-08-31 15:31:36,081] [10422/MainThread] [INFO] [root] Reading with dtypes: None [2022-08-31 15:31:36,081] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Age: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2022-08-31 15:31:36,081] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Sex: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2022-08-31 15:31:36,081] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for ChestPainType: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2022-08-31 15:31:36,081] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for RestingBP: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2022-08-31 15:31:36,081] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Cholesterol: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2022-08-31 15:31:36,081] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for FastingBS: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2022-08-31 15:31:36,082] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for RestingECG: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2022-08-31 15:31:36,082] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for MaxHR: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2022-08-31 15:31:36,082] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for ExerciseAngina: str (schema_type=boolean feature_type=CATEGORY feature_role=INPUT) [2022-08-31 15:31:36,082] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Oldpeak: (schema_type=double feature_type=NUMERIC feature_role=INPUT) [2022-08-31 15:31:36,082] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for ST_Slope: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2022-08-31 15:31:36,082] [10422/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for HeartDisease: (schema_type=boolean feature_type=CATEGORY feature_role=TARGET) [2022-08-31 15:31:36,082] [10422/MainThread] [INFO] [root] Reading with FIXED dtypes: {'Age': , 'Sex': 'str', 'ChestPainType': 'str', 'RestingBP': , 'Cholesterol': , 'FastingBS': , 'RestingECG': 'str', 'MaxHR': , 'ExerciseAngina': 'str', 'Oldpeak': , 'ST_Slope': 'str', 'HeartDisease': } [2022-08-31 15:31:36,084] [10422/MainThread] [INFO] [root] Loaded table [2022-08-31 15:31:36,085] [10422/MainThread] [INFO] [dataiku.doctor.utils] Coercion done [2022-08-31 15:31:36,085] [10422/MainThread] [INFO] [dataiku.doctor.utils.split] Loaded test df: shape=(180,12) [2022-08-31 15:31:36,085] [10422/MainThread] [INFO] [dataiku.doctor.utils.listener] END - Loading test set [2022-08-31 15:31:36,085] [10422/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Collecting statistics [2022-08-31 15:31:36,087] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at RestingBP... (type=NUMERIC) [2022-08-31 15:31:36,087] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False) [2022-08-31 15:31:36,092] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at ST_Slope... (type=CATEGORY) [2022-08-31 15:31:36,094] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at RestingECG... (type=CATEGORY) [2022-08-31 15:31:36,096] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at MaxHR... (type=NUMERIC) [2022-08-31 15:31:36,096] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False) [2022-08-31 15:31:36,097] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at ExerciseAngina... (type=CATEGORY) [2022-08-31 15:31:36,099] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at FastingBS... (type=NUMERIC) [2022-08-31 15:31:36,099] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False) [2022-08-31 15:31:36,100] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at Sex... (type=CATEGORY) [2022-08-31 15:31:36,105] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at HeartDisease... (type=CATEGORY) [2022-08-31 15:31:36,105] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at Oldpeak... (type=NUMERIC) [2022-08-31 15:31:36,105] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False) [2022-08-31 15:31:36,106] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at ChestPainType... (type=CATEGORY) [2022-08-31 15:31:36,108] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at Age... (type=NUMERIC) [2022-08-31 15:31:36,108] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False) [2022-08-31 15:31:36,109] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at Cholesterol... (type=NUMERIC) [2022-08-31 15:31:36,109] [10422/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False) [2022-08-31 15:31:36,110] [10422/MainThread] [INFO] [dataiku.doctor.utils.listener] END - Collecting statistics [2022-08-31 15:31:36,110] [10422/MainThread] [INFO] [dataiku.doctor.multiframe] generating interactions [2022-08-31 15:31:36,110] [10422/MainThread] [INFO] [dataiku.doctor.multiframe] {'feature_selection_params': {'method': 'NONE', 'random_forest_params': {'n_trees': 30, 'depth': 10, 'n_features': 25}, 'lasso_params': {'alpha': [0.01, 0.1, 1.0, 10.0, 100.0], 'cross_validate': True}, 'pca_params': {'n_features': 25, 'variance_proportion': 0.9}, 'correlation_params': {'min_abs_correlation': 0.0, 'max_abs_correlation': 1.0, 'n_features': 25}, 'custom_params': {'code': '# type your code here'}}, 'preprocessingFitSampleRatio': 1.0, 'preprocessingFitSampleSeed': 1337, 'target_remapping': [{'sourceValue': '0', 'mappedValue': 0, 'sampleFreq': 410}, {'sourceValue': '1', 'mappedValue': 1, 'sampleFreq': 508}], 'skipPreprocessing': False, 'per_feature': {'RestingBP': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main', 'isSpecialFeature': False}, 'ST_Slope': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main', 'isSpecialFeature': False}, 'RestingECG': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main', 'isSpecialFeature': False}, 'MaxHR': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main', 'isSpecialFeature': False}, 'ExerciseAngina': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Boolean'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main', 'isSpecialFeature': False}, 'FastingBS': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main', 'isSpecialFeature': False}, 'Sex': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Gender'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main', 'isSpecialFeature': False}, 'HeartDisease': {'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'TARGET', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Boolean'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main', 'isSpecialFeature': False}, 'Oldpeak': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'DoubleMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main', 'isSpecialFeature': False}, 'ChestPainType': {'category_handling': 'DUMMIFY', 'missing_handling': 'NONE', 'missing_impute_with': 'MODE', 'dummy_clip': 'MAX_NB_CATEGORIES', 'cumulative_proportion': 0.95, 'min_samples': 10, 'max_nb_categories': 100, 'max_cat_safety': 200, 'nb_bins_hashing': 1048576, 'hash_whole_categories': True, 'dummy_drop': 'NONE', 'impact_method': 'M_ESTIMATOR', 'impact_m': 10, 'impact_kfold': True, 'impact_kfold_k': 5, 'impact_kfold_seed': 1337, 'ordinal_order': 'COUNT', 'ordinal_ascending': False, 'ordinal_default_mode': 'HIGHEST', 'ordinal_default_value': 0, 'frequency_default_mode': 'EXPLICIT', 'frequency_default_value': 0.0, 'frequency_normalized': True, 'role': 'INPUT', 'type': 'CATEGORY', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'Text'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main', 'isSpecialFeature': False}, 'Age': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main', 'isSpecialFeature': False}, 'Cholesterol': {'generate_derivative': False, 'numerical_handling': 'REGULAR', 'missing_handling': 'IMPUTE', 'missing_impute_with': 'MEAN', 'impute_constant_value': 0.0, 'keep_regular': False, 'rescaling': 'AVGSTD', 'quantile_bin_nb_bins': 4, 'binarize_threshold_mode': 'MEDIAN', 'binarize_constant_threshold': 0.0, 'datetime_cyclical_periods': [], 'role': 'INPUT', 'type': 'NUMERIC', 'state': {'userModified': False, 'autoModifiedByDSS': False, 'recordedMeaning': 'LongMeaning'}, 'customHandlingCode': '', 'customProcessorWantsMatrix': False, 'sendToInput': 'main', 'isSpecialFeature': False}}, 'reduce': {'enabled': False, 'kept_variance': 0.0}, 'feature_generation': {'pairwise_linear': {'behavior': 'DISABLED'}, 'polynomial_combinations': {'behavior': 'DISABLED'}, 'manual_interactions': {'interactions': []}, 'numericals_clustering': {'k': 0, 'all_features': False, 'input_features': [], 'behavior': 'DISABLED'}, 'categoricals_count_transformer': {'all_features': False, 'input_features': [], 'behavior': 'DISABLED'}}} [2022-08-31 15:31:36,110] [10422/MainThread] [INFO] [dataiku.doctor.multiframe] No feature selection to perform [2022-08-31 15:31:36,110] [10422/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Fitting preprocessors [2022-08-31 15:31:36,112] [10422/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 738 [2022-08-31 15:31:36,112] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RemapValueToOutput [2022-08-31 15:31:36,113] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput [2022-08-31 15:31:36,113] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {'RestingBP': 131.95257452574526, 'MaxHR': 137.3008130081301, 'FastingBS': 0.22628726287262874, 'Oldpeak': 0.8831978319783197, 'Age': 53.204607046070464, 'Cholesterol': 201.13685636856368} [2022-08-31 15:31:36,114] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput [2022-08-31 15:31:36,114] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {} [2022-08-31 15:31:36,114] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (RestingBP) [2022-08-31 15:31:36,114] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Rescale RestingBP (avg=131.95257452574526 std=18.674943491053227 shift=131.95257452574526 inv_scale=0.05354768545774069) [2022-08-31 15:31:36,116] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Rescaled RestingBP (avg=-2.936524856461119e-16 std=1.0) nulls=0 [2022-08-31 15:31:36,116] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (MaxHR) [2022-08-31 15:31:36,116] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Rescale MaxHR (avg=137.3008130081301 std=25.52183250100235 shift=137.3008130081301 inv_scale=0.03918213944710772) [2022-08-31 15:31:36,118] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Rescaled MaxHR (avg=-4.3325776570737815e-16 std=1.0) nulls=0 [2022-08-31 15:31:36,118] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (FastingBS) [2022-08-31 15:31:36,118] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Rescale FastingBS (avg=0.22628726287262874 std=0.4187109946195503 shift=0.22628726287262874 inv_scale=2.3882821632343836) [2022-08-31 15:31:36,119] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Rescaled FastingBS (avg=-3.851180139621139e-17 std=1.0) nulls=0 [2022-08-31 15:31:36,119] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (Oldpeak) [2022-08-31 15:31:36,120] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Rescale Oldpeak (avg=0.8831978319783197 std=1.044239644769169 shift=0.8831978319783197 inv_scale=0.9576345860925934) [2022-08-31 15:31:36,120] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Rescaled Oldpeak (avg=1.155354041886342e-16 std=1.0000000000000002) nulls=0 [2022-08-31 15:31:36,120] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (Age) [2022-08-31 15:31:36,120] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Rescale Age (avg=53.204607046070464 std=9.402365845642779 shift=53.204607046070464 inv_scale=0.10635621038543375) [2022-08-31 15:31:36,121] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Rescaled Age (avg=-3.6586211326400824e-16 std=1.0) nulls=0 [2022-08-31 15:31:36,121] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (Cholesterol) [2022-08-31 15:31:36,121] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Rescale Cholesterol (avg=201.13685636856368 std=109.24035949756431 shift=201.13685636856368 inv_scale=0.009154125861534688) [2022-08-31 15:31:36,122] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Rescaled Cholesterol (avg=-1.9255900698105696e-17 std=1.0) nulls=0 [2022-08-31 15:31:36,122] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(num_flagonly) [2022-08-31 15:31:36,122] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(datetime_cyclical) [2022-08-31 15:31:36,122] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (ST_Slope) [2022-08-31 15:31:36,126] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(738, 5) nnz=738 [2022-08-31 15:31:36,126] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (RestingECG) [2022-08-31 15:31:36,127] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(738, 5) nnz=738 [2022-08-31 15:31:36,127] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (ExerciseAngina) [2022-08-31 15:31:36,129] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(738, 4) nnz=738 [2022-08-31 15:31:36,129] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (Sex) [2022-08-31 15:31:36,130] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(738, 4) nnz=738 [2022-08-31 15:31:36,130] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (ChestPainType) [2022-08-31 15:31:36,131] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(738, 6) nnz=738 [2022-08-31 15:31:36,131] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput [2022-08-31 15:31:36,131] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {} [2022-08-31 15:31:36,131] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(cat_flagpresence) [2022-08-31 15:31:36,131] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput [2022-08-31 15:31:36,131] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {} [2022-08-31 15:31:36,131] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput [2022-08-31 15:31:36,131] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {} [2022-08-31 15:31:36,131] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(interaction) [2022-08-31 15:31:36,131] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RealignTarget [2022-08-31 15:31:36,131] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Realign target series = (738,) [2022-08-31 15:31:36,132] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] After realign target: (738,) [2022-08-31 15:31:36,132] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DropRowsWhereNoTarget [2022-08-31 15:31:36,132] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] Deleting 0 rows because one of ['target'] is missing [2022-08-31 15:31:36,132] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] MF before = (738, 30) [2022-08-31 15:31:36,132] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] target before = (738,) [2022-08-31 15:31:36,132] [10422/MainThread] [INFO] [dataiku.doctor.multiframe] MultiFrame, dropping rows: [] [2022-08-31 15:31:36,138] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] After DRWNT input_df=(738, 12) [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] MF after = (738, 30) [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] target after = (738,) [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DumpPipelineState [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (Before feature selection) [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] input_df= (738, 12) [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] current_mf=(738, 30) [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] PPR: [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] target = ((738,)) [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:EmitCurrentMFAsResult [2022-08-31 15:31:36,139] [10422/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 738 [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DumpPipelineState [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (At end) [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] input_df= (738, 12) [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] current_mf=(0, 0) [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] PPR: [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] target = ((738,)) [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] TRAIN = ((738, 30)) [2022-08-31 15:31:36,139] [10422/MainThread] [DEBUG] [dku.ml.preprocessing] UNPROCESSED = ((738, 12)) [2022-08-31 15:31:36,140] [10422/MainThread] [INFO] [dataiku.doctor.utils.listener] END - Fitting preprocessors [2022-08-31 15:31:36,140] [10422/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Preprocessing train set [2022-08-31 15:31:36,141] [10422/MainThread] [INFO] [dataiku.doctor.utils.listener] END - Preprocessing train set [2022-08-31 15:31:36,142] [10422/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Preprocessing test set [2022-08-31 15:31:36,142] [10422/MainThread] [INFO] [dataiku.doctor.utils.listener] END - Preprocessing test set [2022-08-31 15:31:36,143] [10422/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Fitting model [2022/08/31-15:31:36.314] [KNL-python-single-command-kernel-monitor-1358] [INFO] [dku.kernels] - Process done with code 132 [2022/08/31-15:31:36.314] [KNL-python-single-command-kernel-monitor-1358] [INFO] [dip.tickets] - Destroying API ticket for analysis-ml-DEEPLEARNING-1spcFrH on behalf of admin [2022/08/31-15:31:36.315] [KNL-python-single-command-kernel-monitor-1358] [WARN] [dku.resource] - stat file for pid 10422 does not exist. Process died? [2022/08/31-15:31:36.315] [KNL-python-single-command-kernel-monitor-1358] [DEBUG] [dku.resourceusage] - Reporting completion of CRU:{"context":{"type":"ANALYSIS_ML_TRAIN","authIdentifier":"admin","projectKey":"DEEPLEARNING","analysisId":"m0wnmYdk","mlTaskId":"6TiIkKTR","sessionId":"s1"},"type":"LOCAL_PROCESS","id":"pU3tu2HiJMXhMH9M","startTime":1661959894846,"localProcess":{"pid":10422,"commandName":"/home/dataiku/dss/code-envs/python/Deeplearning/bin/python","cpuUserTimeMS":30,"cpuSystemTimeMS":30,"cpuChildrenUserTimeMS":0,"cpuChildrenSystemTimeMS":0,"cpuTotalMS":60,"cpuCurrent":0.0,"vmSizeMB":246,"vmRSSMB":16,"vmHWMMB":16,"vmRSSAnonMB":10,"vmDataMB":9,"vmSizePeakMB":246,"vmRSSPeakMB":16,"vmRSSTotalMBS":0,"majorFaults":0,"childrenMajorFaults":0}} [2022/08/31-15:31:36.315] [MRT-1354] [INFO] [dku.kernels] - Getting kernel tail [2022/08/31-15:31:36.319] [MRT-1354] [INFO] [dku.kernels] - Trying to enrich exception: com.dataiku.dip.io.SocketBlockLinkIOException: Failed to get result from kernel from kernel com.dataiku.dip.analysis.coreservices.AnalysisMLKernel@1aa17531 process=null pid=?? retcode=132 [2022/08/31-15:31:36.420] [MRT-1354] [INFO] [dku.kernels] - Getting kernel tail [2022/08/31-15:31:36.421] [MRT-1354] [WARN] [dku.analysis.ml.python] - Training failed com.dataiku.dip.exceptions.ProcessDiedException: Process died (exit code: 132) at com.dataiku.dip.kernels.DSSKernelBase.maybeRethrowAsProcessDied(DSSKernelBase.java:284) at com.dataiku.dip.analysis.ml.prediction.PredictionTrainAdditionalThread.process(PredictionTrainAdditionalThread.java:78) at com.dataiku.dip.analysis.ml.shared.PRNSTrainThread.run(PRNSTrainThread.java:173) [2022/08/31-15:31:36.421] [MRT-1354] [INFO] [dku.block.link] - Closed socket [2022/08/31-15:31:36.421] [MRT-1354] [INFO] [dku.block.link] - Closed socket [2022/08/31-15:31:36.422] [MRT-1354] [INFO] [dku.block.link] - Closed serverSocket [2022/08/31-15:31:36.422] [MRT-1354] [ERROR] [dku.analysis.ml.python] - Processing failed com.dataiku.dip.exceptions.ProcessDiedException: Process died (exit code: 132) at com.dataiku.dip.kernels.DSSKernelBase.maybeRethrowAsProcessDied(DSSKernelBase.java:284) at com.dataiku.dip.analysis.ml.prediction.PredictionTrainAdditionalThread.process(PredictionTrainAdditionalThread.java:78) at com.dataiku.dip.analysis.ml.shared.PRNSTrainThread.run(PRNSTrainThread.java:173) [2022/08/31-15:31:36.422] [MRT-1354] [INFO] [dku.analysis.ml] - Locking model train info file /home/dataiku/dss/analysis-data/DEEPLEARNING/m0wnmYdk/6TiIkKTR/sessions/s1/pp1/m1/train_info.json [2022/08/31-15:31:36.422] [MRT-1354] [INFO] [dku.analysis.ml] - Unlocking model train info file /home/dataiku/dss/analysis-data/DEEPLEARNING/m0wnmYdk/6TiIkKTR/sessions/s1/pp1/m1/train_info.json [2022/08/31-15:31:36.422] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.ml.python] T-6TiIkKTR - [ct: 1753] Processing thread joined ... [2022/08/31-15:31:36.423] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.ml.python] T-6TiIkKTR - [ct: 1754] Joining processing thread ... [2022/08/31-15:31:36.423] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.ml.python] T-6TiIkKTR - [ct: 1754] Processing thread joined ... [2022/08/31-15:31:36.423] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis] T-6TiIkKTR - [ct: 1754] Train done [2022/08/31-15:31:36.424] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.prediction] T-6TiIkKTR - Train done [2022/08/31-15:31:36.479] [FT-TrainWorkThread-Fnjgqoah-1353] [INFO] [dku.analysis.trainingdetails] T-6TiIkKTR - Publishing mltask-train-done reflected event