[2020/05/13-07:52:19.234] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.prediction] - ****************************************** [2020/05/13-07:52:19.234] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.prediction] - ** Start train session s8 [2020/05/13-07:52:19.234] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.prediction] - ****************************************** [2020/05/13-07:52:19.235] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.splits] T-hrRkST3I - [ct: 1] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=customers_labeled,sel=(method=head-s,records=100000),r=0.8,s=1337 i=3eb33c15252255001bc38241aa60a4d7-0 [2020/05/13-07:52:19.237] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.splits] T-hrRkST3I - [ct: 3] Checking if splits are up to date. Policy: type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=customers_labeled,sel=(method=head-s,records=100000),r=0.8,s=1337, instance id: 3eb33c15252255001bc38241aa60a4d7-0 [2020/05/13-07:52:19.238] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.splits] T-hrRkST3I - [ct: 4] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=customers_labeled,sel=(method=head-s,records=100000),r=0.8,s=1337 i=3eb33c15252255001bc38241aa60a4d7-0 [2020/05/13-07:52:19.239] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.splits] T-hrRkST3I - [ct: 5] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=customers_labeled,sel=(method=head-s,records=100000),r=0.8,s=1337 i=3eb33c15252255001bc38241aa60a4d7-0 [2020/05/13-07:52:19.244] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.ml.python] T-hrRkST3I - Joining processing thread ... [2020/05/13-07:52:19.244] [MRT-552] [INFO] [dku.analysis.ml.python] - Running a preprocessing set: pp1 in /home/vinhdiesal/DATA_DIR/analysis-data/DKU_TUTORIAL_ML/yqhRRBMm/hrRkST3I/sessions/s8/pp1 [2020/05/13-07:52:19.246] [MRT-552] [INFO] [dku.block.link] - Started a socket on port 46709 [2020/05/13-07:52:19.246] [MRT-552] [INFO] [dku.ml.kernel] - Writing output of python-single-command-kernel to /home/vinhdiesal/DATA_DIR/analysis-data/DKU_TUTORIAL_ML/yqhRRBMm/hrRkST3I/sessions/s8/pp1/train.log [2020/05/13-07:52:19.247] [MRT-552] [INFO] [dku.code.envs.resolution] - Executing Python activity in env: python2_7 [2020/05/13-07:52:19.247] [MRT-552] [WARN] [dku.code.projectLibs] - External libraries file not found: /home/vinhdiesal/DATA_DIR/config/projects/DKU_TUTORIAL_ML/lib/external-libraries.json [2020/05/13-07:52:19.247] [MRT-552] [INFO] [dku.code.projectLibs] - EXTERNAL LIBS FROM DKU_TUTORIAL_ML is {"gitReferences":{},"pythonPath":["python"],"rsrcPath":["R"],"importLibrariesFromProjects":[]} [2020/05/13-07:52:19.247] [MRT-552] [INFO] [dku.code.projectLibs] - chunkFolder is /home/vinhdiesal/DATA_DIR/config/projects/DKU_TUTORIAL_ML/lib/R [2020/05/13-07:52:19.247] [MRT-552] [INFO] [dku.python.single_command.kernel] - Starting Python process for kernel python-single-command-kernel [2020/05/13-07:52:19.248] [MRT-552] [INFO] [dip.tickets] - Creating API ticket for analysis-ml-DKU_TUTORIAL_ML-vUiuP5J on behalf of admin id=analysis-ml-DKU_TUTORIAL_ML-vUiuP5J_heKWLZxFXo54 [2020/05/13-07:52:19.248] [MRT-552] [INFO] [dku.security.process] - Starting process (regular) [2020/05/13-07:52:19.249] [MRT-552] [INFO] [dku.security.process] - Process started with pid=6211 [2020/05/13-07:52:19.250] [MRT-552] [INFO] [dku.processes.cgroups] - Will use cgroups [] [2020/05/13-07:52:19.250] [MRT-552] [INFO] [dku.processes.cgroups] - Applying rules to used cgroups: [] [2020/05/13-07:52:19.271] [MRT-553] [INFO] [dku.analysis.ml.python] - TrainAdditionalThread done Installing debugging signal handler [2020-05-13 07:52:19,747] [6211/MainThread] [INFO] [root] Connecting to parent at port 46709 [2020-05-13 07:52:19,748] [6211/MainThread] [INFO] [root] Connected to parent at port 46709 [2020/05/13-07:52:19.748] [MRT-552] [INFO] [dku.link.secret_protected] - Connected to kernel [2020/05/13-07:52:19.748] [MRT-552] [INFO] [dku.block.link.interaction] - Execute link command respClazz=true respTypeToken=false respIsString=false is=false asyncInputStream=false os=false [2020-05-13 07:52:20,048] [6211/MainThread] [INFO] [root] Running analysis command: train_prediction_keras [2020-05-13 07:52:20,049] [6211/MainThread] [INFO] [dataiku.doctor.commands] PPS is {u'preprocessingFitSampleSeed': 1337, u'feature_selection_params': {u'custom_params': {u'code': u'# type your code here'}, u'pca_params': {u'variance_proportion': 0.9, u'n_features': 25}, u'random_forest_params': {u'depth': 10, u'n_features': 25, u'n_trees': 30}, u'lasso_params': {u'alpha': [0.01, 0.1, 1.0, 10.0, 100.0], u'cross_validate': True}, u'method': u'NONE', u'correlation_params': {u'max_abs_correlation': 1.0, u'n_features': 25, u'min_abs_correlation': 0.0}}, u'preprocessingFitSampleRatio': 1.0, u'reduce': {u'enabled': False, u'kept_variance': 0.0}, u'skipPreprocessing': False, u'target_remapping': [{u'mappedValue': 0, u'sourceValue': u'False', u'sampleFreq': 9078}, {u'mappedValue': 1, u'sourceValue': u'True', u'sampleFreq': 922}], u'per_feature': {u'ip_address_geopoint': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'GeoPoint', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'user_agent_os': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'campaign': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Boolean', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'user_agent_brand': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'age_first_order': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'ip_address_country': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'CountryMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'high_revenue': {u'min_samples': 10, u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Boolean', u'autoModifiedByDSS': False}, u'role': u'TARGET', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'max_cat_safety': 200, u'cumulative_proportion': 0.95, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'gender': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Gender', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'pages_visited_avg': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'customerID': {u'missing_impute_with': u'MODE', u'autoReason': u'REJECT_IDENTIFIER', u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'customProcessorWantsMatrix': False, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'REJECT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}}, u'feature_generation': {u'manual_interactions': {u'interactions': []}, u'pairwise_linear': {u'behavior': u'DISABLED'}, u'categoricals_count_transformer': {u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}, u'polynomial_combinations': {u'behavior': u'DISABLED'}, u'numericals_clustering': {u'k': 0, u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}}} [2020-05-13 07:52:20,049] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Loading train set [2020-05-13 07:52:20,049] [6211/MainThread] [INFO] [root] Reading with dtypes: None [2020-05-13 07:52:20,049] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for customerID: None (schema_type=string feature_type=CATEGORY feature_role=REJECT) [2020-05-13 07:52:20,049] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for gender: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2020-05-13 07:52:20,049] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for age_first_order: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2020-05-13 07:52:20,049] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for user_agent_brand: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2020-05-13 07:52:20,049] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for user_agent_os: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2020-05-13 07:52:20,050] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for ip_address_country: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2020-05-13 07:52:20,050] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for ip_address_geopoint: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2020-05-13 07:52:20,050] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for campaign: str (schema_type=boolean feature_type=CATEGORY feature_role=INPUT) [2020-05-13 07:52:20,050] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for pages_visited_avg: (schema_type=double feature_type=NUMERIC feature_role=INPUT) [2020-05-13 07:52:20,050] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for high_revenue: (schema_type=boolean feature_type=CATEGORY feature_role=TARGET) [2020-05-13 07:52:20,050] [6211/MainThread] [INFO] [root] Reading with FIXED dtypes: {u'ip_address_geopoint': 'str', u'user_agent_os': 'str', u'campaign': 'str', u'gender': 'str', u'age_first_order': , u'ip_address_country': 'str', u'high_revenue': , u'user_agent_brand': 'str', u'pages_visited_avg': } [2020-05-13 07:52:20,067] [6211/MainThread] [INFO] [root] Loaded table [2020-05-13 07:52:20,068] [6211/MainThread] [INFO] [dataiku.doctor.utils] Coercion done [2020-05-13 07:52:20,068] [6211/MainThread] [INFO] [dataiku.doctor.commands] Loaded train df: shape=(8610,10) [2020-05-13 07:52:20,068] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] END - Loading train set [2020-05-13 07:52:20,068] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Loading test set [2020-05-13 07:52:20,069] [6211/MainThread] [INFO] [root] Reading with dtypes: None [2020-05-13 07:52:20,069] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for customerID: None (schema_type=string feature_type=CATEGORY feature_role=REJECT) [2020-05-13 07:52:20,069] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for gender: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2020-05-13 07:52:20,069] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for age_first_order: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2020-05-13 07:52:20,069] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for user_agent_brand: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2020-05-13 07:52:20,069] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for user_agent_os: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2020-05-13 07:52:20,069] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for ip_address_country: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2020-05-13 07:52:20,069] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for ip_address_geopoint: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2020-05-13 07:52:20,069] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for campaign: str (schema_type=boolean feature_type=CATEGORY feature_role=INPUT) [2020-05-13 07:52:20,069] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for pages_visited_avg: (schema_type=double feature_type=NUMERIC feature_role=INPUT) [2020-05-13 07:52:20,069] [6211/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for high_revenue: (schema_type=boolean feature_type=CATEGORY feature_role=TARGET) [2020-05-13 07:52:20,069] [6211/MainThread] [INFO] [root] Reading with FIXED dtypes: {u'ip_address_geopoint': 'str', u'user_agent_os': 'str', u'campaign': 'str', u'gender': 'str', u'age_first_order': , u'ip_address_country': 'str', u'high_revenue': , u'user_agent_brand': 'str', u'pages_visited_avg': } [2020-05-13 07:52:20,075] [6211/MainThread] [INFO] [root] Loaded table [2020-05-13 07:52:20,076] [6211/MainThread] [INFO] [dataiku.doctor.utils] Coercion done [2020-05-13 07:52:20,076] [6211/MainThread] [INFO] [dataiku.doctor.commands] Loaded test df: shape=(2177,10) [2020-05-13 07:52:20,076] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] END - Loading test set [2020-05-13 07:52:20,076] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Collecting statistics [2020-05-13 07:52:20,076] [6211/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at ip_address_geopoint... (type=CATEGORY) [2020-05-13 07:52:20,080] [6211/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at user_agent_os... (type=CATEGORY) [2020-05-13 07:52:20,083] [6211/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at campaign... (type=CATEGORY) [2020-05-13 07:52:20,086] [6211/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at user_agent_brand... (type=CATEGORY) [2020-05-13 07:52:20,088] [6211/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at age_first_order... (type=NUMERIC) [2020-05-13 07:52:20,088] [6211/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False) [2020-05-13 07:52:20,090] [6211/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at ip_address_country... (type=CATEGORY) [2020-05-13 07:52:20,092] [6211/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at high_revenue... (type=CATEGORY) [2020-05-13 07:52:20,092] [6211/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at gender... (type=CATEGORY) [2020-05-13 07:52:20,094] [6211/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at pages_visited_avg... (type=NUMERIC) [2020-05-13 07:52:20,095] [6211/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False) [2020-05-13 07:52:20,096] [6211/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at customerID... (type=CATEGORY) [2020-05-13 07:52:20,096] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] END - Collecting statistics [2020-05-13 07:52:20,096] [6211/MainThread] [INFO] [dataiku.doctor.multiframe] generating interactions [2020-05-13 07:52:20,096] [6211/MainThread] [INFO] [dataiku.doctor.multiframe] {u'preprocessingFitSampleSeed': 1337, u'feature_selection_params': {u'custom_params': {u'code': u'# type your code here'}, u'pca_params': {u'variance_proportion': 0.9, u'n_features': 25}, u'random_forest_params': {u'depth': 10, u'n_features': 25, u'n_trees': 30}, u'lasso_params': {u'alpha': [0.01, 0.1, 1.0, 10.0, 100.0], u'cross_validate': True}, u'method': u'NONE', u'correlation_params': {u'max_abs_correlation': 1.0, u'n_features': 25, u'min_abs_correlation': 0.0}}, u'preprocessingFitSampleRatio': 1.0, u'reduce': {u'enabled': False, u'kept_variance': 0.0}, u'skipPreprocessing': False, u'target_remapping': [{u'mappedValue': 0, u'sourceValue': u'False', u'sampleFreq': 9078}, {u'mappedValue': 1, u'sourceValue': u'True', u'sampleFreq': 922}], u'per_feature': {u'ip_address_geopoint': {'isSpecialFeature': False, u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'GeoPoint', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'user_agent_os': {'isSpecialFeature': False, u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'campaign': {'isSpecialFeature': False, u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Boolean', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'user_agent_brand': {'isSpecialFeature': False, u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'age_first_order': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, 'isSpecialFeature': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'ip_address_country': {'isSpecialFeature': False, u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'CountryMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'high_revenue': {'isSpecialFeature': False, u'min_samples': 10, u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Boolean', u'autoModifiedByDSS': False}, u'role': u'TARGET', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'max_cat_safety': 200, u'cumulative_proportion': 0.95, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'gender': {'isSpecialFeature': False, u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Gender', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'pages_visited_avg': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, 'isSpecialFeature': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'customerID': {'isSpecialFeature': False, u'missing_impute_with': u'MODE', u'autoReason': u'REJECT_IDENTIFIER', u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'customProcessorWantsMatrix': False, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'REJECT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}}, u'feature_generation': {u'manual_interactions': {u'interactions': []}, u'pairwise_linear': {u'behavior': u'DISABLED'}, u'categoricals_count_transformer': {u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}, u'polynomial_combinations': {u'behavior': u'DISABLED'}, u'numericals_clustering': {u'k': 0, u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}}} [2020-05-13 07:52:20,097] [6211/MainThread] [INFO] [dataiku.doctor.multiframe] No feature selection to perform [2020-05-13 07:52:20,097] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Fitting preprocessors [2020-05-13 07:52:20,097] [6211/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 8610 [2020-05-13 07:52:20,097] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RemapValueToOutput [2020-05-13 07:52:20,099] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput [2020-05-13 07:52:20,099] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {u'age_first_order': 48.06687777074097, u'pages_visited_avg': 11.29058660591608} [2020-05-13 07:52:20,100] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (age_first_order) [2020-05-13 07:52:20,101] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] Rescale age_first_order (avg=48.06687777074098 std=15.657119563420252 shift=48.0668777707 inv_scale=0.0611589948427) [2020-05-13 07:52:20,102] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] Rescaled age_first_order (avg=1.269857879966149e-16 std=0.9575736946304232) nulls=0 [2020-05-13 07:52:20,102] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (pages_visited_avg) [2020-05-13 07:52:20,103] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] Rescale pages_visited_avg (avg=11.29058660591608 std=2.7594693034135367 shift=11.2905866059 inv_scale=0.362388521142) [2020-05-13 07:52:20,103] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] Rescaled pages_visited_avg (avg=-5.4776160378718526e-17 std=0.9999999999999999) nulls=0 [2020-05-13 07:52:20,103] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(num_flagonly) [2020-05-13 07:52:20,103] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (ip_address_geopoint) [2020-05-13 07:52:20,107] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(8610, 102) nnz=8610 [2020-05-13 07:52:20,107] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (user_agent_os) [2020-05-13 07:52:20,111] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(8610, 5) nnz=8610 [2020-05-13 07:52:20,111] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (campaign) [2020-05-13 07:52:20,115] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(8610, 4) nnz=8610 [2020-05-13 07:52:20,115] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (user_agent_brand) [2020-05-13 07:52:20,119] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(8610, 7) nnz=8610 [2020-05-13 07:52:20,119] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (ip_address_country) [2020-05-13 07:52:20,123] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(8610, 102) nnz=8610 [2020-05-13 07:52:20,123] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (gender) [2020-05-13 07:52:20,126] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(8610, 4) nnz=8610 [2020-05-13 07:52:20,126] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput [2020-05-13 07:52:20,126] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {} [2020-05-13 07:52:20,126] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(cat_flagpresence) [2020-05-13 07:52:20,126] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput [2020-05-13 07:52:20,126] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {} [2020-05-13 07:52:20,126] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(interaction) [2020-05-13 07:52:20,126] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RealignTarget [2020-05-13 07:52:20,126] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] Realign target series = (8610,) [2020-05-13 07:52:20,127] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] After realign target: (8610,) [2020-05-13 07:52:20,127] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DropRowsWhereNoTarget [2020-05-13 07:52:20,127] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] Deleting 0 rows because no target [2020-05-13 07:52:20,127] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] MF before = (8610, 226) target before = (8610,) [2020-05-13 07:52:20,128] [6211/MainThread] [INFO] [dataiku.doctor.multiframe] MultiFrame, dropping rows: [] [2020-05-13 07:52:20,133] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] After DRWNT input_df=(8610, 10) [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] MF after = (8610, 226) target after = (8610,) [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DumpPipelineState [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (Before feature selection) [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] input_df= (8610, 10) [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] current_mf=(8610, 226) [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] PPR: [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] target = ((8610,)) [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:EmitCurrentMFAsResult [2020-05-13 07:52:20,134] [6211/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 8610 [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DumpPipelineState [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (At end) [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] input_df= (8610, 10) [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] current_mf=(0, 0) [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] PPR: [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] UNPROCESSED = ((8610, 10)) [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] TRAIN = ((8610, 226)) [2020-05-13 07:52:20,134] [6211/MainThread] [DEBUG] [dku.ml.preprocessing] target = ((8610,)) [2020-05-13 07:52:20,135] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] END - Fitting preprocessors [2020-05-13 07:52:20,135] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Preprocessing train set [2020-05-13 07:52:20,136] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] END - Preprocessing train set [2020-05-13 07:52:20,136] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Preprocessing test set [2020-05-13 07:52:20,136] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] END - Preprocessing test set [2020-05-13 07:52:20,142] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] START - Fitting model /home/vinhdiesal/DATA_DIR/code-envs/python/python2_7/local/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`. from ._conv import register_converters as _register_converters Using TensorFlow backend. [2020-05-13 07:52:20,970] [6211/MainThread] [WARNING] [tensorflow] From /home/vinhdiesal/dataiku-dss-7.0.2/python/dataiku/doctor/deep_learning/gpu.py:20: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead. [2020-05-13 07:52:20,970] [6211/MainThread] [WARNING] [tensorflow] From /home/vinhdiesal/dataiku-dss-7.0.2/python/dataiku/doctor/deep_learning/gpu.py:24: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead. 2020-05-13 07:52:20.971325: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2020-05-13 07:52:20.991927: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1999965000 Hz 2020-05-13 07:52:20.992117: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55baa6b05210 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-05-13 07:52:20.992129: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2020-05-13 07:52:20.993770: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2020-05-13 07:52:21.392333: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-05-13 07:52:21.587849: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-05-13 07:52:21.589319: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55baa3d46a50 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2020-05-13 07:52:21.589340: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): GeForce MX150, Compute Capability 6.1 2020-05-13 07:52:21.589344: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (1): GeForce RTX 2080 Ti, Compute Capability 7.5 2020-05-13 07:52:21.589607: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-05-13 07:52:21.590358: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: name: GeForce MX150 major: 6 minor: 1 memoryClockRate(GHz): 1.5315 pciBusID: 0000:02:00.0 2020-05-13 07:52:21.590425: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-05-13 07:52:21.591497: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 1 with properties: name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.635 pciBusID: 0000:08:00.0 2020-05-13 07:52:21.591579: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory 2020-05-13 07:52:21.591616: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory 2020-05-13 07:52:21.591651: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory 2020-05-13 07:52:21.591684: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory 2020-05-13 07:52:21.591718: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory 2020-05-13 07:52:21.591751: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory 2020-05-13 07:52:21.593816: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2020-05-13 07:52:21.593829: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform. Skipping registering GPU devices... 2020-05-13 07:52:21.593850: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-05-13 07:52:21.593855: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0 1 2020-05-13 07:52:21.593859: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N N 2020-05-13 07:52:21.593861: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 1: N N [2020-05-13 07:52:21,594] [6211/MainThread] [WARNING] [tensorflow] From /home/vinhdiesal/DATA_DIR/code-envs/python/python2_7/local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:507: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead. [2020-05-13 07:52:21,595] [6211/MainThread] [WARNING] [tensorflow] From /home/vinhdiesal/DATA_DIR/code-envs/python/python2_7/local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:68: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead. [2020-05-13 07:52:21,595] [6211/MainThread] [WARNING] [tensorflow] From /home/vinhdiesal/DATA_DIR/code-envs/python/python2_7/local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:3831: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead. [2020-05-13 07:52:21,615] [6211/MainThread] [WARNING] [tensorflow] From /home/vinhdiesal/DATA_DIR/code-envs/python/python2_7/local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:167: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead. [2020-05-13 07:52:21,615] [6211/MainThread] [WARNING] [tensorflow] From /home/vinhdiesal/DATA_DIR/code-envs/python/python2_7/local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:183: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead. [2020-05-13 07:52:21,615] [6211/MainThread] [WARNING] [tensorflow] From /home/vinhdiesal/DATA_DIR/code-envs/python/python2_7/local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:192: The name tf.is_variable_initialized is deprecated. Please use tf.compat.v1.is_variable_initialized instead. [2020-05-13 07:52:21,626] [6211/MainThread] [WARNING] [tensorflow] From /home/vinhdiesal/DATA_DIR/code-envs/python/python2_7/local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py:199: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead. [2020-05-13 07:52:21,638] [6211/MainThread] [INFO] [dataiku.doctor.utils.listener] END - Fitting model [2020/05/13-07:52:21.639] [MRT-552] [INFO] [dku.block.link.interaction] - Check result for nullity exceptionIfNull=true result=null Traceback (most recent call last): File "/home/vinhdiesal/dataiku-dss-7.0.2/python/dataiku/doctor/server.py", line 47, in serve ret = api_command(arg) File "/home/vinhdiesal/dataiku-dss-7.0.2/python/dataiku/doctor/dkuapi.py", line 45, in aux return api(**kwargs) File "/home/vinhdiesal/dataiku-dss-7.0.2/python/dataiku/doctor/commands.py", line 453, in train_prediction_keras pipeline.generated_features_mapping) File "/home/vinhdiesal/dataiku-dss-7.0.2/python/dataiku/doctor/prediction_entrypoints.py", line 437, in prediction_train_model_keras generated_features_mapping) File "/home/vinhdiesal/dataiku-dss-7.0.2/python/dataiku/doctor/deep_learning/keras_support.py", line 81, in get_keras_model model = multi_gpu_model(base_model, num_gpus) File "/home/vinhdiesal/DATA_DIR/code-envs/python/python2_7/local/lib/python2.7/site-packages/keras/utils/multi_gpu_utils.py", line 138, in multi_gpu_model available_devices)) ValueError: To call `multi_gpu_model` with `gpus=2`, we expect the following devices to be available: ['/cpu:0', '/gpu:0', '/gpu:1']. However this machine only has: ['/cpu:0', '/xla_cpu:0', '/xla_gpu:0', '/xla_gpu:1']. Try reducing `gpus`. [2020/05/13-07:52:22.050] [KNL-python-single-command-kernel-monitor-556] [INFO] [dku.kernels] - Process done with code 0 [2020/05/13-07:52:22.051] [KNL-python-single-command-kernel-monitor-556] [INFO] [dip.tickets] - Destroying API ticket for analysis-ml-DKU_TUTORIAL_ML-vUiuP5J on behalf of admin [2020/05/13-07:52:22.051] [MRT-552] [INFO] [dku.kernels] - Getting kernel tail [2020/05/13-07:52:22.052] [MRT-552] [INFO] [dku.kernels] - Trying to enrich exception: com.dataiku.dip.io.SocketBlockLinkKernelException: Failed to train : : To call `multi_gpu_model` with `gpus=2`, we expect the following devices to be available: ['/cpu:0', '/gpu:0', '/gpu:1']. However this machine only has: ['/cpu:0', '/xla_cpu:0', '/xla_gpu:0', '/xla_gpu:1']. Try reducing `gpus`. from kernel com.dataiku.dip.analysis.coreservices.AnalysisMLKernel@2bf974a3 process=null pid=?? retcode=0 [2020/05/13-07:52:22.052] [MRT-552] [WARN] [dku.analysis.ml.python] - Training failed com.dataiku.dip.io.SocketBlockLinkKernelException: Failed to train : : To call `multi_gpu_model` with `gpus=2`, we expect the following devices to be available: ['/cpu:0', '/gpu:0', '/gpu:1']. However this machine only has: ['/cpu:0', '/xla_cpu:0', '/xla_gpu:0', '/xla_gpu:1']. Try reducing `gpus`. at com.dataiku.dip.io.SocketBlockLinkInteraction.throwExceptionFromPython(SocketBlockLinkInteraction.java:302) at com.dataiku.dip.io.SocketBlockLinkInteraction$AsyncResult.checkException(SocketBlockLinkInteraction.java:215) at com.dataiku.dip.io.SocketBlockLinkInteraction$AsyncResult.get(SocketBlockLinkInteraction.java:190) at com.dataiku.dip.io.SingleCommandKernelLink$1.call(SingleCommandKernelLink.java:208) at com.dataiku.dip.analysis.ml.prediction.PredictionTrainAdditionalThread.process(PredictionTrainAdditionalThread.java:74) at com.dataiku.dip.analysis.ml.shared.PRNSTrainThread.run(PRNSTrainThread.java:137) [2020/05/13-07:52:22.054] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.ml.python] T-hrRkST3I - Processing thread joined ... [2020/05/13-07:52:22.055] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.ml.python] T-hrRkST3I - Joining processing thread ... [2020/05/13-07:52:22.055] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.ml.python] T-hrRkST3I - Processing thread joined ... [2020/05/13-07:52:22.055] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.prediction] T-hrRkST3I - Train done [2020/05/13-07:52:22.055] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.prediction] T-hrRkST3I - Train done [2020/05/13-07:52:22.057] [FT-TrainWorkThread-RYn9Xj41-551] [INFO] [dku.analysis.prediction] T-hrRkST3I - Publishing mltask-train-done reflected event