[01:11:07] [INFO] [dku.utils] - F1 Score: 0.9352678571428572 [01:11:07] [INFO] [dku.utils] - ROC AUC Score: 0.8291498642087071 [01:11:07] [INFO] [dku.utils] - Accuracy Score: 0.8784067085953878 [01:11:07] [INFO] [dku.utils] - Recall Score: 1.0 [01:11:07] [INFO] [dku.utils] - Precision Score: 0.8784067085953878 [01:11:09] [INFO] [dku.utils] - /data/dataiku/dss_data/code-envs/python/test_snowparkML/lib/python3.8/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils. [01:11:09] [INFO] [dku.utils] - warnings.warn("Setuptools is replacing distutils.") [01:11:10] [INFO] [dku.utils] - Successfully registered model 'loan_default_snowpark_models_code'. [01:11:10] [INFO] [dku.utils] - *************** Recipe code failed ************** [01:11:10] [INFO] [dku.utils] - Begin Python stack [01:11:10] [INFO] [dku.utils] - Traceback (most recent call last): [01:11:10] [INFO] [dku.utils] - File "/data/dataiku/dss_data/jobs/TESTPROJECTNAKHOEKIM/Build_LOAN_REQUESTS_TRAIN_SET__NP___snowpark_ml_trained_models__NP___LOAN_REQUESTS_TEST_SET__NP__2024-01-09T01-10-08.992/compute_LOAN_REQUESTS_TRAIN_SET_NP/python-recipe/pyoutFc162fvqH8sj/python-exec-wrapper.py", line 204, in [01:11:10] [INFO] [dku.utils] - exec(f.read()) [01:11:10] [INFO] [dku.utils] - File "", line 383, in [01:11:10] [INFO] [dku.utils] - File "/data/dataiku/dss_data/code-envs/python/test_snowparkML/lib/python3.8/site-packages/mlflow/sklearn/__init__.py", line 407, in log_model [01:11:10] [INFO] [dku.utils] - return Model.log( [01:11:10] [INFO] [dku.utils] - File "/data/dataiku/dss_data/code-envs/python/test_snowparkML/lib/python3.8/site-packages/mlflow/models/model.py", line 384, in log [01:11:10] [INFO] [dku.utils] - mlflow.register_model( [01:11:10] [INFO] [dku.utils] - File "/data/dataiku/dss_data/code-envs/python/test_snowparkML/lib/python3.8/site-packages/mlflow/tracking/_model_registry/fluent.py", line 79, in register_model [01:11:10] [INFO] [dku.utils] - create_version_response = client.create_model_version( [01:11:10] [INFO] [dku.utils] - File "/data/dataiku/dss_data/code-envs/python/test_snowparkML/lib/python3.8/site-packages/mlflow/tracking/client.py", line 2446, in create_model_version [01:11:10] [INFO] [dku.utils] - return self._get_registry_client().create_model_version( [01:11:10] [INFO] [dku.utils] - File "/data/dataiku/dss_data/code-envs/python/test_snowparkML/lib/python3.8/site-packages/mlflow/tracking/_model_registry/client.py", line 203, in create_model_version [01:11:10] [INFO] [dku.utils] - mv = self.store.create_model_version(name, source, run_id, tags, run_link, description) [01:11:10] [INFO] [dku.utils] - File "/data/dataiku/dss_data/code-envs/python/test_snowparkML/lib/python3.8/site-packages/mlflow/store/model_registry/rest_store.py", line 254, in create_model_version [01:11:10] [INFO] [dku.utils] - response_proto = self._call_endpoint(CreateModelVersion, req_body) [01:11:10] [INFO] [dku.utils] - File "/data/dataiku/dss_data/code-envs/python/test_snowparkML/lib/python3.8/site-packages/mlflow/store/model_registry/rest_store.py", line 64, in _call_endpoint [01:11:10] [INFO] [dku.utils] - return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto) [01:11:10] [INFO] [dku.utils] - File "/data/dataiku/dss_data/code-envs/python/test_snowparkML/lib/python3.8/site-packages/mlflow/utils/rest_utils.py", line 280, in call_endpoint [01:11:10] [INFO] [dku.utils] - response = verify_rest_response(response, endpoint) [01:11:10] [INFO] [dku.utils] - File "/data/dataiku/dss_data/code-envs/python/test_snowparkML/lib/python3.8/site-packages/mlflow/utils/rest_utils.py", line 206, in verify_rest_response [01:11:10] [INFO] [dku.utils] - raise RestException(json.loads(response.text)) [01:11:10] [INFO] [dku.utils] - mlflow.exceptions.RestException: INTERNAL_ERROR: Not Found: /dip/publicapi/api/2.0/preview/mlflow/model-versions/create [01:11:10] [INFO] [dku.utils] - End Python stack [01:11:10] [INFO] [dku.utils] - 2024-01-09 01:11:10,699 INFO Check if spark is available [01:11:10] [INFO] [dku.utils] - 2024-01-09 01:11:10,700 INFO Not stopping a spark context: No module named 'pyspark' [01:11:11] [INFO] [dku.utils] - 2024-01-09 01:11:11,190 210224 INFO [Child] Process 210226 exited with exit=1 signal=0 [01:11:11] [INFO] [dku.utils] - 2024-01-09 01:11:11,190 210224 INFO Full child code: 1 [01:11:11] [WARN] [dku.resource] - stat file for pid 210226 does not exist. Process died? [01:11:11] [DEBUG] [dku.resourceusage] - Reporting completion of CRU:{"context":{"type":"JOB_ACTIVITY","authIdentifier":"nakhoe.kim","projectKey":"TESTPROJECTNAKHOEKIM","jobId":"Build_LOAN_REQUESTS_TRAIN_SET__NP___snowpark_ml_trained_models__NP___LOAN_REQUESTS_TEST_SET__NP__2024-01-09T01-10-08.992","activityId":"compute_LOAN_REQUESTS_TRAIN_SET_NP","activityType":"recipe","recipeType":"python","recipeName":"compute_LOAN_REQUESTS_TRAIN_SET"},"type":"LOCAL_PROCESS","id":"6MfWnfnl1sFJQXwp","startTime":1704762616355,"localProcess":{"pid":210226,"commandName":"/data/dataiku/dss_data/code-envs/python/test_snowparkML/bin/python","cpuUserTimeMS":5120,"cpuSystemTimeMS":1500,"cpuChildrenUserTimeMS":0,"cpuChildrenSystemTimeMS":0,"cpuTotalMS":6620,"cpuCurrent":0.007998400319936013,"cpuAverageOverPast60Seconds":0.0,"vmSizeMB":3406,"vmRSSMB":266,"vmHWMMB":266,"vmRSSAnonMB":168,"vmDataMB":1429,"vmSizePeakMB":3470,"vmRSSPeakMB":266,"vmRSSTotalMBS":12990,"majorFaults":6,"childrenMajorFaults":0}} [01:11:11] [INFO] [dip.exec.resultHandler] - Error file found, trying to throw it: /data/dataiku/dss_data/jobs/TESTPROJECTNAKHOEKIM/Build_LOAN_REQUESTS_TRAIN_SET__NP___snowpark_ml_trained_models__NP___LOAN_REQUESTS_TEST_SET__NP__2024-01-09T01-10-08.992/compute_LOAN_REQUESTS_TRAIN_SET_NP/python-recipe/pyoutFc162fvqH8sj/error.json [01:11:11] [INFO] [dip.exec.resultHandler] - Raw error is{"errorType":"\u003cclass \u0027mlflow.exceptions.RestException\u0027\u003e","message":"INTERNAL_ERROR: Not Found: /dip/publicapi/api/2.0/preview/mlflow/model-versions/create","detailedMessage":"At line 383: \u003cclass \u0027mlflow.exceptions.RestException\u0027\u003e: INTERNAL_ERROR: Not Found: /dip/publicapi/api/2.0/preview/mlflow/model-versions/create","stackTrace":[]} [01:11:11] [INFO] [dip.exec.resultHandler] - After enrichment of error file, error is: {"errorType":"\u003cclass \u0027mlflow.exceptions.RestException\u0027\u003e","message":"Error in Python process: INTERNAL_ERROR: Not Found: /dip/publicapi/api/2.0/preview/mlflow/model-versions/create","detailedMessage":"Error in Python process: At line 383: \u003cclass \u0027mlflow.exceptions.RestException\u0027\u003e: INTERNAL_ERROR: Not Found: /dip/publicapi/api/2.0/preview/mlflow/model-versions/create","stackTrace":[]} [01:11:11] [INFO] [dku.ml.distributed.pool] - Closing worker pool pool-8ohetq8cnpv5b6sp [01:11:11] [INFO] [dku.ml.distributed.service] - Unregistered worker pool: pool-8ohetq8cnpv5b6sp [01:11:11] [INFO] [dku.flow.activity] - Run thread failed for activity compute_LOAN_REQUESTS_TRAIN_SET_NP com.dataiku.common.server.APIError$SerializedErrorException: Error in Python process: At line 383: : INTERNAL_ERROR: Not Found: /dip/publicapi/api/2.0/preview/mlflow/model-versions/create at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.throwFromErrorFileIfPossible(JobExecutionResultHandler.java:106) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.throwFromErrorFileOrLogs(JobExecutionResultHandler.java:39) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.throwFromErrorFileOrLogs(JobExecutionResultHandler.java:34) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.handleExecutionResult(JobExecutionResultHandler.java:26) at com.dataiku.dip.dataflow.exec.AbstractCodeBasedActivityRunner.execute(AbstractCodeBasedActivityRunner.java:70) at com.dataiku.dip.dataflow.exec.AbstractPythonRecipeRunner.executeScript(AbstractPythonRecipeRunner.java:64) at com.dataiku.dip.recipes.code.python.PythonRecipeRunner.run(PythonRecipeRunner.java:70) at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:374) [01:11:11] [INFO] [dku.flow.activity] running compute_LOAN_REQUESTS_TRAIN_SET_NP - activity is finished [01:11:11] [ERROR] [dku.flow.activity] running compute_LOAN_REQUESTS_TRAIN_SET_NP - Activity failed com.dataiku.common.server.APIError$SerializedErrorException: Error in Python process: At line 383: : INTERNAL_ERROR: Not Found: /dip/publicapi/api/2.0/preview/mlflow/model-versions/create at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.throwFromErrorFileIfPossible(JobExecutionResultHandler.java:106) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.throwFromErrorFileOrLogs(JobExecutionResultHandler.java:39) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.throwFromErrorFileOrLogs(JobExecutionResultHandler.java:34) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.handleExecutionResult(JobExecutionResultHandler.java:26) at com.dataiku.dip.dataflow.exec.AbstractCodeBasedActivityRunner.execute(AbstractCodeBasedActivityRunner.java:70) at com.dataiku.dip.dataflow.exec.AbstractPythonRecipeRunner.executeScript(AbstractPythonRecipeRunner.java:64) at com.dataiku.dip.recipes.code.python.PythonRecipeRunner.run(PythonRecipeRunner.java:70) at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:374) [01:11:11] [INFO] [dku.flow.activity] running compute_LOAN_REQUESTS_TRAIN_SET_NP - Executing default post-activity lifecycle hook [01:11:11] [INFO] [dku.flow.activity] running compute_LOAN_REQUESTS_TRAIN_SET_NP - Done post-activity tasks