TypeError: copy() got an unexpected keyword argument 'deep'
currently we have 2 prediction model and we setup the prediction model in the next recipe
# setup efficiency function
efficiency = lambda b: pred_1.predict(np.array([b]))[0] /pred_2.predict(np.array([b]))[0]
# run optimization algorithm
result = optimize.minimize(efficiency,x_init.iloc[0], constraints=constraints, method='COBYLA')
with error msg
2022/12/16-02:21:52.610] [null-err-40] [INFO] [dku.utils] - File "/opt/dataiku-dss-11.0.2/python/dataiku/core/saved_model.py", line 232, in _renormalize_dates [2022/12/16-02:21:52.610] [null-err-40] [INFO] [dku.utils] - df = df.copy(deep=False) [2022/12/16-02:21:52.610] [null-err-40] [INFO] [dku.utils] - TypeError: copy() got an unexpected keyword argument 'deep' [2022/12/16-02:21:52.611] [null-err-40] [INFO] [dku.utils] - End Python stack [2022/12/16-02:21:52.611] [null-err-40] [INFO] [dku.utils] - 2022-12-16 02:21:52,606 INFO Check if spark is available [2022/12/16-02:21:52.611] [null-err-40] [INFO] [dku.utils] - 2022-12-16 02:21:52,607 INFO Not stopping a spark context: No module named 'pyspark' [2022/12/16-02:21:52.839] [FRT-37-FlowRunnable] [WARN] [dku.resource] act.compute_NL9Ayu9Z_NP - stat file for pid 4623 does not exist. Process died? [2022/12/16-02:21:52.840] [FRT-37-FlowRunnable] [DEBUG] [dku.resourceusage] act.compute_NL9Ayu9Z_NP - Reporting completion of CRU:{"context":{"type":"JOB_ACTIVITY","authIdentifier":"sucipto_hardinata","projectKey":"SPOPTIMIZATION","jobId":"Build_projection_output__NP__2022-12-16T02-21-49.030","activityId":"compute_NL9Ayu9Z_NP","activityType":"recipe","recipeType":"python","recipeName":"compute_NL9Ayu9Z"},"type":"LOCAL_PROCESS","id":"pt38VccsgtPeZhdd","startTime":1671157310112,"localProcess":{"pid":4623,"commandName":"/opt/DATA_DIR/code-envs/python/Python37/bin/python","cpuUserTimeMS":20,"cpuSystemTimeMS":0,"cpuChildrenUserTimeMS":0,"cpuChildrenSystemTimeMS":0,"cpuTotalMS":20,"cpuCurrent":0.0,"vmSizeMB":11,"vmRSSMB":7,"vmHWMMB":7,"vmRSSAnonMB":3,"vmDataMB":3,"vmSizePeakMB":11,"vmRSSPeakMB":7,"vmRSSTotalMBS":0,"majorFaults":0,"childrenMajorFaults":0}} [2022/12/16-02:21:52.841] [FRT-37-FlowRunnable] [INFO] [dip.exec.resultHandler] act.compute_NL9Ayu9Z_NP - Error file found, trying to throw it: /opt/DATA_DIR/jobs/SPOPTIMIZATION/Build_projection_output__NP__2022-12-16T02-21-49.030/compute_NL9Ayu9Z_NP/python-recipe/pyout455zZxomC3EN/error.json [2022/12/16-02:21:52.841] [FRT-37-FlowRunnable] [INFO] [dip.exec.resultHandler] act.compute_NL9Ayu9Z_NP - Raw error is{"errorType":"\u003cclass \u0027TypeError\u0027\u003e","message":"copy() got an unexpected keyword argument \u0027deep\u0027","detailedMessage":"At line 81: \u003cclass \u0027TypeError\u0027\u003e: copy() got an unexpected keyword argument \u0027deep\u0027","stackTrace":[]} [2022/12/16-02:21:52.841] [FRT-37-FlowRunnable] [INFO] [dip.exec.resultHandler] act.compute_NL9Ayu9Z_NP - After enrichment of error file, error is: {"errorType":"\u003cclass \u0027TypeError\u0027\u003e","message":"Error in Python process: copy() got an unexpected keyword argument \u0027deep\u0027","detailedMessage":"Error in Python process: At line 81: \u003cclass \u0027TypeError\u0027\u003e: copy() got an unexpected keyword argument \u0027deep\u0027","stackTrace":[]} [2022/12/16-02:21:52.845] [FRT-37-FlowRunnable] [INFO] [dku.flow.activity] act.compute_NL9Ayu9Z_NP - Run thread failed for activity compute_NL9Ayu9Z_NP com.dataiku.common.server.APIError$SerializedErrorException: Error in Python process: At line 81: <class 'TypeError'>: copy() got an unexpected keyword argument 'deep' at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.throwFromErrorFileIfPossible(JobExecutionResultHandler.java:106) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.throwFromErrorFileOrLogs(JobExecutionResultHandler.java:39) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.throwFromErrorFileOrLogs(JobExecutionResultHandler.java:34) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.handleExecutionResult(JobExecutionResultHandler.java:26) at com.dataiku.dip.dataflow.exec.AbstractCodeBasedActivityRunner.execute(AbstractCodeBasedActivityRunner.java:73) at com.dataiku.dip.dataflow.exec.AbstractPythonRecipeRunner.executeScript(AbstractPythonRecipeRunner.java:57) at com.dataiku.dip.recipes.code.python.PythonRecipeRunner.run(PythonRecipeRunner.java:72) at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:374) [2022/12/16-02:21:53.022] [ActivityExecutor-33] [INFO] [dku.flow.activity] running compute_NL9Ayu9Z_NP - activity is finished [2022/12/16-02:21:53.023] [ActivityExecutor-33] [ERROR] [dku.flow.activity] running compute_NL9Ayu9Z_NP - Activity failed com.dataiku.common.server.APIError$SerializedErrorException: Error in Python process: At line 81: <class 'TypeError'>: copy() got an unexpected keyword argument 'deep' at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.throwFromErrorFileIfPossible(JobExecutionResultHandler.java:106) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.throwFromErrorFileOrLogs(JobExecutionResultHandler.java:39) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.throwFromErrorFileOrLogs(JobExecutionResultHandler.java:34) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.handleExecutionResult(JobExecutionResultHandler.java:26) at com.dataiku.dip.dataflow.exec.AbstractCodeBasedActivityRunner.execute(AbstractCodeBasedActivityRunner.java:73) at com.dataiku.dip.dataflow.exec.AbstractPythonRecipeRunner.executeScript(AbstractPythonRecipeRunner.java:57) at com.dataiku.dip.recipes.code.python.PythonRecipeRunner.run(PythonRecipeRunner.java:72) at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:374) [2022/12/16-02:21:53.023] [ActivityExecutor-33] [INFO] [dku.flow.activity] running compute_NL9Ayu9Z_NP - Executing default post-activity lifecycle hook [2022/12/16-02:21:53.026] [ActivityExecutor-33] [INFO] [dku.flow.activity] running compute_NL9Ayu9Z_NP - Done post-activity tasks
Operating system used: ubuntu
Answers
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,226 Dataiker
Hi @Sup
,
The error :
[2022/12/16-02:21:52.610] [null-err-40] [INFO] [dku.utils] - df = df.copy(deep=False)
[2022/12/16-02:21:52.610] [null-err-40] [INFO] [dku.utils] - TypeError: copy() got an unexpected keyword argument 'deep'
This seems related to the panda's version and should be fixed with pandas 1.0.1+
Note you can use pandas 1.1-1.3 in DSS 0.0.4+ https://doc.dataiku.com/dss/latest/release_notes/10.0.html#version-10-0-4-march-7th-2022Can you try updating the panda's version if that doesn't help please raise a support ticket with the job diagnostics and code env diagnostics.
Thanks