[2021/12/21-13:40:07.863] [ActivityExecutor-44] [INFO] [dku] running compute_output_gen_python_NP - ---------------------------------------- [2021/12/21-13:40:07.863] [ActivityExecutor-44] [INFO] [dku] running compute_output_gen_python_NP - DSS startup: jek version:9.0.4 [2021/12/21-13:40:07.863] [ActivityExecutor-44] [INFO] [dku] running compute_output_gen_python_NP - DSS home: /dataiku/dss_data [2021/12/21-13:40:07.863] [ActivityExecutor-44] [INFO] [dku] running compute_output_gen_python_NP - OS: Linux 4.18.0-305.19.1.el8_4.x86_64 amd64 - Java: Red Hat, Inc. 1.8.0_275 [2021/12/21-13:40:07.863] [ActivityExecutor-44] [INFO] [dku.flow.jobrunner] running compute_output_gen_python_NP - Allocated a slot for this activity! [2021/12/21-13:40:07.864] [ActivityExecutor-44] [INFO] [dku.flow.jobrunner] running compute_output_gen_python_NP - Run activity [2021/12/21-13:40:07.879] [ActivityExecutor-44] [INFO] [dku.flow.activity] running compute_output_gen_python_NP - Executing default pre-activity lifecycle hook [2021/12/21-13:40:07.894] [ActivityExecutor-44] [INFO] [dku.flow.activity] running compute_output_gen_python_NP - Checking if sources are ready [2021/12/21-13:40:07.907] [ActivityExecutor-44] [INFO] [dku.flow.activity] running compute_output_gen_python_NP - Checked source readiness SPINETTA_FLUIDS_FINGERPRINTER.Outputs -> true [2021/12/21-13:40:07.942] [ActivityExecutor-44] [INFO] [dku.dataset.hash] running compute_output_gen_python_NP - managed non-FS dataset, computed new hash for SPINETTA_FLUIDS_FINGERPRINTER.batches_types_copy/NP: uyBcAIMqpIibcSfBLXAy9g-giSb5jQ/PMPSf0lyd+Deiw, based on 2 propagated from upstream [2021/12/21-13:40:07.943] [ActivityExecutor-44] [INFO] [dku.flow.activity] running compute_output_gen_python_NP - Checked source readiness SPINETTA_FLUIDS_FINGERPRINTER.batches_types_copy -> true [2021/12/21-13:40:07.953] [ActivityExecutor-44] [INFO] [dku.dataset.hash] running compute_output_gen_python_NP - managed non-FS dataset, computed new hash for SPINETTA_FLUIDS_FINGERPRINTER.last_hour_extract/NP: d+s8dL9Kyozq7akgb1Fn0g-mCC+3QAPjDXINo+7rEUPBw, based on 2 propagated from upstream [2021/12/21-13:40:07.954] [ActivityExecutor-44] [INFO] [dku.flow.activity] running compute_output_gen_python_NP - Checked source readiness SPINETTA_FLUIDS_FINGERPRINTER.last_hour_extract -> true [2021/12/21-13:40:07.955] [ActivityExecutor-44] [DEBUG] [dku.flow.activity] running compute_output_gen_python_NP - Computing hashes to propagate BEFORE activity [2021/12/21-13:40:07.956] [ActivityExecutor-44] [DEBUG] [dku.flow.activity] running compute_output_gen_python_NP - Recorded 3 hashes before activity run [2021/12/21-13:40:07.956] [ActivityExecutor-44] [DEBUG] [dku.flow.activity] running compute_output_gen_python_NP - Building recipe runner of type [2021/12/21-13:40:07.970] [ActivityExecutor-44] [DEBUG] [dku.job.activity] running compute_output_gen_python_NP - Filling source sizes [2021/12/21-13:40:07.991] [ActivityExecutor-44] [DEBUG] [dku.job.activity] running compute_output_gen_python_NP - Done filling source sizes [2021/12/21-13:40:07.992] [ActivityExecutor-44] [DEBUG] [dku.flow.activity] running compute_output_gen_python_NP - Recipe runner built, will use 1 thread(s) [2021/12/21-13:40:07.992] [ActivityExecutor-44] [DEBUG] [dku.flow.activity] running compute_output_gen_python_NP - Starting execution thread: com.dataiku.dip.recipes.code.python.PythonRecipeRunner@49e17993 [2021/12/21-13:40:07.994] [ActivityExecutor-44] [DEBUG] [dku.flow.activity] running compute_output_gen_python_NP - Execution threads started, waiting for activity end [2021/12/21-13:40:07.996] [FRT-49-FlowRunnable] [INFO] [dku.flow.activity] - Run thread for activity compute_output_gen_python_NP starting [2021/12/21-13:40:07.997] [FRT-49-FlowRunnable] [INFO] [dku.flow.python] - Starting execution of user's Python code [2021/12/21-13:40:08.005] [FRT-49-FlowRunnable] [INFO] [dku.venv.selector] - Select code env lang=PYTHON projectSelection={"mode":"INHERIT","preventOverride":false} globalDefault=Python3_6 [2021/12/21-13:40:08.007] [FRT-49-FlowRunnable] [INFO] [dku.flow.abstract.python] - Dumping Python script to /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/script.py [2021/12/21-13:40:08.024] [FRT-49-FlowRunnable] [WARN] [dku.code.projectLibs] - External libraries file not found: /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/localconfig/projects/SPINETTA_FLUIDS_FINGERPRINTER/lib/external-libraries.json [2021/12/21-13:40:08.026] [FRT-49-FlowRunnable] [INFO] [dku.code.projectLibs] - EXTERNAL LIBS FROM SPINETTA_FLUIDS_FINGERPRINTER is {"gitReferences":{},"pythonPath":["python"],"rsrcPath":["R"],"importLibrariesFromProjects":[]} [2021/12/21-13:40:08.027] [FRT-49-FlowRunnable] [INFO] [dku.code.projectLibs] - chunkFolder is /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/localconfig/projects/SPINETTA_FLUIDS_FINGERPRINTER/lib/R [2021/12/21-13:40:08.029] [FRT-49-FlowRunnable] [INFO] [dku.recipes.code.base] - Writing dku-exec-env for local execution in /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/remote-run-env-def.json [2021/12/21-13:40:08.038] [FRT-49-FlowRunnable] [INFO] [dku.code.envs.resolution] - Executing Python activity in env: Python3_6 [2021/12/21-13:40:08.043] [FRT-49-FlowRunnable] [INFO] [dku.flow.abstract.python] - Execute activity command: ["/dataiku/dataiku-dss-9.0.4/resources/code-envs/conda-support/conda-run.sh","/dataiku/dss_data/code-envs/python/Python3_6","python","-u","/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/python-exec-wrapper.py","/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/script.py"] [2021/12/21-13:40:08.048] [FRT-49-FlowRunnable] [INFO] [dku.recipes.code.base] - Run command insecurely, from user dataiku [2021/12/21-13:40:08.054] [FRT-49-FlowRunnable] [INFO] [dku.security.process] - Setting exclusive access on /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V [2021/12/21-13:40:08.056] [FRT-49-FlowRunnable] [DEBUG] [dku.security.fsacl] - exec chown: sudo -n /etc/dataiku-security/y5nrSSq74Fzg7kQNTQrqj6zw/execwrapper.sh acls chown --path /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V --owner khou2892 [2021/12/21-13:40:08.133] [null-err-51] [INFO] [dku.utils] - 2021-12-21 13:40:08,132 2914043 INFO Reading security conf from /etc/dataiku-security/y5nrSSq74Fzg7kQNTQrqj6zw [2021/12/21-13:40:08.134] [null-err-51] [INFO] [dku.utils] - 2021-12-21 13:40:08,133 2914043 INFO Allowed path: /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V in /dataiku/dss_data [2021/12/21-13:40:08.135] [null-err-51] [INFO] [dku.utils] - 2021-12-21 13:40:08,133 2914043 INFO Reading security conf from /etc/dataiku-security/y5nrSSq74Fzg7kQNTQrqj6zw [2021/12/21-13:40:08.135] [null-err-51] [INFO] [dku.utils] - 2021-12-21 13:40:08,134 2914043 INFO Allowed group ids: [1004] [2021/12/21-13:40:08.156] [null-err-51] [INFO] [dku.utils] - 2021-12-21 13:40:08,156 2914043 INFO User khou2892 belongs to groups: [1360, 1004] [2021/12/21-13:40:08.157] [null-err-51] [INFO] [dku.utils] - 2021-12-21 13:40:08,156 2914043 INFO User khou2892 belongs to group 1004 (dss_users) -> allowed [2021/12/21-13:40:08.157] [null-err-51] [INFO] [dku.utils] - 2021-12-21 13:40:08,156 2914043 INFO Executing chown -Rh: /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V -> khou2892 [2021/12/21-13:40:08.170] [Thread-26] [DEBUG] [dku.utils] - Process sudo done (return code 0) [2021/12/21-13:40:08.178] [FRT-49-FlowRunnable] [DEBUG] [dku.security.fsacl] - exec setfacl: sudo -n /etc/dataiku-security/y5nrSSq74Fzg7kQNTQrqj6zw/execwrapper.sh acls setfacl --path /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V --perm u::rwx --perm g::--- --perm o:--- --perm d:u:dataiku:rwx --perm u:dataiku:rwx --recursive [2021/12/21-13:40:08.253] [null-err-54] [INFO] [dku.utils] - 2021-12-21 13:40:08,252 2914051 INFO Reading security conf from /etc/dataiku-security/y5nrSSq74Fzg7kQNTQrqj6zw [2021/12/21-13:40:08.254] [null-err-54] [INFO] [dku.utils] - 2021-12-21 13:40:08,253 2914051 INFO Allowed path: /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V in /dataiku/dss_data [2021/12/21-13:40:08.254] [null-err-54] [INFO] [dku.utils] - 2021-12-21 13:40:08,253 2914051 INFO Executing: ['setfacl', '-RP', '-m', 'u::rwx,g::---,o:---,d:u:dataiku:rwx,u:dataiku:rwx', '/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V'] [2021/12/21-13:40:08.268] [Thread-29] [DEBUG] [dku.utils] - Process sudo done (return code 0) [2021/12/21-13:40:08.276] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - cgroup memory/DSS/${user}/recipes expanded to memory/DSS/khou2892/recipes [2021/12/21-13:40:08.277] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - cgroup cpu/DSS/recipes expanded to cpu/DSS/recipes [2021/12/21-13:40:08.277] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Will use cgroups ["memory/DSS/khou2892/recipes","cpu/DSS/recipes"] [2021/12/21-13:40:08.278] [FRT-49-FlowRunnable] [WARN] [dku.processes.cgroups] - Failed to create cgroup /sys/fs/cgroup/memory/DSS/khou2892/recipes [2021/12/21-13:40:08.278] [FRT-49-FlowRunnable] [WARN] [dku.processes.cgroups] - Failed to create cgroup /sys/fs/cgroup/cpu/DSS/recipes [2021/12/21-13:40:08.278] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Applying rules to used cgroups: [memory/DSS/khou2892/recipes, cpu/DSS/recipes] [2021/12/21-13:40:08.279] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Rule path memory/DSS/khou2892/ [2021/12/21-13:40:08.279] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Checking match of cgroup setting memory/DSS/khou2892/ vs used memory/DSS/khou2892/recipes/ [2021/12/21-13:40:08.280] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Used cgroupPath memory/DSS/khou2892/recipes/ matches definition for cgroup memory/DSS/${user} [2021/12/21-13:40:08.280] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Setting rules [{"key":"memory.limit_in_bytes","value":"40G"}] on /sys/fs/cgroup/memory/DSS/khou2892 [2021/12/21-13:40:08.280] [FRT-49-FlowRunnable] [WARN] [dku.processes.cgroups] - Ignoring set of limit on non-existing limit file: /sys/fs/cgroup/memory/DSS/khou2892/memory.limit_in_bytes [2021/12/21-13:40:08.281] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Checking match of cgroup setting memory/DSS/khou2892/ vs used cpu/DSS/recipes/ [2021/12/21-13:40:08.281] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Rule path memory/DSS/notebooks/khou2892/ [2021/12/21-13:40:08.282] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Checking match of cgroup setting memory/DSS/notebooks/khou2892/ vs used memory/DSS/khou2892/recipes/ [2021/12/21-13:40:08.282] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Checking match of cgroup setting memory/DSS/notebooks/khou2892/ vs used cpu/DSS/recipes/ [2021/12/21-13:40:08.282] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Rule path cpu/DSS/notebooks/ [2021/12/21-13:40:08.283] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Checking match of cgroup setting cpu/DSS/notebooks/ vs used memory/DSS/khou2892/recipes/ [2021/12/21-13:40:08.283] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Checking match of cgroup setting cpu/DSS/notebooks/ vs used cpu/DSS/recipes/ [2021/12/21-13:40:08.283] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Rule path memory/DSS/notebooks/ [2021/12/21-13:40:08.283] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Checking match of cgroup setting memory/DSS/notebooks/ vs used memory/DSS/khou2892/recipes/ [2021/12/21-13:40:08.284] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Checking match of cgroup setting memory/DSS/notebooks/ vs used cpu/DSS/recipes/ [2021/12/21-13:40:08.284] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Rule path memory/DSS/ [2021/12/21-13:40:08.284] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Checking match of cgroup setting memory/DSS/ vs used memory/DSS/khou2892/recipes/ [2021/12/21-13:40:08.285] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Used cgroupPath memory/DSS/khou2892/recipes/ matches definition for cgroup memory/DSS [2021/12/21-13:40:08.285] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Setting rules [{"key":"memory.limit_in_bytes","value":"100G"}] on /sys/fs/cgroup/memory/DSS [2021/12/21-13:40:08.285] [FRT-49-FlowRunnable] [WARN] [dku.processes.cgroups] - Ignoring set of limit on non-existing limit file: /sys/fs/cgroup/memory/DSS/memory.limit_in_bytes [2021/12/21-13:40:08.286] [FRT-49-FlowRunnable] [INFO] [dku.processes.cgroups] - Checking match of cgroup setting memory/DSS/ vs used cpu/DSS/recipes/ [2021/12/21-13:40:08.288] [FRT-49-FlowRunnable] [INFO] [dku.security.process] - Written config to /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/config.json [2021/12/21-13:40:08.291] [FRT-49-FlowRunnable] [INFO] [dku.security.process] - Starting wrapper: ["sudo","-n","/etc/dataiku-security/y5nrSSq74Fzg7kQNTQrqj6zw/execwrapper.sh","execute","/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/config.json"] [2021/12/21-13:40:08.363] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,362 2914056 INFO Reading security conf from /etc/dataiku-security/y5nrSSq74Fzg7kQNTQrqj6zw [2021/12/21-13:40:08.363] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,363 2914056 INFO Allowed group ids: [1004] [2021/12/21-13:40:08.380] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,379 2914056 INFO User khou2892 belongs to groups: [1360, 1004] [2021/12/21-13:40:08.381] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,380 2914056 INFO User khou2892 belongs to group 1004 (dss_users) -> allowed [2021/12/21-13:40:08.381] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,380 2914056 INFO Reading security conf from /etc/dataiku-security/y5nrSSq74Fzg7kQNTQrqj6zw [2021/12/21-13:40:08.381] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,381 2914056 INFO Allowed path: /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V in /dataiku/dss_data [2021/12/21-13:40:08.381] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,381 2914056 INFO [BF] Setting up process environment [2021/12/21-13:40:08.382] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,381 2914056 INFO Will set process in cgroup memory/DSS/khou2892/recipes [2021/12/21-13:40:08.382] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,381 2914056 INFO Reading security conf from /etc/dataiku-security/y5nrSSq74Fzg7kQNTQrqj6zw [2021/12/21-13:40:08.382] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,382 2914056 INFO Allowed path: /sys/fs/cgroup/memory/DSS/khou2892/recipes/tasks in /sys/fs/cgroup/memory/DSS [2021/12/21-13:40:08.382] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,382 2914056 INFO Will set process in cgroup cpu/DSS/recipes [2021/12/21-13:40:08.382] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,382 2914056 INFO Reading security conf from /etc/dataiku-security/y5nrSSq74Fzg7kQNTQrqj6zw [2021/12/21-13:40:08.383] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,382 2914056 INFO Allowed path: /sys/fs/cgroup/cpu,cpuacct/DSS/recipes/tasks in /sys/fs/cgroup/cpu,cpuacct/DSS [2021/12/21-13:40:08.383] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,382 2914056 INFO [BF] Writing wrapper pipes and pid to wrapper stdout [2021/12/21-13:40:08.384] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,383 2914056 INFO [Parent] Started process 2914058 [2021/12/21-13:40:08.384] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,383 2914058 INFO [Child] pgroup is set [2021/12/21-13:40:08.384] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,384 2914056 INFO [Parent] Setting process 2914058 in cgroup tasks file /sys/fs/cgroup/memory/DSS/khou2892/recipes/tasks [2021/12/21-13:40:08.385] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,384 2914058 INFO [Child] opening stdin pipe: /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/.stdin.w3kauqw1 [2021/12/21-13:40:08.385] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,384 2914056 ERROR [Parent] Failed to set process in cgroup tasks file [2021/12/21-13:40:08.385] [wrapper-stderr-56] [INFO] [dku.utils] - Traceback (most recent call last): [2021/12/21-13:40:08.386] [wrapper-stderr-56] [INFO] [dku.utils] - File "/etc/dataiku-security/y5nrSSq74Fzg7kQNTQrqj6zw/python/exec.py", line 189, in start [2021/12/21-13:40:08.386] [wrapper-stderr-56] [INFO] [dku.utils] - with open(cgroup_task_file, "w") as f: [2021/12/21-13:40:08.386] [wrapper-stderr-56] [INFO] [dku.utils] - IOError: [Errno 2] No such file or directory: u'/sys/fs/cgroup/memory/DSS/khou2892/recipes/tasks' [2021/12/21-13:40:08.386] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,385 2914056 INFO [Parent] Setting process 2914058 in cgroup tasks file /sys/fs/cgroup/cpu/DSS/recipes/tasks [2021/12/21-13:40:08.387] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,385 2914056 ERROR [Parent] Failed to set process in cgroup tasks file [2021/12/21-13:40:08.387] [wrapper-stderr-56] [INFO] [dku.utils] - Traceback (most recent call last): [2021/12/21-13:40:08.387] [wrapper-stderr-56] [INFO] [dku.utils] - File "/etc/dataiku-security/y5nrSSq74Fzg7kQNTQrqj6zw/python/exec.py", line 189, in start [2021/12/21-13:40:08.388] [wrapper-stderr-56] [INFO] [dku.utils] - with open(cgroup_task_file, "w") as f: [2021/12/21-13:40:08.388] [wrapper-stderr-56] [INFO] [dku.utils] - IOError: [Errno 2] No such file or directory: u'/sys/fs/cgroup/cpu/DSS/recipes/tasks' [2021/12/21-13:40:08.395] [FRT-49-FlowRunnable] [INFO] [dku.security.process] - Reading pipes from wrapper [2021/12/21-13:40:08.396] [FRT-49-FlowRunnable] [INFO] [dku.security.process] - Pipes read as bytes: 849 [2021/12/21-13:40:08.396] [FRT-49-FlowRunnable] [INFO] [dku.security.process] - Pipes read as bytes: {"pipes": {"stdin": "/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/.stdin.w3kauqw1", "stderr": "/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/.stderr.w3kauqw1", "stdout": "/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/.stdout.w3kauqw1"}, "pid": 2914056, "childPidFile": "/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/child.pid"} [2021/12/21-13:40:08.397] [FRT-49-FlowRunnable] [INFO] [dku.security.process] - Pid read: 2914056 [2021/12/21-13:40:08.397] [FRT-49-FlowRunnable] [INFO] [dku.security.process] - Pipes read: { "stdin": "/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/.stdin.w3kauqw1", "stderr": "/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/.stderr.w3kauqw1", "stdout": "/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/.stdout.w3kauqw1" } [2021/12/21-13:40:08.398] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,398 2914058 INFO [Child] opened stdin pipe [2021/12/21-13:40:08.399] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,398 2914058 INFO [Child] opening stdout pipe: /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/.stdout.w3kauqw1 [2021/12/21-13:40:08.399] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,398 2914058 INFO [Child] opened stdout [2021/12/21-13:40:08.400] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:08,398 2914058 INFO [Child] opening stderr pipe: /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/.stderr.w3kauqw1 [2021/12/21-13:40:08.400] [FRT-49-FlowRunnable] [INFO] [dku.recipes.code.base] - Process reads from nothing [2021/12/21-13:40:08.401] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:08,398 2914058 INFO [Child] opened stderr [2021/12/21-13:40:08.402] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:08,398 2914058 INFO [Child] about to close other fd [2021/12/21-13:40:08.402] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:08,399 2914058 INFO [Child] closed other fd [2021/12/21-13:40:08.403] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:08,399 2914058 INFO [Child] chdired [2021/12/21-13:40:08.403] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:08,399 2914058 INFO setting username=khou2892 uid=1358 gid=1360 [2021/12/21-13:40:08.404] [FRT-49-FlowRunnable] [DEBUG] [dku.resourceusage] - Reporting start of CRU:{"context":{"type":"JOB_ACTIVITY","authIdentifier":"khou2892","projectKey":"SPINETTA_FLUIDS_FINGERPRINTER","jobId":"Build_output_gen_python__NP__2021-12-21T13-40-02.474","activityId":"compute_output_gen_python_NP","activityType":"recipe","recipeType":"python","recipeName":"compute_output_gen_python"},"type":"LOCAL_PROCESS","id":"MUKg6PpK9etLpVcO","startTime":1640094008401,"localProcess":{"cpuCurrent":0.0}} [2021/12/21-13:40:08.407] [process-resource-monitor-unkpid-60] [INFO] [dku.security.process] - GET PID CPF=/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/child.pid [2021/12/21-13:40:08.409] [process-resource-monitor-2914058-60] [DEBUG] [dku.resource] - Process stats for pid 2914058: {"pid":2914058,"commandName":"/dataiku/dataiku-dss-9.0.4/resources/code-envs/conda-support/conda-run.sh","cpuUserTimeMS":0,"cpuSystemTimeMS":0,"cpuChildrenUserTimeMS":0,"cpuChildrenSystemTimeMS":0,"cpuTotalMS":0,"cpuCurrent":0.0,"vmSizeMB":171,"vmRSSMB":12,"vmHWMMB":12,"vmRSSAnonMB":6,"vmDataMB":5,"vmSizePeakMB":171,"vmRSSPeakMB":12,"vmRSSTotalMBS":0,"majorFaults":0,"childrenMajorFaults":0} [2021/12/21-13:40:08.413] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:08,413 2914058 INFO [Child] dropped privileges [2021/12/21-13:40:08.414] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:08,413 2914058 INFO [Child] Checking access to DKUINSTALLDIR and DIP_HOME directories [2021/12/21-13:40:08.414] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:08,413 2914058 INFO [Child] Executing: /dataiku/dataiku-dss-9.0.4/resources/code-envs/conda-support/conda-run.sh : /dataiku/dataiku-dss-9.0.4/resources/code-envs/conda-support/conda-run.sh /dataiku/dss_data/code-envs/python/Python3_6 python -u /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/python-exec-wrapper.py /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/script.py [2021/12/21-13:40:09.048] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:09,048 INFO -------------------- [2021/12/21-13:40:09.049] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:09,048 INFO Dataiku Python entrypoint starting up [2021/12/21-13:40:09.049] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:09,048 INFO executable = /dataiku/dss_data/code-envs/python/Python3_6/bin/python [2021/12/21-13:40:09.050] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:09,048 INFO argv = ['/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/python-exec-wrapper.py', '/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/script.py'] [2021/12/21-13:40:09.050] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:09,048 INFO -------------------- [2021/12/21-13:40:09.051] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:09,048 INFO Looking for RemoteRunEnvDef in ./remote-run-env-def.json [2021/12/21-13:40:09.051] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:09,048 INFO Found RemoteRunEnvDef environment: ./remote-run-env-def.json [2021/12/21-13:40:09.051] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:09,049 INFO Running a DSS Python recipe locally, uinsetting env [2021/12/21-13:40:09.052] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:09,050 INFO Setup complete, ready to execute Python code [2021/12/21-13:40:09.052] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:09,050 INFO Sys path: ['/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V', '/dataiku/dss_data/lib/python', '/dataiku/dataiku-dss-9.0.4/python', '/dataiku/dss_data/code-envs/python/Python3_6/lib/python36.zip', '/dataiku/dss_data/code-envs/python/Python3_6/lib/python3.6', '/dataiku/dss_data/code-envs/python/Python3_6/lib/python3.6/lib-dynload', '/dataiku/dss_data/code-envs/python/Python3_6/lib/python3.6/site-packages', '/dataiku/dss_data/code-envs/python/Python3_6/lib/python3.6/site-packages/scikit_optimize-0.7.4-py3.6.egg'] [2021/12/21-13:40:09.053] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:09,050 INFO Script file: /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/script.py [2021/12/21-13:40:14.572] [null-err-58] [INFO] [dku.utils] - sys:1: DtypeWarning: Columns (19,38,41,43,44,45,67,68,69,70,73,79,80,81,87,89,98,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134) have mixed types.Specify dtype option on import or set low_memory=False. [2021/12/21-13:40:21.113] [null-err-58] [INFO] [dku.utils] - *************** Recipe code failed ************** [2021/12/21-13:40:21.113] [null-err-58] [INFO] [dku.utils] - Begin Python stack [2021/12/21-13:40:21.113] [null-err-58] [INFO] [dku.utils] - Traceback (most recent call last): [2021/12/21-13:40:21.114] [null-err-58] [INFO] [dku.utils] - File "/dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/python-exec-wrapper.py", line 208, in [2021/12/21-13:40:21.114] [null-err-58] [INFO] [dku.utils] - exec(f.read()) [2021/12/21-13:40:21.114] [null-err-58] [INFO] [dku.utils] - File "", line 155, in [2021/12/21-13:40:21.114] [null-err-58] [INFO] [dku.utils] - NameError: name 'data_df' is not defined [2021/12/21-13:40:21.114] [null-err-58] [INFO] [dku.utils] - End Python stack [2021/12/21-13:40:21.115] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:21,113 INFO Check if spark is available [2021/12/21-13:40:21.115] [null-err-58] [INFO] [dku.utils] - 2021-12-21 13:40:21,114 INFO Not stopping a spark context: No module named 'pyspark' [2021/12/21-13:40:21.377] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:21,376 2914056 INFO [Child] Process 2914058 exited with exit=1 signal=0 [2021/12/21-13:40:21.377] [wrapper-stderr-56] [INFO] [dku.utils] - 2021-12-21 13:40:21,377 2914056 INFO Full child code: 1 [2021/12/21-13:40:21.386] [FRT-49-FlowRunnable] [WARN] [dku.resource] - stat file for pid 2914058 does not exist. Process died? [2021/12/21-13:40:21.387] [FRT-49-FlowRunnable] [DEBUG] [dku.resourceusage] - Reporting completion of CRU:{"context":{"type":"JOB_ACTIVITY","authIdentifier":"khou2892","projectKey":"SPINETTA_FLUIDS_FINGERPRINTER","jobId":"Build_output_gen_python__NP__2021-12-21T13-40-02.474","activityId":"compute_output_gen_python_NP","activityType":"recipe","recipeType":"python","recipeName":"compute_output_gen_python"},"type":"LOCAL_PROCESS","id":"MUKg6PpK9etLpVcO","startTime":1640094008401,"localProcess":{"pid":2914058,"commandName":"/dataiku/dataiku-dss-9.0.4/resources/code-envs/conda-support/conda-run.sh","cpuUserTimeMS":3220,"cpuSystemTimeMS":3020,"cpuChildrenUserTimeMS":240,"cpuChildrenSystemTimeMS":330,"cpuTotalMS":6810,"cpuCurrent":0.5178964207158568,"vmSizeMB":1121,"vmRSSMB":405,"vmHWMMB":443,"vmRSSAnonMB":377,"vmDataMB":953,"vmSizePeakMB":1160,"vmRSSPeakMB":405,"vmRSSTotalMBS":2955,"majorFaults":0,"childrenMajorFaults":0}} [2021/12/21-13:40:21.388] [FRT-49-FlowRunnable] [INFO] [dip.exec.resultHandler] - Error file found, trying to throw it: /dataiku/dss_data/jobs/SPINETTA_FLUIDS_FINGERPRINTER/Build_output_gen_python__NP__2021-12-21T13-40-02.474/compute_output_gen_python_NP/cpython-recipe/pyoutLHggEvT0HH8V/error.json [2021/12/21-13:40:21.389] [FRT-49-FlowRunnable] [INFO] [dip.exec.resultHandler] - Raw error is{"errorType":"\u003cclass \u0027NameError\u0027\u003e","message":"name \u0027data_df\u0027 is not defined","detailedMessage":"At line 155: \u003cclass \u0027NameError\u0027\u003e: name \u0027data_df\u0027 is not defined","stackTrace":[]} [2021/12/21-13:40:21.389] [FRT-49-FlowRunnable] [INFO] [dip.exec.resultHandler] - Now err: {"errorType":"\u003cclass \u0027NameError\u0027\u003e","message":"Error in Python process: name \u0027data_df\u0027 is not defined","detailedMessage":"Error in Python process: At line 155: \u003cclass \u0027NameError\u0027\u003e: name \u0027data_df\u0027 is not defined","stackTrace":[]} [2021/12/21-13:40:21.392] [FRT-49-FlowRunnable] [INFO] [dku.flow.activity] - Run thread failed for activity compute_output_gen_python_NP com.dataiku.common.server.APIError$SerializedErrorException: Error in Python process: At line 155: : name 'data_df' is not defined at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.handleErrorFile(JobExecutionResultHandler.java:65) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.handleExecutionResultNoProcessDiedException(JobExecutionResultHandler.java:32) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.handleExecutionResult(JobExecutionResultHandler.java:25) at com.dataiku.dip.dataflow.exec.AbstractCodeBasedActivityRunner.execute(AbstractCodeBasedActivityRunner.java:74) at com.dataiku.dip.dataflow.exec.AbstractPythonRecipeRunner.executeScript(AbstractPythonRecipeRunner.java:48) at com.dataiku.dip.recipes.code.python.PythonRecipeRunner.run(PythonRecipeRunner.java:64) at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:374) [2021/12/21-13:40:21.403] [ActivityExecutor-44] [INFO] [dku.flow.activity] running compute_output_gen_python_NP - activity is finished [2021/12/21-13:40:21.404] [ActivityExecutor-44] [ERROR] [dku.flow.activity] running compute_output_gen_python_NP - Activity failed com.dataiku.common.server.APIError$SerializedErrorException: Error in Python process: At line 155: : name 'data_df' is not defined at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.handleErrorFile(JobExecutionResultHandler.java:65) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.handleExecutionResultNoProcessDiedException(JobExecutionResultHandler.java:32) at com.dataiku.dip.dataflow.exec.JobExecutionResultHandler.handleExecutionResult(JobExecutionResultHandler.java:25) at com.dataiku.dip.dataflow.exec.AbstractCodeBasedActivityRunner.execute(AbstractCodeBasedActivityRunner.java:74) at com.dataiku.dip.dataflow.exec.AbstractPythonRecipeRunner.executeScript(AbstractPythonRecipeRunner.java:48) at com.dataiku.dip.recipes.code.python.PythonRecipeRunner.run(PythonRecipeRunner.java:64) at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:374) [2021/12/21-13:40:21.405] [ActivityExecutor-44] [INFO] [dku.flow.activity] running compute_output_gen_python_NP - Executing default post-activity lifecycle hook [2021/12/21-13:40:21.408] [ActivityExecutor-44] [INFO] [dku.flow.activity] running compute_output_gen_python_NP - Removing samples for SPINETTA_FLUIDS_FINGERPRINTER.output_gen_python [2021/12/21-13:40:21.410] [ActivityExecutor-44] [INFO] [dku.flow.activity] running compute_output_gen_python_NP - Done post-activity tasks