[2022/09/07-18:45:11.957] [ActivityExecutor-41] [INFO] [dku] running compute_O2C_VESTA_FR_NP - ---------------------------------------- [2022/09/07-18:45:11.957] [ActivityExecutor-41] [INFO] [dku] running compute_O2C_VESTA_FR_NP - DSS startup: jek version:10.0.5 [2022/09/07-18:45:11.957] [ActivityExecutor-41] [INFO] [dku] running compute_O2C_VESTA_FR_NP - DSS home: /data/dataiku/dss_data [2022/09/07-18:45:11.957] [ActivityExecutor-41] [INFO] [dku] running compute_O2C_VESTA_FR_NP - OS: Linux 3.10.0-1160.59.1.el7.x86_64 amd64 - Java: Red Hat, Inc. 1.8.0_342 [2022/09/07-18:45:11.957] [ActivityExecutor-41] [INFO] [dku.flow.jobrunner] running compute_O2C_VESTA_FR_NP - Allocated a slot for this activity! [2022/09/07-18:45:11.957] [ActivityExecutor-41] [INFO] [dku.flow.jobrunner] running compute_O2C_VESTA_FR_NP - Run activity [2022/09/07-18:45:11.964] [ActivityExecutor-41] [INFO] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Executing default pre-activity lifecycle hook [2022/09/07-18:45:11.970] [ActivityExecutor-41] [INFO] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Checking if sources are ready [2022/09/07-18:45:11.981] [ActivityExecutor-41] [INFO] [dku.datasets.file] running compute_O2C_VESTA_FR_NP - Building Filesystem handler config: {"path":"/data/dataiku/dss_data/uploads/02CKPIINDUSTRIALIZATIONBSFINANCE/datasets/REF_OC_PERIMETRE_ERNERGY_BE","notReadyIfEmpty":false,"filesSelectionRules":{"mode":"ALL","excludeRules":[],"includeRules":[],"explicitFiles":[]}} [2022/09/07-18:45:11.982] [ActivityExecutor-41] [INFO] [dku.datasets.ftplike] running compute_O2C_VESTA_FR_NP - Enumerating Filesystem dataset prefix= [2022/09/07-18:45:11.987] [ActivityExecutor-41] [DEBUG] [dku.fs.local] running compute_O2C_VESTA_FR_NP - Enumerating local filesystem prefix=/ [2022/09/07-18:45:11.988] [ActivityExecutor-41] [DEBUG] [dku.fs.local] running compute_O2C_VESTA_FR_NP - Enumeration done nb_paths=1 size=19493 [2022/09/07-18:45:11.990] [ActivityExecutor-41] [INFO] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Checked source readiness 02CKPIINDUSTRIALIZATIONBSFINANCE.REF_OC_PERIMETRE_ERNERGY_BE -> true [2022/09/07-18:45:12.007] [ActivityExecutor-41] [INFO] [com.dataiku.dip.datasets.fs.FilesInFolderDatasetHandler] running compute_O2C_VESTA_FR_NP - Build real handler with filter {"bucket":"cdh-kpibsfinanceinputdatasourcenoprod-786117","metastoreSynchronizationEnabled":true,"metastoreTableName":"f3SIqHI4","connection":"kpi_bs_finance_connection","path":"/R001_VESTA_BS_FINANCE","notReadyIfEmpty":false,"filesSelectionRules":{"mode":"ALL","excludeRules":[],"includeRules":[],"explicitFiles":[]}} [2022/09/07-18:45:12.016] [ActivityExecutor-41] [INFO] [dku.datasets.bloblike] running compute_O2C_VESTA_FR_NP - Enumerating Filesystem dataset prefix= [2022/09/07-18:45:12.020] [ActivityExecutor-41] [INFO] [dku.fs.s3] running compute_O2C_VESTA_FR_NP - S3 provider bucket=cdh-kpibsfinanceinputdatasourcenoprod-786117 pathInBucket=/R001_VESTA_BS_FINANCE (connectionChroot=/) [2022/09/07-18:45:12.291] [ActivityExecutor-41] [INFO] [dku.aws.connection] running compute_O2C_VESTA_FR_NP - AWS connection=kpi_bs_finance_connection authCtx=ZF6372 assuming role=arn:aws:iam::418791513031:role/cdh_kpiindustrializationbsf_33147 [2022/09/07-18:45:12.328] [ActivityExecutor-41] [INFO] [dku.aws.connection] running compute_O2C_VESTA_FR_NP - Using context creds access=ASIAWDAPLLPD3LDMTEN7 [2022/09/07-18:45:13.136] [ActivityExecutor-41] [INFO] [dku.fs.s3] running compute_O2C_VESTA_FR_NP - Retrieving location from bucket [2022/09/07-18:45:13.273] [ActivityExecutor-41] [INFO] [dku.fs.s3] running compute_O2C_VESTA_FR_NP - Bucket is in location eu-west-1 [2022/09/07-18:45:13.426] [ActivityExecutor-41] [INFO] [dku.fs.s3] running compute_O2C_VESTA_FR_NP - Start S3 Enumeration ON bucketPath=R001_VESTA_BS_FINANCE/ prefix= fullPath=R001_VESTA_BS_FINANCE/ [2022/09/07-18:45:13.454] [ActivityExecutor-41] [INFO] [dku.fs.s3] running compute_O2C_VESTA_FR_NP - S3 enumeration done, found 6 items, 0 bytes [2022/09/07-18:45:13.455] [ActivityExecutor-41] [INFO] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Checked source readiness 02CKPIINDUSTRIALIZATIONBSFINANCE.VESTA -> true [2022/09/07-18:45:13.455] [ActivityExecutor-41] [DEBUG] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Computing hashes to propagate BEFORE activity [2022/09/07-18:45:13.456] [ActivityExecutor-41] [DEBUG] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Recorded 2 hashes before activity run [2022/09/07-18:45:13.456] [ActivityExecutor-41] [DEBUG] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Building recipe runner of type [2022/09/07-18:45:13.464] [ActivityExecutor-41] [INFO] [dku.recipe.fuzzyjoin.runner] running compute_O2C_VESTA_FR_NP - SET PAYLOAD: { "joins": [ { "table2": 1, "table1": 0, "conditionsMode": "AND", "type": "FULL", "on": [ { "column1": { "name": "Code_OC", "table": 0 }, "column2": { "name": "CoCo", "table": 1 }, "fuzzyMatchDesc": { "distanceType": "EXACT", "threshold": 1 } } ] } ], "selectedColumns": [ { "name": "SOURCESYS", "type": "string", "table": 0 }, { "name": "Mandant", "type": "string", "table": 0 }, { "name": "Code_GBU1", "type": "string", "table": 0 }, { "name": "GBU_level_1", "type": "string", "table": 0 }, { "name": "Code_GBU2", "type": "string", "table": 0 }, { "name": "GBU_level_2", "type": "string", "table": 0 }, { "name": "Code_GBU3", "type": "string", "table": 0 }, { "name": "GBU_level_3", "type": "string", "table": 0 }, { "name": "Pays", "type": "string", "table": 0 }, { "name": "Code_SMART", "type": "string", "table": 0 }, { "name": "Libellé_SMART", "type": "string", "table": 0 }, { "name": "Code_OC", "type": "string", "table": 0 }, { "name": "Libellé_OC", "type": "string", "table": 0 }, { "name": "Code_juridique", "type": "string", "table": 0 }, { "name": "Segment", "type": "string", "table": 0 }, { "name": "Division", "type": "string", "table": 0 }, { "name": "Exercice", "type": "string", "table": 0 }, { "name": "Période", "type": "string", "table": 0 }, { "name": "Cle_Comptabilisation", "type": "string", "table": 0 }, { "name": "Type_Pièce", "type": "string", "table": 0 }, { "name": "Libellé_Type_Pièce", "type": "string", "table": 0 }, { "name": "Code_Transaction", "type": "string", "table": 0 }, { "name": "ID_Utilisateur", "type": "string", "table": 0 }, { "name": "Code_Fournisseur", "type": "string", "table": 0 }, { "name": "Fournisseur", "type": "string", "table": 0 }, { "name": "Siret", "type": "string", "table": 0 }, { "name": "Compte_General", "type": "string", "table": 0 }, { "name": "Type_Compte_General", "type": "string", "table": 0 }, { "name": "ID_FMFI", "type": "string", "table": 0 }, { "name": "Lettrage", "type": "string", "table": 0 }, { "name": "Contre_Passation", "type": "string", "table": 0 }, { "name": "Montant_Total_EUR", "type": "string", "table": 0 }, { "name": "Montant_Total_Devise_Interne", "type": "string", "table": 0 }, { "name": "Nombre", "type": "string", "table": 0 }, { "name": "SAP", "type": "string", "table": 1 }, { "name": "CoCo", "type": "string", "table": 1 }, { "name": "BU", "type": "string", "table": 1 }, { "name": "group", "type": "string", "table": 1 }, { "name": "Sub groep", "type": "string", "table": 1 }, { "name": "Entity", "type": "string", "table": 1 }, { "name": "col_7", "type": "string", "table": 1 }, { "name": "col_8", "type": "string", "table": 1 }, { "name": "SAP_1", "type": "string", "table": 1 }, { "name": "GL", "type": "string", "table": 1 } ], "resolvedSelectedColumns": [], "engineParams": { "hive": { "skipPrerunValidate": false, "hiveconf": [], "inheritConf": "default", "addDkuUdf": false, "executionEngine": "HIVESERVER2" }, "sqlPipelineParams": { "pipelineAllowMerge": true, "pipelineAllowStart": true }, "impala": { "forceStreamMode": true }, "lowerCaseSchemaIfEngineRequiresIt": true, "sparkSQL": { "skipPrerunValidate": false, "pipelineAllowMerge": true, "useGlobalMetastore": false, "pipelineAllowStart": true, "readParams": { "mode": "AUTO", "autoModeRepartitionInto": 10, "map": {} }, "overwriteOutputSchema": false, "executionEngine": "SPARK_SUBMIT", "sparkConfig": { "inheritConf": "default", "conf": [] } } }, "virtualInputs": [ { "autoSelectColumns": false, "index": 1 }, { "preFilter": { "distinct": false, "enabled": false }, "autoSelectColumns": false, "originLabel": "REF_OC_PERIMETRE_ERNERGY_BE", "index": 0, "computedColumns": [] } ], "withMetaColumn": false, "debugMode": false, "computedColumns": [], "postFilter": { "$status": { "schema": { "columns": [ { "name": "SOURCESYS", "type": "string" }, { "name": "Mandant", "type": "string" }, { "name": "Code_GBU1", "type": "string" }, { "name": "GBU_level_1", "type": "string" }, { "name": "Code_GBU2", "type": "string" }, { "name": "GBU_level_2", "type": "string" }, { "name": "Code_GBU3", "type": "string" }, { "name": "GBU_level_3", "type": "string" }, { "name": "Pays", "type": "string" }, { "name": "Code_SMART", "type": "string" }, { "name": "Libellé_SMART", "type": "string" }, { "name": "Code_OC", "type": "string" }, { "name": "Libellé_OC", "type": "string" }, { "name": "Code_juridique", "type": "string" }, { "name": "Segment", "type": "string" }, { "name": "Division", "type": "string" }, { "meaning": "LongMeaning", "name": "Exercice", "type": "string" }, { "meaning": "LongMeaning", "name": "Période", "type": "string" }, { "meaning": "LongMeaning", "name": "Cle_Comptabilisation", "type": "string" }, { "name": "Type_Pièce", "type": "string" }, { "name": "Libellé_Type_Pièce", "type": "string" }, { "name": "Code_Transaction", "type": "string" }, { "name": "ID_Utilisateur", "type": "string" }, { "name": "Code_Fournisseur", "type": "string" }, { "name": "Fournisseur", "type": "string" }, { "name": "Siret", "type": "string" }, { "name": "Compte_General", "type": "string" }, { "name": "Type_Compte_General", "type": "string" }, { "name": "ID_FMFI", "type": "string" }, { "name": "Lettrage", "type": "string" }, { "name": "Contre_Passation", "type": "string" }, { "name": "Montant_Total_EUR", "type": "string" }, { "name": "Montant_Total_Devise_Interne", "type": "string" }, { "name": "Nombre", "type": "string" }, { "name": "SAP", "type": "string" }, { "name": "CoCo", "type": "string" }, { "name": "BU", "type": "string" }, { "name": "group", "type": "string" }, { "name": "Sub groep", "type": "string" }, { "name": "Entity", "type": "string" }, { "name": "col_7", "type": "string" }, { "name": "col_8", "type": "string" }, { "name": "SAP_1", "type": "string" }, { "name": "GL", "type": "string" } ], "userModified": false } } } } [2022/09/07-18:45:13.474] [Thread-22] [INFO] [dku.datasets.pull] - pull background thread starting for REF_OC_PERIMETRE_ERNERGY_BE [2022/09/07-18:45:13.475] [Thread-23] [INFO] [dku.datasets.pull] - pull background thread starting for VESTA [2022/09/07-18:45:13.481] [ActivityExecutor-41] [INFO] [dku.datasets.bloblike] running compute_O2C_VESTA_FR_NP - Clear partitions [2022/09/07-18:45:13.482] [ActivityExecutor-41] [INFO] [dku.fs.s3] running compute_O2C_VESTA_FR_NP - S3 provider bucket=cdh-kpibsfinanceinputdatasourcenoprod-786117 pathInBucket=/O2C_VESTA_FR_WITH_PERIMETRE_OC/managed (connectionChroot=/) [2022/09/07-18:45:13.482] [ActivityExecutor-41] [INFO] [dku.aws.connection] running compute_O2C_VESTA_FR_NP - AWS connection=kpi_bs_finance_connection authCtx=ZF6372 assuming role=arn:aws:iam::418791513031:role/cdh_kpiindustrializationbsf_33147 [2022/09/07-18:45:13.483] [ActivityExecutor-41] [INFO] [dku.aws.connection] running compute_O2C_VESTA_FR_NP - Using context creds access=ASIAWDAPLLPD3LDMTEN7 [2022/09/07-18:45:13.483] [Thread-23] [INFO] [com.dataiku.dip.datasets.fs.FilesInFolderDatasetHandler] - Build real handler with filter {"bucket":"cdh-kpibsfinanceinputdatasourcenoprod-786117","metastoreSynchronizationEnabled":true,"metastoreTableName":"f3SIqHI4","connection":"kpi_bs_finance_connection","path":"/R001_VESTA_BS_FINANCE","notReadyIfEmpty":false,"filesSelectionRules":{"mode":"ALL","excludeRules":[],"includeRules":[],"explicitFiles":[]}} [2022/09/07-18:45:13.487] [Thread-23] [INFO] [dku.datasets.bloblike] - Enumerating Filesystem dataset prefix= [2022/09/07-18:45:13.487] [Thread-22] [INFO] [dku.datasets.file] - Building Filesystem handler config: {"path":"/data/dataiku/dss_data/uploads/02CKPIINDUSTRIALIZATIONBSFINANCE/datasets/REF_OC_PERIMETRE_ERNERGY_BE","notReadyIfEmpty":false,"filesSelectionRules":{"mode":"ALL","excludeRules":[],"includeRules":[],"explicitFiles":[]}} [2022/09/07-18:45:13.488] [Thread-22] [INFO] [dku.datasets.ftplike] - Enumerating Filesystem dataset prefix= [2022/09/07-18:45:13.488] [Thread-23] [INFO] [dku.fs.s3] - S3 provider bucket=cdh-kpibsfinanceinputdatasourcenoprod-786117 pathInBucket=/R001_VESTA_BS_FINANCE (connectionChroot=/) [2022/09/07-18:45:13.489] [Thread-22] [DEBUG] [dku.fs.local] - Enumerating local filesystem prefix=/ [2022/09/07-18:45:13.489] [Thread-23] [INFO] [dku.aws.connection] - AWS connection=kpi_bs_finance_connection authCtx=ZF6372 assuming role=arn:aws:iam::418791513031:role/cdh_kpiindustrializationbsf_33147 [2022/09/07-18:45:13.489] [Thread-22] [DEBUG] [dku.fs.local] - Enumeration done nb_paths=1 size=19493 [2022/09/07-18:45:13.489] [Thread-23] [INFO] [dku.aws.connection] - Using context creds access=ASIAWDAPLLPD3LDMTEN7 [2022/09/07-18:45:13.490] [Thread-22] [INFO] [dku.input.push] - USTP: push selection.method=FULL records=-1 ratio=0.02 col=null [2022/09/07-18:45:13.504] [Thread-22] [INFO] [dku.format] - Extractor run: limit={"maxBytes":-1,"maxRecords":-1,"ordering":{"enabled":false,"rules":[]}} totalRecords=0 [2022/09/07-18:45:13.505] [Thread-22] [INFO] [dku] - getCompression filename=**REF_OC_PERIMETRE_ERNERGY_BE_02_09_2022.xlsx** [2022/09/07-18:45:13.506] [Thread-22] [INFO] [dku] - getCompression filename=**REF_OC_PERIMETRE_ERNERGY_BE_02_09_2022.xlsx** [2022/09/07-18:45:13.506] [Thread-22] [INFO] [dku.format] - Start uncompressed stream: /data/dataiku/dss_data/uploads/02CKPIINDUSTRIALIZATIONBSFINANCE/datasets/REF_OC_PERIMETRE_ERNERGY_BE/REF_OC_PERIMETRE_ERNERGY_BE_02_09_2022.xlsx / totalRecsBefore=0 [2022/09/07-18:45:13.507] [Thread-22] [INFO] [dku] - getCompression filename=**REF_OC_PERIMETRE_ERNERGY_BE_02_09_2022.xlsx** [2022/09/07-18:45:13.507] [Thread-22] [INFO] [dku.format.excel] - Excel starting to process one stream: {"xlsx":true,"preserveNumberFormatting":false,"parseDatesToISO":false,"skipRowsBeforeHeader":0,"parseHeaderRow":true,"skipRowsAfterHeader":0,"sheets":"*STAR"} [2022/09/07-18:45:13.514] [Thread-22] [DEBUG] [com.monitorjbl.xlsx.impl.StreamingWorkbookReader] - Created temp file [/data/dataiku/dss_data/tmp/jeks/jek-maVFRrwi/tmp-7754732215141412180.xlsx] [2022/09/07-18:45:13.794] [ActivityExecutor-41] [INFO] [dku.fs.s3] running compute_O2C_VESTA_FR_NP - Bucket is in location eu-west-1 [2022/09/07-18:45:13.809] [Thread-23] [INFO] [dku.fs.s3] - Bucket is in location eu-west-1 [2022/09/07-18:45:13.848] [ActivityExecutor-41] [INFO] [dku.datasets.bloblike] running compute_O2C_VESTA_FR_NP - Clearing partition as a folder : 'NP' [2022/09/07-18:45:13.887] [ActivityExecutor-41] [INFO] [dku.datasets.bloblike] running compute_O2C_VESTA_FR_NP - Done clearing partition 'NP' [2022/09/07-18:45:13.889] [ActivityExecutor-41] [INFO] [dku.fs.s3] running compute_O2C_VESTA_FR_NP - S3 provider bucket=cdh-kpibsfinanceinputdatasourcenoprod-786117 pathInBucket=/O2C_VESTA_FR_WITH_PERIMETRE_OC/managed (connectionChroot=/) [2022/09/07-18:45:13.890] [ActivityExecutor-41] [INFO] [dku.aws.connection] running compute_O2C_VESTA_FR_NP - AWS connection=kpi_bs_finance_connection authCtx=ZF6372 assuming role=arn:aws:iam::418791513031:role/cdh_kpiindustrializationbsf_33147 [2022/09/07-18:45:13.890] [ActivityExecutor-41] [INFO] [dku.aws.connection] running compute_O2C_VESTA_FR_NP - Using context creds access=ASIAWDAPLLPD3LDMTEN7 [2022/09/07-18:45:13.925] [Thread-22] [INFO] [dku.format.excel] - Parse as header row: colIdx=0 cellValue=SAP [2022/09/07-18:45:13.925] [Thread-22] [INFO] [dku.format.excel] - Parse as header row: colIdx=1 cellValue=SOURCESYS [2022/09/07-18:45:13.925] [Thread-22] [INFO] [dku.format.excel] - Parse as header row: colIdx=2 cellValue=CoCo [2022/09/07-18:45:13.926] [Thread-22] [INFO] [dku.format.excel] - Parse as header row: colIdx=3 cellValue=BU [2022/09/07-18:45:13.926] [Thread-22] [INFO] [dku.format.excel] - Parse as header row: colIdx=4 cellValue=group [2022/09/07-18:45:13.926] [Thread-22] [INFO] [dku.format.excel] - Parse as header row: colIdx=5 cellValue=Sub groep [2022/09/07-18:45:13.927] [Thread-22] [INFO] [dku.format.excel] - Parse as header row: colIdx=6 cellValue=Entity [2022/09/07-18:45:13.927] [Thread-22] [INFO] [dku.format.excel] - Parse as header row: colIdx=9 cellValue=SAP [2022/09/07-18:45:13.927] [Thread-22] [INFO] [dku.format.excel] - Parse as header row: colIdx=10 cellValue=GL [2022/09/07-18:45:13.948] [Thread-22] [DEBUG] [com.monitorjbl.xlsx.impl.StreamingWorkbookReader] - Deleting tmp file [/data/dataiku/dss_data/tmp/jeks/jek-maVFRrwi/tmp-7754732215141412180.xlsx] [2022/09/07-18:45:14.077] [Thread-22] [INFO] [dku.format] - after stream totalComp=19493 totalUncomp=19493 totalRec=48 [2022/09/07-18:45:14.078] [Thread-22] [INFO] [dku.format] - Extractor run done, totalCompressed=19493 totalRecords=48 [2022/09/07-18:45:14.079] [Thread-22] [DEBUG] [dku.datasets.pull] - pull background thread: ending queue, cursize=48 [2022/09/07-18:45:14.082] [Thread-22] [INFO] [dku.datasets.pull] - pull background thread finished for REF_OC_PERIMETRE_ERNERGY_BE [2022/09/07-18:45:14.101] [Thread-23] [INFO] [dku.fs.s3] - Start S3 Enumeration ON bucketPath=R001_VESTA_BS_FINANCE/ prefix= fullPath=R001_VESTA_BS_FINANCE/ [2022/09/07-18:45:14.127] [Thread-23] [INFO] [dku.fs.s3] - S3 enumeration done, found 6 items, 0 bytes [2022/09/07-18:45:14.127] [Thread-23] [INFO] [dku.input.push] - USTP: push selection.method=FULL records=-1 ratio=0.02 col=null [2022/09/07-18:45:14.128] [Thread-23] [INFO] [dku.format] - Extractor run: limit={"maxBytes":-1,"maxRecords":-1,"ordering":{"enabled":false,"rules":[]}} totalRecords=0 [2022/09/07-18:45:14.129] [Thread-23] [INFO] [dku.fs.s3] - Getting S3 stream on R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_01_2022.csv [2022/09/07-18:45:14.165] [Thread-23] [INFO] [dku.fs.s3] - Path to read /R001_VESTA_BS_FINANCE|/R001_VESTA_BS_FINANCE_01_01_2022.csv -> R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_01_2022.csv [2022/09/07-18:45:14.166] [Thread-23] [INFO] [dku] - getCompression filename=**R001_VESTA_BS_FINANCE_01_01_2022.csv** [2022/09/07-18:45:14.166] [Thread-23] [INFO] [dku.fs.s3] - Getting range on -1 [2022/09/07-18:45:14.196] [Thread-23] [INFO] [dku] - getCompression filename=**R001_VESTA_BS_FINANCE_01_01_2022.csv** [2022/09/07-18:45:14.196] [Thread-23] [INFO] [dku.format] - Start uncompressed stream: /R001_VESTA_BS_FINANCE_01_01_2022.csv / totalRecsBefore=0 [2022/09/07-18:45:14.196] [Thread-23] [INFO] [dku] - getCompression filename=**R001_VESTA_BS_FINANCE_01_01_2022.csv** [2022/09/07-18:45:14.314] [ActivityExecutor-41] [INFO] [dku.fs.s3] running compute_O2C_VESTA_FR_NP - Bucket is in location eu-west-1 [2022/09/07-18:45:14.397] [ActivityExecutor-41] [INFO] [dku.output.file] running compute_O2C_VESTA_FR_NP - INIT Resplittable [2022/09/07-18:45:14.411] [ActivityExecutor-41] [INFO] [dku.output.file] running compute_O2C_VESTA_FR_NP - Writing base=/ split=0 chunk=0 -> target = out-s0.csv.gz [2022/09/07-18:45:14.412] [ActivityExecutor-41] [INFO] [dku.fs.s3] running compute_O2C_VESTA_FR_NP - Writing S3 stream on O2C_VESTA_FR_WITH_PERIMETRE_OC/managed/out-s0.csv.gz [2022/09/07-18:45:14.413] [ActivityExecutor-41] [INFO] [dku.fs.s3.upload] running compute_O2C_VESTA_FR_NP - Initiate multipart upload [bucket=cdh-kpibsfinanceinputdatasourcenoprod-786117, path=O2C_VESTA_FR_WITH_PERIMETRE_OC/managed/out-s0.csv.gz, encrypt=no]. [2022/09/07-18:45:14.482] [ActivityExecutor-41] [DEBUG] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Recipe runner built, will use 1 thread(s) [2022/09/07-18:45:14.482] [ActivityExecutor-41] [DEBUG] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Starting execution thread: com.dataiku.dip.dataflow.exec.fuzzyjoin.FuzzyJoinRecipeRunner@9bb447f [2022/09/07-18:45:14.483] [ActivityExecutor-41] [DEBUG] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Execution threads started, waiting for activity end [2022/09/07-18:45:14.484] [FRT-49-FlowRunnable] [INFO] [dku.flow.activity] act.compute_O2C_VESTA_FR_NP - Run thread for activity compute_O2C_VESTA_FR_NP starting [2022/09/07-18:45:14.498] [FRT-49-FlowRunnable] [INFO] [dku.h2db] act.compute_O2C_VESTA_FR_NP - Creating a temporary H2 database: /data/dataiku/dss_data/jobs/02CKPIINDUSTRIALIZATIONBSFINANCE/Build_O2C_VESTA_FR_WITH_PERIMETRE_OC__NP__2022-09-07T18-45-09.083/compute_O2C_VESTA_FR_NP/dataset-to-h2/bpIBKGwumi2HYXuadCMN [2022/09/07-18:45:14.501] [FRT-49-FlowRunnable] [INFO] [dku.connections.sql.provider] act.compute_O2C_VESTA_FR_NP - Connecting to jdbc:h2:/data/dataiku/dss_data/jobs/02CKPIINDUSTRIALIZATIONBSFINANCE/Build_O2C_VESTA_FR_WITH_PERIMETRE_OC__NP__2022-09-07T18-45-09.083/compute_O2C_VESTA_FR_NP/dataset-to-h2/bpIBKGwumi2HYXuadCMN/dataset with props: {"USER":"h2_admin"} conn=internal-h2-connection-for-recipe-1EwLbjf [2022/09/07-18:45:14.511] [FRT-49-FlowRunnable] [DEBUG] [dku.connections.sql.driver] act.compute_O2C_VESTA_FR_NP - Driver version 1.4 [2022/09/07-18:45:14.644] [FRT-49-FlowRunnable] [INFO] [dku.connections.sql.provider] act.compute_O2C_VESTA_FR_NP - Driver: H2 JDBC Driver (JDBC 4.0) 1.4.195 (2017-04-23) (1.4) [2022/09/07-18:45:14.644] [FRT-49-FlowRunnable] [INFO] [dku.connections.sql.provider] act.compute_O2C_VESTA_FR_NP - Database: H2 1.4.195 (2017-04-23) (1.4) rowSize=0 stmts=0 [2022/09/07-18:45:14.646] [FRT-49-FlowRunnable] [DEBUG] [dku.resourceusage] act.compute_O2C_VESTA_FR_NP - Reporting start of CRU:{"context":{"type":"JOB_ACTIVITY","authIdentifier":"ZF6372","projectKey":"02CKPIINDUSTRIALIZATIONBSFINANCE","jobId":"Build_O2C_VESTA_FR_WITH_PERIMETRE_OC__NP__2022-09-07T18-45-09.083","activityId":"compute_O2C_VESTA_FR_NP","activityType":"recipe","recipeType":"fuzzyjoin","recipeName":"compute_O2C_VESTA_FR"},"type":"SQL_CONNECTION","id":"DqKyGrpyeZGybny8","startTime":1662576314641,"sqlConnection":{"connection":"internal-h2-connection-for-recipe"}} [2022/09/07-18:45:14.647] [FRT-49-FlowRunnable] [INFO] [dku.dataset.sql] act.compute_O2C_VESTA_FR_NP - Executing statement: [2022/09/07-18:45:14.647] [FRT-49-FlowRunnable] [INFO] [dku.dataset.sql] act.compute_O2C_VESTA_FR_NP - CREATE USER h2_user PASSWORD '' [2022/09/07-18:45:14.651] [FRT-49-FlowRunnable] [INFO] [dku.dataset.sql] act.compute_O2C_VESTA_FR_NP - Statement done [2022/09/07-18:45:14.652] [FRT-49-FlowRunnable] [INFO] [dku.dataset.sql] act.compute_O2C_VESTA_FR_NP - Executing statement: [2022/09/07-18:45:14.652] [FRT-49-FlowRunnable] [INFO] [dku.dataset.sql] act.compute_O2C_VESTA_FR_NP - GRANT ALL ON SCHEMA public TO h2_user [2022/09/07-18:45:14.653] [FRT-49-FlowRunnable] [INFO] [dku.dataset.sql] act.compute_O2C_VESTA_FR_NP - Statement done [2022/09/07-18:45:14.653] [FRT-49-FlowRunnable] [DEBUG] [dku.connections.sql.provider] act.compute_O2C_VESTA_FR_NP - Close conn=internal-h2-connection-for-recipe-1EwLbjf [2022/09/07-18:45:14.657] [FRT-49-FlowRunnable] [DEBUG] [dku.resourceusage] act.compute_O2C_VESTA_FR_NP - Reporting completion of CRU:{"context":{"type":"JOB_ACTIVITY","authIdentifier":"ZF6372","projectKey":"02CKPIINDUSTRIALIZATIONBSFINANCE","jobId":"Build_O2C_VESTA_FR_WITH_PERIMETRE_OC__NP__2022-09-07T18-45-09.083","activityId":"compute_O2C_VESTA_FR_NP","activityType":"recipe","recipeType":"fuzzyjoin","recipeName":"compute_O2C_VESTA_FR"},"type":"SQL_CONNECTION","id":"DqKyGrpyeZGybny8","startTime":1662576314641,"sqlConnection":{"connection":"internal-h2-connection-for-recipe"}} [2022/09/07-18:45:14.659] [FRT-49-FlowRunnable] [INFO] [dku.connections.sql.provider] act.compute_O2C_VESTA_FR_NP - Connecting to jdbc:h2:/data/dataiku/dss_data/jobs/02CKPIINDUSTRIALIZATIONBSFINANCE/Build_O2C_VESTA_FR_WITH_PERIMETRE_OC__NP__2022-09-07T18-45-09.083/compute_O2C_VESTA_FR_NP/dataset-to-h2/bpIBKGwumi2HYXuadCMN/dataset with props: {"USER":"h2_user"} conn=internal-h2-connection-for-recipe-q25CM1I [2022/09/07-18:45:14.659] [FRT-49-FlowRunnable] [DEBUG] [dku.connections.sql.driver] act.compute_O2C_VESTA_FR_NP - Driver version 1.4 [2022/09/07-18:45:14.667] [FRT-49-FlowRunnable] [INFO] [dku.connections.sql.provider] act.compute_O2C_VESTA_FR_NP - Driver: H2 JDBC Driver (JDBC 4.0) 1.4.195 (2017-04-23) (1.4) [2022/09/07-18:45:14.667] [FRT-49-FlowRunnable] [INFO] [dku.connections.sql.provider] act.compute_O2C_VESTA_FR_NP - Database: H2 1.4.195 (2017-04-23) (1.4) rowSize=0 stmts=0 [2022/09/07-18:45:14.667] [FRT-49-FlowRunnable] [DEBUG] [dku.resourceusage] act.compute_O2C_VESTA_FR_NP - Reporting start of CRU:{"context":{"type":"JOB_ACTIVITY","authIdentifier":"ZF6372","projectKey":"02CKPIINDUSTRIALIZATIONBSFINANCE","jobId":"Build_O2C_VESTA_FR_WITH_PERIMETRE_OC__NP__2022-09-07T18-45-09.083","activityId":"compute_O2C_VESTA_FR_NP","activityType":"recipe","recipeType":"fuzzyjoin","recipeName":"compute_O2C_VESTA_FR"},"type":"SQL_CONNECTION","id":"eAJN5H3k3WJ1OpEX","startTime":1662576314667,"sqlConnection":{"connection":"internal-h2-connection-for-recipe"}} [2022/09/07-18:45:14.668] [FRT-49-FlowRunnable] [INFO] [dku.fuzzyjoin.RowsDAO] act.compute_O2C_VESTA_FR_NP - Creating table 0 [2022/09/07-18:45:14.670] [FRT-49-FlowRunnable] [INFO] [dku.h2loader] act.compute_O2C_VESTA_FR_NP - Dumping dataset to CSV for H2... [2022/09/07-18:45:14.671] [FRT-49-FlowRunnable] [INFO] [com.dataiku.dip.datasets.fs.FilesInFolderDatasetHandler] act.compute_O2C_VESTA_FR_NP - Build real handler with filter {"bucket":"cdh-kpibsfinanceinputdatasourcenoprod-786117","metastoreSynchronizationEnabled":true,"metastoreTableName":"f3SIqHI4","connection":"kpi_bs_finance_connection","path":"/R001_VESTA_BS_FINANCE","notReadyIfEmpty":false,"filesSelectionRules":{"mode":"ALL","excludeRules":[],"includeRules":[],"explicitFiles":[]}} [2022/09/07-18:45:14.672] [FRT-49-FlowRunnable] [INFO] [dku.datasets.bloblike] act.compute_O2C_VESTA_FR_NP - Enumerating Filesystem dataset prefix= [2022/09/07-18:45:14.672] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - S3 provider bucket=cdh-kpibsfinanceinputdatasourcenoprod-786117 pathInBucket=/R001_VESTA_BS_FINANCE (connectionChroot=/) [2022/09/07-18:45:14.673] [FRT-49-FlowRunnable] [INFO] [dku.aws.connection] act.compute_O2C_VESTA_FR_NP - AWS connection=kpi_bs_finance_connection authCtx=ZF6372 assuming role=arn:aws:iam::418791513031:role/cdh_kpiindustrializationbsf_33147 [2022/09/07-18:45:14.673] [FRT-49-FlowRunnable] [INFO] [dku.aws.connection] act.compute_O2C_VESTA_FR_NP - Using context creds access=ASIAWDAPLLPD3LDMTEN7 [2022/09/07-18:45:14.990] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Bucket is in location eu-west-1 [2022/09/07-18:45:15.173] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Start S3 Enumeration ON bucketPath=R001_VESTA_BS_FINANCE/ prefix= fullPath=R001_VESTA_BS_FINANCE/ [2022/09/07-18:45:15.203] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - S3 enumeration done, found 6 items, 0 bytes [2022/09/07-18:45:15.203] [FRT-49-FlowRunnable] [INFO] [dku.input.push] act.compute_O2C_VESTA_FR_NP - USTP: push selection.method=FULL records=-1 ratio=0.02 col=null [2022/09/07-18:45:15.203] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - Extractor run: limit={"maxBytes":-1,"maxRecords":-1,"ordering":{"enabled":false,"rules":[]}} totalRecords=0 [2022/09/07-18:45:15.203] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Getting S3 stream on R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_01_2022.csv [2022/09/07-18:45:15.233] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Path to read /R001_VESTA_BS_FINANCE|/R001_VESTA_BS_FINANCE_01_01_2022.csv -> R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_01_2022.csv [2022/09/07-18:45:15.233] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_01_2022.csv** [2022/09/07-18:45:15.234] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Getting range on -1 [2022/09/07-18:45:15.265] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_01_2022.csv** [2022/09/07-18:45:15.265] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - Start uncompressed stream: /R001_VESTA_BS_FINANCE_01_01_2022.csv / totalRecsBefore=0 [2022/09/07-18:45:15.265] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_01_2022.csv** [2022/09/07-18:45:15.562] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - after stream totalComp=4993150 totalUncomp=4993150 totalRec=16998 [2022/09/07-18:45:15.563] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Getting S3 stream on R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_02_2022.csv [2022/09/07-18:45:15.612] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Path to read /R001_VESTA_BS_FINANCE|/R001_VESTA_BS_FINANCE_01_02_2022.csv -> R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_02_2022.csv [2022/09/07-18:45:15.612] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_02_2022.csv** [2022/09/07-18:45:15.613] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Getting range on -1 [2022/09/07-18:45:15.667] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_02_2022.csv** [2022/09/07-18:45:15.667] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - Start uncompressed stream: /R001_VESTA_BS_FINANCE_01_02_2022.csv / totalRecsBefore=16998 [2022/09/07-18:45:15.668] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_02_2022.csv** [2022/09/07-18:45:15.921] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - after stream totalComp=9993999 totalUncomp=9993999 totalRec=34159 [2022/09/07-18:45:15.922] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Getting S3 stream on R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_03_2022.csv [2022/09/07-18:45:15.946] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Path to read /R001_VESTA_BS_FINANCE|/R001_VESTA_BS_FINANCE_01_03_2022.csv -> R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_03_2022.csv [2022/09/07-18:45:15.946] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_03_2022.csv** [2022/09/07-18:45:15.946] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Getting range on -1 [2022/09/07-18:45:15.982] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_03_2022.csv** [2022/09/07-18:45:15.982] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - Start uncompressed stream: /R001_VESTA_BS_FINANCE_01_03_2022.csv / totalRecsBefore=34159 [2022/09/07-18:45:15.982] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_03_2022.csv** [2022/09/07-18:45:16.268] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - after stream totalComp=16682112 totalUncomp=16682112 totalRec=57310 [2022/09/07-18:45:16.268] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Getting S3 stream on R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_04_2022.csv [2022/09/07-18:45:16.294] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Path to read /R001_VESTA_BS_FINANCE|/R001_VESTA_BS_FINANCE_01_04_2022.csv -> R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_04_2022.csv [2022/09/07-18:45:16.294] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_04_2022.csv** [2022/09/07-18:45:16.295] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Getting range on -1 [2022/09/07-18:45:16.329] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_04_2022.csv** [2022/09/07-18:45:16.330] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - Start uncompressed stream: /R001_VESTA_BS_FINANCE_01_04_2022.csv / totalRecsBefore=57310 [2022/09/07-18:45:16.330] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_04_2022.csv** [2022/09/07-18:45:16.503] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - after stream totalComp=22002718 totalUncomp=22002718 totalRec=75423 [2022/09/07-18:45:16.503] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Getting S3 stream on R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_05_2022.csv [2022/09/07-18:45:16.534] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Path to read /R001_VESTA_BS_FINANCE|/R001_VESTA_BS_FINANCE_01_05_2022.csv -> R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_05_2022.csv [2022/09/07-18:45:16.535] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_05_2022.csv** [2022/09/07-18:45:16.535] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Getting range on -1 [2022/09/07-18:45:16.582] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_05_2022.csv** [2022/09/07-18:45:16.582] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - Start uncompressed stream: /R001_VESTA_BS_FINANCE_01_05_2022.csv / totalRecsBefore=75423 [2022/09/07-18:45:16.582] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_05_2022.csv** [2022/09/07-18:45:16.789] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - after stream totalComp=28078716 totalUncomp=28078716 totalRec=96208 [2022/09/07-18:45:16.789] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Getting S3 stream on R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_06_2022.csv [2022/09/07-18:45:16.814] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Path to read /R001_VESTA_BS_FINANCE|/R001_VESTA_BS_FINANCE_01_06_2022.csv -> R001_VESTA_BS_FINANCE/R001_VESTA_BS_FINANCE_01_06_2022.csv [2022/09/07-18:45:16.814] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_06_2022.csv** [2022/09/07-18:45:16.814] [FRT-49-FlowRunnable] [INFO] [dku.fs.s3] act.compute_O2C_VESTA_FR_NP - Getting range on -1 [2022/09/07-18:45:16.854] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_06_2022.csv** [2022/09/07-18:45:16.855] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - Start uncompressed stream: /R001_VESTA_BS_FINANCE_01_06_2022.csv / totalRecsBefore=96208 [2022/09/07-18:45:16.855] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**R001_VESTA_BS_FINANCE_01_06_2022.csv** [2022/09/07-18:45:17.052] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - after stream totalComp=34340070 totalUncomp=34340070 totalRec=117711 [2022/09/07-18:45:17.053] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - Extractor run done, totalCompressed=34340070 totalRecords=117711 [2022/09/07-18:45:17.053] [FRT-49-FlowRunnable] [INFO] [dku.h2loader] act.compute_O2C_VESTA_FR_NP - Done: 2384ms [2022/09/07-18:45:17.056] [FRT-49-FlowRunnable] [INFO] [dku.h2loader] act.compute_O2C_VESTA_FR_NP - Creating H2 table from CSV. Statement: [2022/09/07-18:45:17.056] [FRT-49-FlowRunnable] [INFO] [dku.h2loader] act.compute_O2C_VESTA_FR_NP - CREATE TABLE "TABLE_0" ( "SOURCESYS" varchar, "Mandant" varchar, "Code_GBU1" varchar, "GBU_level_1" varchar, "Code_GBU2" varchar, "GBU_level_2" varchar, "Code_GBU3" varchar, "GBU_level_3" varchar, "Pays" varchar, "Code_SMART" varchar, "Libellé_SMART" varchar, "Code_OC" varchar, "Libellé_OC" varchar, "Code_juridique" varchar, "Segment" varchar, "Division" varchar, "Exercice" varchar, "Période" varchar, "Cle_Comptabilisation" varchar, "Type_Pièce" varchar, "Libellé_Type_Pièce" varchar, "Code_Transaction" varchar, "ID_Utilisateur" varchar, "Code_Fournisseur" varchar, "Fournisseur" varchar, "Siret" varchar, "Compte_General" varchar, "Type_Compte_General" varchar, "ID_FMFI" varchar, "Lettrage" varchar, "Contre_Passation" varchar, "Montant_Total_EUR" varchar, "Montant_Total_Devise_Interne" varchar, "Nombre" varchar ) AS SELECT "SOURCESYS","Mandant","Code_GBU1","GBU_level_1","Code_GBU2","GBU_level_2","Code_GBU3","GBU_level_3","Pays","Code_SMART","Libellé_SMART","Code_OC","Libellé_OC","Code_juridique","Segment","Division","Exercice","Période","Cle_Comptabilisation","Type_Pièce","Libellé_Type_Pièce","Code_Transaction","ID_Utilisateur","Code_Fournisseur","Fournisseur","Siret","Compte_General","Type_Compte_General","ID_FMFI","Lettrage","Contre_Passation","Montant_Total_EUR","Montant_Total_Devise_Interne","Nombre" FROM CSVREAD('/data/dataiku/dss_data/jobs/02CKPIINDUSTRIALIZATIONBSFINANCE/Build_O2C_VESTA_FR_WITH_PERIMETRE_OC__NP__2022-09-07T18-45-09.083/compute_O2C_VESTA_FR_NP/dataset-to-h2/bpIBKGwumi2HYXuadCMN/TABLE_0.csv', 'SOURCESYS,Mandant,Code_GBU1,GBU_level_1,Code_GBU2,GBU_level_2,Code_GBU3,GBU_level_3,Pays,Code_SMART,Libellé_SMART,Code_OC,Libellé_OC,Code_juridique,Segment,Division,Exercice,Période,Cle_Comptabilisation,Type_Pièce,Libellé_Type_Pièce,Code_Transaction,ID_Utilisateur,Code_Fournisseur,Fournisseur,Siret,Compte_General,Type_Compte_General,ID_FMFI,Lettrage,Contre_Passation,Montant_Total_EUR,Montant_Total_Devise_Interne,Nombre', 'charset=UTF-8 escape=\\ fieldSeparator=, fieldDelimiter="') [2022/09/07-18:45:19.558] [FRT-49-FlowRunnable] [INFO] [dku.h2loader] act.compute_O2C_VESTA_FR_NP - Done: 2502ms [2022/09/07-18:45:19.559] [FRT-49-FlowRunnable] [INFO] [dku.fuzzyjoin.RowsDAO] act.compute_O2C_VESTA_FR_NP - Creating table 1 [2022/09/07-18:45:19.560] [FRT-49-FlowRunnable] [INFO] [dku.h2loader] act.compute_O2C_VESTA_FR_NP - Dumping dataset to CSV for H2... [2022/09/07-18:45:19.562] [FRT-49-FlowRunnable] [INFO] [dku.datasets.file] act.compute_O2C_VESTA_FR_NP - Building Filesystem handler config: {"path":"/data/dataiku/dss_data/uploads/02CKPIINDUSTRIALIZATIONBSFINANCE/datasets/REF_OC_PERIMETRE_ERNERGY_BE","notReadyIfEmpty":false,"filesSelectionRules":{"mode":"ALL","excludeRules":[],"includeRules":[],"explicitFiles":[]}} [2022/09/07-18:45:19.562] [FRT-49-FlowRunnable] [INFO] [dku.datasets.ftplike] act.compute_O2C_VESTA_FR_NP - Enumerating Filesystem dataset prefix= [2022/09/07-18:45:19.562] [FRT-49-FlowRunnable] [DEBUG] [dku.fs.local] act.compute_O2C_VESTA_FR_NP - Enumerating local filesystem prefix=/ [2022/09/07-18:45:19.563] [FRT-49-FlowRunnable] [DEBUG] [dku.fs.local] act.compute_O2C_VESTA_FR_NP - Enumeration done nb_paths=1 size=19493 [2022/09/07-18:45:19.563] [FRT-49-FlowRunnable] [INFO] [dku.input.push] act.compute_O2C_VESTA_FR_NP - USTP: push selection.method=FULL records=-1 ratio=0.02 col=null [2022/09/07-18:45:19.563] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - Extractor run: limit={"maxBytes":-1,"maxRecords":-1,"ordering":{"enabled":false,"rules":[]}} totalRecords=0 [2022/09/07-18:45:19.564] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**REF_OC_PERIMETRE_ERNERGY_BE_02_09_2022.xlsx** [2022/09/07-18:45:19.564] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**REF_OC_PERIMETRE_ERNERGY_BE_02_09_2022.xlsx** [2022/09/07-18:45:19.564] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - Start uncompressed stream: /data/dataiku/dss_data/uploads/02CKPIINDUSTRIALIZATIONBSFINANCE/datasets/REF_OC_PERIMETRE_ERNERGY_BE/REF_OC_PERIMETRE_ERNERGY_BE_02_09_2022.xlsx / totalRecsBefore=0 [2022/09/07-18:45:19.564] [FRT-49-FlowRunnable] [INFO] [dku] act.compute_O2C_VESTA_FR_NP - getCompression filename=**REF_OC_PERIMETRE_ERNERGY_BE_02_09_2022.xlsx** [2022/09/07-18:45:19.564] [FRT-49-FlowRunnable] [INFO] [dku.format.excel] act.compute_O2C_VESTA_FR_NP - Excel starting to process one stream: {"xlsx":true,"preserveNumberFormatting":false,"parseDatesToISO":false,"skipRowsBeforeHeader":0,"parseHeaderRow":true,"skipRowsAfterHeader":0,"sheets":"*STAR"} [2022/09/07-18:45:19.565] [FRT-49-FlowRunnable] [DEBUG] [com.monitorjbl.xlsx.impl.StreamingWorkbookReader] act.compute_O2C_VESTA_FR_NP - Created temp file [/data/dataiku/dss_data/tmp/jeks/jek-maVFRrwi/tmp-6065332789419691273.xlsx] [2022/09/07-18:45:19.602] [FRT-49-FlowRunnable] [INFO] [dku.format.excel] act.compute_O2C_VESTA_FR_NP - Parse as header row: colIdx=0 cellValue=SAP [2022/09/07-18:45:19.603] [FRT-49-FlowRunnable] [INFO] [dku.format.excel] act.compute_O2C_VESTA_FR_NP - Parse as header row: colIdx=1 cellValue=SOURCESYS [2022/09/07-18:45:19.603] [FRT-49-FlowRunnable] [INFO] [dku.format.excel] act.compute_O2C_VESTA_FR_NP - Parse as header row: colIdx=2 cellValue=CoCo [2022/09/07-18:45:19.603] [FRT-49-FlowRunnable] [INFO] [dku.format.excel] act.compute_O2C_VESTA_FR_NP - Parse as header row: colIdx=3 cellValue=BU [2022/09/07-18:45:19.603] [FRT-49-FlowRunnable] [INFO] [dku.format.excel] act.compute_O2C_VESTA_FR_NP - Parse as header row: colIdx=4 cellValue=group [2022/09/07-18:45:19.603] [FRT-49-FlowRunnable] [INFO] [dku.format.excel] act.compute_O2C_VESTA_FR_NP - Parse as header row: colIdx=5 cellValue=Sub groep [2022/09/07-18:45:19.603] [FRT-49-FlowRunnable] [INFO] [dku.format.excel] act.compute_O2C_VESTA_FR_NP - Parse as header row: colIdx=6 cellValue=Entity [2022/09/07-18:45:19.604] [FRT-49-FlowRunnable] [INFO] [dku.format.excel] act.compute_O2C_VESTA_FR_NP - Parse as header row: colIdx=9 cellValue=SAP [2022/09/07-18:45:19.604] [FRT-49-FlowRunnable] [INFO] [dku.format.excel] act.compute_O2C_VESTA_FR_NP - Parse as header row: colIdx=10 cellValue=GL [2022/09/07-18:45:19.608] [FRT-49-FlowRunnable] [DEBUG] [com.monitorjbl.xlsx.impl.StreamingWorkbookReader] act.compute_O2C_VESTA_FR_NP - Deleting tmp file [/data/dataiku/dss_data/tmp/jeks/jek-maVFRrwi/tmp-6065332789419691273.xlsx] [2022/09/07-18:45:19.608] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - after stream totalComp=19493 totalUncomp=19493 totalRec=48 [2022/09/07-18:45:19.608] [FRT-49-FlowRunnable] [INFO] [dku.format] act.compute_O2C_VESTA_FR_NP - Extractor run done, totalCompressed=19493 totalRecords=48 [2022/09/07-18:45:19.609] [FRT-49-FlowRunnable] [INFO] [dku.h2loader] act.compute_O2C_VESTA_FR_NP - Done: 49ms [2022/09/07-18:45:19.610] [FRT-49-FlowRunnable] [INFO] [dku.h2loader] act.compute_O2C_VESTA_FR_NP - Creating H2 table from CSV. Statement: [2022/09/07-18:45:19.610] [FRT-49-FlowRunnable] [INFO] [dku.h2loader] act.compute_O2C_VESTA_FR_NP - CREATE TABLE "TABLE_1" ( "SAP" varchar, "SOURCESYS" varchar, "CoCo" varchar, "BU" varchar, "group" varchar, "Sub groep" varchar, "Entity" varchar, "col_7" varchar, "col_8" varchar, "SAP_1" varchar, "GL" varchar ) AS SELECT "SAP","SOURCESYS","CoCo","BU","group","Sub groep","Entity","col_7","col_8","SAP_1","GL" FROM CSVREAD('/data/dataiku/dss_data/jobs/02CKPIINDUSTRIALIZATIONBSFINANCE/Build_O2C_VESTA_FR_WITH_PERIMETRE_OC__NP__2022-09-07T18-45-09.083/compute_O2C_VESTA_FR_NP/dataset-to-h2/bpIBKGwumi2HYXuadCMN/TABLE_1.csv', 'SAP,SOURCESYS,CoCo,BU,group,Sub groep,Entity,col_7,col_8,SAP_1,GL', 'charset=UTF-8 escape=\\ fieldSeparator=, fieldDelimiter="') [2022/09/07-18:45:19.613] [FRT-49-FlowRunnable] [INFO] [dku.h2loader] act.compute_O2C_VESTA_FR_NP - Done: 3ms [2022/09/07-18:45:19.613] [FRT-49-FlowRunnable] [INFO] [dku.fuzzyjoin.RowsDAO] act.compute_O2C_VESTA_FR_NP - Creating index for table 0 [2022/09/07-18:45:28.740] [FRT-49-FlowRunnable] [INFO] [dku.fuzzyjoin.RowsDAO] act.compute_O2C_VESTA_FR_NP - Creating index for table 1 [2022/09/07-18:45:28.743] [FRT-49-FlowRunnable] [INFO] [dku.fuzzyjoin.RowsDAO] act.compute_O2C_VESTA_FR_NP - Creating match flag for table 0 [2022/09/07-18:45:36.782] [FRT-49-FlowRunnable] [INFO] [dku.fuzzyjoin.RowsDAO] act.compute_O2C_VESTA_FR_NP - Creating match flag for table 1 [2022/09/07-18:45:39.302] [FRT-49-FlowRunnable] [INFO] [dku.recipe.fuzzyjoin.builtin] act.compute_O2C_VESTA_FR_NP - Init Fuzzy Join engine [2022/09/07-18:45:39.306] [FRT-49-FlowRunnable] [INFO] [dku.flow.activity] act.compute_O2C_VESTA_FR_NP - Run thread failed for activity compute_O2C_VESTA_FR_NP java.lang.NullPointerException at com.dataiku.dip.datalayer.streamimpl.StreamRow.get(StreamRow.java:94) at com.dataiku.dip.dataflow.exec.fuzzyjoin.builtinengine.io.FuzzyJoinRecords$RecordsIterator.makeRecord(FuzzyJoinRecords.java:86) at com.dataiku.dip.dataflow.exec.fuzzyjoin.builtinengine.io.FuzzyJoinRecords$RecordsIterator.next(FuzzyJoinRecords.java:70) at com.dataiku.dip.dataflow.exec.fuzzyjoin.builtinengine.io.FuzzyJoinRecords$RecordsIterator.next(FuzzyJoinRecords.java:48) at com.dataiku.dip.dataflow.exec.fuzzyjoin.builtinengine.selector.CandidateSelectorPipeline.populateWithCandidates(CandidateSelectorPipeline.java:126) at com.dataiku.dip.dataflow.exec.fuzzyjoin.builtinengine.selector.CandidateSelectorPipeline.(CandidateSelectorPipeline.java:31) at com.dataiku.dip.dataflow.exec.fuzzyjoin.builtinengine.FuzzyJoinBuiltinRecipeExecutor.(FuzzyJoinBuiltinRecipeExecutor.java:40) at com.dataiku.dip.dataflow.exec.fuzzyjoin.FuzzyJoinRecipeRunner.run(FuzzyJoinRecipeRunner.java:61) at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:374) [2022/09/07-18:45:39.496] [ActivityExecutor-41] [INFO] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - activity is finished [2022/09/07-18:45:39.497] [ActivityExecutor-41] [ERROR] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Activity failed java.lang.NullPointerException at com.dataiku.dip.datalayer.streamimpl.StreamRow.get(StreamRow.java:94) at com.dataiku.dip.dataflow.exec.fuzzyjoin.builtinengine.io.FuzzyJoinRecords$RecordsIterator.makeRecord(FuzzyJoinRecords.java:86) at com.dataiku.dip.dataflow.exec.fuzzyjoin.builtinengine.io.FuzzyJoinRecords$RecordsIterator.next(FuzzyJoinRecords.java:70) at com.dataiku.dip.dataflow.exec.fuzzyjoin.builtinengine.io.FuzzyJoinRecords$RecordsIterator.next(FuzzyJoinRecords.java:48) at com.dataiku.dip.dataflow.exec.fuzzyjoin.builtinengine.selector.CandidateSelectorPipeline.populateWithCandidates(CandidateSelectorPipeline.java:126) at com.dataiku.dip.dataflow.exec.fuzzyjoin.builtinengine.selector.CandidateSelectorPipeline.(CandidateSelectorPipeline.java:31) at com.dataiku.dip.dataflow.exec.fuzzyjoin.builtinengine.FuzzyJoinBuiltinRecipeExecutor.(FuzzyJoinBuiltinRecipeExecutor.java:40) at com.dataiku.dip.dataflow.exec.fuzzyjoin.FuzzyJoinRecipeRunner.run(FuzzyJoinRecipeRunner.java:61) at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:374) [2022/09/07-18:45:39.497] [ActivityExecutor-41] [INFO] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Executing default post-activity lifecycle hook [2022/09/07-18:45:39.499] [ActivityExecutor-41] [INFO] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Removing samples for 02CKPIINDUSTRIALIZATIONBSFINANCE.O2C_VESTA_FR_WITH_PERIMETRE_OC [2022/09/07-18:45:39.500] [ActivityExecutor-41] [INFO] [dku.flow.activity] running compute_O2C_VESTA_FR_NP - Done post-activity tasks