---------------------------------------- DSS startup: jek version:5.1.2 DSS home: /opt/dataiku/data_dir OS: Linux 3.10.0-1062.9.1.el7.x86_64 amd64 - Java: Oracle Corporation 1.8.0_201 [2020/02/24-16:13:52.440] [qtp128028301-29] [INFO] [dku.jobs] - Connects using Shared secret [2020/02/24-16:13:52.440] [qtp128028301-29] [DEBUG] [dku.jobs] - Received command : /pintercom/start_session [2020/02/24-16:13:52.451] [qtp128028301-29] [INFO] [dku.jobs] - Start Session: {"jobProjectKey":"SERVICEMANAGERWORKLOAD","jobId":"Build_ServiceManager_scored_2020-02-24T21-13-52.407","jobTicketSecret":"BhDmma5gMFw8ezbg5eGQ1qrbAMa7HbdruSn4JUvPqub8szdNwEWpbf4SWcF80KYb","initiatorAuthContext":{"via":[],"authSource":"USER_FROM_UI","realUserLogin":"212435620","userGroups":["AC_Projects"],"userGroupLevelPermissions":{"admin":true,"mayManageUDM":true,"mayCreateProjects":true,"mayWriteUnsafeCode":true,"mayWriteSafeCode":true,"mayCreateAuthenticatedConnections":true,"mayCreateCodeEnvs":true,"mayCreateClusters":true,"mayDevelopPlugins":true,"mayEditLibFolders":true,"mayManageCodeEnvs":true,"mayManageClusters":true,"mayViewIndexedHiveConnections":true,"mayCreatePublishedAPIServices":true},"userProfile":"DATA_SCIENTIST"}} [2020/02/24-16:13:52.454] [qtp128028301-29] [INFO] [dip.tickets.fixed] - Creating service for single API ticket in auth context: 212435620 [2020/02/24-16:13:52.463] [qtp128028301-29] [DEBUG] [dku.jobs] - Command /pintercom/start_session processed in 23ms [2020/02/24-16:13:52.468] [qtp128028301-30] [INFO] [dku.jobs] - Connects using Shared secret [2020/02/24-16:13:52.469] [qtp128028301-30] [DEBUG] [dku.jobs] - Received command : /pintercom/resolve_job [2020/02/24-16:13:52.469] [qtp128028301-30] [INFO] [dku.job.slave] - Starting to resolve job [2020/02/24-16:13:52.539] [qtp128028301-30] [INFO] [dku.job.slave] - { "type": "NON_RECURSIVE_FORCED_BUILD", "projectKey": "SERVICEMANAGERWORKLOAD", "id": "Build_ServiceManager_scored_2020-02-24T21-13-52.407", "name": "Build ServiceManager_scored", "initiator": "212435620", "initiationTimestamp": 1582578832407, "mailNotification": false, "outputs": [ { "type": "DATASET", "targetDatasetProjectKey": "SERVICEMANAGERWORKLOAD", "targetDataset": "ServiceManager_scored" } ], "refreshHiveMetastore": true } [2020/02/24-16:13:52.540] [qtp128028301-30] [INFO] [dku.job.slave] - Loading recipes [2020/02/24-16:13:52.546] [qtp128028301-30] [DEBUG] [dku.flow.recipes] - Building global graph [2020/02/24-16:13:52.647] [qtp128028301-30] [DEBUG] [dku.flow.recipes] - Done getting global graph [2020/02/24-16:13:52.648] [qtp128028301-30] [INFO] [dku.job.slave] - Recipes loaded, computing job details [2020/02/24-16:13:52.658] [qtp128028301-30] [INFO] [dku.flow.compute] - Job output type: DATASET [2020/02/24-16:13:52.670] [qtp128028301-30] [INFO] [dku.flow.compute] - computing job from target SERVICEMANAGERWORKLOAD.ServiceManager_scored partition NP [2020/02/24-16:13:52.674] [qtp128028301-30] [INFO] [dku.flow.compute] - Refreshing SERVICEMANAGERWORKLOAD.ServiceManager_scored/NP with recipe score_ServiceManager [2020/02/24-16:13:52.686] [qtp128028301-30] [INFO] [dku.flow.compute.op] activity score_ServiceManager_ - Compute output partitions for subgraph, with target : SERVICEMANAGERWORKLOAD.ServiceManager_scored and partition: NP [2020/02/24-16:13:52.687] [qtp128028301-30] [INFO] [dku.flow.compute.op] activity score_ServiceManager_ - Target SERVICEMANAGERWORKLOAD.ServiceManager_scored is not partitioned [2020/02/24-16:13:52.687] [qtp128028301-30] [INFO] [dku.flow.compute.op] activity score_ServiceManager_ - Add output partition for SERVICEMANAGERWORKLOAD.ServiceManager_scored : NP [2020/02/24-16:13:52.688] [qtp128028301-30] [INFO] [dku.flow.compute] activity score_ServiceManager_ - computed output partitions: {SERVICEMANAGERWORKLOAD.ServiceManager_scored=} [2020/02/24-16:13:52.694] [qtp128028301-30] [INFO] [dku.flow.compute.ip] activity score_ServiceManager_ - 0 dependency for SERVICEMANAGERWORKLOAD.ServiceManager [2020/02/24-16:13:52.695] [qtp128028301-30] [INFO] [dku.flow.compute.ip] activity score_ServiceManager_ - source SERVICEMANAGERWORKLOAD.ServiceManager is not partitioned, including the global partition [2020/02/24-16:13:52.696] [qtp128028301-30] [INFO] [dku.flow.compute] activity score_ServiceManager_ - computed input partitions : {SERVICEMANAGERWORKLOAD.ServiceManager=[], SERVICEMANAGERWORKLOAD.e3DDStGd=[]} [2020/02/24-16:13:52.706] [qtp128028301-30] [INFO] [dku.flow.compute] - Before prune, job looks like: JOB ACTIVITY ROOT DEPS ACTIVITY score_ServiceManager_NP SGID: score_ServiceManager_NP SRC: SERVICEMANAGERWORKLOAD.e3DDStGd (partitions: NP) SRC: SERVICEMANAGERWORKLOAD.ServiceManager (partitions: NP) DST: SERVICEMANAGERWORKLOAD.ServiceManager_scored (partition: NP) [2020/02/24-16:13:52.707] [qtp128028301-30] [INFO] [dku.flow.compute] - After cleanup of implicit recipes, job looks like: JOB ACTIVITY ROOT DEPS ACTIVITY score_ServiceManager_NP SGID: score_ServiceManager_NP SRC: SERVICEMANAGERWORKLOAD.e3DDStGd (partitions: NP) SRC: SERVICEMANAGERWORKLOAD.ServiceManager (partitions: NP) DST: SERVICEMANAGERWORKLOAD.ServiceManager_scored (partition: NP) [2020/02/24-16:13:52.710] [qtp128028301-30] [INFO] [dku.flow.compute] - Job project: SERVICEMANAGERWORKLOAD [2020/02/24-16:13:52.730] [qtp128028301-30] [INFO] [dku.flow.compute] - Doing job pruning according to settings [2020/02/24-16:13:52.737] [qtp128028301-30] [INFO] [dku.flow.compute] - not pruning activity, job type is NON_RECURSIVE_FORCED_BUILD [2020/02/24-16:13:52.738] [qtp128028301-30] [INFO] [dku.flow.compute] - After prune, job looks like that [2020/02/24-16:13:52.739] [qtp128028301-30] [INFO] [dku.flow.compute] - JOB ACTIVITY ROOT DEPS ACTIVITY score_ServiceManager_NP SGID: score_ServiceManager_NP SRC: SERVICEMANAGERWORKLOAD.e3DDStGd (partitions: NP) SRC: SERVICEMANAGERWORKLOAD.ServiceManager (partitions: NP) DST: SERVICEMANAGERWORKLOAD.ServiceManager_scored (partition: NP) [2020/02/24-16:13:52.739] [qtp128028301-30] [INFO] [dku.flow.compute] - Done building final job graph [2020/02/24-16:13:52.740] [qtp128028301-30] [INFO] [dku.job.slave] - Done computing job data, dumping graph [2020/02/24-16:13:52.740] [qtp128028301-30] [INFO] [dku.job.slave] - JOB ACTIVITY ROOT DEPS ACTIVITY score_ServiceManager_NP SGID: score_ServiceManager_NP SRC: SERVICEMANAGERWORKLOAD.e3DDStGd (partitions: NP) SRC: SERVICEMANAGERWORKLOAD.ServiceManager (partitions: NP) DST: SERVICEMANAGERWORKLOAD.ServiceManager_scored (partition: NP) [2020/02/24-16:13:52.744] [qtp128028301-30] [INFO] [dku.job.slave] - Job has the following sources: [{"projectKey":"SERVICEMANAGERWORKLOAD","type":"SAVED_MODEL","id":"e3DDStGd"},{"projectKey":"SERVICEMANAGERWORKLOAD","type":"DATASET","id":"ServiceManager"}] [2020/02/24-16:13:52.744] [qtp128028301-30] [INFO] [dku.job.slave] - Trying to obtain grant: datasetread::READ_DATA::SERVICEMANAGERWORKLOAD.ServiceManager [2020/02/24-16:13:52.781] [qtp128028301-30] [DEBUG] [dku.jobs] - Command /pintercom/resolve_job processed in 313ms [2020/02/24-16:13:52.787] [qtp128028301-29] [INFO] [dku.jobs] - Connects using Shared secret [2020/02/24-16:13:52.788] [qtp128028301-29] [DEBUG] [dku.jobs] - Received command : /pintercom/run_fully 47.974: [GC (Allocation Failure) [PSYoungGen: 258048K->16210K(279040K)] 264154K->22324K(470528K), 0.0215882 secs] [Times: user=0.06 sys=0.02, real=0.03 secs] [2020/02/24-16:13:52.832] [qtp128028301-29] [INFO] [dku.job.slave] - Job run started in kernel with pid 8839 [2020/02/24-16:13:52.835] [qtp128028301-29] [INFO] [dku.flow.jobrunner] - Creating 5 activity runner thread(s). [2020/02/24-16:13:52.836] [qtp128028301-29] [INFO] [dku.flow.jobrunner] - Starting activity executor threads [2020/02/24-16:13:52.838] [qtp128028301-29] [INFO] [dku.flow.jobrunner] - Waiting for activity executor threads [2020/02/24-16:13:52.845] [ActivityExecutor-34] [INFO] [dku.flow.jobrunner] - About to run score_ServiceManager_NP [2020/02/24-16:13:52.861] [ActivityExecutor-34] [INFO] [dku.flow.jobrunner] running score_ServiceManager_NP - Allocated a slot for this activity! [2020/02/24-16:13:52.867] [ActivityExecutor-34] [INFO] [dku.flow.jobrunner] running score_ServiceManager_NP - Run activity [2020/02/24-16:13:52.883] [ActivityExecutor-34] [INFO] [dku.flow.activity] running score_ServiceManager_NP - Executing default pre-activity lifecycle hook [2020/02/24-16:13:52.898] [ActivityExecutor-34] [INFO] [dku.flow.activity] running score_ServiceManager_NP - Checking if sources are ready [2020/02/24-16:13:52.938] [ActivityExecutor-34] [INFO] [dku.flow.activity] running score_ServiceManager_NP - Checked source readiness SERVICEMANAGERWORKLOAD.e3DDStGd -> true [2020/02/24-16:13:52.945] [ActivityExecutor-34] [DEBUG] [dku.dataset.hash] running score_ServiceManager_NP - Readiness cache miss for dataset__212435620__SERVICEMANAGERWORKLOAD.ServiceManager__NP [2020/02/24-16:13:53.125] [ActivityExecutor-34] [INFO] [dku.connections.sql.provider] running score_ServiceManager_NP - Connecting to jdbc:sqlserver://tnwp09931.logon.ds.ge.com with props: {"user":"Public123","password":"***","portNumber":"1433","databaseName":"DW_ODS"} conn=DW_ODS_Read_Only-8L3iPFs [2020/02/24-16:13:53.126] [ActivityExecutor-34] [DEBUG] [dku.connections.sql.provider] running score_ServiceManager_NP - Driver version 7.2 [2020/02/24-16:13:53.477] [ActivityExecutor-34] [INFO] [dku.connections.sql.provider] running score_ServiceManager_NP - Driver: Microsoft JDBC Driver 7.2 for SQL Server (JDBC 4.2) 7.2.1.0 (7.2) [2020/02/24-16:13:53.478] [ActivityExecutor-34] [INFO] [dku.connections.sql.provider] running score_ServiceManager_NP - Database: Microsoft SQL Server 10.50.6560 (10.50) rowSize=8060 stmts=0 [2020/02/24-16:13:53.534] [ActivityExecutor-34] [INFO] [dip.input.sql] running score_ServiceManager_NP - Using optimized method for getQuerySchema on query --select SM.Name, Year, FiscalWeek, T.PREndDate Date, sum(Hours) Hours, count(distinct T.Job) Jobs, count(distinct Employee) Employees, LOB.NewLOB, -6.86342439758949+count(distinct Employee)+count(distinct T.Job)*11.6679304433033 Workload, case when count(distinct Accident) >=1 then 1 else 0 end AccidentOccured from JCJM inner join VP_zzPRTH2015 T on JCJM.Job collate Database_Default = T.Job and JCJM.JCCo = T.PRCo inner join AACalendar C on T.PREndDate = C.Date inner join VP_zzudServiceManager SM on SM.DatabaseValue = JCJM.udServiceManager left join (select distinct PRTH.PRCo, PRTH.Job, case when Year(PREndDate)=2019 then udLineOfBusiness when JCCI.JCCo = 6 then 'APM Canada & Specialty Services' when udJobRegion in ('SS','RIG') then 'APM Canada & Specialty Services' when (CC.udSiteColl is not null or (JCCI.JCCo = 1 and udFrameSize in ('3','5','6B','6F','7B','7E','7F','7FB','7H','7HA','9E','9F','F5','5','6B','7E','7F','7FB'))) then 'APM Gas & Combined Cycle' when (udLineOfBusiness = 'Nuclear' or (JCCI.JCCo = 1 and udFrameSize in ('A10','A14','D11','ST','ST-Large','ST'))) then 'APM Steam & Nuclear' when JCCI.JCCo in (7,8) then 'Boiler - Canada and US'else 'Other' end as NewLOB from VP_zzPRTH2015 PRTH inner join VP_zzJCCICritical JCCI on JCCI.ID = PRTH.JCCI_ID left join vwCombinedCycleSites CC on JCCI.udSiteColl = CC.udSiteColl ) LOB on LOB.Job = JCJM.Contract collate Database_Default left join (select Accident, AccidentDate, JCCI_ID, Job collate Database_Default Job, Super, 'Recordable' as Recordability from AAAccidents where Recordability = 'Recordable' union all select [ENTRY #] as Accident, EventDate as AccidentDate, JCCI_ID, Job, ApmSuperEmpNo, case when FirstAid = 'Yes' then 'First Aid' else 'Near Miss' end as Recordability from VP_zzSERDataNew where (FirstAid='Yes' or [Safety Issue] in ('Near Miss','Incident') or (Issue='Quality' and ManHrsLost>0) )) Events on C.Date = Events.AccidentDate and JCJM.Job = Events.Job where udServiceManager is not null and Year(PREndDate)>=2017 group by SM.Name, Year, FiscalWeek, T.PREndDate, LOB.NewLOB-- [2020/02/24-16:13:53.535] [ActivityExecutor-34] [INFO] [dip.input.sql] running score_ServiceManager_NP - SELECT statement is n.0: --select SM.Name, Year, FiscalWeek, T.PREndDate Date, sum(Hours) Hours, count(distinct T.Job) Jobs, count(distinct Employee) Employees, LOB.NewLOB, -6.86342439758949+count(distinct Employee)+count(distinct T.Job)*11.6679304433033 Workload, case when count(distinct Accident) >=1 then 1 else 0 end AccidentOccured from JCJM inner join VP_zzPRTH2015 T on JCJM.Job collate Database_Default = T.Job and JCJM.JCCo = T.PRCo inner join AACalendar C on T.PREndDate = C.Date inner join VP_zzudServiceManager SM on SM.DatabaseValue = JCJM.udServiceManager left join (select distinct PRTH.PRCo, PRTH.Job, case when Year(PREndDate)=2019 then udLineOfBusiness when JCCI.JCCo = 6 then 'APM Canada & Specialty Services' when udJobRegion in ('SS','RIG') then 'APM Canada & Specialty Services' when (CC.udSiteColl is not null or (JCCI.JCCo = 1 and udFrameSize in ('3','5','6B','6F','7B','7E','7F','7FB','7H','7HA','9E','9F','F5','5','6B','7E','7F','7FB'))) then 'APM Gas & Combined Cycle' when (udLineOfBusiness = 'Nuclear' or (JCCI.JCCo = 1 and udFrameSize in ('A10','A14','D11','ST','ST-Large','ST'))) then 'APM Steam & Nuclear' when JCCI.JCCo in (7,8) then 'Boiler - Canada and US'else 'Other' end as NewLOB from VP_zzPRTH2015 PRTH inner join VP_zzJCCICritical JCCI on JCCI.ID = PRTH.JCCI_ID left join vwCombinedCycleSites CC on JCCI.udSiteColl = CC.udSiteColl ) LOB on LOB.Job = JCJM.Contract collate Database_Default left join (select Accident, AccidentDate, JCCI_ID, Job collate Database_Default Job, Super, 'Recordable' as Recordability from AAAccidents where Recordability = 'Recordable' union all select [ENTRY #] as Accident, EventDate as AccidentDate, JCCI_ID, Job, ApmSuperEmpNo, case when FirstAid = 'Yes' then 'First Aid' else 'Near Miss' end as Recordability from VP_zzSERDataNew where (FirstAid='Yes' or [Safety Issue] in ('Near Miss','Incident') or (Issue='Quality' and ManHrsLost>0) )) Events on C.Date = Events.AccidentDate and JCJM.Job = Events.Job where udServiceManager is not null and Year(PREndDate)>=2017 group by SM.Name, Year, FiscalWeek, T.PREndDate, LOB.NewLOB-- [2020/02/24-16:13:53.677] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Name label=Name sqlType=varchar [12(100,0)] sqlSize=100 dkuType=STRING (len:100) [2020/02/24-16:13:53.678] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Year label=Year sqlType=int [4(10,0)] sqlSize=11 dkuType=INT (len:-1) [2020/02/24-16:13:53.678] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=FiscalWeek label=FiscalWeek sqlType=int [4(10,0)] sqlSize=11 dkuType=INT (len:-1) [2020/02/24-16:13:53.679] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Date label=Date sqlType=datetime [93(23,3)] sqlSize=23 dkuType=STRING (len:-1) [2020/02/24-16:13:53.679] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Hours label=Hours sqlType=numeric [2(38,2)] sqlSize=40 dkuType=DOUBLE (len:-1) [2020/02/24-16:13:53.680] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Jobs label=Jobs sqlType=int [4(10,0)] sqlSize=11 dkuType=INT (len:-1) [2020/02/24-16:13:53.680] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Employees label=Employees sqlType=int [4(10,0)] sqlSize=11 dkuType=INT (len:-1) [2020/02/24-16:13:53.681] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=NewLOB label=NewLOB sqlType=varchar [12(50,0)] sqlSize=50 dkuType=STRING (len:50) [2020/02/24-16:13:53.681] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Workload label=Workload sqlType=numeric [2(28,14)] sqlSize=30 dkuType=DOUBLE (len:-1) [2020/02/24-16:13:53.682] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=AccidentOccured label=AccidentOccured sqlType=int [4(10,0)] sqlSize=11 dkuType=INT (len:-1) [2020/02/24-16:13:53.683] [ActivityExecutor-34] [DEBUG] [dku.connections.sql.provider] running score_ServiceManager_NP - Close conn=DW_ODS_Read_Only-8L3iPFs [2020/02/24-16:13:53.685] [ActivityExecutor-34] [INFO] [dku.dataset.hash] running score_ServiceManager_NP - Caching readiness for dataset__212435620__SERVICEMANAGERWORKLOAD.ServiceManager__NP s=READY h=null [2020/02/24-16:13:53.685] [ActivityExecutor-34] [INFO] [dku.flow.activity] running score_ServiceManager_NP - Checked source readiness SERVICEMANAGERWORKLOAD.ServiceManager -> true [2020/02/24-16:13:53.686] [ActivityExecutor-34] [DEBUG] [dku.flow.activity] running score_ServiceManager_NP - Computing hashes to propagate BEFORE activity [2020/02/24-16:13:53.709] [ActivityExecutor-34] [DEBUG] [dku.dataset.hash] running score_ServiceManager_NP - Readiness cache miss for dataset__212435620__SERVICEMANAGERWORKLOAD.ServiceManager__NP [2020/02/24-16:13:53.715] [ActivityExecutor-34] [INFO] [dku.connections.sql.provider] running score_ServiceManager_NP - Connecting to jdbc:sqlserver://tnwp09931.logon.ds.ge.com with props: {"user":"Public123","password":"***","portNumber":"1433","databaseName":"DW_ODS"} conn=DW_ODS_Read_Only-jTI6G1Z [2020/02/24-16:13:53.716] [ActivityExecutor-34] [DEBUG] [dku.connections.sql.provider] running score_ServiceManager_NP - Driver version 7.2 [2020/02/24-16:13:53.961] [ActivityExecutor-34] [INFO] [dku.connections.sql.provider] running score_ServiceManager_NP - Driver: Microsoft JDBC Driver 7.2 for SQL Server (JDBC 4.2) 7.2.1.0 (7.2) [2020/02/24-16:13:53.961] [ActivityExecutor-34] [INFO] [dku.connections.sql.provider] running score_ServiceManager_NP - Database: Microsoft SQL Server 10.50.6560 (10.50) rowSize=8060 stmts=0 [2020/02/24-16:13:54.014] [ActivityExecutor-34] [INFO] [dip.input.sql] running score_ServiceManager_NP - Using optimized method for getQuerySchema on query --select SM.Name, Year, FiscalWeek, T.PREndDate Date, sum(Hours) Hours, count(distinct T.Job) Jobs, count(distinct Employee) Employees, LOB.NewLOB, -6.86342439758949+count(distinct Employee)+count(distinct T.Job)*11.6679304433033 Workload, case when count(distinct Accident) >=1 then 1 else 0 end AccidentOccured from JCJM inner join VP_zzPRTH2015 T on JCJM.Job collate Database_Default = T.Job and JCJM.JCCo = T.PRCo inner join AACalendar C on T.PREndDate = C.Date inner join VP_zzudServiceManager SM on SM.DatabaseValue = JCJM.udServiceManager left join (select distinct PRTH.PRCo, PRTH.Job, case when Year(PREndDate)=2019 then udLineOfBusiness when JCCI.JCCo = 6 then 'APM Canada & Specialty Services' when udJobRegion in ('SS','RIG') then 'APM Canada & Specialty Services' when (CC.udSiteColl is not null or (JCCI.JCCo = 1 and udFrameSize in ('3','5','6B','6F','7B','7E','7F','7FB','7H','7HA','9E','9F','F5','5','6B','7E','7F','7FB'))) then 'APM Gas & Combined Cycle' when (udLineOfBusiness = 'Nuclear' or (JCCI.JCCo = 1 and udFrameSize in ('A10','A14','D11','ST','ST-Large','ST'))) then 'APM Steam & Nuclear' when JCCI.JCCo in (7,8) then 'Boiler - Canada and US'else 'Other' end as NewLOB from VP_zzPRTH2015 PRTH inner join VP_zzJCCICritical JCCI on JCCI.ID = PRTH.JCCI_ID left join vwCombinedCycleSites CC on JCCI.udSiteColl = CC.udSiteColl ) LOB on LOB.Job = JCJM.Contract collate Database_Default left join (select Accident, AccidentDate, JCCI_ID, Job collate Database_Default Job, Super, 'Recordable' as Recordability from AAAccidents where Recordability = 'Recordable' union all select [ENTRY #] as Accident, EventDate as AccidentDate, JCCI_ID, Job, ApmSuperEmpNo, case when FirstAid = 'Yes' then 'First Aid' else 'Near Miss' end as Recordability from VP_zzSERDataNew where (FirstAid='Yes' or [Safety Issue] in ('Near Miss','Incident') or (Issue='Quality' and ManHrsLost>0) )) Events on C.Date = Events.AccidentDate and JCJM.Job = Events.Job where udServiceManager is not null and Year(PREndDate)>=2017 group by SM.Name, Year, FiscalWeek, T.PREndDate, LOB.NewLOB-- [2020/02/24-16:13:54.015] [ActivityExecutor-34] [INFO] [dip.input.sql] running score_ServiceManager_NP - SELECT statement is n.0: --select SM.Name, Year, FiscalWeek, T.PREndDate Date, sum(Hours) Hours, count(distinct T.Job) Jobs, count(distinct Employee) Employees, LOB.NewLOB, -6.86342439758949+count(distinct Employee)+count(distinct T.Job)*11.6679304433033 Workload, case when count(distinct Accident) >=1 then 1 else 0 end AccidentOccured from JCJM inner join VP_zzPRTH2015 T on JCJM.Job collate Database_Default = T.Job and JCJM.JCCo = T.PRCo inner join AACalendar C on T.PREndDate = C.Date inner join VP_zzudServiceManager SM on SM.DatabaseValue = JCJM.udServiceManager left join (select distinct PRTH.PRCo, PRTH.Job, case when Year(PREndDate)=2019 then udLineOfBusiness when JCCI.JCCo = 6 then 'APM Canada & Specialty Services' when udJobRegion in ('SS','RIG') then 'APM Canada & Specialty Services' when (CC.udSiteColl is not null or (JCCI.JCCo = 1 and udFrameSize in ('3','5','6B','6F','7B','7E','7F','7FB','7H','7HA','9E','9F','F5','5','6B','7E','7F','7FB'))) then 'APM Gas & Combined Cycle' when (udLineOfBusiness = 'Nuclear' or (JCCI.JCCo = 1 and udFrameSize in ('A10','A14','D11','ST','ST-Large','ST'))) then 'APM Steam & Nuclear' when JCCI.JCCo in (7,8) then 'Boiler - Canada and US'else 'Other' end as NewLOB from VP_zzPRTH2015 PRTH inner join VP_zzJCCICritical JCCI on JCCI.ID = PRTH.JCCI_ID left join vwCombinedCycleSites CC on JCCI.udSiteColl = CC.udSiteColl ) LOB on LOB.Job = JCJM.Contract collate Database_Default left join (select Accident, AccidentDate, JCCI_ID, Job collate Database_Default Job, Super, 'Recordable' as Recordability from AAAccidents where Recordability = 'Recordable' union all select [ENTRY #] as Accident, EventDate as AccidentDate, JCCI_ID, Job, ApmSuperEmpNo, case when FirstAid = 'Yes' then 'First Aid' else 'Near Miss' end as Recordability from VP_zzSERDataNew where (FirstAid='Yes' or [Safety Issue] in ('Near Miss','Incident') or (Issue='Quality' and ManHrsLost>0) )) Events on C.Date = Events.AccidentDate and JCJM.Job = Events.Job where udServiceManager is not null and Year(PREndDate)>=2017 group by SM.Name, Year, FiscalWeek, T.PREndDate, LOB.NewLOB-- [2020/02/24-16:13:54.062] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Name label=Name sqlType=varchar [12(100,0)] sqlSize=100 dkuType=STRING (len:100) [2020/02/24-16:13:54.063] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Year label=Year sqlType=int [4(10,0)] sqlSize=11 dkuType=INT (len:-1) [2020/02/24-16:13:54.063] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=FiscalWeek label=FiscalWeek sqlType=int [4(10,0)] sqlSize=11 dkuType=INT (len:-1) [2020/02/24-16:13:54.063] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Date label=Date sqlType=datetime [93(23,3)] sqlSize=23 dkuType=STRING (len:-1) [2020/02/24-16:13:54.064] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Hours label=Hours sqlType=numeric [2(38,2)] sqlSize=40 dkuType=DOUBLE (len:-1) [2020/02/24-16:13:54.064] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Jobs label=Jobs sqlType=int [4(10,0)] sqlSize=11 dkuType=INT (len:-1) [2020/02/24-16:13:54.064] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Employees label=Employees sqlType=int [4(10,0)] sqlSize=11 dkuType=INT (len:-1) [2020/02/24-16:13:54.065] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=NewLOB label=NewLOB sqlType=varchar [12(50,0)] sqlSize=50 dkuType=STRING (len:50) [2020/02/24-16:13:54.065] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=Workload label=Workload sqlType=numeric [2(28,14)] sqlSize=30 dkuType=DOUBLE (len:-1) [2020/02/24-16:13:54.065] [ActivityExecutor-34] [DEBUG] [dip.input.sql] running score_ServiceManager_NP - metadataToSchema: name=AccidentOccured label=AccidentOccured sqlType=int [4(10,0)] sqlSize=11 dkuType=INT (len:-1) [2020/02/24-16:13:54.066] [ActivityExecutor-34] [DEBUG] [dku.connections.sql.provider] running score_ServiceManager_NP - Close conn=DW_ODS_Read_Only-jTI6G1Z [2020/02/24-16:13:54.066] [ActivityExecutor-34] [INFO] [dku.dataset.hash] running score_ServiceManager_NP - Caching readiness for dataset__212435620__SERVICEMANAGERWORKLOAD.ServiceManager__NP s=READY h=null [2020/02/24-16:13:54.067] [ActivityExecutor-34] [DEBUG] [dku.flow.activity] running score_ServiceManager_NP - Recorded 2 hashes before activity run [2020/02/24-16:13:54.068] [ActivityExecutor-34] [DEBUG] [dku.flow.activity] running score_ServiceManager_NP - Building recipe runner of type [2020/02/24-16:13:54.088] [ActivityExecutor-34] [DEBUG] [dku.flow.activity] running score_ServiceManager_NP - Recipe runner built, will use 1 thread(s) [2020/02/24-16:13:54.089] [ActivityExecutor-34] [DEBUG] [dku.flow.activity] running score_ServiceManager_NP - Starting execution thread: com.dataiku.dip.analysis.ml.prediction.flow.PredictionScoringRecipeRunner@cda951 [2020/02/24-16:13:54.090] [ActivityExecutor-34] [DEBUG] [dku.flow.activity] running score_ServiceManager_NP - Execution threads started, waiting for activity end [2020/02/24-16:13:54.090] [FRT-39-FlowRunnable] [INFO] [dku.flow.activity] act.score_ServiceManager_NP - Run thread for activity score_ServiceManager_NP starting 49.301: [GC (Metadata GC Threshold) [PSYoungGen: 74696K->13644K(387584K)] 80811K->19767K(579072K), 0.0117489 secs] [Times: user=0.04 sys=0.00, real=0.01 secs] 49.313: [Full GC (Metadata GC Threshold) [PSYoungGen: 13644K->0K(387584K)] [ParOldGen: 6122K->18231K(279552K)] 19767K->18231K(667136K), [Metaspace: 34871K->34871K(1081344K)], 0.0738104 secs] [Times: user=0.36 sys=0.01, real=0.08 secs] [2020/02/24-16:13:54.224] [FRT-39-FlowRunnable] [INFO] [dku.shaker.data] act.score_ServiceManager_NP - Need to compute sampleId before checking memory cache [2020/02/24-16:13:54.230] [FRT-39-FlowRunnable] [DEBUG] [dip.shaker.runner] act.score_ServiceManager_NP - Script settings sampleMax=104857600 processedMax=-1 [2020/02/24-16:13:54.233] [FRT-39-FlowRunnable] [DEBUG] [dip.shaker.runner] act.score_ServiceManager_NP - Processing with sampleMax=104857600 processedMax=524288000 [2020/02/24-16:13:54.246] [FRT-39-FlowRunnable] [DEBUG] [dip.shaker.runner] act.score_ServiceManager_NP - Computed required sample id : ebd08810a11da06f67d132fe37d99128-NA-31c57f559ab672625afa057353f0de860--d751713988987e9331980363e24189ce [2020/02/24-16:13:54.250] [FRT-39-FlowRunnable] [DEBUG] [dku.shaker.cache] act.score_ServiceManager_NP - Shaker MemoryCache get on SERVICEMANAGERWORKLOAD.ServiceManager key=ds=68f5ed57b2103c88228c312fdfa7cc0a--scr=a006f9dc626bd17543236aead8ac2ab1--samp=ebd08810a11da06f67d132fe37d99128-NA-31c57f559ab672625afa057353f0de860--d751713988987e9331980363e24189ce: miss [2020/02/24-16:13:54.251] [FRT-39-FlowRunnable] [INFO] [dku.shaker.data] act.score_ServiceManager_NP - Memory cache miss [2020/02/24-16:13:54.254] [FRT-39-FlowRunnable] [INFO] [dip.shaker.runner] act.score_ServiceManager_NP - Getting sample: null [2020/02/24-16:13:54.255] [FRT-39-FlowRunnable] [DEBUG] [dip.shaker.runner] act.score_ServiceManager_NP - Script settings sampleMax=104857600 processedMax=-1 [2020/02/24-16:13:54.255] [FRT-39-FlowRunnable] [DEBUG] [dip.shaker.runner] act.score_ServiceManager_NP - Processing with sampleMax=104857600 processedMax=524288000 [2020/02/24-16:13:54.258] [FRT-39-FlowRunnable] [INFO] [dip.shaker.runner] act.score_ServiceManager_NP - Computed required sample id : ebd08810a11da06f67d132fe37d99128-NA-31c57f559ab672625afa057353f0de860--d751713988987e9331980363e24189ce [2020/02/24-16:13:54.259] [FRT-39-FlowRunnable] [INFO] [dip.shaker.runner] act.score_ServiceManager_NP - cache hit [2020/02/24-16:13:54.259] [FRT-39-FlowRunnable] [INFO] [dip.shaker.runner] act.score_ServiceManager_NP - Opening sample ebd08810a11da06f67d132fe37d99128-NA-31c57f559ab672625afa057353f0de860--d751713988987e9331980363e24189ce fileSize=257831 [2020/02/24-16:13:54.274] [FRT-39-FlowRunnable] [INFO] [dip.shaker.runner] act.score_ServiceManager_NP - Reading sample done, read 2618 rows: MemTable stats nrows=2618 ncols=10 allocChars=228949 usefulChars=228949 estimatedMem=738024 [2020/02/24-16:13:54.276] [FRT-39-FlowRunnable] [INFO] [dku.table] act.score_ServiceManager_NP - MemTable stats nrows=2618 ncols=10 allocChars=228949 usefulChars=228949 estimatedMem=738024 [2020/02/24-16:13:54.277] [FRT-39-FlowRunnable] [INFO] [dip.shaker.runner] act.score_ServiceManager_NP - Serialized warnings com.dataiku.dip.warnings.WarningsContext@278cae64 -> {"warnings":{},"totalCount":0} [2020/02/24-16:13:54.278] [FRT-39-FlowRunnable] [INFO] [dku.shaker.types.inferrer] act.score_ServiceManager_NP - Starting ANALYSIS type inference on 2618 rows [2020/02/24-16:13:54.329] [analysis-typeinfer-2] [INFO] [dku.meanings.service] - Loading list of meanings [2020/02/24-16:13:54.420] [FRT-39-FlowRunnable] [INFO] [dku.shaker.types.inferrer] act.score_ServiceManager_NP - Type inference elapsed=142 detect=10 verify=0 colCacheHit=0 colCacheMiss=10 ht=93 pt=197 dt=316 [2020/02/24-16:13:54.421] [FRT-39-FlowRunnable] [INFO] [dku.shaker.data] act.score_ServiceManager_NP - DataService done, initTime=1 runTime=23 inferTime=144 [2020/02/24-16:13:54.425] [FRT-39-FlowRunnable] [DEBUG] [dku.shaker.cache] act.score_ServiceManager_NP - Shaker MemoryCache put on SERVICEMANAGERWORKLOAD.ServiceManager key=ds=68f5ed57b2103c88228c312fdfa7cc0a--scr=a006f9dc626bd17543236aead8ac2ab1--samp=ebd08810a11da06f67d132fe37d99128-NA-31c57f559ab672625afa057353f0de860--d751713988987e9331980363e24189ce [2020/02/24-16:13:54.429] [FRT-39-FlowRunnable] [INFO] [dku.shaker.schema] act.score_ServiceManager_NP - Column Name meaning=Text fail=0 [2020/02/24-16:13:54.431] [FRT-39-FlowRunnable] [INFO] [dku.shaker.schema] act.score_ServiceManager_NP - Column Year meaning=LongMeaning fail=0 [2020/02/24-16:13:54.431] [FRT-39-FlowRunnable] [INFO] [dku.shaker.schema] act.score_ServiceManager_NP - Column FiscalWeek meaning=LongMeaning fail=0 [2020/02/24-16:13:54.431] [FRT-39-FlowRunnable] [INFO] [dku.shaker.schema] act.score_ServiceManager_NP - Column Date meaning=DateSource fail=0 [2020/02/24-16:13:54.432] [FRT-39-FlowRunnable] [INFO] [dku.shaker.schema] act.score_ServiceManager_NP - Column Hours meaning=DoubleMeaning fail=0 [2020/02/24-16:13:54.432] [FRT-39-FlowRunnable] [INFO] [dku.shaker.schema] act.score_ServiceManager_NP - Column Jobs meaning=LongMeaning fail=0 [2020/02/24-16:13:54.432] [FRT-39-FlowRunnable] [INFO] [dku.shaker.schema] act.score_ServiceManager_NP - Column Employees meaning=LongMeaning fail=0 [2020/02/24-16:13:54.433] [FRT-39-FlowRunnable] [INFO] [dku.shaker.schema] act.score_ServiceManager_NP - Column NewLOB meaning=FreeText fail=0 [2020/02/24-16:13:54.433] [FRT-39-FlowRunnable] [INFO] [dku.shaker.schema] act.score_ServiceManager_NP - Column Workload meaning=DoubleMeaning fail=0 [2020/02/24-16:13:54.433] [FRT-39-FlowRunnable] [INFO] [dku.shaker.schema] act.score_ServiceManager_NP - Column AccidentOccured meaning=LongMeaning fail=0 [2020/02/24-16:13:54.434] [FRT-39-FlowRunnable] [DEBUG] [dku.recipes.ml] act.score_ServiceManager_NP - Preprocessing is not compatible with Java scoring : Feature NewLOB cannot be preprocessed in Java: Text handling uses hashing on a Python model [2020/02/24-16:13:54.434] [FRT-39-FlowRunnable] [DEBUG] [dku.recipes.ml] act.score_ServiceManager_NP - Preprocessing is not compatible with SQL scoring : Not compatible with optimized scoring: Feature NewLOB cannot be preprocessed in Java: Text handling uses hashing on a Python model [2020/02/24-16:13:54.479] [FRT-39-FlowRunnable] [INFO] [dku.recipes.ml] act.score_ServiceManager_NP - Selected engine: {"type":"DSS","typeLabel":"DSS","variant":"STREAM","variantLabel":"Stream","label":"DSS stream (single-thread)","isSelectable":true,"statusWarnLevel":"OK","recommended":false} [2020/02/24-16:13:54.486] [FRT-39-FlowRunnable] [INFO] [dku.recipes.prediction.scoring] act.score_ServiceManager_NP - Will predict with threshold = 0.625 [2020/02/24-16:13:54.492] [FRT-39-FlowRunnable] [INFO] [dku.recipes.prediction.scoring] act.score_ServiceManager_NP - Selected engine type: DSS [2020/02/24-16:13:54.493] [FRT-39-FlowRunnable] [INFO] [dku.recipes.prediction.scoring] act.score_ServiceManager_NP - Running with original engine [2020/02/24-16:13:54.493] [FRT-39-FlowRunnable] [INFO] [dku.recipes.prediction.scoring] act.score_ServiceManager_NP - Running with a python backend [2020/02/24-16:13:54.503] [FRT-39-FlowRunnable] [INFO] [dku.recipes.prediction.scoring] act.score_ServiceManager_NP - Run scoring in code env built-in (set at deploy-time) [2020/02/24-16:13:54.514] [FRT-39-FlowRunnable] [WARN] [dku.code.projectLibs] act.score_ServiceManager_NP - External libraries file not found: /opt/dataiku/data_dir/jobs/SERVICEMANAGERWORKLOAD/Build_ServiceManager_scored_2020-02-24T21-13-52.407/localconfig/projects/SERVICEMANAGERWORKLOAD/lib/external-libraries.json [2020/02/24-16:13:54.515] [FRT-39-FlowRunnable] [INFO] [dku.code.projectLibs] act.score_ServiceManager_NP - EXTERNAL LIBS FROM SERVICEMANAGERWORKLOAD is {"gitReferences":{},"pythonPath":["python"],"rsrcPath":["R"],"importLibrariesFromProjects":[]} [2020/02/24-16:13:54.516] [FRT-39-FlowRunnable] [INFO] [dku.code.projectLibs] act.score_ServiceManager_NP - chunkFolder is /opt/dataiku/data_dir/jobs/SERVICEMANAGERWORKLOAD/Build_ServiceManager_scored_2020-02-24T21-13-52.407/localconfig/projects/SERVICEMANAGERWORKLOAD/lib/R [2020/02/24-16:13:54.517] [FRT-39-FlowRunnable] [INFO] [xxx] act.score_ServiceManager_NP - RSRC PATH: ["/opt/dataiku/data_dir/jobs/SERVICEMANAGERWORKLOAD/Build_ServiceManager_scored_2020-02-24T21-13-52.407/localconfig/projects/SERVICEMANAGERWORKLOAD/lib/R"] [2020/02/24-16:13:54.518] [FRT-39-FlowRunnable] [INFO] [dku.recipes.code.base] act.score_ServiceManager_NP - Writing dku-exec-env for local execution in /opt/dataiku/data_dir/jobs/SERVICEMANAGERWORKLOAD/Build_ServiceManager_scored_2020-02-24T21-13-52.407/score_ServiceManager_NP/prediction-scoring-recipe/runNTrk6wEaNDbq/remote-run-env-def.json [2020/02/24-16:13:54.522] [FRT-39-FlowRunnable] [INFO] [dku.code.envs.resolution] act.score_ServiceManager_NP - Executing Python activity in builtin env [2020/02/24-16:13:54.522] [FRT-39-FlowRunnable] [INFO] [dku.flow.abstract.python] act.score_ServiceManager_NP - Execute activity command: ["/opt/dataiku/data_dir/bin/python","-u","-m","dataiku.doctor.prediction.reg_scoring_recipe","/opt/dataiku/data_dir/saved_models/SERVICEMANAGERWORKLOAD/e3DDStGd/versions/initial","ServiceManager","ServiceManager_scored","/opt/dataiku/data_dir/jobs/SERVICEMANAGERWORKLOAD/Build_ServiceManager_scored_2020-02-24T21-13-52.407/score_ServiceManager_NP/prediction-scoring-recipe/runNTrk6wEaNDbq/desc.json","/opt/dataiku/data_dir/jobs/SERVICEMANAGERWORKLOAD/Build_ServiceManager_scored_2020-02-24T21-13-52.407/score_ServiceManager_NP/prediction-scoring-recipe/runNTrk6wEaNDbq/script.json","/opt/dataiku/data_dir/jobs/SERVICEMANAGERWORKLOAD/Build_ServiceManager_scored_2020-02-24T21-13-52.407/score_ServiceManager_NP/prediction-scoring-recipe/runNTrk6wEaNDbq/preparation_output_schema.json","/opt/dataiku/data_dir/jobs/SERVICEMANAGERWORKLOAD/Build_ServiceManager_scored_2020-02-24T21-13-52.407/score_ServiceManager_NP/prediction-scoring-recipe/runNTrk6wEaNDbq/conditional_outputs.json"] [2020/02/24-16:13:54.523] [FRT-39-FlowRunnable] [INFO] [dku.recipes.code.base] act.score_ServiceManager_NP - Run command insecurely, from user dssacct [2020/02/24-16:13:54.524] [FRT-39-FlowRunnable] [INFO] [dku.security.process] act.score_ServiceManager_NP - Starting process (regular) [2020/02/24-16:13:54.526] [FRT-39-FlowRunnable] [INFO] [dku.security.process] act.score_ServiceManager_NP - Process started with pid=9113 [2020/02/24-16:13:54.527] [FRT-39-FlowRunnable] [INFO] [dku.processes.cgroups] act.score_ServiceManager_NP - Will use cgroups [] [2020/02/24-16:13:54.528] [FRT-39-FlowRunnable] [INFO] [dku.processes.cgroups] act.score_ServiceManager_NP - Applying rules to used cgroups: [] [2020/02/24-16:13:54.531] [FRT-39-FlowRunnable] [INFO] [dku.recipes.code.base] act.score_ServiceManager_NP - Process reads from nothing [2020/02/24-16:13:55.763] [null-out-45] [INFO] [dku.utils] - Installing debugging signal handler [2020/02/24-16:13:55.763] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,762 INFO Will do preparation, output schema: {u'userModified': False, u'columns': [{u'type': u'string', u'name': u'Name'}, {u'type': u'bigint', u'name': u'Year'}, {u'type': u'bigint', u'name': u'FiscalWeek'}, {u'type': u'string', u'name': u'Date'}, {u'type': u'double', u'name': u'Hours'}, {u'type': u'bigint', u'name': u'Jobs'}, {u'type': u'bigint', u'name': u'Employees'}, {u'type': u'string', u'name': u'NewLOB'}, {u'type': u'double', u'name': u'Workload'}, {u'type': u'bigint', u'name': u'AccidentOccured'}]} [2020/02/24-16:13:55.947] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,945 INFO generating interactions [2020/02/24-16:13:55.949] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,946 INFO {u'preprocessingFitSampleSeed': 1337, u'feature_selection_params': {u'custom_params': {u'code': u'# type your code here'}, u'pca_params': {u'variance_proportion': 0.9, u'n_features': 25}, u'random_forest_params': {u'depth': 10, u'n_features': 25, u'n_trees': 30}, u'lasso_params': {u'alpha': [0.01, 0.1, 1.0, 10.0, 100.0], u'cross_validate': True}, u'method': u'NONE', u'correlation_params': {u'n_features': 25, u'min_abs_correlation': 0.0}}, u'preprocessingFitSampleRatio': 1.0, u'reduce': {u'enabled': False, u'kept_variance': 0.0}, u'skipPreprocessing': False, u'target_remapping': [{u'mappedValue': 0, u'sourceValue': u'0', u'sampleFreq': 2591}, {u'mappedValue': 1, u'sourceValue': u'1', u'sampleFreq': 27}], u'per_feature': {u'Hours': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'Workload': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'Jobs': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'Name': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'Employees': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'FiscalWeek': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'REJECT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'AccidentOccured': {u'generate_derivative': False, u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'sendToInput': u'main', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'role': u'TARGET', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'Year': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'REJECT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'Date': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'DateSource', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'REJECT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'NewLOB': {u'hashSVDHashSize': 200000, u'useCustomVectorizer': False, u'customHandlingCode': u'', u'hashSVDSVDLimit': 50000, u'customProcessorWantsMatrix': False, u'hashSVDSVDComponents': 100, u'sendToInput': u'main', u'maxWords': 0, u'state': {u'userModified': False, u'recordedMeaning': u'FreeText', u'autoModifiedByDSS': False}, u'ngramMaxSize': 1, u'text_handling': u'TOKENIZE_HASHING_SVD', u'ngramMinSize': 1, u'stopWordsMode': u'NONE', u'minRowsRatio': 0.001, u'role': u'INPUT', u'type': u'TEXT', u'maxRowsRatio': 0.8, u'name': u'NewLOB'}}, u'feature_generation': {u'manual_interactions': {u'interactions': []}, u'pairwise_linear': {u'behavior': u'DISABLED'}, u'categoricals_count_transformer': {u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}, u'polynomial_combinations': {u'behavior': u'DISABLED'}, u'numericals_clustering': {u'k': 0, u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}}} [2020/02/24-16:13:55.949] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,946 INFO No feature selection to perform [2020/02/24-16:13:55.949] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,946 INFO Scoring with batch size: 100000 [2020/02/24-16:13:55.957] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,957 INFO Starting writer [2020/02/24-16:13:55.957] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,957 INFO Initializing dataset writer for dataset SERVICEMANAGERWORKLOAD.ServiceManager_scored [2020/02/24-16:13:55.958] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,957 INFO Starting to iterate [2020/02/24-16:13:55.958] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,957 INFO Start output generator ... [2020/02/24-16:13:55.958] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,957 INFO Reading with INITIAL dtypes: {u'Hours': , u'Workload': , u'Jobs': , u'Name': , u'Employees': , u'FiscalWeek': , u'AccidentOccured': , u'Year': , u'Date': , u'NewLOB': } [2020/02/24-16:13:55.958] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,957 INFO Computed dtype for Name: str (schema_type=string feature_type=CATEGORY feature_role=INPUT) [2020/02/24-16:13:55.959] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,957 INFO Computed dtype for Year: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT) [2020/02/24-16:13:55.959] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,957 INFO Computed dtype for FiscalWeek: None (schema_type=bigint feature_type=NUMERIC feature_role=REJECT) [2020/02/24-16:13:55.959] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,957 INFO Computed dtype for Date: None (schema_type=string feature_type=CATEGORY feature_role=REJECT) [2020/02/24-16:13:55.960] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,957 INFO Computed dtype for Hours: (schema_type=double feature_type=NUMERIC feature_role=INPUT) [2020/02/24-16:13:55.960] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,957 INFO Computed dtype for Jobs: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2020/02/24-16:13:55.960] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,957 INFO Computed dtype for Employees: (schema_type=bigint feature_type=NUMERIC feature_role=INPUT) [2020/02/24-16:13:55.960] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Computed dtype for NewLOB: str (schema_type=string feature_type=TEXT feature_role=INPUT) [2020/02/24-16:13:55.961] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Computed dtype for Workload: (schema_type=double feature_type=NUMERIC feature_role=INPUT) [2020/02/24-16:13:55.961] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Computed dtype for AccidentOccured: (schema_type=bigint feature_type=NUMERIC feature_role=TARGET) [2020/02/24-16:13:55.961] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Reading with dtypes: {u'Workload': , u'Jobs': , u'Name': 'str', u'Employees': , u'Hours': , u'AccidentOccured': , u'NewLOB': 'str'} [2020/02/24-16:13:55.962] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Column 0 = Name (dtype=str) [2020/02/24-16:13:55.963] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Column 1 = Year (dtype=None) [2020/02/24-16:13:55.963] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Column 2 = FiscalWeek (dtype=None) [2020/02/24-16:13:55.963] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Column 3 = Date (dtype=None) [2020/02/24-16:13:55.963] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Column 4 = Hours (dtype=) [2020/02/24-16:13:55.964] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Column 5 = Jobs (dtype=) [2020/02/24-16:13:55.964] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Column 6 = Employees (dtype=) [2020/02/24-16:13:55.964] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Column 7 = NewLOB (dtype=str) [2020/02/24-16:13:55.965] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Column 8 = Workload (dtype=) [2020/02/24-16:13:55.965] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:13:55,958 INFO Column 9 = AccidentOccured (dtype=) [2020/02/24-16:13:57.837] [Thread-17] [INFO] [dku.job.slave] - Sending status update [2020/02/24-16:13:57.845] [Thread-17] [INFO] [dku.job.slave] - Status update sent [2020/02/24-16:14:02.845] [Thread-17] [INFO] [dku.job.slave] - Sending status update [2020/02/24-16:14:02.852] [Thread-17] [INFO] [dku.job.slave] - Status update sent [2020/02/24-16:14:07.853] [Thread-17] [INFO] [dku.job.slave] - Sending status update [2020/02/24-16:14:07.860] [Thread-17] [INFO] [dku.job.slave] - Status update sent [2020/02/24-16:14:12.861] [Thread-17] [INFO] [dku.job.slave] - Sending status update [2020/02/24-16:14:12.868] [Thread-17] [INFO] [dku.job.slave] - Status update sent [2020/02/24-16:14:17.868] [Thread-17] [INFO] [dku.job.slave] - Sending status update [2020/02/24-16:14:17.875] [Thread-17] [INFO] [dku.job.slave] - Status update sent [2020/02/24-16:14:22.875] [Thread-17] [INFO] [dku.job.slave] - Sending status update [2020/02/24-16:14:22.881] [Thread-17] [INFO] [dku.job.slave] - Status update sent [2020/02/24-16:14:27.881] [Thread-17] [INFO] [dku.job.slave] - Sending status update [2020/02/24-16:14:27.888] [Thread-17] [INFO] [dku.job.slave] - Status update sent [2020/02/24-16:14:28.705] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,704 INFO Starting dataframes iterator [2020/02/24-16:14:28.714] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,713 INFO Got a dataframe : (2618, 10) [2020/02/24-16:14:28.714] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,714 INFO Coercion done [2020/02/24-16:14:28.715] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,714 INFO NORMALIZED: Name -> object [2020/02/24-16:14:28.715] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,714 INFO NORMALIZED: Year -> int64 [2020/02/24-16:14:28.715] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,715 INFO NORMALIZED: FiscalWeek -> int64 [2020/02/24-16:14:28.715] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,715 INFO NORMALIZED: Date -> object [2020/02/24-16:14:28.716] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,715 INFO NORMALIZED: Hours -> float64 [2020/02/24-16:14:28.716] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,715 INFO NORMALIZED: Jobs -> float64 [2020/02/24-16:14:28.716] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,715 INFO NORMALIZED: Employees -> float64 [2020/02/24-16:14:28.717] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,715 INFO NORMALIZED: NewLOB -> object [2020/02/24-16:14:28.717] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,715 INFO NORMALIZED: Workload -> float64 [2020/02/24-16:14:28.717] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,715 INFO NORMALIZED: AccidentOccured -> object [2020/02/24-16:14:28.718] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,715 INFO Processing it [2020/02/24-16:14:28.718] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,715 INFO Predicting it [2020/02/24-16:14:28.718] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,715 INFO Prepare to predict ... [2020/02/24-16:14:28.719] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,716 INFO Set MF index len 2618 [2020/02/24-16:14:28.719] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,716 DEBUG PROCESS WITH Step:MultipleImputeMissingFromInput [2020/02/24-16:14:28.719] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,716 DEBUG MIMIFI: Imputing with map {u'Hours': 1958.1162278602203, u'Employees': 36.40928674006702, u'Workload': 71.40295700121932, u'Jobs': 3.58736237434179} [2020/02/24-16:14:28.720] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,717 DEBUG PROCESS WITH Step:RescalingProcessor2 (Hours) [2020/02/24-16:14:28.720] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,718 DEBUG Rescale Hours (avg=1942.16398014 std=3564.88503741 shift=1958.11622786 inv_scale=0.000279062395027) [2020/02/24-16:14:28.723] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,723 DEBUG Rescaled Hours (avg=-0.00445167245556 std=0.994825356536) nulls=0 [2020/02/24-16:14:28.723] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,723 DEBUG PROCESS WITH Step:RescalingProcessor2 (Workload) [2020/02/24-16:14:28.723] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,723 DEBUG Rescale Workload (avg=71.2431018894 std=95.5116381039 shift=71.4029570012 inv_scale=0.0104377212415) [2020/02/24-16:14:28.725] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,724 DEBUG Rescaled Workload (avg=-0.00166852309649 std=0.996923853852) nulls=0 [2020/02/24-16:14:28.725] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,724 DEBUG PROCESS WITH Step:RescalingProcessor2 (Jobs) [2020/02/24-16:14:28.726] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,724 DEBUG Rescale Jobs (avg=3.59243697479 std=3.72774918244 shift=3.58736237434 inv_scale=0.267141066582) [2020/02/24-16:14:28.727] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,727 DEBUG Rescaled Jobs (avg=0.00135563417619 std=0.995834892547) nulls=0 [2020/02/24-16:14:28.727] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,727 DEBUG PROCESS WITH Step:RescalingProcessor2 (Employees) [2020/02/24-16:14:28.727] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,727 DEBUG Rescale Employees (avg=36.1902215432 std=59.4253698149 shift=36.4092867401 inv_scale=0.0168496935127) [2020/02/24-16:14:28.729] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,728 DEBUG Rescaled Employees (avg=-0.00369118142713 std=1.00129926826) nulls=0 [2020/02/24-16:14:28.729] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,728 DEBUG PROCESS WITH Step:FlushDFBuilder(num_flagonly) [2020/02/24-16:14:28.729] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,728 DEBUG PROCESS WITH Step:FastSparseDummifyProcessor (Name) [2020/02/24-16:14:28.733] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,732 DEBUG Dummifier: Append a sparse block shape=(2618, 24) nnz=2618 [2020/02/24-16:14:28.733] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,732 DEBUG PROCESS WITH Step:MultipleImputeMissingFromInput [2020/02/24-16:14:28.734] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,732 DEBUG MIMIFI: Imputing with map {} [2020/02/24-16:14:28.734] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,733 DEBUG PROCESS WITH Step:FlushDFBuilder(cat_flagpresence) [2020/02/24-16:14:28.734] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,733 DEBUG PROCESS WITH Step:TextHashingVectorizerWithSVDProcessor (NewLOB hs=200000 sl=50000 sc=100) [2020/02/24-16:14:28.849] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,848 DEBUG PROCESS WITH Step:MultipleImputeMissingFromInput [2020/02/24-16:14:28.849] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 DEBUG MIMIFI: Imputing with map {} [2020/02/24-16:14:28.849] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 DEBUG PROCESS WITH Step:FlushDFBuilder(interaction) [2020/02/24-16:14:28.850] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 DEBUG PROCESS WITH Step:DumpPipelineState [2020/02/24-16:14:28.850] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 INFO ********* Pipieline state (Before feature selection) [2020/02/24-16:14:28.850] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 INFO input_df= (2618, 10) [2020/02/24-16:14:28.850] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 INFO current_mf=(2618, 128) [2020/02/24-16:14:28.850] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 INFO PPR: [2020/02/24-16:14:28.851] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 DEBUG PROCESS WITH Step:EmitCurrentMFAsResult [2020/02/24-16:14:28.851] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 INFO Set MF index len 2618 [2020/02/24-16:14:28.851] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 DEBUG PROCESS WITH Step:DumpPipelineState [2020/02/24-16:14:28.851] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 INFO ********* Pipieline state (At end) [2020/02/24-16:14:28.851] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 INFO input_df= (2618, 10) [2020/02/24-16:14:28.852] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 INFO current_mf=(0, 0) [2020/02/24-16:14:28.852] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,849 INFO PPR: [2020/02/24-16:14:28.852] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,850 INFO UNPROCESSED = ((2618, 10)) [2020/02/24-16:14:28.853] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,850 INFO TRAIN = ((2618, 128)) [2020/02/24-16:14:28.854] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,853 INFO prepare multiframe shape=(2618,128) tn=335104 nnz=274890 fill_ratio=0.82 [2020/02/24-16:14:28.854] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,853 INFO too small, using array [2020/02/24-16:14:28.856] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,855 INFO Start actual predict (threshold=0.625) [2020/02/24-16:14:28.856] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,856 INFO F Employees = 1.5601251405508862 [2020/02/24-16:14:28.856] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,856 INFO F Hours = 1.6175806441885276 [2020/02/24-16:14:28.856] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,856 INFO F Jobs = 2.7816431212496786 [2020/02/24-16:14:28.857] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,856 INFO F Workload = 2.2345558445957243 [2020/02/24-16:14:28.857] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,856 INFO F dummy:Name:Mike Benacquisto = 1.0 [2020/02/24-16:14:28.857] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,857 INFO F dummy:Name:Craig Mosier = 0.0 [2020/02/24-16:14:28.857] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,857 INFO F dummy:Name:Gary Vanderloo = 0.0 [2020/02/24-16:14:28.858] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,857 INFO F dummy:Name:Chris Jones = 0.0 [2020/02/24-16:14:28.858] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,857 INFO F dummy:Name:Mike Kehl = 0.0 [2020/02/24-16:14:28.858] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,857 INFO F dummy:Name:Don Lacroix = 0.0 [2020/02/24-16:14:28.858] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,857 INFO F dummy:Name:Dax Reed = 0.0 [2020/02/24-16:14:28.859] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,857 INFO F dummy:Name:Matt Kunzelman = 0.0 [2020/02/24-16:14:28.859] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,858 INFO F dummy:Name:Ron Mayor = 0.0 [2020/02/24-16:14:28.859] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,858 INFO F dummy:Name:Jason Guthrie = 0.0 [2020/02/24-16:14:28.859] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,858 INFO F dummy:Name:Jonathan Sautelet = 0.0 [2020/02/24-16:14:28.860] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,858 INFO F dummy:Name:Henry Geerts = 0.0 [2020/02/24-16:14:28.860] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,858 INFO F dummy:Name:Brett Kennard = 0.0 [2020/02/24-16:14:28.860] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,858 INFO F dummy:Name:Matt McCrary = 0.0 [2020/02/24-16:14:28.861] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,859 INFO F dummy:Name:James Ray = 0.0 [2020/02/24-16:14:28.861] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,859 INFO F dummy:Name:John Baierlein = 0.0 [2020/02/24-16:14:28.861] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,859 INFO F dummy:Name:Joe Giglio = 0.0 [2020/02/24-16:14:28.861] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,859 INFO F dummy:Name:James Gehring = 0.0 [2020/02/24-16:14:28.862] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,859 INFO F dummy:Name:Ed Deitman = 0.0 [2020/02/24-16:14:28.862] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,859 INFO F dummy:Name:Michel Frappier = 0.0 [2020/02/24-16:14:28.862] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,860 INFO F dummy:Name:Ben Storie = 0.0 [2020/02/24-16:14:28.862] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,860 INFO F dummy:Name:Stacy Mullis = 0.0 [2020/02/24-16:14:28.862] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,860 INFO F dummy:Name:N/A = 0.0 [2020/02/24-16:14:28.863] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,860 INFO F dummy:Name:__Others__ = 0.0 [2020/02/24-16:14:28.863] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,860 INFO F thsvd:NewLOB:0 = 0.9515651813009924 [2020/02/24-16:14:28.863] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,860 INFO F thsvd:NewLOB:1 = -0.17977777237146203 [2020/02/24-16:14:28.863] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,860 INFO F thsvd:NewLOB:2 = -0.11129585624477575 [2020/02/24-16:14:28.863] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,861 INFO F thsvd:NewLOB:3 = 0.13901588540722537 [2020/02/24-16:14:28.863] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,861 INFO F thsvd:NewLOB:4 = -4.2671773211996007e-16 [2020/02/24-16:14:28.864] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,861 INFO F thsvd:NewLOB:5 = -0.15743143357194345 [2020/02/24-16:14:28.864] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,861 INFO F thsvd:NewLOB:6 = 0.07554348421560977 [2020/02/24-16:14:28.864] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,861 INFO F thsvd:NewLOB:7 = 1.760155976662118e-19 [2020/02/24-16:14:28.864] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,861 INFO F thsvd:NewLOB:8 = 0.0 [2020/02/24-16:14:28.864] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,862 INFO F thsvd:NewLOB:9 = -2.7755575615628914e-17 [2020/02/24-16:14:28.864] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,862 INFO F thsvd:NewLOB:10 = -1.3877787807814457e-17 [2020/02/24-16:14:28.865] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,862 INFO F thsvd:NewLOB:11 = -3.469446951953614e-18 [2020/02/24-16:14:28.865] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,862 INFO F thsvd:NewLOB:12 = -6.938893903907228e-18 [2020/02/24-16:14:28.865] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,862 INFO F thsvd:NewLOB:13 = 0.0 [2020/02/24-16:14:28.865] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,862 INFO F thsvd:NewLOB:14 = -3.469446951953614e-18 [2020/02/24-16:14:28.865] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,862 INFO F thsvd:NewLOB:15 = 0.0 [2020/02/24-16:14:28.865] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,863 INFO F thsvd:NewLOB:16 = 1.951563910473908e-18 [2020/02/24-16:14:28.866] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,863 INFO F thsvd:NewLOB:17 = 6.938893903907228e-18 [2020/02/24-16:14:28.866] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,863 INFO F thsvd:NewLOB:18 = -4.163336342344337e-17 [2020/02/24-16:14:28.866] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,863 INFO F thsvd:NewLOB:19 = -2.1141942363467336e-18 [2020/02/24-16:14:28.866] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,863 INFO F thsvd:NewLOB:20 = 3.469446951953614e-18 [2020/02/24-16:14:28.866] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,863 INFO F thsvd:NewLOB:21 = 6.938893903907228e-18 [2020/02/24-16:14:28.867] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,864 INFO F thsvd:NewLOB:22 = 2.7755575615628914e-17 [2020/02/24-16:14:28.867] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,864 INFO F thsvd:NewLOB:23 = 6.938893903907228e-18 [2020/02/24-16:14:28.867] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,864 INFO F thsvd:NewLOB:24 = -1.3877787807814457e-17 [2020/02/24-16:14:28.867] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,864 INFO F thsvd:NewLOB:25 = 3.469446951953614e-18 [2020/02/24-16:14:28.867] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,864 INFO F thsvd:NewLOB:26 = 0.0 [2020/02/24-16:14:28.867] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,864 INFO F thsvd:NewLOB:27 = -6.071532165918825e-18 [2020/02/24-16:14:28.868] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,864 INFO F thsvd:NewLOB:28 = 0.0 [2020/02/24-16:14:28.868] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,865 INFO F thsvd:NewLOB:29 = -3.469446951953614e-18 [2020/02/24-16:14:28.868] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,865 INFO F thsvd:NewLOB:30 = 0.0 [2020/02/24-16:14:28.868] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,865 INFO F thsvd:NewLOB:31 = 3.469446951953614e-18 [2020/02/24-16:14:28.868] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,865 INFO F thsvd:NewLOB:32 = -1.3010426069826053e-18 [2020/02/24-16:14:28.868] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,865 INFO F thsvd:NewLOB:33 = -1.734723475976807e-18 [2020/02/24-16:14:28.869] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,866 INFO F thsvd:NewLOB:34 = 0.0 [2020/02/24-16:14:28.869] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,866 INFO F thsvd:NewLOB:35 = 8.673617379884035e-19 [2020/02/24-16:14:28.869] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,866 INFO F thsvd:NewLOB:36 = -4.336808689942018e-19 [2020/02/24-16:14:28.869] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,866 INFO F thsvd:NewLOB:37 = 0.0 [2020/02/24-16:14:28.869] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,866 INFO F thsvd:NewLOB:38 = 4.336808689942018e-19 [2020/02/24-16:14:28.870] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,866 INFO F thsvd:NewLOB:39 = -8.673617379884035e-19 [2020/02/24-16:14:28.870] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,866 INFO F thsvd:NewLOB:40 = 4.336808689942018e-19 [2020/02/24-16:14:28.870] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,867 INFO F thsvd:NewLOB:41 = 0.0 [2020/02/24-16:14:28.870] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,867 INFO F thsvd:NewLOB:42 = 0.0 [2020/02/24-16:14:28.870] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,867 INFO F thsvd:NewLOB:43 = 0.0 [2020/02/24-16:14:28.871] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,867 INFO F thsvd:NewLOB:44 = 4.336808689942018e-19 [2020/02/24-16:14:28.871] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,867 INFO F thsvd:NewLOB:45 = 4.336808689942018e-19 [2020/02/24-16:14:28.871] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,867 INFO F thsvd:NewLOB:46 = -1.0842021724855044e-19 [2020/02/24-16:14:28.871] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,868 INFO F thsvd:NewLOB:47 = 0.0 [2020/02/24-16:14:28.871] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,868 INFO F thsvd:NewLOB:48 = 0.0 [2020/02/24-16:14:28.871] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,868 INFO F thsvd:NewLOB:49 = 0.0 [2020/02/24-16:14:28.872] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,868 INFO F thsvd:NewLOB:50 = 0.0 [2020/02/24-16:14:28.872] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,868 INFO F thsvd:NewLOB:51 = -2.0679515313825692e-25 [2020/02/24-16:14:28.872] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,868 INFO F thsvd:NewLOB:52 = 5.048709793414476e-29 [2020/02/24-16:14:28.872] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,868 INFO F thsvd:NewLOB:53 = 0.0 [2020/02/24-16:14:28.872] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,869 INFO F thsvd:NewLOB:54 = 0.0 [2020/02/24-16:14:28.873] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,869 INFO F thsvd:NewLOB:55 = -3.1554436208840472e-30 [2020/02/24-16:14:28.873] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,869 INFO F thsvd:NewLOB:56 = 1.5777218104420236e-30 [2020/02/24-16:14:28.873] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,869 INFO F thsvd:NewLOB:57 = 1.5777218104420236e-30 [2020/02/24-16:14:28.873] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,869 INFO F thsvd:NewLOB:58 = -6.310887241768095e-30 [2020/02/24-16:14:28.873] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,869 INFO F thsvd:NewLOB:59 = 0.0 [2020/02/24-16:14:28.873] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,870 INFO F thsvd:NewLOB:60 = 0.0 [2020/02/24-16:14:28.874] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,870 INFO F thsvd:NewLOB:61 = 1.5777218104420236e-30 [2020/02/24-16:14:28.874] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,870 INFO F thsvd:NewLOB:62 = 0.0 [2020/02/24-16:14:28.874] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,870 INFO F thsvd:NewLOB:63 = 0.0 [2020/02/24-16:14:28.874] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,870 INFO F thsvd:NewLOB:64 = -7.888609052210118e-31 [2020/02/24-16:14:28.874] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,870 INFO F thsvd:NewLOB:65 = 0.0 [2020/02/24-16:14:28.875] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,871 INFO F thsvd:NewLOB:66 = 0.0 [2020/02/24-16:14:28.875] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,871 INFO F thsvd:NewLOB:67 = 3.944304526105059e-31 [2020/02/24-16:14:28.875] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,871 INFO F thsvd:NewLOB:68 = -1.9721522630525295e-31 [2020/02/24-16:14:28.875] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,871 INFO F thsvd:NewLOB:69 = 1.9721522630525295e-31 [2020/02/24-16:14:28.875] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,871 INFO F thsvd:NewLOB:70 = 0.0 [2020/02/24-16:14:28.875] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,871 INFO F thsvd:NewLOB:71 = 0.0 [2020/02/24-16:14:28.876] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,871 INFO F thsvd:NewLOB:72 = 2.619264724366641e-32 [2020/02/24-16:14:28.876] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,872 INFO F thsvd:NewLOB:73 = 0.0 [2020/02/24-16:14:28.876] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,872 INFO F thsvd:NewLOB:74 = 0.0 [2020/02/24-16:14:28.876] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,872 INFO F thsvd:NewLOB:75 = 0.0 [2020/02/24-16:14:28.876] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,872 INFO F thsvd:NewLOB:76 = 0.0 [2020/02/24-16:14:28.876] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,872 INFO F thsvd:NewLOB:77 = 0.0 [2020/02/24-16:14:28.877] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,873 INFO F thsvd:NewLOB:78 = 0.0 [2020/02/24-16:14:28.877] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,873 INFO F thsvd:NewLOB:79 = -6.162975822039155e-33 [2020/02/24-16:14:28.877] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,873 INFO F thsvd:NewLOB:80 = -4.930380657631324e-32 [2020/02/24-16:14:28.877] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,873 INFO F thsvd:NewLOB:81 = 0.0 [2020/02/24-16:14:28.877] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,873 INFO F thsvd:NewLOB:82 = 2.465190328815662e-32 [2020/02/24-16:14:28.878] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,874 INFO F thsvd:NewLOB:83 = 0.0 [2020/02/24-16:14:28.878] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,874 INFO F thsvd:NewLOB:84 = 0.0 [2020/02/24-16:14:28.878] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,874 INFO F thsvd:NewLOB:85 = 0.0 [2020/02/24-16:14:28.878] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,874 INFO F thsvd:NewLOB:86 = -3.0814879110195774e-33 [2020/02/24-16:14:28.878] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,874 INFO F thsvd:NewLOB:87 = -3.851859888774472e-34 [2020/02/24-16:14:28.878] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,875 INFO F thsvd:NewLOB:88 = -1.925929944387236e-34 [2020/02/24-16:14:28.879] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,875 INFO F thsvd:NewLOB:89 = 0.0 [2020/02/24-16:14:28.879] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,875 INFO F thsvd:NewLOB:90 = 9.62964972193618e-35 [2020/02/24-16:14:28.879] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,875 INFO F thsvd:NewLOB:91 = 0.0 [2020/02/24-16:14:28.879] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,875 INFO F thsvd:NewLOB:92 = 0.0 [2020/02/24-16:14:28.879] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,875 INFO F thsvd:NewLOB:93 = -9.183549615799121e-41 [2020/02/24-16:14:28.879] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,876 INFO F thsvd:NewLOB:94 = 7.174648137343064e-43 [2020/02/24-16:14:28.880] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,876 INFO F thsvd:NewLOB:95 = 0.0 [2020/02/24-16:14:28.880] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,876 INFO F thsvd:NewLOB:96 = -1.1210387714598537e-44 [2020/02/24-16:14:28.880] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,876 INFO F thsvd:NewLOB:97 = 1.1210387714598537e-44 [2020/02/24-16:14:28.880] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,876 INFO F thsvd:NewLOB:98 = 0.0 [2020/02/24-16:14:28.880] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,876 INFO F thsvd:NewLOB:99 = 0.0 [2020/02/24-16:14:28.995] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,995 INFO Done actual predict [2020/02/24-16:14:28.995] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,995 INFO Probas raw shape 2618/2 target_map=2 [2020/02/24-16:14:28.997] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,997 INFO pred df debug : [2020/02/24-16:14:29.016] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:28,997 INFO prediction proba_0 proba_1 [2020/02/24-16:14:29.017] [null-err-46] [INFO] [dku.utils] - 0 1 0.325943 0.674057 [2020/02/24-16:14:29.017] [null-err-46] [INFO] [dku.utils] - 1 0 0.666141 0.333859 [2020/02/24-16:14:29.017] [null-err-46] [INFO] [dku.utils] - 2 0 0.930486 0.069514 [2020/02/24-16:14:29.017] [null-err-46] [INFO] [dku.utils] - 3 0 0.982296 0.017704 [2020/02/24-16:14:29.018] [null-err-46] [INFO] [dku.utils] - 4 0 0.952043 0.047957 [2020/02/24-16:14:29.018] [null-err-46] [INFO] [dku.utils] - 5 0 0.737420 0.262580 [2020/02/24-16:14:29.018] [null-err-46] [INFO] [dku.utils] - 6 0 0.434107 0.565893 [2020/02/24-16:14:29.018] [null-err-46] [INFO] [dku.utils] - 7 0 0.518870 0.481130 [2020/02/24-16:14:29.018] [null-err-46] [INFO] [dku.utils] - 8 0 0.908514 0.091486 [2020/02/24-16:14:29.018] [null-err-46] [INFO] [dku.utils] - 9 0 0.752077 0.247923 [2020/02/24-16:14:29.019] [null-err-46] [INFO] [dku.utils] - 10 0 0.963701 0.036299 [2020/02/24-16:14:29.019] [null-err-46] [INFO] [dku.utils] - 11 0 0.958466 0.041534 [2020/02/24-16:14:29.019] [null-err-46] [INFO] [dku.utils] - 12 0 0.958252 0.041748 [2020/02/24-16:14:29.019] [null-err-46] [INFO] [dku.utils] - 13 0 0.970054 0.029946 [2020/02/24-16:14:29.019] [null-err-46] [INFO] [dku.utils] - 14 0 0.788481 0.211519 [2020/02/24-16:14:29.020] [null-err-46] [INFO] [dku.utils] - 15 0 0.742506 0.257494 [2020/02/24-16:14:29.020] [null-err-46] [INFO] [dku.utils] - 16 0 0.960581 0.039419 [2020/02/24-16:14:29.020] [null-err-46] [INFO] [dku.utils] - 17 0 0.901237 0.098763 [2020/02/24-16:14:29.020] [null-err-46] [INFO] [dku.utils] - 18 0 0.770763 0.229237 [2020/02/24-16:14:29.020] [null-err-46] [INFO] [dku.utils] - 19 0 0.858394 0.141606 [2020/02/24-16:14:29.020] [null-err-46] [INFO] [dku.utils] - 20 0 0.923098 0.076902 [2020/02/24-16:14:29.021] [null-err-46] [INFO] [dku.utils] - 21 0 0.982296 0.017704 [2020/02/24-16:14:29.021] [null-err-46] [INFO] [dku.utils] - 22 0 0.972205 0.027795 [2020/02/24-16:14:29.021] [null-err-46] [INFO] [dku.utils] - 23 0 0.955332 0.044668 [2020/02/24-16:14:29.021] [null-err-46] [INFO] [dku.utils] - 24 0 0.990380 0.009620 [2020/02/24-16:14:29.021] [null-err-46] [INFO] [dku.utils] - 25 0 0.970045 0.029955 [2020/02/24-16:14:29.021] [null-err-46] [INFO] [dku.utils] - 26 0 0.985919 0.014081 [2020/02/24-16:14:29.022] [null-err-46] [INFO] [dku.utils] - 27 1 0.318232 0.681768 [2020/02/24-16:14:29.022] [null-err-46] [INFO] [dku.utils] - 28 0 0.646955 0.353045 [2020/02/24-16:14:29.022] [null-err-46] [INFO] [dku.utils] - 29 0 0.915053 0.084947 [2020/02/24-16:14:29.022] [null-err-46] [INFO] [dku.utils] - ... ... ... ... [2020/02/24-16:14:29.022] [null-err-46] [INFO] [dku.utils] - 2588 0 0.970054 0.029946 [2020/02/24-16:14:29.022] [null-err-46] [INFO] [dku.utils] - 2589 0 0.949370 0.050630 [2020/02/24-16:14:29.023] [null-err-46] [INFO] [dku.utils] - 2590 0 0.939357 0.060643 [2020/02/24-16:14:29.023] [null-err-46] [INFO] [dku.utils] - 2591 0 0.970054 0.029946 [2020/02/24-16:14:29.023] [null-err-46] [INFO] [dku.utils] - 2592 0 0.892547 0.107453 [2020/02/24-16:14:29.024] [null-err-46] [INFO] [dku.utils] - 2593 0 0.894462 0.105538 [2020/02/24-16:14:29.024] [qtp128028301-30] [INFO] [dku.jobs] - Connects using API ticket [2020/02/24-16:14:29.024] [null-err-46] [INFO] [dku.utils] - 2594 0 0.951693 0.048307 [2020/02/24-16:14:29.024] [qtp128028301-30] [DEBUG] [dku.jobs] - Received command : /tintercom/datasets/init-write-session [2020/02/24-16:14:29.025] [null-err-46] [INFO] [dku.utils] - 2595 0 0.927761 0.072239 [2020/02/24-16:14:29.025] [null-err-46] [INFO] [dku.utils] - 2596 0 0.985919 0.014081 [2020/02/24-16:14:29.025] [null-err-46] [INFO] [dku.utils] - 2597 0 0.742574 0.257426 [2020/02/24-16:14:29.025] [null-err-46] [INFO] [dku.utils] - 2598 0 0.904711 0.095289 [2020/02/24-16:14:29.026] [null-err-46] [INFO] [dku.utils] - 2599 0 0.963878 0.036122 [2020/02/24-16:14:29.026] [null-err-46] [INFO] [dku.utils] - 2600 0 0.892547 0.107453 [2020/02/24-16:14:29.026] [null-err-46] [INFO] [dku.utils] - 2601 0 0.896390 0.103610 [2020/02/24-16:14:29.026] [null-err-46] [INFO] [dku.utils] - 2602 1 0.320444 0.679556 [2020/02/24-16:14:29.026] [null-err-46] [INFO] [dku.utils] - 2603 0 0.963701 0.036299 [2020/02/24-16:14:29.027] [null-err-46] [INFO] [dku.utils] - 2604 0 0.419172 0.580828 [2020/02/24-16:14:29.027] [null-err-46] [INFO] [dku.utils] - 2605 0 0.627646 0.372354 [2020/02/24-16:14:29.027] [null-err-46] [INFO] [dku.utils] - 2606 0 0.984257 0.015743 [2020/02/24-16:14:29.027] [null-err-46] [INFO] [dku.utils] - 2607 0 0.915053 0.084947 [2020/02/24-16:14:29.027] [null-err-46] [INFO] [dku.utils] - 2608 0 0.759415 0.240585 [2020/02/24-16:14:29.028] [null-err-46] [INFO] [dku.utils] - 2609 1 0.346398 0.653602 [2020/02/24-16:14:29.028] [null-err-46] [INFO] [dku.utils] - 2610 0 0.865262 0.134738 [2020/02/24-16:14:29.028] [null-err-46] [INFO] [dku.utils] - 2611 0 0.984398 0.015602 [2020/02/24-16:14:29.028] [null-err-46] [INFO] [dku.utils] - 2612 0 0.434459 0.565541 [2020/02/24-16:14:29.028] [null-err-46] [INFO] [dku.utils] - 2613 0 0.584744 0.415256 [2020/02/24-16:14:29.028] [null-err-46] [INFO] [dku.utils] - 2614 0 0.910249 0.089751 [2020/02/24-16:14:29.029] [null-err-46] [INFO] [dku.utils] - 2615 0 0.959958 0.040042 [2020/02/24-16:14:29.029] [null-err-46] [INFO] [dku.utils] - 2616 0 0.979336 0.020664 [2020/02/24-16:14:29.029] [null-err-46] [INFO] [dku.utils] - 2617 0 0.832870 0.167130 [2020/02/24-16:14:29.029] [null-err-46] [INFO] [dku.utils] - [2618 rows x 3 columns] [2020/02/24-16:14:29.030] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,016 INFO Done predicting it [2020/02/24-16:14:29.030] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,020 INFO Generator generated a df (2618, 13) [2020/02/24-16:14:29.030] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,021 INFO Start_once writer [2020/02/24-16:14:29.030] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,021 INFO Initializing write session [2020/02/24-16:14:29.041] [qtp128028301-30] [INFO] [com.dataiku.dip.dataflow.streaming.DatasetWritingService] - Init write session: gX8SBcaGck [2020/02/24-16:14:29.042] [qtp128028301-30] [DEBUG] [dku.jobs] - Command /tintercom/datasets/init-write-session processed in 18ms [2020/02/24-16:14:29.046] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,045 INFO Starting RemoteStreamWriter [2020/02/24-16:14:29.047] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,047 INFO Initializing write data stream (gX8SBcaGck) [2020/02/24-16:14:29.068] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,068 INFO Remote Stream Writer: start generate [2020/02/24-16:14:29.068] [qtp128028301-30] [INFO] [dku.jobs] - Connects using API ticket [2020/02/24-16:14:29.069] [qtp128028301-48] [INFO] [dku.jobs] - Connects using API ticket [2020/02/24-16:14:29.069] [qtp128028301-48] [DEBUG] [dku.jobs] - Received command : /tintercom/datasets/push-data [2020/02/24-16:14:29.068] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,068 INFO Waiting for data to send ... [2020/02/24-16:14:29.069] [qtp128028301-30] [DEBUG] [dku.jobs] - Received command : /tintercom/datasets/wait-write-session [2020/02/24-16:14:29.076] [qtp128028301-48] [INFO] [com.dataiku.dip.dataflow.streaming.DatasetWriter] - Creating output writer [2020/02/24-16:14:29.079] [qtp128028301-48] [INFO] [com.dataiku.dip.dataflow.streaming.DatasetWriter] - Initializing output writer [2020/02/24-16:14:29.081] [qtp128028301-48] [INFO] [dku.connections.sql.provider] - Connecting to jdbc:sqlserver://tnwp09931.logon.ds.ge.com with props: {"user":"Public123","password":"***","portNumber":"1433","databaseName":"DW_ODS"} conn=DW_ODS_Read_Only-YK83Vui [2020/02/24-16:14:29.082] [qtp128028301-48] [DEBUG] [dku.connections.sql.provider] - Driver version 7.2 [2020/02/24-16:14:29.101] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,101 INFO Output df written [2020/02/24-16:14:29.101] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,101 INFO Start_once writer [2020/02/24-16:14:29.103] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,103 INFO Sending data (369306) [2020/02/24-16:14:29.104] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,104 INFO Waiting for data to send ... [2020/02/24-16:14:29.105] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,105 INFO Remote Stream Writer closed [2020/02/24-16:14:29.107] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,107 INFO Got end mark, ending send [2020/02/24-16:14:29.297] [qtp128028301-48] [INFO] [dku.connections.sql.provider] - Driver: Microsoft JDBC Driver 7.2 for SQL Server (JDBC 4.2) 7.2.1.0 (7.2) [2020/02/24-16:14:29.297] [qtp128028301-48] [INFO] [dku.connections.sql.provider] - Database: Microsoft SQL Server 10.50.6560 (10.50) rowSize=8060 stmts=0 [2020/02/24-16:14:29.338] [qtp128028301-48] [INFO] [dku.sql.generic] - Dropping table [2020/02/24-16:14:29.338] [qtp128028301-48] [INFO] [dku.dataset.sql] - Executing statement: [2020/02/24-16:14:29.338] [qtp128028301-48] [INFO] [dku.dataset.sql] - DROP TABLE "SERVICEMANAGERWORKLOAD_servicemanager_scored" [2020/02/24-16:14:29.381] [qtp128028301-48] [INFO] [dku.sql.generic] - Drop table failed, table probably did not exist: Cannot drop the table 'SERVICEMANAGERWORKLOAD_servicemanager_scored', because it does not exist or you do not have permission. [2020/02/24-16:14:29.381] [qtp128028301-48] [DEBUG] [dku.connections.sql.provider] - Rollback conn=DW_ODS_Read_Only-YK83Vui [2020/02/24-16:14:29.424] [qtp128028301-48] [INFO] [dku.sql.generic] - Creating table [2020/02/24-16:14:29.428] [qtp128028301-48] [INFO] [dku.dataset.sql] - Executing statements: [2020/02/24-16:14:29.428] [qtp128028301-48] [INFO] [dku.dataset.sql] - CREATE TABLE "SERVICEMANAGERWORKLOAD_servicemanager_scored" ( "Name" nvarchar(max), "Year" bigint, "FiscalWeek" bigint, "Date" nvarchar(max), "Hours" float, "Jobs" bigint, "Employees" bigint, "NewLOB" nvarchar(max), "Workload" float, "AccidentOccured" bigint, "proba_0" float, "proba_1" float, "prediction" nvarchar(max) ) [2020/02/24-16:14:29.428] [qtp128028301-48] [INFO] [dku.dataset.sql] - Executing statement: CREATE TABLE "SERVICEMANAGERWORKLOAD_servicemanager_scored" ( "Name" nvarchar(max), "Year" bigint, "FiscalWeek" bigint, "Date" nvarchar(max), "Hours" float, "Jobs" bigint, "Employees" bigint, "NewLOB" nvarchar(max), "Workload" float, "AccidentOccured" bigint, "proba_0" float, "proba_1" float, "prediction" nvarchar(max) ) [2020/02/24-16:14:29.470] [qtp128028301-30] [ERROR] [com.dataiku.dip.dataflow.streaming.DatasetWritingService] - Wait session error: CREATE TABLE permission denied in database 'DW_ODS'. com.microsoft.sqlserver.jdbc.SQLServerException: CREATE TABLE permission denied in database 'DW_ODS'. at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:256) at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1621) at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:868) at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:768) at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2930) at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:248) at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:223) at com.microsoft.sqlserver.jdbc.SQLServerStatement.execute(SQLServerStatement.java:744) at com.dataiku.dip.sql.SQLUtils.safeSplitAndExec(SQLUtils.java:239) at com.dataiku.dip.sql.GenericSQLDialect.dropAndRecreateTableOrPartition(GenericSQLDialect.java:531) at com.dataiku.dip.datasets.sql.SQLTableOutput$SQLTableOutputWriter.init(SQLTableOutput.java:115) at com.dataiku.dip.dataflow.streaming.DatasetWriter.(DatasetWriter.java:77) at com.dataiku.dip.dataflow.streaming.DatasetWriter.build(DatasetWriter.java:114) at com.dataiku.dip.dataflow.streaming.DatasetWriter.build(DatasetWriter.java:106) at com.dataiku.dip.dataflow.streaming.DatasetWritingService.pushData(DatasetWritingService.java:251) at com.dataiku.dip.dataflow.kernel.slave.KernelSession.pushData(KernelSession.java:235) at com.dataiku.dip.dataflow.kernel.slave.KernelServlet.service(KernelServlet.java:228) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:462) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536) at java.lang.Thread.run(Thread.java:748) [2020/02/24-16:14:29.470] [qtp128028301-48] [ERROR] [com.dataiku.dip.dataflow.streaming.DatasetWritingService] - Push data error during streaming:CREATE TABLE permission denied in database 'DW_ODS'. com.microsoft.sqlserver.jdbc.SQLServerException: CREATE TABLE permission denied in database 'DW_ODS'. at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:256) at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1621) at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:868) at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:768) at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7194) at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2930) at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:248) at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:223) at com.microsoft.sqlserver.jdbc.SQLServerStatement.execute(SQLServerStatement.java:744) at com.dataiku.dip.sql.SQLUtils.safeSplitAndExec(SQLUtils.java:239) at com.dataiku.dip.sql.GenericSQLDialect.dropAndRecreateTableOrPartition(GenericSQLDialect.java:531) at com.dataiku.dip.datasets.sql.SQLTableOutput$SQLTableOutputWriter.init(SQLTableOutput.java:115) at com.dataiku.dip.dataflow.streaming.DatasetWriter.(DatasetWriter.java:77) at com.dataiku.dip.dataflow.streaming.DatasetWriter.build(DatasetWriter.java:114) at com.dataiku.dip.dataflow.streaming.DatasetWriter.build(DatasetWriter.java:106) at com.dataiku.dip.dataflow.streaming.DatasetWritingService.pushData(DatasetWritingService.java:251) at com.dataiku.dip.dataflow.kernel.slave.KernelSession.pushData(KernelSession.java:235) at com.dataiku.dip.dataflow.kernel.slave.KernelServlet.service(KernelServlet.java:228) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:551) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1111) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:462) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536) at java.lang.Thread.run(Thread.java:748) [2020/02/24-16:14:29.474] [qtp128028301-48] [DEBUG] [dku.jobs] - Command /tintercom/datasets/push-data processed in 405ms [2020/02/24-16:14:29.474] [qtp128028301-30] [DEBUG] [dku.jobs] - Command /tintercom/datasets/wait-write-session processed in 406ms [2020/02/24-16:14:29.477] [null-err-46] [INFO] [dku.utils] - 2020-02-24 16:14:29,476 ERROR Exception caught while writing [2020/02/24-16:14:29.478] [null-err-46] [INFO] [dku.utils] - Traceback (most recent call last): [2020/02/24-16:14:29.478] [null-err-46] [INFO] [dku.utils] - File "/opt/dataiku/install_dir/dataiku-dss-5.1.2/python/dataiku/core/dataset_write.py", line 217, in run [2020/02/24-16:14:29.478] [null-err-46] [INFO] [dku.utils] - self.streaming_api.wait_write_session(self.session_id) [2020/02/24-16:14:29.478] [null-err-46] [INFO] [dku.utils] - File "/opt/dataiku/install_dir/dataiku-dss-5.1.2/python/dataiku/core/dataset_write.py", line 180, in wait_write_session [2020/02/24-16:14:29.479] [null-err-46] [INFO] [dku.utils] - raise Exception(u'An error occurred during dataset write (%s): %s' % (id, decoded_resp["message"])) [2020/02/24-16:14:29.479] [null-err-46] [INFO] [dku.utils] - Exception: An error occurred during dataset write (gX8SBcaGck): SQLServerException: CREATE TABLE permission denied in database 'DW_ODS'. [2020/02/24-16:14:29.520] [null-err-46] [INFO] [dku.utils] - Traceback (most recent call last): [2020/02/24-16:14:29.521] [null-err-46] [INFO] [dku.utils] - File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main [2020/02/24-16:14:29.521] [null-err-46] [INFO] [dku.utils] - "__main__", fname, loader, pkg_name) [2020/02/24-16:14:29.521] [null-err-46] [INFO] [dku.utils] - File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code [2020/02/24-16:14:29.522] [null-err-46] [INFO] [dku.utils] - exec code in run_globals [2020/02/24-16:14:29.522] [null-err-46] [INFO] [dku.utils] - File "/opt/dataiku/install_dir/dataiku-dss-5.1.2/python/dataiku/doctor/prediction/reg_scoring_recipe.py", line 138, in [2020/02/24-16:14:29.522] [null-err-46] [INFO] [dku.utils] - dkujson.load_from_filepath(sys.argv[7])) [2020/02/24-16:14:29.523] [null-err-46] [INFO] [dku.utils] - File "/opt/dataiku/install_dir/dataiku-dss-5.1.2/python/dataiku/doctor/prediction/reg_scoring_recipe.py", line 126, in main [2020/02/24-16:14:29.523] [null-err-46] [INFO] [dku.utils] - logging.info("Output df written") [2020/02/24-16:14:29.523] [null-err-46] [INFO] [dku.utils] - File "/opt/dataiku/install_dir/dataiku-dss-5.1.2/python/dataiku/core/dataset_write.py", line 413, in __exit__ [2020/02/24-16:14:29.523] [null-err-46] [INFO] [dku.utils] - self.close() [2020/02/24-16:14:29.524] [null-err-46] [INFO] [dku.utils] - File "/opt/dataiku/install_dir/dataiku-dss-5.1.2/python/dataiku/core/dataset_write.py", line 407, in close [2020/02/24-16:14:29.524] [null-err-46] [INFO] [dku.utils] - self.waiter.wait_end() [2020/02/24-16:14:29.524] [null-err-46] [INFO] [dku.utils] - File "/opt/dataiku/install_dir/dataiku-dss-5.1.2/python/dataiku/core/dataset_write.py", line 211, in wait_end [2020/02/24-16:14:29.525] [null-err-46] [INFO] [dku.utils] - self.raise_on_failure() [2020/02/24-16:14:29.525] [null-err-46] [INFO] [dku.utils] - File "/opt/dataiku/install_dir/dataiku-dss-5.1.2/python/dataiku/core/dataset_write.py", line 204, in raise_on_failure [2020/02/24-16:14:29.525] [null-err-46] [INFO] [dku.utils] - exec("raise self.exception_type, self.exception, self.traceback") [2020/02/24-16:14:29.525] [null-err-46] [INFO] [dku.utils] - File "/opt/dataiku/install_dir/dataiku-dss-5.1.2/python/dataiku/core/dataset_write.py", line 217, in run [2020/02/24-16:14:29.526] [null-err-46] [INFO] [dku.utils] - self.streaming_api.wait_write_session(self.session_id) [2020/02/24-16:14:29.526] [null-err-46] [INFO] [dku.utils] - File "/opt/dataiku/install_dir/dataiku-dss-5.1.2/python/dataiku/core/dataset_write.py", line 180, in wait_write_session [2020/02/24-16:14:29.526] [null-err-46] [INFO] [dku.utils] - raise Exception(u'An error occurred during dataset write (%s): %s' % (id, decoded_resp["message"])) [2020/02/24-16:14:29.527] [null-err-46] [INFO] [dku.utils] - Exception: An error occurred during dataset write (gX8SBcaGck): SQLServerException: CREATE TABLE permission denied in database 'DW_ODS'. [2020/02/24-16:14:29.675] [FRT-39-FlowRunnable] [INFO] [dku.flow.activity] act.score_ServiceManager_NP - Run thread failed for activity score_ServiceManager_NP com.dataiku.dip.exceptions.ProcessDiedException: The Python process failed (exit code: 1). More info might be available in the logs. at com.dataiku.dip.dataflow.exec.AbstractCodeBasedActivityRunner.throwSubprocessError(AbstractCodeBasedActivityRunner.java:195) at com.dataiku.dip.dataflow.exec.AbstractCodeBasedActivityRunner.handleExecutionResult(AbstractCodeBasedActivityRunner.java:168) at com.dataiku.dip.dataflow.exec.AbstractCodeBasedActivityRunner.execute(AbstractCodeBasedActivityRunner.java:102) at com.dataiku.dip.dataflow.exec.AbstractPythonRecipeRunner.executeModule(AbstractPythonRecipeRunner.java:64) at com.dataiku.dip.analysis.ml.prediction.flow.PredictionScoringRecipeRunner$3.run(PredictionScoringRecipeRunner.java:515) at com.dataiku.dip.analysis.ml.prediction.flow.PredictionScoringRecipeRunner.runPython(PredictionScoringRecipeRunner.java:551) at com.dataiku.dip.analysis.ml.prediction.flow.PredictionScoringRecipeRunner.runWithOriginalEngine(PredictionScoringRecipeRunner.java:308) at com.dataiku.dip.analysis.ml.prediction.flow.PredictionScoringRecipeRunner.run(PredictionScoringRecipeRunner.java:221) at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:357) [2020/02/24-16:14:29.723] [ActivityExecutor-34] [INFO] [dku.flow.activity] running score_ServiceManager_NP - activity is finished [2020/02/24-16:14:29.724] [ActivityExecutor-34] [ERROR] [dku.flow.activity] running score_ServiceManager_NP - Activity failed com.dataiku.dip.exceptions.ProcessDiedException: The Python process failed (exit code: 1). More info might be available in the logs. at com.dataiku.dip.dataflow.exec.AbstractCodeBasedActivityRunner.throwSubprocessError(AbstractCodeBasedActivityRunner.java:195) at com.dataiku.dip.dataflow.exec.AbstractCodeBasedActivityRunner.handleExecutionResult(AbstractCodeBasedActivityRunner.java:168) at com.dataiku.dip.dataflow.exec.AbstractCodeBasedActivityRunner.execute(AbstractCodeBasedActivityRunner.java:102) at com.dataiku.dip.dataflow.exec.AbstractPythonRecipeRunner.executeModule(AbstractPythonRecipeRunner.java:64) at com.dataiku.dip.analysis.ml.prediction.flow.PredictionScoringRecipeRunner$3.run(PredictionScoringRecipeRunner.java:515) at com.dataiku.dip.analysis.ml.prediction.flow.PredictionScoringRecipeRunner.runPython(PredictionScoringRecipeRunner.java:551) at com.dataiku.dip.analysis.ml.prediction.flow.PredictionScoringRecipeRunner.runWithOriginalEngine(PredictionScoringRecipeRunner.java:308) at com.dataiku.dip.analysis.ml.prediction.flow.PredictionScoringRecipeRunner.run(PredictionScoringRecipeRunner.java:221) at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:357) [2020/02/24-16:14:29.725] [ActivityExecutor-34] [INFO] [dku.flow.activity] running score_ServiceManager_NP - Executing default post-activity lifecycle hook [2020/02/24-16:14:29.726] [ActivityExecutor-34] [INFO] [dku.flow.activity] running score_ServiceManager_NP - Removing samples for SERVICEMANAGERWORKLOAD.ServiceManager_scored [2020/02/24-16:14:29.727] [ActivityExecutor-34] [INFO] [dku.flow.activity] running score_ServiceManager_NP - Done post-activity tasks [2020/02/24-16:14:29.860] [ActivityExecutor-38] [DEBUG] [dku.flow.jobrunner] - No activity is running, and none is available --> End of ActivityExecutorThread [2020/02/24-16:14:29.861] [ActivityExecutor-36] [DEBUG] [dku.flow.jobrunner] - No activity is running, and none is available --> End of ActivityExecutorThread [2020/02/24-16:14:29.863] [ActivityExecutor-37] [DEBUG] [dku.flow.jobrunner] - No activity is running, and none is available --> End of ActivityExecutorThread [2020/02/24-16:14:29.866] [ActivityExecutor-35] [DEBUG] [dku.flow.jobrunner] - No activity is running, and none is available --> End of ActivityExecutorThread [2020/02/24-16:14:30.029] [ActivityExecutor-34] [DEBUG] [dku.flow.jobrunner] - No activity is running, and none is available --> End of ActivityExecutorThread [2020/02/24-16:14:30.029] [qtp128028301-29] [INFO] [dku.flow.jobrunner] - All activity executor threads joined, end of job [2020/02/24-16:14:30.029] [qtp128028301-29] [INFO] [dku.flow.jobrunner] - JOB IS COMPLETE [2020/02/24-16:14:30.032] [qtp128028301-29] [DEBUG] [dku.jobs] - Command /pintercom/run_fully processed in 37245ms [2020/02/24-16:14:30.037] [qtp128028301-30] [INFO] [dku.jobs] - Connects using Shared secret [2020/02/24-16:14:30.037] [qtp128028301-30] [DEBUG] [dku.jobs] - Received command : /pintercom/stop_session [2020/02/24-16:14:30.039] [qtp128028301-30] [DEBUG] [dku.jobs] - Command /pintercom/stop_session processed in 2ms