[2021/06/04-07:40:33.174] [ActivityExecutor-38] [INFO] [dku] running download_to_departures_NP - ---------------------------------------- [2021/06/04-07:40:33.174] [ActivityExecutor-38] [INFO] [dku] running download_to_departures_NP - DSS startup: jek version:8.0.2 [2021/06/04-07:40:33.174] [ActivityExecutor-38] [INFO] [dku] running download_to_departures_NP - DSS home: /opt/dataiku [2021/06/04-07:40:33.174] [ActivityExecutor-38] [INFO] [dku] running download_to_departures_NP - OS: Linux 4.15.0-1111-azure amd64 - Java: Private Build 1.8.0_282 [2021/06/04-07:40:33.173] [ActivityExecutor-38] [INFO] [dku.flow.jobrunner] running download_to_departures_NP - Allocated a slot for this activity! [2021/06/04-07:40:33.174] [ActivityExecutor-38] [INFO] [dku.flow.jobrunner] running download_to_departures_NP - Run activity [2021/06/04-07:40:33.190] [ActivityExecutor-38] [INFO] [dku.flow.activity] running download_to_departures_NP - Executing default pre-activity lifecycle hook [2021/06/04-07:40:33.196] [ActivityExecutor-38] [INFO] [dku.managedfolders.handler] running download_to_departures_NP - Create provider for INTERNATIONALFLIGHTREPORTING.9Rxas5GZ with path INTERNATIONALFLIGHTREPORTING/9Rxas5GZ [2021/06/04-07:40:33.197] [ActivityExecutor-38] [INFO] [dku.managedfolders.handler] running download_to_departures_NP - Ensured folder for root of managed folder [2021/06/04-07:40:33.198] [ActivityExecutor-38] [INFO] [dku.flow.activity] running download_to_departures_NP - Checking if sources are ready [2021/06/04-07:40:33.199] [ActivityExecutor-38] [DEBUG] [dku.flow.activity] running download_to_departures_NP - Computing hashes to propagate BEFORE activity [2021/06/04-07:40:33.200] [ActivityExecutor-38] [DEBUG] [dku.flow.activity] running download_to_departures_NP - Recorded 0 hashes before activity run [2021/06/04-07:40:33.201] [ActivityExecutor-38] [DEBUG] [dku.flow.activity] running download_to_departures_NP - Building recipe runner of type [2021/06/04-07:40:33.206] [ActivityExecutor-38] [DEBUG] [dku.flow.activity] running download_to_departures_NP - Recipe runner built, will use 1 thread(s) [2021/06/04-07:40:33.206] [ActivityExecutor-38] [DEBUG] [dku.flow.activity] running download_to_departures_NP - Starting execution thread: com.dataiku.dip.recipes.download.DownloadRecipeRunner@3e973286 [2021/06/04-07:40:33.207] [ActivityExecutor-38] [DEBUG] [dku.flow.activity] running download_to_departures_NP - Execution threads started, waiting for activity end [2021/06/04-07:40:33.209] [FRT-43-FlowRunnable] [INFO] [dku.flow.activity] act.download_to_departures_NP - Run thread for activity download_to_departures_NP starting [2021/06/04-07:40:33.211] [FRT-43-FlowRunnable] [INFO] [dku.managedfolders.handler] act.download_to_departures_NP - Create provider for INTERNATIONALFLIGHTREPORTING.9Rxas5GZ with path INTERNATIONALFLIGHTREPORTING/9Rxas5GZ [2021/06/04-07:40:33.211] [FRT-43-FlowRunnable] [DEBUG] [dku.fs.local] act.download_to_departures_NP - Enumerating local filesystem prefix=/ [2021/06/04-07:40:33.212] [FRT-43-FlowRunnable] [DEBUG] [dku.fs.local] act.download_to_departures_NP - Enumeration done nb_paths=1 size=70247037 [2021/06/04-07:40:33.213] [FRT-43-FlowRunnable] [INFO] [dku.recipes.download.service] act.download_to_departures_NP - Probe ability to set modificationtime on output [2021/06/04-07:40:33.213] [FRT-43-FlowRunnable] [INFO] [dku.recipes.download.service] act.download_to_departures_NP - Handle source on URL [2021/06/04-07:40:33.214] [FRT-43-FlowRunnable] [INFO] [dku.recipes.download.service] act.download_to_departures_NP - Check URL source [2021/06/04-07:40:33.216] [FRT-43-FlowRunnable] [INFO] [dku.recipes.download.service] act.download_to_departures_NP - Enumerate '/rows.csv' : '/rows.csv?accessType=DOWNLOAD' [2021/06/04-07:40:33.216] [FRT-43-FlowRunnable] [INFO] [dku.fsproviders.http] act.download_to_departures_NP - Enumerate HTTP URI: https://data.transportation.gov/api/views/innc-gbgc/rows.csv?accessType=DOWNLOAD [2021/06/04-07:40:33.291] [FRT-43-FlowRunnable] [DEBUG] [dku.fsproviders.http] act.download_to_departures_NP - HTTP response code: 400 [2021/06/04-07:40:33.291] [FRT-43-FlowRunnable] [WARN] [dku.fsproviders.http] act.download_to_departures_NP - HEAD method not working, trying GET com.dataiku.dip.datasets.fs.HTTPSourceException: in act.download_to_departures_NP: HTTP 400: Bad Request at com.dataiku.dip.datasets.fs.HTTPFSProvider.execute(HTTPFSProvider.java:333) at com.dataiku.dip.datasets.fs.HTTPFSProvider.getResponseForHeadersWithHEADorGET(HTTPFSProvider.java:228) at com.dataiku.dip.datasets.fs.HTTPFSProvider.enumerateRecursive(HTTPFSProvider.java:247) at com.dataiku.dip.datasets.fs.HTTPFSProvider.enumerateRecursive(HTTPFSProvider.java:51) at com.dataiku.dip.server.recipes.KernelsDownloadRecipeService.enumerateSource(KernelsDownloadRecipeService.java:205) at com.dataiku.dip.server.recipes.KernelsDownloadRecipeService.copyFromSourceToFolder(KernelsDownloadRecipeService.java:419) at com.dataiku.dip.server.recipes.KernelsDownloadRecipeService.copyFromSourcesToFolder(KernelsDownloadRecipeService.java:321) at com.dataiku.dip.recipes.download.DownloadRecipeRunner.run(DownloadRecipeRunner.java:75) at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:374) [2021/06/04-07:40:33.574] [FRT-43-FlowRunnable] [DEBUG] [dku.fsproviders.http] act.download_to_departures_NP - HTTP response code: 200 [2021/06/04-07:40:33.575] [FRT-43-FlowRunnable] [INFO] [dku.recipes.download.service] act.download_to_departures_NP - Handle '/rows.csv?accessType=DOWNLOAD' ('' -> '/rows.csv'): exists=true changed=true identifying=/rows.csv [2021/06/04-07:40:33.575] [FRT-43-FlowRunnable] [INFO] [dku.recipes.download.service] act.download_to_departures_NP - Copy /rows.csv?accessType=DOWNLOAD to /rows.csv [2021/06/04-07:40:33.897] [FRT-43-FlowRunnable] [DEBUG] [dku.fsproviders.http] act.download_to_departures_NP - HTTP response code: 200 [2021/06/04-07:41:04.486] [FRT-43-FlowRunnable] [INFO] [dku.flow.activity] act.download_to_departures_NP - Run thread done for activity download_to_departures_NP [2021/06/04-07:41:04.629] [ActivityExecutor-38] [INFO] [dku.flow.activity] running download_to_departures_NP - activity is finished [2021/06/04-07:41:04.629] [ActivityExecutor-38] [INFO] [dku.flow.activity] running download_to_departures_NP - Executing default post-activity lifecycle hook [2021/06/04-07:41:04.638] [ActivityExecutor-38] [INFO] [dku.flow.activity] running download_to_departures_NP - Activity successful, computing metrics and running checks [2021/06/04-07:41:04.701] [ActivityExecutor-38] [INFO] [dku.flow.activity] running download_to_departures_NP - Activity checks produced 0 values [2021/06/04-07:41:04.702] [ActivityExecutor-38] [INFO] [dku.flow.activity] running download_to_departures_NP - Activity is successful, computing timestamps to propagate [2021/06/04-07:41:04.711] [ActivityExecutor-38] [INFO] [dku.flow.activity] running download_to_departures_NP - Activity is successful, propagating 3 timestamps [2021/06/04-07:41:04.712] [ActivityExecutor-38] [INFO] [dku.timestamps.sql] running download_to_departures_NP - Starting to propagate 3 hashes [2021/06/04-07:41:04.717] [ActivityExecutor-38] [DEBUG] [dku.connections.sql.provider] running download_to_departures_NP - Commit conn=dssdb-h2-flow_state-9dW5aZC [2021/06/04-07:41:04.718] [ActivityExecutor-38] [INFO] [dku.timestamps.sql] running download_to_departures_NP - Hashes propagated in 6 ms [2021/06/04-07:41:04.719] [ActivityExecutor-38] [INFO] [dku.flow.activity] running download_to_departures_NP - Done post-activity tasks