My output datasets are empty after adding condition in existing sample/filter recipe from flow.
My output datasets are empty after adding condition of year 2017 for departures dataset in existing sample/filter recipe for "Hands On : Airport Traffic " in Visual Recipes 101 course.
After adding condition in existing filter recipe from flow, when I right click to "Build Flow outputs reachable from here", Job does execute but subsequent datasets are empty. I am not able to figure out what went wrong and what can I do?
In logs, below section was highlighted in red:
[07:40:33] [WARN] [dku.fsproviders.http] - HEAD method not working, trying GET com.dataiku.dip.datasets.fs.HTTPSourceException: in act.download_to_departures_NP: HTTP 400: Bad Request at com.dataiku.dip.datasets.fs.HTTPFSProvider.execute(HTTPFSProvider.java:333) at com.dataiku.dip.datasets.fs.HTTPFSProvider.getResponseForHeadersWithHEADorGET(HTTPFSProvider.java:228) at com.dataiku.dip.datasets.fs.HTTPFSProvider.enumerateRecursive(HTTPFSProvider.java:247) at com.dataiku.dip.datasets.fs.HTTPFSProvider.enumerateRecursive(HTTPFSProvider.java:51) at com.dataiku.dip.server.recipes.KernelsDownloadRecipeService.enumerateSource(KernelsDownloadRecipeService.java:205) at com.dataiku.dip.server.recipes.KernelsDownloadRecipeService.copyFromSourceToFolder(KernelsDownloadRecipeService.java:419) at com.dataiku.dip.server.recipes.KernelsDownloadRecipeService.copyFromSourcesToFolder(KernelsDownloadRecipeService.java:321) at com.dataiku.dip.recipes.download.DownloadRecipeRunner.run(DownloadRecipeRunner.java:75) at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:374)
I am attaching the complete log file. Please assist.
Thank You
Deepak
Best Answer
-
Sean Dataiker, Alpha Tester, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer Posts: 168 Dataiker
Hi @deepakdhiman
, when you return to the Filter recipe, 2017 should already be present. It's 2016 that you are adding-- is that correct?Is it possible you forgot to change the condition to keep rows that satisfy all conditions to rows that satisfy at least one of the following conditions? This would explain why the following datasets are "empty" (if by empty you mean that they have 0 rows) since no rows can satisfy both conditions at the same time.
Answers
-
Hi Sean
Thank you for replying to help me out. I went back to the filter recipe and checked that "all the following conditions" was selected in "Keep only rows that satisfy" field.
I changed it to "atleast one of the following condition", run and rebuild it. It worked! It was such a silly mistake.
Thank you so much
-
Sean Dataiker, Alpha Tester, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer Posts: 168 Dataiker
Great! No problem at all.