My output datasets are empty after adding condition in existing sample/filter recipe from flow.

Solved!
deepakdhiman
Level 2
My output datasets are empty after adding condition in existing sample/filter recipe from flow.

My output datasets are empty after adding condition of year 2017 for departures dataset in existing sample/filter recipe for "Hands On : Airport Traffic " in Visual Recipes 101 course. 

After adding condition in existing filter recipe from flow, when I right click to "Build Flow outputs reachable from here", Job does execute but subsequent datasets are empty. I am not able to figure out what went wrong and what can I do?

In logs, below section was highlighted in red:

[07:40:33] [WARN] [dku.fsproviders.http] - HEAD method not working, trying GET
com.dataiku.dip.datasets.fs.HTTPSourceException: in act.download_to_departures_NP: HTTP 400: Bad Request
	at com.dataiku.dip.datasets.fs.HTTPFSProvider.execute(HTTPFSProvider.java:333)
	at com.dataiku.dip.datasets.fs.HTTPFSProvider.getResponseForHeadersWithHEADorGET(HTTPFSProvider.java:228)
	at com.dataiku.dip.datasets.fs.HTTPFSProvider.enumerateRecursive(HTTPFSProvider.java:247)
	at com.dataiku.dip.datasets.fs.HTTPFSProvider.enumerateRecursive(HTTPFSProvider.java:51)
	at com.dataiku.dip.server.recipes.KernelsDownloadRecipeService.enumerateSource(KernelsDownloadRecipeService.java:205)
	at com.dataiku.dip.server.recipes.KernelsDownloadRecipeService.copyFromSourceToFolder(KernelsDownloadRecipeService.java:419)
	at com.dataiku.dip.server.recipes.KernelsDownloadRecipeService.copyFromSourcesToFolder(KernelsDownloadRecipeService.java:321)
	at com.dataiku.dip.recipes.download.DownloadRecipeRunner.run(DownloadRecipeRunner.java:75)
	at com.dataiku.dip.dataflow.jobrunner.ActivityRunner$FlowRunnableThread.run(ActivityRunner.java:374)

 

I am attaching the complete log file. Please assist.

Thank You

Deepak

 

0 Kudos
1 Solution
SeanA
Community Manager
Community Manager

Hi @deepakdhiman , when you return to the Filter recipe, 2017 should already be present. It's 2016 that you are adding-- is that correct?

Is it possible you forgot to change the condition to keep rows that satisfy all conditions to rows that satisfy at least one of the following conditions? This would explain why the following datasets are "empty" (if by empty you mean that they have 0 rows) since no rows can satisfy both conditions at the same time.

Screen Shot 2021-06-07 at 11.58.37 AM.png

Dataiku

View solution in original post

3 Replies
SeanA
Community Manager
Community Manager

Hi @deepakdhiman , when you return to the Filter recipe, 2017 should already be present. It's 2016 that you are adding-- is that correct?

Is it possible you forgot to change the condition to keep rows that satisfy all conditions to rows that satisfy at least one of the following conditions? This would explain why the following datasets are "empty" (if by empty you mean that they have 0 rows) since no rows can satisfy both conditions at the same time.

Screen Shot 2021-06-07 at 11.58.37 AM.png

Dataiku
deepakdhiman
Level 2
Author

Hi Sean

Thank you for replying to help me out. I went back to the filter recipe and checked that "all the following conditions" was selected in "Keep only rows that satisfy" field. 

I changed it to "atleast one of the following condition", run and rebuild it. It worked! It was such a silly mistake.

Thank you so much 🙂

SeanA
Community Manager
Community Manager

Great! No problem at all.

Dataiku
0 Kudos