compatibility of the Foreach and transpose transformations
Hi, I have a project which uses Foreach statement and right after Transpose. The Result of the output should be 26 as u see in the picture but we get 19 for some reason. When we spited the prepare statement in two moving the Transpose in a separate prepare the output was correct.
I do believe it has to do something with the column and row indexing of the foreach transformation output.
Operating system used: Windows
Answers
-
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,639 NeuronWelcome to the Dataiku Community we are so glad to have you join us. In looking over your note here I do not see the approximate record count. But based on the image you have provided I think the count is small. So yes two hours seem wildly inappropriate. That said I don’t personally have an answer for you. The Dataiku support team is very good. I’d like to invite you to share this issue with them. I suspect that they will be able to help most effectively. When you resolve your issue. I also invite you to share here the outcome so that others might learn when searching for related challenges. Good luck let us know how this goes.
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023, Circle Member Posts: 2,641 NeuronThe numbers you see in the prepare recipe are based on a "preview" of the execution of the recipe over the data sampled in the last sample taken. You should not consider these to be final counts. To see how your recipe impacts your data you need to execute it so that the recipe runs against all the input dataset. The results will be pushed to the output dataset but then again when you explore this dataset you will be looking at a data sample so if you want to see all rows you may need to use other tools like a SQL Notebook.