I hope you are doing well.
I am writing to you because I am trying to set up an etl. Indeed I have a flow zone with the data preparation steps and a step with the final joins.
The goal is to create an ETL with a sandbox of data on which I can build different tables from the output dataset.
In terms of performance, is a final join with all datasets the best option? Because I have about 200 columns to join and it takes easily 1H for the run.
(Each dataset is stored in a postgreed database)
Thanks for your answers