Insert data into Hive table for every periodic execution of recipe to maintain timeseries of result

vivekkumar

User case : A complicated data-intensive Quantitate model is executed every month. The result of periodic execution should be saved in Hive table for reporting.

Quantitate model is complicated and seems suitable for PySpark recipe

Quantitate model produces multiple result set & each result set should be saved in respective Hive table

Persistence of periodic results in hive table is critical because there is requirement to perform time series analysis of period results

Hive table is required to be partitioned by date (month end execution date)