Insert data into Hive table for every periodic execution of recipe to maintain timeseries of result

vivekkumar
Level 1
Insert data into Hive table for every periodic execution of recipe to maintain timeseries of result

User case : A complicated data-intensive Quantitate model is executed every month. The result of periodic execution should be saved in Hive table for reporting. 

Quantitate model is complicated and seems suitable for PySpark recipe 

Quantitate model produces multiple result set & each result set should be saved in respective Hive table 

Persistence of periodic results in hive table is critical because there is requirement to perform time series analysis of period results

Hive table is required to be partitioned by date (month end execution date)

 

 

0 Kudos
0 Replies