maintaining time series dataset - adding data file every month

vivekkumar Registered Posts: 4 ✭✭✭

We have a use case where we need to maintain monthly data in Hive table for reporting

- Every month data file is sourced manually

- Data file has a column to save date (month end date)

- Requirement is to store monthly data in Hive table

- Hive table should be partitioned by date (month end date)

- Its like stacking new data into Hive table

There is also requirement to occasionally override monthly data if an monthly override data file arrives

Please suggest a suitable solution


Setup Info
      Help me…