Automation with a changing schema
Graham_E
Registered Posts: 14 ✭✭✭✭
I can't find an explicit description of this use case in the docs, so I am asking here. If there is something in the docs I missed, please let me know.
I am using a custom plugin to read data from an external service (via a REST api). This is working great. I am also using a scenario to build the project on request. The caveat is that in the future the schema of the data will change. We have columns of the form "entry_1", "entry_2" ... "entry_n" etc. and the max value of n will most likely change in the future. Ideally, the automation setup we would use would allow the data sets schema change to be propagated when the project is re-built. Is this possible? I would like the flow of data to be as hands off as possible.
Thanks
I am using a custom plugin to read data from an external service (via a REST api). This is working great. I am also using a scenario to build the project on request. The caveat is that in the future the schema of the data will change. We have columns of the form "entry_1", "entry_2" ... "entry_n" etc. and the max value of n will most likely change in the future. Ideally, the automation setup we would use would allow the data sets schema change to be propagated when the project is re-built. Is this possible? I would like the flow of data to be as hands off as possible.
Thanks
Tagged:
Answers
-
Hi,
It is not currently possible to automate schema propagation changes along the Flow. -
Thanks for the quick reply Clément .
Is there a work around we could use in the meantime? Like is it possible to pre-define a schema in conjunction with the custom plugin? If so we could pad the schema so that the value "n" for "entry_n" is arbitrarily large.
