Dynamic Rename of Headers

CurtisC Registered Posts: 8

Hello All,

First Post here, and very new to Dataiku.

I am attempting to consume data from an API call to QuickBase, and I am looking for a solution that accepts a dynamic number of columns, and that replaces any special characters IE ( | , # so that I can output a file to BigQuery without issue. The team that owns the data is not willing to make this change for me on the QB side..

I see that I can make these changes for the columns in the schema, but when additional columns are added, I'm not seeing a way to manage this scenario if a new column is added. Thinking some regex replace function, but where and how is not known to me currently.

The goal is to just ETL the data with the only modification being to address invalid header naming conventions.

Appreciate your assistance.


Operating system used: Windows 11


Best Answer


Setup Info
      Help me…