You now have until September 15th to submit your use case or success story to the 2022 Dataiku Frontrunner Awards!ENTER YOUR SUBMISSION

Google Sheets Plugin import bug

Eldiias
Level 1
Google Sheets Plugin import bug

Hi there!

I encountered a bug (or potentially it is a hardcoded limitation) while using Google Sheets plugin. I have the following flow:

  1. Dataset is transformed and stored into GSheets
  2. Another dataset is transformed, the dataset from Gsheets is loaded and appended to it. New dataset is stored as a Dataiku dataset.

Sounds simple. But it didn't work. After careful review of the flow, I found out the following:

  1. The first dataset has long column names. There is no limitation on the column name length, so not a problem.
  2. Dataset in GSheets is OK, the column names are correct.
  3. While importing a dataset from GSheets, the column names are cropped. So, instead of total_transport_cost_per_ton it is total_transport_cost_per. 

I believe the issue is related with a plugin, though I didn't find any similar comment yet.

I could definitely just update the column names (since I have the same ones in the second dataset), but I would love to retrieve column names correctly.

0 Kudos
1 Reply
HarizoR
Dataiker
Dataiker

Hi Eldiias,

After reviewing the plugin's code it appears that your issue is indeed due to a hardcoded limitation of 25 characters for the slugified column names (see here: https://github.com/dataiku/dataiku-contrib/blob/master/googlesheets/python-connectors/googlesheets-s...). In your case, the easiest workaround is to switch the plugin to development mode and edit that value manually.

Hope this helps!

Best,

Harizo

Labels

?
Labels (2)
A banner prompting to get Dataiku