Partition column values have a pipe symbol

pratikgujral-sf
Level 2
Partition column values have a pipe symbol

Hi Community,

I have a SQL dataset that is partitioned. I am trying to run a code recipe on this dataset, but receiving the error:  An invalid argument has been encountered : For `<partition_value>': Invalid partition identifier, has 2 dimensions, expected 1

A few things to note:

  • There is only one column that has 10 discrete values on which 'Discrete Values' partitioning is done
  • All of the 10 discrete values have a pipe in the string value; that is, each of the 10 values looks like this: abc|xyz

I suspect that the reason for the error is the occurrence of a pipe symbol in the values. If so, how do I tell Dataiku to consider the values "as is" and that the pipe has no special meaning?

 

I do not wish to add an additional Prepare recipe to replace the pipe in the column values, as a downstream Code recipe needs to use this data. 


Operating system used: Red Hat Enterprise Linux

0 Kudos
1 Reply
JordanB
Dataiker

Hi @pratikgujral-sf,

Unfortunately, there is no way to partition by discreet dimension with a pipe value included. Discrete dimensions should only contain letters and numbers: https://doc.dataiku.com/dss/latest/partitions/identifiers.html#discrete-dimension-identifiers

With a pipe value included, you will see an error like "Invalid partition identifier, has 2 dimensions, expected 1". The value will need to be changed prior to partitioning. 

Let me know if you have any questions. 

Thanks,

Jordan

0 Kudos