Partition column values have a pipe symbol

pratikgujral-sf

Hi Community,

I have a SQL dataset that is partitioned. I am trying to run a code recipe on this dataset, but receiving the error: An invalid argument has been encountered : For `<partition_value>': Invalid partition identifier, has 2 dimensions, expected 1

A few things to note:

• There is only one column that has 10 discrete values on which 'Discrete Values' partitioning is done
• All of the 10 discrete values have a pipe in the string value; that is, each of the 10 values looks like this: abc|xyz

I suspect that the reason for the error is the occurrence of a pipe symbol in the values. If so, how do I tell Dataiku to consider the values "as is" and that the pipe has no special meaning?

I do not wish to add an additional Prepare recipe to replace the pipe in the column values, as a downstream Code recipe needs to use this data.

---

Operating system used: Red Hat Enterprise Linux

JordanB

Hi @pratikgujral-sf
,

Unfortunately, there is no way to partition by discreet dimension with a pipe value included. Discrete dimensions should only contain letters and numbers: https://doc.dataiku.com/dss/latest/partitions/identifiers.html#discrete-dimension-identifiers

With a pipe value included, you will see an error like "Invalid partition identifier, has 2 dimensions, expected 1". The value will need to be changed prior to partitioning.

Let me know if you have any questions.

Thanks,

Jordan