Normalize text without lowercase

Dataiker, Alpha Tester Posts: 535 Dataiker
The "Simplify text" processor (and a handful of others) have a "Normalize text" option that "transforms to lowercase, removes accents and performs Unicode normalization (Café -> cafe)." Anyone figure out a way to remove accents and perform unicode normalization but not change the case? Café -> Cafe

Welcome!

It looks like you're new here. Sign in or register to get started.

Answers

  • Dataiker, Dataiku DSS Core Designer, Registered, Moderator Posts: 753 Dataiker
    Hi,

    This is not possible via the Simplify text processor. You could do it with a custom Python processor
  • Dataiker, Alpha Tester Posts: 535 Dataiker
    Is a "custom Python processor" different than just the "Python function" processor? I couldn't get the latter to work with the unicodedata module as described here: https://stackoverflow.com/a/16467505/612166 I suspect that's a limitation of the Jython executor?

    What normalization form does DSS use, anyhow?
  • Dataiker, Registered Posts: 125 Dataiker
    Hi,
    You can use a replace in the python processor with this:
    spec = u"²ÀÁÂÃÄÅàáâãäåĀāĂ㥹ÇçĆćĈĉĊċČčÐðĎďĐđÈÉÊËèéêëĒēĔĕĖėĘęĚěĜĝĞğĠġĢģĤĥĦħÌÍÎÏìíîïĨĩĪīĬĭĮįİıĴĵĶķĸĹĺĻļĽľĿŀŁłÑñŃńŅņŇňʼnŊŋÒÓÔÕÖØòóôõöøŌōŎŏŐőŔŕŖŗŘřŚśŜŝŞşŠšſŢţŤťŦŧÙÚÛÜùúûüŨũŪūŬŭŮůŰűŲųŴŵÝýÿŶŷŸŹźŻżŽž"
    norm = u"2AAAAAAaaaaaaAaAaAaCcCcCcCcCcDdDdDdEEEEeeeeEeEeEeEeEeGgGgGgGgHhHhIIIIiiiiIiIiIiIiIiJjKkkLlLlLlLlLlNnNnNnNnnNnOOOOOOooooooOoOoOoRrRrRrSsSsSsSssTtTtTtUUUUuuuuUuUuUuUuUuUuWwYyyYyYZzZzZz"

Welcome!

It looks like you're new here. Sign in or register to get started.

Welcome!

It looks like you're new here. Sign in or register to get started.