Have the ability to use third-party module for Python UserDefinedFunction in a recipe

[tomtom]

Hello,

Unlike a Python recipe which can use a custom code env with external Python modules, it is not possible to do it from a Python UDF in a preparation recipe.

This is documented in https://doc.dataiku.com/dss/latest/code_recipes/python.html

> "Python UDF can only use a subset of the packages in the base Python installation and cannot use any other package"

I understand that Python UDF is executed with Jython 2.7 and :

- to me it is possible to call an external Python module with Jython as long as the module is compatible with 2.7

- and by the way in the DSS 5.1.2, the Jython version is obsolete 2.7b4, when the 2.7.0 is available



I may be missing some stuff but it would be cool to improve this essential Python UDF function 🙂

Best regards.

[cperdigou · Accepted solution]

Thanks for pointing this out, you'll be happy to learn that this is on our roadmap and we that we are actively working on it!

[tomtom]

great !

[Mark_Treveil]

DSS Version 6, released December 2019, bring native python for preparation. The Python processor in data preparation can now use a real Python process, which allows usage of Python 3 and of any additional package through the usage of the DSS code environments feature.

The Python processor now supports vectorized operation using Pandas for fast operation.
https://doc.dataiku.com/dss/latest/preparation/processors/python-custom.html