Partitioning in Dataiku DSSWatch now

How do I import python code in recipes?

Dataiker
Dataiker
How do I import python code in recipes?

Hi,



I am trying to set up a software architecture in order to be able to navigate easily between a test environment (where I will try out many methods in DSS) and a production environment without DSS.



The architecture that I want to set up is the following:




  • A main config script (main.py) that includes variables such as the modules to run and how to input/output

  • Multiple modules files that import main and do their respective computation



The idea is that, in the production environment, the main file will take into account production-specific I/O, while it will use dataiku I/O in the test environment. However, importing another source file in Python DSS is not supported as far as I know.



 



I saw that you have global variables and scenarii, but they are not flexible enough for me.



 



My question is therefore : is there any way to do what I want that is integrated in DSS, so without messing with the files in the filesystem?



 



Best regards,



Jean Creusefond

0 Kudos
1 Reply
Dataiker
Dataiker

See the documentation on the Python Environment, you can add .py files in DATA_DIR/lib/python.



You can also look into plugins, to develop a custom recipe that has the same name, within the same plugin ID, but with different rreleases for dev & production.

Labels (2)