Import python script in library to code recipe

Solved!
NurulPutri
Level 1
Import python script in library to code recipe

Hello,

I make a script to clean my data. The script uses data from the json files for computation purposes. I store all the script and the json in the library of my project.

So here is how the library editor looks.

Screen Shot 2021-07-21 at 21.26.23.png

I tried to import some function and class in a code recipe, but get the following error.

Screen Shot 2021-07-21 at 21.38.01.png

Is there any way I can import the function defined in the script in the library to a python recipe to make custom transformation? Or, do I have to make a plugin?

Here I attach all files that I upload to the library.

Thank you in advance for your answer.

0 Kudos
2 Solutions
AlexT
Dataiker

Hi Nurul,


Welcome to Dataiku community!

When importing new libraries you just added into a Notebook you will need to restart the notebook kernel so the notebook can load these new libraries.

Screenshot 2021-07-21 at 18.07.25.png

I see you have already created __init__.py in your subfolder.  As per out doc here :

 https://doc.dataiku.com/dss/latest/python/reusing-code.html

 

You also don't need to include the lib.python in your import.

can you try changing the after restarting the notebook Kernel and let us know if you still have issues.

from location_validation_function.synonym import batch_synonim, SynonymEntity

 

 

View solution in original post

0 Kudos
AlexT
Dataiker

Hi Nurul,

Indeed loading JSON files from the python library is not ideal.

You can use Managed folder instead, add the JSON files there and use the Managed folder API to read them, there are several examples here: https://doc.dataiku.com/dss/latest/connecting/managed_folders.html#usage-in-python

Depending on what you are actually storing in these json files you may want to simply define project variables instead since that is also just JSON. You can then read the variables  with: via https://doc.dataiku.com/dss/latest/code_recipes/variables_expansion.html#python

 

View solution in original post

0 Kudos
9 Replies
AlexT
Dataiker

Hi Nurul,


Welcome to Dataiku community!

When importing new libraries you just added into a Notebook you will need to restart the notebook kernel so the notebook can load these new libraries.

Screenshot 2021-07-21 at 18.07.25.png

I see you have already created __init__.py in your subfolder.  As per out doc here :

 https://doc.dataiku.com/dss/latest/python/reusing-code.html

 

You also don't need to include the lib.python in your import.

can you try changing the after restarting the notebook Kernel and let us know if you still have issues.

from location_validation_function.synonym import batch_synonim, SynonymEntity

 

 

0 Kudos
NurulPutri
Level 1
Author

Hi Alex,

The solution works! Thank you so much.

0 Kudos
NurulPutri
Level 1
Author

sorry, but after I can import the module, I got another error

Screen Shot 2021-07-22 at 12.05.43.png

the file is in the same folder as the .py script.

Where should I put these json files?

0 Kudos
AlexT
Dataiker

Hi Nurul,

Indeed loading JSON files from the python library is not ideal.

You can use Managed folder instead, add the JSON files there and use the Managed folder API to read them, there are several examples here: https://doc.dataiku.com/dss/latest/connecting/managed_folders.html#usage-in-python

Depending on what you are actually storing in these json files you may want to simply define project variables instead since that is also just JSON. You can then read the variables  with: via https://doc.dataiku.com/dss/latest/code_recipes/variables_expansion.html#python

 

0 Kudos
NurulPutri
Level 1
Author

Thank you so much for your help. I managed to create and run the recipe.

Once again, thank you๐Ÿ™‚

0 Kudos
hmjk347
Level 2

Hi,

 

I am facing same issue while importing the modules we have created. I am following all the basic rules needed for import but still have the issue . can someone help on what could i be missing here.

0 Kudos
dlastrazeneca
Level 1

This solution is not working for me, :(.

 

 

Screenshot 2024-02-19 at 11.38.06.jpg

โ€ƒ

โ€ƒ

 

Screenshot 2024-02-19 at 11.38.49.jpg

 

This is after restarting the kernel. Any advice?

0 Kudos
AlexT
Dataiker

Hi @dlastrazeneca ,
Did you force reload the notebook, not just restart - from the menu here:Screenshot 2024-02-19 at 1.22.58โ€ฏPM.png

If this doesn't help then please open a support ticket.

Thanks

dlastrazeneca
Level 1

That does the trick!

Or using the option to "shutdown" the kernel, and then using "restart". "Restart" alone does not work.

 

Thanks!

0 Kudos

Labels

?
Labels (3)
A banner prompting to get Dataiku