Discover the winners & finalists of the 2022 Dataiku Frontrunner Awards!READ THEIR USE CASES

"importLibrariesFromProject" in the "external-libraries.json" file not working

stephensunny
Level 3
"importLibrariesFromProject" in the "external-libraries.json" file not working

This is the end of my project URL: /projects/ASTELLAS_MI_RUN/

From here I selected ASTELLAS_MI_RUN as my project key.

In the "external-libraries.json" file I have the following code: 

{
"gitReferences": {},
"pythonPath": [
"python"
],
"rsrcPath": [
"R"
],
"importLibrariesFromProjects": ['ASTELLAS_MI_RUN']
}

Within the ASTELLAS_MI_RUN project, I am trying to call the function "new_MI()" in my new project.

When I do this, I get the following error:

Error in new_MI(): could not find function "new_MI" Traceback:

Everything is spelled correctly, I am using Dataiku version 9.0.3, I have restarted my notebook several times and I have admin, read and write privileges in both projects. Please advise.

0 Kudos
12 Replies
HarizoR
Developer Advocate

Hi Stephen,

The logic behind shared project libraries is that you choose which target projects you want to import code from in the importFromLibrariesProject field.

For example:

- in the project which key is ASTELLAS_MI_RUN, you have a library that contains the new_MI() function.

- in another project which key is OTHER_PROJECT, you should be able to call the new_MI() function if you add "ASTELLAS_MI_RUN" in the importLibrairiesFromProjects field of the external-libraries.json file in OTHER_PROJECT's libraries.

Best,

Harizo

0 Kudos
stephensunny
Level 3
Author

Yes. The key was already in the other project's external-libraries.json file. Notice the project name in the attached screen shot. The project name is in the upper left of the screenshot.

 

0 Kudos
coriBio
Level 2

This works , and later if you are loading the resources in a Jupyter NOTEBOOK or recipe, you have to "Force Reload" (Restarting the Kernel is not enough)

 DSSlibraryexportIssue.PNG

 

EliasH
Dataiker

Hi @stephensunny ,

Can you confirm if your set up of shared libraries looks like mine?

In my source project with the project key TEST, I have one python file with two functions I want to utilize in another project

Screen Shot 2021-09-29 at 10.41.35 AM.png

In another project with the key NOTEBOOKS, I want to use those functions, so I import the TEST project in importFromLibrariesProject field

Screen Shot 2021-09-29 at 10.41.53 AM.png

 And I import them like so.

Screen Shot 2021-09-29 at 10.42.42 AM.png

If so, is your library's folder structure set up differently? Would you mind providing a screenshot of where your new_MI() function is located in?

0 Kudos
stephensunny
Level 3
Author

In my source project with the project key ASTELLAS_MI_RUN, there is an R file ("run_mi.R") with the function new_MI() that I want to utilize in another project. Please see the attachment titled "1.png".

In another project with the key ASTELLAS_PRICE_X_VOL, I want to use the new_MI() function, so I imported the ASTELLAS_MI_RUN project in the importFromLibrariesProject field. Please see the attachment titled "2.png".

When I try to import the function new_MI() from ASTELLAS_MI_RUN to ASTELLAS_PRICE_X_VOL I get an error. Please see the attachment titled "3.png".

The function new_MI() works and has been used multiple times in the source project.

0 Kudos
HarizoR
Developer Advocate

Hi Stephen,

As mentioned in the documentation, you need to import the functions of your run_mi.R file in the global namespace. In practice, you should add dkuSourceLibR("run_mi.R") in your notebook before calling the new_MI() function.

Best,

Harizo

0 Kudos
stephensunny
Level 3
Author

That doesn't work either. Please see the attachment titled "4.png". I get the following error:

Error in path.expand(file): invalid 'path' argument
Traceback:

1. dkuSourceLibR("run_mi.R")
2. source(file_path, local = local, verbose = verbose, chdir = chdir, 
 .     encoding = encoding)
3. withVisible(eval(ei, envir))
4. eval(ei, envir)
5. eval(ei, envir)
6. dkuSourceLibR("data_prep_func.R")
7. source(file_path, local = local, verbose = verbose, chdir = chdir, 
 .     encoding = encoding)
8. withVisible(eval(ei, envir))
9. eval(ei, envir)
10. eval(ei, envir)
11. read.xlsx(dkuCustomVariable("params"), check.names = TRUE, header = TRUE, 
  .     stringsAsFactors = FALSE, sheetName = "Sheet1")
12. loadWorkbook(file, password = password)
13. file.exists(path.expand(file))
14. path.expand(file)

The file "run_mi.R" is in the R folder of my project library in addition to being in the source project's library. Please see the attachment titled "5.png". 

I'd also like to point out that you are combining steps from the Importing libraries from other projects and the Sharing R code globally section.

I'd also like to point out that EliasH did not include the dkuSourceLibR step in his/her response. 

0 Kudos
HarizoR
Developer Advocate

Hi Stephen,

It is expected to encounter issues if you are trying to pull two files with the same name in your R global namespace. If the two files have the same content, you don't need to resort to shared libraries, so you should remove the entry in externalLibraries.json. If the content is different, I suggest renaming one of the files and adjust your sourcing accordingly.

Best,

Harizo

0 Kudos
achett2
Level 1

I came across this same issue and was excited to see that it was already asked. Sadly, nobody at Dataiku knows the answer. Luckily stephensunny was able to help.

Sourcing a file is a very basic thing and Dataiku needs to make it simpler, train their employees on how to do it or create better documentation.

0 Kudos
CoreyS
Dataiker Alumni

Hi @achett2 thank you for your feedback. If you think there should be an easier way to utilize Dataiku then please feel free to utilize the Product Ideas board. The Product Ideas board is here to let you share and exchange your ideas on how to improve Dataiku. Here are some resources to help get you started:


How to suggest Dataiku ideas 
Participating on the Product Ideas board 
Suggest an idea

 

I hope this helps!

Looking for more resources to help you use Dataiku effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as ‘Accepted Solution’ to help others like you!
0 Kudos
vasilisa_dku
Dataiker

I came across the same issue with troubles importing code from another project. In the notebook, you should always "FORCE RELOAD" the server in order to make the shared code visible. Just re-starting the kernel is not enough. 

coriBio
Level 2

Thank you so much! This worked for me! I also was struggling with this and finally I found your comment.

DSSlibraryexportIssue.PNG