Announcing the winners & finalists of the Dataiku Frontrunner Awards 2021! Read their inspiring stories

"importLibrariesFromProject" in the "external-libraries.json" file not working

stephensunny
Level 1
"importLibrariesFromProject" in the "external-libraries.json" file not working

This is the end of my project URL: /projects/ASTELLAS_MI_RUN/

From here I selected ASTELLAS_MI_RUN as my project key.

In the "external-libraries.json" file I have the following code: 

{
"gitReferences": {},
"pythonPath": [
"python"
],
"rsrcPath": [
"R"
],
"importLibrariesFromProjects": ['ASTELLAS_MI_RUN']
}

Within the ASTELLAS_MI_RUN project, I am trying to call the function "new_MI()" in my new project.

When I do this, I get the following error:

Error in new_MI(): could not find function "new_MI" Traceback:

Everything is spelled correctly, I am using Dataiku version 9.0.3, I have restarted my notebook several times and I have admin, read and write privileges in both projects. Please advise.

0 Kudos
7 Replies
HarizoR
Dataiker
Dataiker

Hi Stephen,

The logic behind shared project libraries is that you choose which target projects you want to import code from in the importFromLibrariesProject field.

For example:

- in the project which key is ASTELLAS_MI_RUN, you have a library that contains the new_MI() function.

- in another project which key is OTHER_PROJECT, you should be able to call the new_MI() function if you add "ASTELLAS_MI_RUN" in the importLibrairiesFromProjects field of the external-libraries.json file in OTHER_PROJECT's libraries.

Best,

Harizo

0 Kudos
stephensunny
Level 1
Author

Yes. The key was already in the other project's external-libraries.json file. Notice the project name in the attached screen shot. The project name is in the upper left of the screenshot.

 

0 Kudos
EliasH
Dataiker
Dataiker

Hi @stephensunny ,

Can you confirm if your set up of shared libraries looks like mine?

In my source project with the project key TEST, I have one python file with two functions I want to utilize in another project

Screen Shot 2021-09-29 at 10.41.35 AM.png

In another project with the key NOTEBOOKS, I want to use those functions, so I import the TEST project in importFromLibrariesProject field

Screen Shot 2021-09-29 at 10.41.53 AM.png

 And I import them like so.

Screen Shot 2021-09-29 at 10.42.42 AM.png

If so, is your library's folder structure set up differently? Would you mind providing a screenshot of where your new_MI() function is located in?

0 Kudos
stephensunny
Level 1
Author

In my source project with the project key ASTELLAS_MI_RUN, there is an R file ("run_mi.R") with the function new_MI() that I want to utilize in another project. Please see the attachment titled "1.png".

In another project with the key ASTELLAS_PRICE_X_VOL, I want to use the new_MI() function, so I imported the ASTELLAS_MI_RUN project in the importFromLibrariesProject field. Please see the attachment titled "2.png".

When I try to import the function new_MI() from ASTELLAS_MI_RUN to ASTELLAS_PRICE_X_VOL I get an error. Please see the attachment titled "3.png".

The function new_MI() works and has been used multiple times in the source project.

0 Kudos
HarizoR
Dataiker
Dataiker

Hi Stephen,

As mentioned in the documentation, you need to import the functions of your run_mi.R file in the global namespace. In practice, you should add dkuSourceLibR("run_mi.R") in your notebook before calling the new_MI() function.

Best,

Harizo

0 Kudos
stephensunny
Level 1
Author

That doesn't work either. Please see the attachment titled "4.png". I get the following error:

Error in path.expand(file): invalid 'path' argument
Traceback:

1. dkuSourceLibR("run_mi.R")
2. source(file_path, local = local, verbose = verbose, chdir = chdir, 
 .     encoding = encoding)
3. withVisible(eval(ei, envir))
4. eval(ei, envir)
5. eval(ei, envir)
6. dkuSourceLibR("data_prep_func.R")
7. source(file_path, local = local, verbose = verbose, chdir = chdir, 
 .     encoding = encoding)
8. withVisible(eval(ei, envir))
9. eval(ei, envir)
10. eval(ei, envir)
11. read.xlsx(dkuCustomVariable("params"), check.names = TRUE, header = TRUE, 
  .     stringsAsFactors = FALSE, sheetName = "Sheet1")
12. loadWorkbook(file, password = password)
13. file.exists(path.expand(file))
14. path.expand(file)

The file "run_mi.R" is in the R folder of my project library in addition to being in the source project's library. Please see the attachment titled "5.png". 

I'd also like to point out that you are combining steps from the Importing libraries from other projects and the Sharing R code globally section.

I'd also like to point out that EliasH did not include the dkuSourceLibR step in his/her response. 

0 Kudos
HarizoR
Dataiker
Dataiker

Hi Stephen,

It is expected to encounter issues if you are trying to pull two files with the same name in your R global namespace. If the two files have the same content, you don't need to resort to shared libraries, so you should remove the entry in externalLibraries.json. If the content is different, I suggest renaming one of the files and adjust your sourcing accordingly.

Best,

Harizo

0 Kudos
A banner prompting to get Dataiku DSS