Calling Python code from Shiny webapp using reticulate
Hi,
I want to test out calling Python from a Shiny webapp using https://rstudio.github.io/reticulate/index.html . I believe I will need to specify my `virtualenv`, which i would equate to my dataiku Python code environment. Is there a Shiny function to get the path to a code environment?
thx
Operating system used: Windows 10
Best Answer
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptionsHi,
No function, in particular, could provide the path to the Python Code env. You would need to determine this yourself and hard code this path would be :
DATADIR/code-envs/python/<code-env-name/bin/python3Thanks,
Answers
-
so if my webapp is at /data/dataiku/data_dir/tmp/myproject/web_apps/QxSZRxY, is the data dir == '/data/dataiku/data_dir'?
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptions
Yes, The datadir would be /data/dataiku/data_dir/
-
OK so after confirming the path did the following:
library(reticulate)
use_virtualenv("/data/dataiku/data_dir/code-envs/python/mycodeenv")
Produces the following:
. Warning: Error in : '/data/dataiku/data_dir/code-envs/python/mycodeenv/bin/python' was not built with a shared library.reticulate can only bind to copies of Python built with '--enable-shared'.53: stop52: stopf 51: use_python 50: use_virtualenv 1: shiny::runAppError : '/data/dataiku/data_dir/code-envs/python/mycodeenv/bin/python' was not built with a shared library.reticulate can only bind to copies of Python built with '--enable-shared'.
Any ideas on how to get this working?
**EDIT**: @AlexT
Could this be as simple as changing this command?
command : /data/dataiku/data_dir/bin/python /data/dataiku/dataiku-dss-10.0.3/scripts/virtualenv.pyz -p python3.7 /data/dataiku/data_dir/code-envs/python/mycodeenv
NEW:
command : /data/dataiku/data_dir/bin/python /data/dataiku/dataiku-dss-10.0.3/scripts/virtualenv.pyz -p python3.7 /data/dataiku/data_dir/code-envs/python/mycodeenv --enable-shared -
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptions
Hi,
I believe the issue is from the fact your don't have python3-dev installed on the OS.
You will need to install python3-dev package and rebuild the code env and try again.
Similar issue reported here : https://github.com/rstudio/reticulate/issues/637 -
I rebuilt my code environment with `python-dev-tools` and still get the same error.
But when I check the code environment logs, the log file outdated:
createPythonEnv.log 8.43 KB Tuesday, 16 August Any ideas?
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptions
Hi,
Can you confirm if you ran: yum install -y python3-devel.x86_64
Also did are you using OS python 3 or did you install python from source?
Thanks, -
I am just using dataiku to build my code environment so I am not specifying anything special here. I added `python-dev-tools` to my installed packages and selected 'Rebuild env' and that is it. You keep mentioning the OS. To me, that is all handled by dataiku. Sorry a bit confused here.
thx
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptions
Sorry if I wasn't clear you need to install python3-dev/python3-devel on the OS since the reticulate package requires it for it to work. As explained in the issue here:
https://github.com/rstudio/reticulate/issues/495#issuecomment-489304420
-
@AlexT
Do you mean our IT department that manages dataiku has to install this? I am just a user that owns a project. -
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptions
Yes the OS dependencies would need to be installed by your DSS /Systems Admin
-