Calling Python code from Shiny webapp using reticulate

info-rchitect
info-rchitect Registered Posts: 184 ✭✭✭✭✭✭

Hi,

I want to test out calling Python from a Shiny webapp using https://rstudio.github.io/reticulate/index.html . I believe I will need to specify my `virtualenv`, which i would equate to my dataiku Python code environment. Is there a Shiny function to get the path to a code environment?

thx


Operating system used: Windows 10

Tagged:

Best Answer

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,225 Dataiker
    Answer ✓

    Hi,

    No function, in particular, could provide the path to the Python Code env. You would need to determine this yourself and hard code this path would be :

    DATADIR/code-envs/python/<code-env-name/bin/python3

    Thanks,

Answers

  • info-rchitect
    info-rchitect Registered Posts: 184 ✭✭✭✭✭✭

    so if my webapp is at /data/dataiku/data_dir/tmp/myproject/web_apps/QxSZRxY, is the data dir == '/data/dataiku/data_dir'?

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,225 Dataiker

    Yes, The datadir would be /data/dataiku/data_dir/

  • info-rchitect
    info-rchitect Registered Posts: 184 ✭✭✭✭✭✭

    OK so after confirming the path did the following:

    library(reticulate)

    use_virtualenv("/data/dataiku/data_dir/code-envs/python/mycodeenv")

    Produces the following:

    . Warning: Error in : '/data/dataiku/data_dir/code-envs/python/mycodeenv/bin/python' was not built with a shared library.reticulate can only bind to copies of Python built with '--enable-shared'.53: stop52: stopf 51: use_python 50: use_virtualenv 1: shiny::runAppError : '/data/dataiku/data_dir/code-envs/python/mycodeenv/bin/python' was not built with a shared library.reticulate can only bind to copies of Python built with '--enable-shared'.

    Any ideas on how to get this working?

    **EDIT**: @AlexT
    Could this be as simple as changing this command?

    command : /data/dataiku/data_dir/bin/python /data/dataiku/dataiku-dss-10.0.3/scripts/virtualenv.pyz -p python3.7 /data/dataiku/data_dir/code-envs/python/mycodeenv

    NEW:

    command : /data/dataiku/data_dir/bin/python /data/dataiku/dataiku-dss-10.0.3/scripts/virtualenv.pyz -p python3.7 /data/dataiku/data_dir/code-envs/python/mycodeenv --enable-shared

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,225 Dataiker

    Hi,

    I believe the issue is from the fact your don't have python3-dev installed on the OS.

    You will need to install python3-dev package and rebuild the code env and try again.

    Similar issue reported here : https://github.com/rstudio/reticulate/issues/637

  • info-rchitect
    info-rchitect Registered Posts: 184 ✭✭✭✭✭✭

    @AlexT

    I rebuilt my code environment with `python-dev-tools` and still get the same error.

    rebuild_dku_code_env.png

    But when I check the code environment logs, the log file outdated:

    createPythonEnv.log8.43 KBTuesday, 16 August

    Any ideas?


  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,225 Dataiker

    Hi,

    Can you confirm if you ran: yum install -y python3-devel.x86_64
    Also did are you using OS python 3 or did you install python from source?

    Thanks,

  • info-rchitect
    info-rchitect Registered Posts: 184 ✭✭✭✭✭✭

    @AlexT

    I am just using dataiku to build my code environment so I am not specifying anything special here. I added `python-dev-tools` to my installed packages and selected 'Rebuild env' and that is it. You keep mentioning the OS. To me, that is all handled by dataiku. Sorry a bit confused here.

    thx

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,225 Dataiker

    Sorry if I wasn't clear you need to install python3-dev/python3-devel on the OS since the reticulate package requires it for it to work. As explained in the issue here:

    https://github.com/rstudio/reticulate/issues/495#issuecomment-489304420

  • info-rchitect
    info-rchitect Registered Posts: 184 ✭✭✭✭✭✭

    @AlexT
    Do you mean our IT department that manages dataiku has to install this? I am just a user that owns a project.

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,225 Dataiker

    Yes the OS dependencies would need to be installed by your DSS /Systems Admin

  • info-rchitect
    info-rchitect Registered Posts: 184 ✭✭✭✭✭✭

    @AlexT
    ,

    We were able to get it working by installing python3-dev on DSS.

    Thx!

Setup Info
    Tags
      Help me…