Installing packages in R Code Environment (Conda-forge?)

Options
tgb417
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,595 Neuron

Using Macintosh with High Sierra (10.13.5)

I'm trying to install the following packages into a Code Env and having difficulties.

  • deSolve
  • flexsurv
  • muhaz

If I use Conda to do this. DSS can't find the packages. (Makes sense, they are not in the "standard" list of conda packages)

If I Request the R package directly in the Code Envs setup I'm getting a compilation error.


ld: warning: directory not found for option '-L/opt/conda/conda-bld/r-base_1519528683708/_build_env/lib/gcc/x86_64-apple-darwin11.4.2/4.8.5'
ld: warning: directory not found for option '-L/opt/conda/conda-bld/r-base_1519528683708/_build_env/lib/gcc'
ld: warning: directory not found for option '-L/opt/conda/conda-bld/r-base_1519528683708/_build_env/lib'
ld: library not found for -lquadmath
clang-4.0: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [muhaz.so] Error 1
ERROR: compilation failed for package ‘muhaz’

This happens even though I've had no problems installing with RStudio installed compiling the same packages. (My guess is that I have some sort of path not set in the DSS environment.

Questions:

  • Can one use conda-forge package with DSS? Some of what I'm looking for is there.
  • Anyone work through a situation like the above with this kind of compilation issue?

Answers

  • Clément_Stenac
    Clément_Stenac Dataiker, Dataiku DSS Core Designer, Registered Posts: 753 Dataiker
    Options
    Hi,

    You can use conda-forge by adding options for the "conda" command. In your code env settings, uncheck "inherit global settings", and add "-c" and "conda-forge" as options. Beware that conda-forge very often causes incompatibilities with other packages, though



    Your compilation issue comes from the fact that you don't have the libquadmath development headers, which is required in order to compile code using it. It may be possible to install this via brew
  • tgb417
    tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,595 Neuron
    Options
    Clément thanks so much for the quick reply.

    There are two options when one unchecks "Inherit global settings
    * Extra options for 'conda create'
    * Extra options for 'conda install'

    I presume you are referencing the second option 'conda install'.

    It seems a bit odd that you need two lines one for each of the parameters. I'm not clear if I need the double quotes or not. And the reference documentation is not clear on this point regarding this page. I've tried both and did not achieve a successful install.

    Looks like there were a bunch of other prerequisites that were being attempted to be installed.

    ```Updating code environment according to spec ...
    Installing from Conda spec file
    UnsatisfiableError: The following specifications were found to be in conflict:
    Solving environment: ...working... failed
    - r-desolve
    - rstudio==1.1.423 -> r-essentials[version='>=3.4.3'] -> r-base[version='>=3.4.3,<3.4.4.0a0']<BR />Use "conda info " to see the dependencies for each package.```

    These R packages install just fine on my non-anaconda RStudio on the same machine. What I'm not clear about is what is the best way to get conda to install the library libquadmath. Or have DSS reference the library in my non-anaconda environment.

    I've tried installing
    gcc
    libgcc
    mkl
    mpmath

    from conda.

    libgcc seems to include a libquadmath.0.dylib file.
Setup Info
    Tags
      Help me…