Error while installing sklearn

asadmasad

Hi there,

I created a new Python 3.9 environment in Dataiku installed on GCP and receive the following error when installing the required packages for "Visual Machine Learning with Bayesian search":

    INFO: f2py options: ['--no-wrap-functions']
    INFO:   adding 'build/src.linux-x86_64-3.9/build/src.linux-x86_64-3.9/scipy/stats/fortranobject.c' to sources.
    INFO:   adding 'build/src.linux-x86_64-3.9/build/src.linux-x86_64-3.9/scipy/stats' to include_dirs.
    INFO: building extension "scipy.stats._stats" sources
    INFO: building extension "scipy.stats.mvn" sources
    INFO: f2py options: []
    INFO:   adding 'build/src.linux-x86_64-3.9/build/src.linux-x86_64-3.9/scipy/stats/fortranobject.c' to sources.
    INFO:   adding 'build/src.linux-x86_64-3.9/build/src.linux-x86_64-3.9/scipy/stats' to include_dirs.
    INFO:   adding 'build/src.linux-x86_64-3.9/scipy/stats/mvn-f2pywrappers.f' to sources.
    INFO: building extension "scipy.ndimage._nd_image" sources
    INFO: building extension "scipy.ndimage._ni_label" sources
    INFO: building extension "scipy.ndimage._ctest" sources
    INFO: building extension "scipy.ndimage._ctest_oldapi" sources
    INFO: building extension "scipy.ndimage._cytest" sources
    INFO: building extension "scipy._lib._ccallback_c" sources
    INFO: building extension "scipy._lib._test_ccallback" sources
    INFO: building extension "scipy._lib._fpumode" sources
    INFO: building extension "scipy._lib.messagestream" sources
    WARN: Could not locate executable armflang
    INFO: building data_files sources
    INFO: build_src: building npy-pkg config files
    running build_py
    copying scipy/version.py -> build/lib.linux-x86_64-3.9/scipy
    copying build/src.linux-x86_64-3.9/scipy/__config__.py -> build/lib.linux-x86_64-3.9/scipy
    running build_clib
    INFO: customize UnixCCompiler
    INFO: customize UnixCCompiler using build_clib
    INFO: CCompilerOpt.__init__[773] : load cache from file -> /tmp/pip-install-khr0znww/scipy_5f009fe13e1d4136bd09e356ce091c49/build/temp.linux-x86_64-3.9/ccompiler_opt_cache_clib.py
    INFO: CCompilerOpt.__init__[784] : hit the file cache
    INFO: get_default_fcompiler: matching types: '['arm', 'gnu95', 'intel', 'lahey', 'pg', 'nv', 'absoft', 'nag', 'vast', 'compaq', 'intele', 'intelem', 'gnu', 'g95', 'pathf95', 'nagfor', 'fujitsu']'
    INFO: customize ArmFlangCompiler
    INFO: customize Gnu95FCompiler
    INFO: customize Gnu95FCompiler
    INFO: customize Gnu95FCompiler using build_clib
    INFO: building 'sc_c_misc' library
    INFO: compiling C sources
    INFO: C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC
    
    INFO: compile options: '-I/tmp/pip-install-khr0znww/scipy_5f009fe13e1d4136bd09e356ce091c49/scipy/special -I/home/dataiku/dss/code-envs/python/spec39/lib64/python3.9/site-packages/numpy/core/include -I/usr/include/python3.9 -I/tmp/pip-install-khr0znww/scipy_5f009fe13e1d4136bd09e356ce091c49/scipy/special/c_misc -Iscipy/_lib -I/home/dataiku/dss/code-envs/python/spec39/lib64/python3.9/site-packages/numpy/core/include -Ibuild/src.linux-x86_64-3.9/numpy/distutils/include -c'
    extra options: '-msse -msse2 -msse3'
    INFO: gcc: scipy/special/c_misc/gammasgn.c
    INFO: gcc: scipy/special/c_misc/poch.c
    INFO: gcc: scipy/special/c_misc/struve.c
    scipy/special/c_misc/gammasgn.c:1:10: fatal error: Python.h: No such file or directory
     #include <Python.h>
              ^~~~~~~~~~
    compilation terminated.
    scipy/special/c_misc/poch.c:5:10: fatal error: Python.h: No such file or directory
     #include <Python.h>
              ^~~~~~~~~~
    compilation terminated.
    scipy/special/c_misc/struve.c:82:10: fatal error: Python.h: No such file or directory
     #include <Python.h>
              ^~~~~~~~~~
    compilation terminated.
    INFO: ar: adding 4 object files to build/temp.linux-x86_64-3.9/libsc_c_misc.a
    ar: build/temp.linux-x86_64-3.9/scipy/special/c_misc/gammasgn.o: No such file or directory
    error: Command "ar rcs build/temp.linux-x86_64-3.9/libsc_c_misc.a build/temp.linux-x86_64-3.9/scipy/special/c_misc/besselpoly.o build/temp.linux-x86_64-3.9/scipy/special/c_misc/gammasgn.o build/temp.linux-x86_64-3.9/scipy/special/c_misc/poch.o build/temp.linux-x86_64-3.9/scipy/special/c_misc/struve.o" failed with exit status 1
    INFO:
    ########### CLIB COMPILER OPTIMIZATION ###########
    INFO: Platform      :
      Architecture: x64
      Compiler    : gcc
    
    CPU baseline  :
      Requested   : 'min'
      Enabled     : SSE SSE2 SSE3
      Flags       : -msse -msse2 -msse3
      Extra checks: none
    
    CPU dispatch  :
      Requested   : 'max -xop -fma4'
      Enabled     : SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_KNM AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL
      Generated   : none
    INFO: CCompilerOpt.cache_flush[817] : write cache to path -> /tmp/pip-install-khr0znww/scipy_5f009fe13e1d4136bd09e356ce091c49/build/temp.linux-x86_64-3.9/ccompiler_opt_cache_clib.py
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/dataiku/dss/code-envs/python/spec39/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-khr0znww/scipy_5f009fe13e1d4136bd09e356ce091c49/setup.py'"'"'; __file__='"'"'/tmp/pip-install-khr0znww/scipy_5f009fe13e1d4136bd09e356ce091c49/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-xide3inm/install-record.txt --single-version-externally-managed --compile --install-headers /home/dataiku/dss/code-envs/python/spec39/include/site/python3.9/scipy Check the logs for full command output.
WARNING: You are using pip version 21.3.1; however, version 22.0.4 is available.
You should consider upgrading via the '/home/dataiku/dss/code-envs/python/spec39/bin/python -m pip install --upgrade pip' command.
    Running setup.py install for scipy: finished with status 'error'

---

Any help or feedback will be appreciated. Receive the same error on Python 3.8 too. However, working fine on Python 3.6.

Thanks

Operating system used: AlmaLinux

gnaldi62

Hi,

did you install also the python-devel on your machine ? It seems you're missing the python development package. The way and the name of the package to install depends on the Linux version but should be something like python3-dev, python3-devel

Also, have you installed the code-environment as a "Non managed Path" ? In that case you should take care of all the dependencies and install them onthe system.

Rgds.

Giuseppe

asadmasad

Hi Giuseppe,

Thanks for the quick revert. I indeed had to install python39-devel package since I only had it for python 3.6 version.

Thanks heaps for the quick help!

Asad