Error while installing sklearn

Options
asadmasad
asadmasad Registered Posts: 4 ✭✭✭

Hi there,

I created a new Python 3.9 environment in Dataiku installed on GCP and receive the following error when installing the required packages for "Visual Machine Learning with Bayesian search":

INFO: f2py options: ['--no-wrap-functions']INFO: adding 'build/src.linux-x86_64-3.9/build/src.linux-x86_64-3.9/scipy/stats/fortranobject.c' to sources.INFO: adding 'build/src.linux-x86_64-3.9/build/src.linux-x86_64-3.9/scipy/stats' to include_dirs.INFO: building extension "scipy.stats._stats" sourcesINFO: building extension "scipy.stats.mvn" sourcesINFO: f2py options: []INFO: adding 'build/src.linux-x86_64-3.9/build/src.linux-x86_64-3.9/scipy/stats/fortranobject.c' to sources.INFO: adding 'build/src.linux-x86_64-3.9/build/src.linux-x86_64-3.9/scipy/stats' to include_dirs.INFO: adding 'build/src.linux-x86_64-3.9/scipy/stats/mvn-f2pywrappers.f' to sources.INFO: building extension "scipy.ndimage._nd_image" sourcesINFO: building extension "scipy.ndimage._ni_label" sourcesINFO: building extension "scipy.ndimage._ctest" sourcesINFO: building extension "scipy.ndimage._ctest_oldapi" sourcesINFO: building extension "scipy.ndimage._cytest" sourcesINFO: building extension "scipy._lib._ccallback_c" sourcesINFO: building extension "scipy._lib._test_ccallback" sourcesINFO: building extension "scipy._lib._fpumode" sourcesINFO: building extension "scipy._lib.messagestream" sourcesWARN: Could not locate executable armflangINFO: building data_files sourcesINFO: build_src: building npy-pkg config filesrunning build_pycopying scipy/version.py -> build/lib.linux-x86_64-3.9/scipycopying build/src.linux-x86_64-3.9/scipy/__config__.py -> build/lib.linux-x86_64-3.9/scipyrunning build_clibINFO: customize UnixCCompilerINFO: customize UnixCCompiler using build_clibINFO: CCompilerOpt.__init__[773] : load cache from file -> /tmp/pip-install-khr0znww/scipy_5f009fe13e1d4136bd09e356ce091c49/build/temp.linux-x86_64-3.9/ccompiler_opt_cache_clib.pyINFO: CCompilerOpt.__init__[784] : hit the file cacheINFO: get_default_fcompiler: matching types: '['arm', 'gnu95', 'intel', 'lahey', 'pg', 'nv', 'absoft', 'nag', 'vast', 'compaq', 'intele', 'intelem', 'gnu', 'g95', 'pathf95', 'nagfor', 'fujitsu']'INFO: customize ArmFlangCompilerINFO: customize Gnu95FCompilerINFO: customize Gnu95FCompilerINFO: customize Gnu95FCompiler using build_clibINFO: building 'sc_c_misc' libraryINFO: compiling C sourcesINFO: C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPICINFO: compile options: '-I/tmp/pip-install-khr0znww/scipy_5f009fe13e1d4136bd09e356ce091c49/scipy/special -I/home/dataiku/dss/code-envs/python/spec39/lib64/python3.9/site-packages/numpy/core/include -I/usr/include/python3.9 -I/tmp/pip-install-khr0znww/scipy_5f009fe13e1d4136bd09e356ce091c49/scipy/special/c_misc -Iscipy/_lib -I/home/dataiku/dss/code-envs/python/spec39/lib64/python3.9/site-packages/numpy/core/include -Ibuild/src.linux-x86_64-3.9/numpy/distutils/include -c'extra options: '-msse -msse2 -msse3'INFO: gcc: scipy/special/c_misc/gammasgn.cINFO: gcc: scipy/special/c_misc/poch.cINFO: gcc: scipy/special/c_misc/struve.cscipy/special/c_misc/gammasgn.c:1:10: fatal error: Python.h: No such file or directory#include <Python.h>^~~~~~~~~~compilation terminated.scipy/special/c_misc/poch.c:5:10: fatal error: Python.h: No such file or directory#include <Python.h>^~~~~~~~~~compilation terminated.scipy/special/c_misc/struve.c:82:10: fatal error: Python.h: No such file or directory#include <Python.h>^~~~~~~~~~compilation terminated.INFO: ar: adding 4 object files to build/temp.linux-x86_64-3.9/libsc_c_misc.aar: build/temp.linux-x86_64-3.9/scipy/special/c_misc/gammasgn.o: No such file or directoryerror: Command "ar rcs build/temp.linux-x86_64-3.9/libsc_c_misc.a build/temp.linux-x86_64-3.9/scipy/special/c_misc/besselpoly.o build/temp.linux-x86_64-3.9/scipy/special/c_misc/gammasgn.o build/temp.linux-x86_64-3.9/scipy/special/c_misc/poch.o build/temp.linux-x86_64-3.9/scipy/special/c_misc/struve.o" failed with exit status 1INFO:########### CLIB COMPILER OPTIMIZATION ###########INFO: Platform :Architecture: x64Compiler : gccCPU baseline :Requested : 'min'Enabled : SSE SSE2 SSE3Flags : -msse -msse2 -msse3Extra checks: noneCPU dispatch :Requested : 'max -xop -fma4'Enabled : SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_KNM AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICLGenerated : noneINFO: CCompilerOpt.cache_flush[817] : write cache to path -> /tmp/pip-install-khr0znww/scipy_5f009fe13e1d4136bd09e356ce091c49/build/temp.linux-x86_64-3.9/ccompiler_opt_cache_clib.py----------------------------------------ERROR: Command errored out with exit status 1: /home/dataiku/dss/code-envs/python/spec39/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-khr0znww/scipy_5f009fe13e1d4136bd09e356ce091c49/setup.py'"'"'; __file__='"'"'/tmp/pip-install-khr0znww/scipy_5f009fe13e1d4136bd09e356ce091c49/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-xide3inm/install-record.txt --single-version-externally-managed --compile --install-headers /home/dataiku/dss/code-envs/python/spec39/include/site/python3.9/scipy Check the logs for full command output.WARNING: You are using pip version 21.3.1; however, version 22.0.4 is available.You should consider upgrading via the '/home/dataiku/dss/code-envs/python/spec39/bin/python -m pip install --upgrade pip' command.Running setup.py install for scipy: finished with status 'error'


Any help or feedback will be appreciated. Receive the same error on Python 3.8 too. However, working fine on Python 3.6.

Thanks

Operating system used: AlmaLinux

Tagged:

Best Answer

  • gnaldi62
    gnaldi62 Partner, L2 Designer, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Frontrunner 2022 Participant, Neuron 2023 Posts: 79 Neuron
    Answer ✓
    Options

    Hi,

    did you install also the python-devel on your machine ? It seems you're missing the python development package. The way and the name of the package to install depends on the Linux version but should be something like python3-dev, python3-devel

    Also, have you installed the code-environment as a "Non managed Path" ? In that case you should take care of all the dependencies and install them onthe system.

    Rgds.

    Giuseppe

Answers

  • asadmasad
    asadmasad Registered Posts: 4 ✭✭✭
    Options

    Hi Giuseppe,

    Thanks for the quick revert. I indeed had to install python39-devel package since I only had it for python 3.6 version.

    Thanks heaps for the quick help!

    Asad

Setup Info
    Tags
      Help me…