Missing dependencies for SOCKS support

Options
UserBird
UserBird Dataiker, Alpha Tester Posts: 535 Dataiker
When I try to install Dataiku on Ubuntu, I get the error "Missing dependencies for SOCKS support" during installation. I have already run the script to install dependencies, but this error occurs later during the installation, when the python virtualenv is started.

(ran script "sudo -i "/home/CloudUser/DATAIKU/dataiku-dss-3.1.4/scripts/install/install-deps.sh" -without-java")
Tagged:

Answers

  • pbertin
    pbertin Dataiker Posts: 27 Dataiker
    Options
    Hi,

    this is not a known error, and DSS itself does not use SOCKS.

    Could you send more information so that we can reproduce the issue (like the version of Ubuntu, the command sequence you typed to get there, the output of the command which shows the error, and maybe any information about your setup involving SOCKS?)

    With best regards
    Patrice Bertin
    Dataiku
  • KjellK
    KjellK Registered Posts: 15 ✭✭✭✭
    Options
    I am running Ubuntu 14.04 LTS. Below is the command and the output:

    ./installer.sh -d ../dataiku-config -p 10000

    *********************************************
    * Dataiku DSS installer *
    *********************************************
    [!] *****************************************************
    [!] ../dataiku-config contains a previously failed install of DSS
    [!] Moving it out of the way and proceeding
    [!] *****************************************************
    [+] Creating data directory: ../dataiku-config
    [+] Saving installation log to /home/CORPUSERS/23010473/dataiku/dataiku-config/run/install.log
    [+] Using Java at /usr/bin/java : java version "1.7.0_121"
    [+] Checking required dependencies
    + Detected OS distribution : ubuntu 14.04
    + Checking required packages...
    [+] Installation starting
    [+] Initializing Python environment
    Using virtualenv in /tmp/dku_virtualenv.LFG6UR
    New python executable in /home/CORPUSERS/23010473/dataiku/dataiku-config/pyenv/bin/python2.7
    Also creating executable in /home/CORPUSERS/23010473/dataiku/dataiku-config/pyenv/bin/python
    Installing setuptools, pip, wheel...
    Complete output from command /home/CORPUSERS/2301.../pyenv/bin/python2.7 - setuptools pip wheel:
    Collecting setuptools
    Exception:
    Traceback (most recent call last):
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/commands/install.py", line 324, in run
    requirement_set.prepare_files(finder)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 792, in get_page
    "Cache-Control": "max-age=600",
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/sessions.py", line 488, in get
    return self.request('GET', url, **kwargs)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/download.py", line 386, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/sessions.py", line 596, in send
    r = adapter.send(request, **kwargs)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/cachecontrol/adapter.py", line 47, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/adapters.py", line 390, in send
    conn = self.get_connection(request.url, proxies)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/adapters.py", line 290, in get_connection
    proxy_manager = self.proxy_manager_for(proxy)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/adapters.py", line 184, in proxy_manager_for
    **proxy_kwargs
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/adapters.py", line 43, in SOCKSProxyManager
    raise InvalidSchema("Missing dependencies for SOCKS support.")
    InvalidSchema: Missing dependencies for SOCKS support.
    ----------------------------------------
    ...Installing setuptools, pip, wheel...done.
    Traceback (most recent call last):
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv.py", line 2328, in
    main()
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv.py", line 713, in main
    symlink=options.symlink)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv.py", line 945, in create_environment
    download=download,
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv.py", line 901, in install_wheel
    call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
    File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv.py", line 797, in call_subprocess
    % (cmd_desc, proc.returncode))
    OSError: Command /home/CORPUSERS/2301.../pyenv/bin/python2.7 - setuptools pip wheel failed with error code 2
    [-] Error: could not initialize Python virtualenv in /home/CORPUSERS/23010473/dataiku/dataiku-config/pyenv
  • pbertin
    pbertin Dataiker Posts: 27 Dataiker
    Options
    There indeed seems to be bad interactions between pip and SOCKS
    http://stackoverflow.com/questions/40926018/pip-install-doesnt-work-invalidschema-missing-dependencies-for-socks-support

    I will try to reproduce the issue and find a workaround. Probably not before tonight or tomorrow though. In the meantime, maybe you can try removing all proxy-related variables from your remote session environment for the time of install. Then it would be possible to add extra Python packages to the virtualenv if needed
    (just guessing for now)

    -Patrice Bertin
  • KjellK
    KjellK Registered Posts: 15 ✭✭✭✭
    Options
    Yes, I found that many other appears to have similar problems when running virtualenv. I have tried some of the workarounds suggested (unset ALL_PROXY and then try to install socks), but so far nothing have worked for me. It seems I do not understand enough of how virtualenv works and how that relates to the global context, to resolve the issue.
  • pbertin
    pbertin Dataiker Posts: 27 Dataiker
    Options
    Hi,

    I could indeed reproduce this problem. It comes from the fact that by default, virtualenv tries to connect to the Internet to get the most up-to-date versions of its initial packages when it creates a new virtualenv, but the version of the http code built-in to virtualenv itself lacks support for SOCKS proxies.

    There should be two solutions around that:

    - starting with DSS 4.0.3, we have added the option which disables this default behaviour. If you can upgrade to this DSS version, the problem should not show up

    - otherwise, you should disable the socks proxy for the session which runs the DSS installer. For this:
    . start by checking which proxy environment variables you have defined with: "env | grep -i proxy"
    . unset all of the above with "unset http_proxy" [ replace with the actual name of the variable as found above, could also be uppercase or targeting the https protocol ]
    . run the DSS installer again, which should not trigger this error

    Hope this helps
    Patrice Bertin
    Dataiku
Setup Info
    Tags
      Help me…