Missing dependencies for SOCKS support

UserBird

When I try to install Dataiku on Ubuntu, I get the error "Missing dependencies for SOCKS support" during installation. I have already run the script to install dependencies, but this error occurs later during the installation, when the python virtualenv is started.

(ran script "sudo -i "/home/CloudUser/DATAIKU/dataiku-dss-3.1.4/scripts/install/install-deps.sh" -without-java")

pbertin

Hi,

this is not a known error, and DSS itself does not use SOCKS.

Could you send more information so that we can reproduce the issue (like the version of Ubuntu, the command sequence you typed to get there, the output of the command which shows the error, and maybe any information about your setup involving SOCKS?)

With best regards
Patrice Bertin
Dataiku

KjellK

I am running Ubuntu 14.04 LTS. Below is the command and the output:

./installer.sh -d ../dataiku-config -p 10000

*********************************************
* Dataiku DSS installer *
*********************************************
[!] *****************************************************
[!] ../dataiku-config contains a previously failed install of DSS
[!] Moving it out of the way and proceeding
[!] *****************************************************
[+] Creating data directory: ../dataiku-config
[+] Saving installation log to /home/CORPUSERS/23010473/dataiku/dataiku-config/run/install.log
[+] Using Java at /usr/bin/java : java version "1.7.0_121"
[+] Checking required dependencies
+ Detected OS distribution : ubuntu 14.04
+ Checking required packages...
[+] Installation starting
[+] Initializing Python environment
Using virtualenv in /tmp/dku_virtualenv.LFG6UR
New python executable in /home/CORPUSERS/23010473/dataiku/dataiku-config/pyenv/bin/python2.7
Also creating executable in /home/CORPUSERS/23010473/dataiku/dataiku-config/pyenv/bin/python
Installing setuptools, pip, wheel...
Complete output from command /home/CORPUSERS/2301.../pyenv/bin/python2.7 - setuptools pip wheel:
Collecting setuptools
Exception:
Traceback (most recent call last):
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/commands/install.py", line 324, in run
requirement_set.prepare_files(finder)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_set.py", line 554, in _prepare_file
require_hashes
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_install.py", line 278, in populate_link
self.link = finder.find_requirement(self, upgrade)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 465, in find_requirement
all_candidates = self.find_all_candidates(req.name)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 423, in find_all_candidates
for page in self._get_pages(url_locations, project_name):
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 568, in _get_pages
page = self._get_page(location)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 683, in _get_page
return HTMLPage.get_page(link, session=self.session)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 792, in get_page
"Cache-Control": "max-age=600",
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/sessions.py", line 488, in get
return self.request('GET', url, **kwargs)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/download.py", line 386, in request
return super(PipSession, self).request(method, url, *args, **kwargs)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/sessions.py", line 475, in request
resp = self.send(prep, **send_kwargs)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/sessions.py", line 596, in send
r = adapter.send(request, **kwargs)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/cachecontrol/adapter.py", line 47, in send
resp = super(CacheControlAdapter, self).send(request, **kw)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/adapters.py", line 390, in send
conn = self.get_connection(request.url, proxies)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/adapters.py", line 290, in get_connection
proxy_manager = self.proxy_manager_for(proxy)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/adapters.py", line 184, in proxy_manager_for
**proxy_kwargs
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv_support/pip-9.0.1-py2.py3-none-any.whl/pip/_vendor/requests/adapters.py", line 43, in SOCKSProxyManager
raise InvalidSchema("Missing dependencies for SOCKS support.")
InvalidSchema: Missing dependencies for SOCKS support.
----------------------------------------
...Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv.py", line 2328, in
main()
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv.py", line 713, in main
symlink=options.symlink)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv.py", line 945, in create_environment
download=download,
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv.py", line 901, in install_wheel
call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
File "/tmp/dku_virtualenv.LFG6UR/virtualenv/virtualenv.py", line 797, in call_subprocess
% (cmd_desc, proc.returncode))
OSError: Command /home/CORPUSERS/2301.../pyenv/bin/python2.7 - setuptools pip wheel failed with error code 2
[-] Error: could not initialize Python virtualenv in /home/CORPUSERS/23010473/dataiku/dataiku-config/pyenv

pbertin

There indeed seems to be bad interactions between pip and SOCKS
http://stackoverflow.com/questions/40926018/pip-install-doesnt-work-invalidschema-missing-dependencies-for-socks-support

I will try to reproduce the issue and find a workaround. Probably not before tonight or tomorrow though. In the meantime, maybe you can try removing all proxy-related variables from your remote session environment for the time of install. Then it would be possible to add extra Python packages to the virtualenv if needed
(just guessing for now)

-Patrice Bertin

KjellK

Yes, I found that many other appears to have similar problems when running virtualenv. I have tried some of the workarounds suggested (unset ALL_PROXY and then try to install socks), but so far nothing have worked for me. It seems I do not understand enough of how virtualenv works and how that relates to the global context, to resolve the issue.

pbertin

Hi,

I could indeed reproduce this problem. It comes from the fact that by default, virtualenv tries to connect to the Internet to get the most up-to-date versions of its initial packages when it creates a new virtualenv, but the version of the http code built-in to virtualenv itself lacks support for SOCKS proxies.

There should be two solutions around that:

- starting with DSS 4.0.3, we have added the option which disables this default behaviour. If you can upgrade to this DSS version, the problem should not show up

- otherwise, you should disable the socks proxy for the session which runs the DSS installer. For this:
. start by checking which proxy environment variables you have defined with: "env | grep -i proxy"
. unset all of the above with "unset http_proxy" [ replace with the actual name of the variable as found above, could also be uppercase or targeting the https protocol ]
. run the DSS installer again, which should not trigger this error

Hope this helps
Patrice Bertin
Dataiku