Importing a code env misses some dependencies

tomtom
tomtom Registered Posts: 21 ✭✭✭✭
Hello,

I plan to publish a ready-to-use code env, so I exported it from my DSS instance.

In the exported code env zip, the "python\actual\requirements.txt" contains :

```

backcall==0.1.0
beautifulsoup4==4.7.1
certifi==2019.3.9
chardet==3.0.4
decorator==4.3.0
Deprecated==1.2.5
idna==2.7
ipykernel==4.8.2
ipython==7.4.0
ipython-genutils==0.2.0
jedi==0.13.3
jupyter-client==5.2.3
jupyter-core==4.4.0
lxml==4.3.3
markdownify==0.4.1
numpy==1.16.2
pandas==0.23.4
parso==0.3.4
pexpect==4.6.0
pickleshare==0.7.5
prompt-toolkit==2.0.9
ptyprocess==0.6.0
PyGithub==1.43.6
Pygments==2.3.1
PyJWT==1.7.1
python-dateutil==2.7.4
pytz==2018.6
pyzmq==17.1.2
requests==2.20.1
simplegeneric==0.8.1
six==1.12.0
soupsieve==1.9
tornado==5.1.1
traitlets==4.3.2
urllib3==1.24.1
wcwidth==0.1.7
wrapt==1.11.1

```



But after importing this code env in a new DSS instance, my specific deps (lxml, beautifulsoup4, markdownify and pygithub) are not installed....

(if you want to try my code env : https://ufile.io/zip59u9l)

Can you fix this please ? because as is, this useful feature is not usable...



Best regards.

Answers

  • Clément_Stenac
    Clément_Stenac Dataiker, Dataiku DSS Core Designer, Registered Posts: 753 Dataiker
    Hi,

    Because it is not generally possible nor even desirable to reproduce the exact same code environment (think of bugfixes and security updates for example), when exporting/importing a code environment, DSS uses the "requested" state of the packages, not the "current" state.

    The concept of code envs is that you give a list of packages with optional version constraints that you want installed. When you click update, Pip installs the missing packages so that your requirements are matched.

    Here, your requirements are only "PyGithub". In other words, you don't require your code env to have lxml, bs4 or markdownify, so DSS won't ask pip to install them. Add them to the "Requested packages" list before exporting.
  • tomtom
    tomtom Registered Posts: 21 ✭✭✭✭
    Thanks Clément, I totally missed the "Required packages" option.

    But even after specifying my 4 packages on "Required" before exporting, after the import phase I still have to go into the imported code-env and click on "Update" to have them installed. I would really think that everything (including required) would be installed seamlessly when I import.

    This is a pain and it is really counter-intuitive : if I give to someone my custom code-env, he would think that everything is already pre-packaged without the need to click on something _after_ the import...

    And overall, I understand the "Required packages" design, but again, if I build a custom code-env, it is because the native Python env is missing some of my wanted packages, there is no need for "custom but not mandatory" stuff (at least for me, I don't need distinction (required or not) for a custom code-env)
Setup Info
    Tags
      Help me…