Remove old Versioned Environments and Kernels after importing bundle

marawan
marawan Partner, Registered Posts: 19 Partner

Hello, I noticed that when importing bundles to an automation node, environments are versioned and all old kernels are available for use in Jupyter notebooks, along with all the old environments.

versioned_jupyter_kernels.PNGversioned_envs.PNG

Now I understand that this versioning is done so that if projects/bundles which are using the same environment name can continue to function even when that environment is updated. However, short of me going in through the shell and removing those old versions myself, I don't see a way in the UI to delete old versions of code environments, even ones that are not referenced by any bundles / projects

delete_env_version.PNG

The same also applies for old jupyter kernels.

Now, depending on the content of the environment, a single version can easily exceed 1.5GB for our projects (when we use deep learning libraries), so on the long run, this can become a problem when operating the automation node. Old Jupyter kernels are more of a nuisance, but it would still be nice to get rid of the old unused ones.

Is there a way through the UI or the SDK to delete old versions? I have tried to use set_definition on the DSSCodeEnv object representing the automation node env, but this function cannot modify the contents of the versions array.

Best Answer

  • Clément_Stenac
    Clément_Stenac Dataiker, Dataiku DSS Core Designer, Registered Posts: 753 Dataiker
    Answer ✓

    Hi,

    Unfortunately, there is no API method for this, so you'll indeed need to do it via command-line. We'll take note of your request to inform future developments.

Answers

Setup Info
    Tags
      Help me…