Cleanup up session using Lots of diskspace.

tgb417
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,598 Neuron

I've been using DSS since V3 or V4 on my little Macbook Air.

The computer is running out of disk space.

As I've done some investigations I've discovered that the

DSS_HOME/analytics_date is using 12.5 GB. I'd like to get a bunch of this space back.

of which in the analytics-data directory there are two older projects each eating up GB of disk space.

These projects were likely created or updated in versions V5 and V6

I'd like to selectively understand where the space is being used and cleanup.

Thoughts?

Answers

  • Clément_Stenac
    Clément_Stenac Dataiker, Dataiku DSS Core Designer, Registered Posts: 753 Dataiker

    Hi,

    Are these projects that you are not using anymore ? In this case, you can simply remove these folders.

    Else, you'll need to assess for each analysis whether you still need the trained models and saved splits data. You can remove them either directly from the UI or by removing the corresponding folders.

    In analysis-data, within each project's folder, thre is one project per visual analysis, and the id matches the id of the analysis visible in the URL in the DSS UI.

  • tgb417
    tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,598 Neuron

    @Clément_Stenac

    First thanks so much for the insights. I hope you are well.

    You insights have made it clear which components in the visual interface connect with stuff stored in the data directories.

    I'm intending to use the projects again.

    Over the years, I've deleted a number of the sessions from one of my larger projects.

    For example, in this project/visual-analysis there is only 1 session with 3 models in the client.

    Visual Analytics .jpg

    However when I look into the two directories (splits & sessions). There seem to be 115 split files even though I seem to have only 3 models left in the UI. I think that I don't understand how when these are being created.

    splits.jpg

    And in the Session Folder, there are lots of sessions hanging out. All which I think I've deleted from the UI. Some of which are taking a fair bit of space.

    sessions.jpg

    Have I run into a bug or missing feature in an earlier version?

    What would I lose if I deleted session directories s1 - s80?

    Thanks for your help.

Setup Info
    Tags
      Help me…