How can I install latest version of Pandas

NikhilS
NikhilS Dataiku DSS Core Designer, Registered Posts: 1 ✭✭✭

Hi,

In my instance, pandas version 0.23.4 is installed. How can I install the latest pandas version. As its giving me error due to conflict of earlier version

Answers

  • Sergey
    Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker

    Hi @NikhilS

    The main reason we hard-code pandas 0.23.4 is that internal DSS code uses this version and installing other versions might break things.

    If you are on DSS8.0.x you can create a code-env using python3.6 which will automatically install Pandas v1.0.5. If you are on DSS version lower than 8.0.0 there is no way doing this except using non-managed code env or a code environment, without the "core packages" and put pandas 1.0, but you won't be able to "import dataiku" in it.

  • UgoD
    UgoD Registered Posts: 9 ✭✭✭

    Hi @sergeyd

    I just have a question about your answer.

    If I'm using Dataiku 10 and I'm trying to upgrade pandas from 0.23 to 1.0 on a managed code env, is that possible? Or should I recreate a new managed code env?

    I'm struggling to update my env because of a conflict in pandas version.

    Best regards

    Ugo

  • Ignacio_Toledo
    Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 415 Neuron

    Hi @UgoD
    ,

    If your python environment is based on python27, you will not be able to upgrade pandas to 1.0. You need to use python 3.6 or higher

    (With python 3.7 or higher, you can even update pandas to 1.3)

    Cheers

  • UgoD
    UgoD Registered Posts: 9 ✭✭✭
    edited July 17

    Hi @Ignacio_Toledo
    ,

    My env is based on python36, but it's possible that the managed plugin was created on DATAIKU 8.0 or lower.

    If I try with nothing in requirements.txt, I have pandas 0.23.4 installed and if I try with nothing but pandas 1.0.5, I have this conflict

    ERROR: Cannot install pandas==0.23.4 and pandas==1.0.5 because these package versions have conflicting dependencies

    Hope that can help to understand the issue.

    Best regards,

    Ugo

  • Sergey
    Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
    edited July 17

    Hi @UgoD

    If we are talking about the plugin (I assume the development one), you need to update desc.json instead of adding pandas==1.0.5 in the requirements.txt to contain the next line:

        "corePackagesSet": "PANDAS10"

  • UgoD
    UgoD Registered Posts: 9 ✭✭✭

    Hi @sergeyd
    ,

    Thanks for this quick answer !

    When I add this line to desc.json and I remove pandas==1.0.5 in requirements.txt, I still have pandas 0.23.4 download and installed. And if I try without removing pandas==1.0.5 in requirements.txt, I have the same error than previously..

    Ugo

  • Sergey
    Sergey Dataiker, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts Posts: 365 Dataiker
    edited July 17

    Hi @UgoD

    Are you sure you have something like this in the desc.json:

    {
        "acceptedPythonInterpreters": [
            "PYTHON36"
        ],
        "corePackagesSet": "PANDAS10",
        "forceConda": false,
        "installCorePackages": true,
        "installJupyterSupport": true
    }

    The key here is to have python3.6+ as well to have pandas==1.0.5 installed.

Setup Info
    Tags
      Help me…