Pandas v1.0.0 is coming!

Options
Ignacio_Toledo
Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 411 Neuron

Hi,

The release candidate of pandas 1.0 is already online, and it might become the next stable release. What are the plans to move DSS into this new version? Could it be any workaround for test it before it becomes supported by the main python environment?

Thanks!

I.

Best Answer

  • Clément_Stenac
    Clément_Stenac Dataiker, Dataiku DSS Core Designer Posts: 753 Dataiker
    Answer ✓
    Options

    Hi,

    Bringing in new versions of pandas has historically been quite a lot of work because of various internal behavior changes. This is the reason why we're still at 0.23 and not yet at 0.24 which includes a behavior change on integers that would have far-rippling impacts on customer code.

    We are looking into ways to be able to incorporate multiple versions of pandas but cannot announce any ETA at the moment.

    You can create a new code environment, without the "core packages" and put pandas 1.0, but you won't be able to "import dataiku" in it.

Answers

  • Ignacio_Toledo
    Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 411 Neuron
    Options

    Thanks Clément for the quick reply. I understand that is going to be difficult, but I'm happy to hear you are working towards a way to keep up to date with pandas.

    A little off topic, but also related to Dataiku future plans: will you try to use jupyter labs in the future?

    Thanks again!

    Ignacio.

  • Clément_Stenac
    Clément_Stenac Dataiker, Dataiku DSS Core Designer Posts: 753 Dataiker
    Options

    We're looking into it. It is challenging.

  • casper
    casper Registered Posts: 42 ✭✭✭✭
    Options

    Hi @Clément_Stenac
    , any advice on how I would access my local filesystem Dataiku CSVs without the Dataiku package and lots of troublesome directory work?

  • Suman
    Suman Partner, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Registered Posts: 8 Partner
    Options

    I am facing challenges using Pystan>2.14 & Fbprophet>0.6 due to this older version of Pandas=0.23.4 in DSS-7.0.2.

    As I have tested within DSS-7.0.2, Pystan & Fbprophet works with Pandas1.1.4

  • Ignacio_Toledo
    Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 411 Neuron
    Options

    @Suman
    , pandas 1 started to be supported with DSS 8.0.0 (https://doc.dataiku.com/dss/latest/release_notes/8.0.html#support-for-pandas-1-0). For version 7 you can't have pandas 1 at the same time you have dataiku libraries installed.

Setup Info
    Tags
      Help me…