Float64 failing to write/read?

Dryopes
Dryopes Registered Posts: 1
edited July 16 in Using Dataiku

Hi,

I am working with a database with plenty of float64 operations, and I have trouble comparing these floats which each other when reading/writing from the database. A simple example of this is as follows:

import dataiku
import pandas as pd

df = pd.DataFrame(data = {'a' : [3222.232343243847737]})
db = dataiku.Dataset("test")

db.write_with_schema(df)
df2 = db.get_dataframe()

print(df.values[0][0] == df2.values[0][0])

1 rows successfully written (L26vwn5Fzw)
False

Here it seems that the floating value seem to change in the process of reading/writing the database. Am I doing something wrong, is this expected behavior?

Answers

  • tgb417
    tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,598 Neuron

    @Dryopes
    ,

    Welcome to the Dataiku Community, glad to have you here.

    A while back I ran into something similar. Not clear if it will be your problem or not.

    What was explained to me was a binary encoding rounding error. But I think in my example I had a division.

    See if you put a round() function around things out to say 15 places if the problem resolves.

    That said I’d also encourage you to put in a support ticket. If you have found a defect it would be great to get this in the backlog with the development team.

    Here is a somewhat longer discussion on this topic. https://community.dataiku.com/t5/General-Discussion/Precision-on-Mathematical-formulas-on-a-recipe/td-p/18058

Setup Info
    Tags
      Help me…