This website uses cookies. By clicking OK, you consent to the use of cookies. Read our cookie policy.

Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

Solved!

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Rounding issues

Hi,

I don't understand why in a prepare recipe, the sum of 2 values rounded at a single decimal can be a number with more than one decimal ?

1 Solution

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

DSS, like many other software, uses floating-point arithmetics (https://en.wikipedia.org/wiki/Floating-point_arithmetic).

One of the main characteristics of this kind of computation is that it uses a fixed amount of memory to represent all numbers, and thus cannot represent numbers with an infinite precision.

Due to details of how these numbers are represented, some rounding issues can always happen when doing computations, even very simple ones, because the result might not be representable exactly.

A quite famous example of this is 0.1 + 0.2, which is not equal to 0.3 in floating point arithmetic but 0.300000000004

You will need to reformat your number, as shown here: http://answers.dataiku.com/1027/change-precision-of-double-number

Solutions shown first - Read whole discussion

1 Reply

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

DSS, like many other software, uses floating-point arithmetics (https://en.wikipedia.org/wiki/Floating-point_arithmetic).

One of the main characteristics of this kind of computation is that it uses a fixed amount of memory to represent all numbers, and thus cannot represent numbers with an infinite precision.

Due to details of how these numbers are represented, some rounding issues can always happen when doing computations, even very simple ones, because the result might not be representable exactly.

A quite famous example of this is 0.1 + 0.2, which is not equal to 0.3 in floating point arithmetic but 0.300000000004

You will need to reformat your number, as shown here: http://answers.dataiku.com/1027/change-precision-of-double-number