Question cannot sum 0.7 with 0.1 !!!

RRdaS

New member
Joined
May 11, 2008
Messages
3
Programming Experience
1-3
Hello guys,

I am using Visual Basic 2008 Express Edition under Windows Vista SP1 and I cannot believe in my eyes. In my program 0.7 + 0.1 is not 0.8, but 0.8000001.

Could someone clarify the whole situation for me? I created a small program to reproduce the problem. I tried to keep the same syntax of the original program I'm working with.

Please see this simple code:

VB.NET:
Public Class Form1
    Private NCursor As Byte
    Private r1, y1, b1, InkQty As Single
    Private CupInkStatus As Boolean

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        NCursor = 2
        CupInkStatus = False
        InkQty = 0.1
        Me.Label1.Text = "r1=" & r1 & "  :  y1=" & y1 & "  :  b1=" & b1
    End Sub

    Private Sub Cup_Ink_MouseDown(ByVal sender As Object, ByVal e As System.Windows.Forms.MouseEventArgs) Handles Cup_Ink.MouseDown
        If Not CupInkStatus Then
            Select Case NCursor
                Case 2
                    r1 = 1
                    y1 = 0
                    b1 = 0
                    CupInkStatus = True
                    NCursor = 3
            End Select
        Else
            Select Case NCursor
                Case 3
                    If y1 < 1 Then
                        y1 = y1 + InkQty
                    End If
            End Select
        End If
        Me.Label1.Text = "r1=" & r1 & "  :  y1=" & y1 & "  :  b1=" & b1
    End Sub

End Class

If you keep clicking the PictureBox, you will see the variable y1 increasing. Instead of grow to 0.8, y1 grows to 0.8000001.

Source code here: http://rapidshare.com/files/114191560/Test.zip.html


What is the explanation?


Thank you very much.

RRdaS
 
Last edited:
The problem is in the internal binary representation of the numbers. Floating points (float and double) are very bad with precision because they translate your number into base 2 numbers. It produces rounding errors just like 1/3 in base 10 does. This is independent from VB.NET and you'll have the same problem with C++, C#, Java, etc.

If you need to calculate bank stuff or important decimal calculation, NEVER use floating points! One thing you can do is to use the Decimal type which (I think it works that way...) simulates a base 10 number and so, will not lead to the same rounding error. In other languages, you would just use an integer and use a fixed point location, as if you used a long to represent the number of cents in the account instead of dollars.

Basically, what I would suggest is to just switch to Decimals instead of Doubles.
 
Back
Top