decimal values

[davide128]

Hello .

I'm trying to limit the number of decimal values to 3 in numbers such
as .3125 but I also have numbers such as .5 which I need to leave as is.

how can I determine if a number is 4 decimal places and cut off the last number.. I don't want to round up either.. .3125 should be .312 not .313 and .5 should stay at .5 and not .500. any help is appreciated..

 

[MattP]

This should do it for you:

        Dim d1 As Double = 0.3125
        Dim d2 As Double = 0.5
        d1 = Math.Round(d1 - 0.5 / Math.Pow(10, 3), 3)
        d2 = Math.Round(d2 - 0.5 / Math.Pow(10, 3), 3)

You may want to make an extension method to shorten your code and make it a bit more legible. Here's a quick example.

Imports System.Runtime.CompilerServices

Public Class Form1

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        Dim d1 As Double = 0.3125
        Dim d2 As Double = 0.5
        d1.Truncate(3)
        d2.Truncate(3)
    End Sub
End Class

Friend Module MathHelpers
    <Extension()> _
    Public Sub Truncate(ByRef theDouble As Double, ByVal numDecimalPlaces As Integer)
        theDouble = Math.Round(theDouble - 0.5 / Math.Pow(10, numDecimalPlaces), numDecimalPlaces)
    End Sub
End Module

 

[davide128]

what's the minus 0.5 doing (d1 - 0.5 )? can you explain ur logic here

 

[MattP]

It's .5 / 10^x that you're subtracting. In your case you want to keep 3 decimal places so .5/10^3 becomes .0005.

When you subtract that from your number and then round it makes sure that you're always rounding down.

.3129 --> .3124
.5 --> .4995
.3121 --> .3116

As you can see by offsetting the number before rounding .3124 will round to .312 while .3129 would have rounded to .313. Numbers that originally rounded down already such as .3121 will instead round up to the same number.

The nice thing about that function is that if you decide in the future you need to truncate at another decimal place you just provide a different number of decimal places you want to keep and it will work.

 

[davide128]

interesting..I like that..I took more of a string approach but I'm thinking of changing it to yours

     If (System.Convert.ToString(decGap).Length - System.Convert.ToString(decGap).IndexOf(".")) > 4 Then ' greater than 4 because of Zero before decimal 0.3125
                        decGap = System.Convert.ToDecimal(Mid(System.Convert.ToString(decGap), 1, System.Convert.ToString(decGap).Length - 1))
                    End If

 

[davide128]

hmm.getting .624 when using .625

 

[MattP]

I would rather use the math functions than converting it to string 4x, performing a VB6 Mid function (substring would be better) then converting the result back to decimal.

If you decide you're going to do it that way do the string conversion once, use substring to take the character off the end, then convert it back.

        Dim decGap As Double = 0.3125
        Dim decGapString As String = decGap.ToString
        If decGapString.Length - decGapString.IndexOf("0") > 4 Then
            decGap = CDbl(decGapString.Substring(0, decGapString.Length - 1))
        End If

The code I provided runs fine if you get input of a different length than you suspect.

0.3124999 becomes 0.312 using Math.Round while it becomes 0.312499 using substring/mid.

 

[davide128]

I agree about using math functions but it's not working for me


Dim testGap As Double = Math.Round(0.625 - 0.5 / Math.Pow(10, 3), 3)

returns 0.624

 

[Solitaire]

Try the following: No strings involved and very simple:

		Dim dnum As Double = 0.345678
		dnum = dnum * 1000
		dnum = Math.Floor(dnum)
		dnum = dnum / 1000

Or in a single line:

dnum = (Math.Floor(dnum * 1000)) / 1000

 

[MattP]

I agree about using math functions but it's not working for me


Dim testGap As Double = Math.Round(0.625 - 0.5 / Math.Pow(10, 3), 3)

returns 0.624

Oops Math.Round defaults that if a number is midway between 2 numbers it will return the even one. Change the function to this and it will work for you.

Friend Module MathHelpers
    <Extension()> _
    Public Sub Truncate(ByRef theDouble As Double, ByVal numDecimalPlaces As Integer)
        theDouble = Math.Round(theDouble - 0.5 / Math.Pow(10, numDecimalPlaces), numDecimalPlaces, MidpointRounding.AwayFromZero)
    End Sub
End Module

 

[davide128]

Thx..Both very helpful...both ways work.. so I should get rid of MID and
use substring? not in this case But i',m using MID in othe parts of my code..Is MID not good?

 

[MattP]

Mid is an legacy VB6 function kept in the language to help lower the learning curve for developers just upgrading. Substring is the .NET way to do things.

 

[Solitaire]

Note: The code I posted above will work perfectly with any number according to the requirements. Here is it again in a single line; no need to revert to strings:

		Dim dnum As Double
		dnum = (Math.Floor(dnum * 1000)) / 1000

 

[JuggaloBrotha]

How about you guys make this a function that accepts 1 double and spits out another double (do the math on the incoming number and return the result)

 

[davide128]

you mentioned to use substr instead of mid. I went to microsoft website to finds out how to use and can't find that fuction..Am I looking in the wrong place? I'm looking here:

String Functions (Visual Basic)
isn't there a link that has a complete reference?