Number Handling...Please help

[Kris Jacyna]

Is there a way to round numbers to significant figures?

say i want the number 254937 to be rounded to 2.s.f to 250000 ?

Thanks in advance,

Kris

 

[JohnH]

Const significantFigure As Integer = 1000
Dim X As Integer = 254937
Dim rounded As Integer = CInt(X / significantFigure) * significantFigure

 

[Kris Jacyna]

fantasic,
thanks john

 

[cjard]

John.. Does that round, or floor?

24999 / 1000 = 24.999

in c#, casting that to an int would leave 24 (hence 24999 to 2 sf via this method would be 24000).
Does CInt round, or cast? More significantly, if CInt is a wrapper for Convert.ToInt32 - this apparently uses banker's rounding. Double ugh!

 

[cjard]

Ahh.. Just googled it and yep, apparently CInt rounds, in addition to conv.i4, whereas a C# cast will just do conv.i4

http://www.thescripts.com/forum/threadnav377806-2-10.html

yuck! Though at least CInt rounds normally, unlike Convert.ToInt32.. Too many things to bear in mind! (Or amybe I'm just thinking of this from a C# point of view.. I'd have preferred DirectCast(double, integer) to be allowed)

Here's an additional, how to calculate John's significantFigure (i've called this "divisor") from the operand:

    Public Shared Function ToSigFig(ByVal val As Integer, ByVal sigFig As Integer) As Integer
        Dim divisor As Integer = CInt(Math.Pow(10, Math.Floor(Math.Log(val, 10)) - sigFig + 1))
 
        Return CInt(val / divisor) * divisor
    End Function

 

[JohnH]

When I ran the code CInt rounded. Actually I was a bit surprised, because I associated it with "integer division" and "\" operator which floors/trucates, but CInt did round like Math.Round.