 |
BorlandTalk.com
Borland discussion newsgroups
|
| View previous topic :: View next topic |
| Author |
Message |
Olivier Sauterel
Guest
|
 Posted: Tue Feb 24, 2004 2:25 pm Post subject: Precision rounding |
 |
|
Hi,
I' ve some problem with my round routine, because in test "tmp >= precision
/ 2" does not work in sample:
RoundMultiple(0.725, 0.01)
It will return 0.72 because tmp after Modulo represent 0.00499999999996 and
should
be 0.005 ! And i think the problem is the same with expression "precision /
2".
I've already try to set the PrecisionMode with pmDouble, and it will work in
some case, but not everytime ...
Anyone can help me and maybe explain me why i've this problem ?
Thanks
Olivier
---------------------------------------------------------------
function RoundMultiple(value, precision: Double): Double;
var
tmp: Double;
begin
tmp := Modulo(value, precision);
if tmp <> 0 then
begin
result := value - tmp;
if tmp >= precision / 2 then
result := result + precision;
end
else
result := value;
end;
---------------------------------------------------------------
function Modulo(value, divisor: Double): Double;
begin
result := value - Trunc(value / divisor) * divisor;
end;
---------------------------------------------------------------
|
|
| Back to top |
|
 |
John Herbster (TeamB)
Guest
|
| Posted: Tue Feb 24, 2004 9:25 pm Post subject: Re: Precision rounding |
|
|
"Olivier Sauterel" <sauterel (AT) ioware (DOT) ch> wrote
| Quote: | I've some problem with my round routine, ...
|
Oliver, Did you mean to post in a database group?
This does not seem like a data base problem. It
probably should have been in the *.language.general
group. Nevertheless about your RoundMultiple():
Am I guessing correctly that RoundMultiple(v,p) is
supposed to return a value r such that
"r = n*p", where n is chosen to minimize the absolute
difference of "r - v"?
If so, then I would define RoundMultiple by
function RoundMultiple(v, p: double): double;
var n: int64;
begin
n := round(v/p);
Result := n*p;
end;
If that is not what RoudMultiple is supposed to do,
then please ask again. Regards, JohnH
|
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
