BUG: Incorrect Results When Comparing Large Numbers (189866)



The information in this article applies to:

  • Microsoft Visual FoxPro for Windows 5.0
  • Microsoft Visual FoxPro for Windows 5.0a
  • Microsoft Visual FoxPro for Windows 6.0

This article was previously published under Q189866

SYMPTOMS

When you compare two numbers for equality, you might receive an erroneous result when the resulting numbers are very large.

NOTE: This behavior does not occur in Visual FoxPro 3.0x.

RESOLUTION

Use the ROUND() function when checking the equality of very large floating- point numbers. For example, the following line of code produces the correct result:
   ?ROUND(1+(1048575+(i/1000)),5)=1048576.001
				

STATUS

Microsoft has confirmed this to be a bug in the Microsoft products listed at the beginning of this article. We are researching this bug and will post new information here in the Microsoft Knowledge Base as it becomes available.

MORE INFORMATION

Steps to Reproduce Behavior

Type the following commands in the Command window:
   SET DECIMAL to 5
   i=1
   ?(1+(1048575+(i/1000)))=1048576.001
   ?1+(1048575+(1/1000))=1048576.001
				
Notice that the first calculation, using the variable i, evaluates as false while the calculation using the integer evaluates true.

Modification Type:MajorLast Reviewed:12/11/1999
Keywords:kbbug KB189866