FIX: Rounding Error When Saving Calculated Value into Integer Field (299341)


The information in this article applies to:

  • Microsoft Visual FoxPro for Windows 3.0
  • Microsoft Visual FoxPro for Windows 3.0b
  • Microsoft Visual FoxPro for Windows 5.0
  • Microsoft Visual FoxPro for Windows 5.0a
  • Microsoft Visual FoxPro for Windows 6.0
This article was previously published under Q299341

SYMPTOMS

If a calculated value is written into an integer field in a table, the value may be rounded improperly and stored incorrectly. For example, if X = 2.05 * 100, when X is added to an integer field, the actual value stored to the table is 204.

STATUS

This problem was corrected in Microsoft Visual FoxPro version 7.0 for Windows.

MORE INFORMATION

Steps to Reproduce Behavior

  1. Copy the following code to a new program:
    CLEAR
LOCAL lnCalcedVal
lnCalcedVal = 2.05 * 100

CREATE CURSOR TstCurs (iFLD1 i)
INSERT INTO TstCurs VALUES(lnCalcedVal)
? "2.05 * 100"
? "----------------"
? "Calculated Value:",lnCalcedVal
? "Value inserted into table:",TstCurs.iFLD1

USE IN TstCurs
  2. Save and run the code (the program name does not matter). When the code runs, 2.05 * 100 is stored to a local variable, lnCalcedVal. This variable is then inserted into an integer field in a cursor. The output on the screen shows the value of lnCalcedVal and the contents of the cursor's integer field. Note that the two values are different.
(c) Microsoft Corporation 2001, All Rights Reserved. Contributions by Trevor Hancock, Microsoft Corporation.

Modification Type:MajorLast Reviewed:5/12/2003
Keywords:kbBug kbDatabase kbfix kbvfp700fix KB299341
©2002 Microsoft Corporation. All rights reserved.