PRB: Use /Op to Avoid Loss of Precision from Optimization (42761)

The information in this article applies to:

Microsoft C for MS-DOS 6.0
Microsoft C for MS-DOS 6.0a
Microsoft C for MS-DOS 6.0ax
Microsoft C for OS/2 6.0
Microsoft C for OS/2 6.0a
Microsoft C/C++ for MS-DOS 7.0
Microsoft Visual C++ 1.0
Microsoft Visual C++ 1.5

This article was previously published under Q42761

SYMPTOMS

The program below fails when compiled with default optimizations with C 6.x and with any optimizations except /Op under the later versions of the compiler. The program prints the following with optimizations enabled:

   23.31 * 100 = 2330

However, if compiled with /Od, the following is printed:

   23.31 * 100 = 2331

The problem seems to be in the conversion of the double to int; the rounding apparently is done incorrectly.

CAUSE

When the code is optimized, the result of the division is kept on the floating-point stack rather than being stored into memory and reloaded onto the stack. This causes precision problems, because the numbers on the stack are stored in extended (10- byte) precision, while doubles in memory are stored using 8 bytes.

NOTE: The constant 23.31 cannot be represented exactly in the binary floating-point scheme used by Microsoft C. The fact that it is represented slightly small and then multiplied by 100 (magnifying the error) contributes to this problem.

RESOLUTION

To avoid this problem, use the /Op option when compiling. The /Op switch forces in-memory storage of intermediate results. /Op can be combined with other optimizations (for example, /Oxp).

This problem does not occur in Visual C++ 32-bit Edition.

MORE INFORMATION

Sample Code

/* Compile options needed: none
*/ 
#include <stdio.h>
void main( )
{
   int a;
   double f;
   f = 23.31;
   f = f * 100;
   a = (int) f;
   printf( "23.31 * 100 = %d\n", a );
}

Modification Type:	Minor	Last Reviewed:	7/5/2005
Keywords:	kb16bitonly kbprb KB42761