FLT_EVAL_METHOD is the modern term. What he calls the Java semantics we now call FLT_EVAL_METHOD==0, and what he calls K&R C we now call FLT_EVAL_METHOD==2. Contrary to that essay, for years now C on the most popular architectures by default follows the Java semantics.
His argument is pretty much "hey, you lose precision when you evaluate in fewer bits". Well, yeah, of course you do. The obvious counterargument is "if you need long double, write 'long double'; don't count on the compiler to automatically widen in expressions". The minor precision benefits in code that was fragile to begin with if written in C are by no means worth the giant refactoring hazard of "introducing temporaries can break your code".
> It's: if the programmer doesn't specify what the sizes of the calculations in the expressions are, do these calculations with the biggest sizes (but predictably!), and if he specifies, respect what he specified(1).
The programmer is specifying. When you say "tan(x)" with x as a float, then most programmers would expect to get a float out. In Haskell speak, people expect the type of "tan" to be "Floating a => a -> a" (which is, unsurprisingly, exactly what the type of "tan" is in Haskell [1]). Why? Because it's consistent with the rest of the language: everyone knows that x/y with x and y both ints yields an int, not a double. Likewise, x/y with x and y both floats intuitively yields a float, not a double (and with FLT_EVAL_METHOD==0, the norm in most cases nowadays, this is in fact how it works).
Saying that FLT_EVAL_METHOD==0 robs you of the ability to choose to evaluate in high precision makes no more sense than saying that the way that x/y truncates the remainder robs you of the ability to do floating point division. As all C programmers know, if you want a fractional result from integer division, you write (double)x/y. Likewise, if you want the double-precision tangent of the single-precision x, you would expect to write tan((double)x).
There is a slightly stronger argument that Java doesn't expose access to the 80-bit precision in the x86 FPU. That's true. But the right solution to that is to introduce "long double" in Java (which is what C did), not to complicate the semantics. An implementation of 80-bit floating point that doesn't allow you to store an 80-bit value in a variable is pretty much a toy implementation no matter what. You will need first-class "long double" in order to claim to have true 80-bit support, and once you have it, you have no need for FLT_EVAL_METHOD==2.
> Saying that FLT_EVAL_METHOD==0 robs you of the ability to choose
Kahan never defined something like FLT_EVAL_METHOD==0, he wrote the PDF you linked to, and the PDF doesn't contain claims that in the form you say it does, and I'm not making them either. He says in his suggestion to "fix" Java, among other things: calculate the partial results which aren't specified explicitly to higher precision, the target sizes should be respected as the programmers specified them. That is not what the example program produced with -O flag. And I don't see anywhere that what Kahan suggested has equivalence to
"FLT_EVAL_METHOD==2 all operations and constants evaluate in the range and precision of long double. Additionally, both float_t and double_t are equivalent to long double"
He never mentions something like "float_t" and "double_t" there as far as I saw? He mentions K&R C which never had any of these. And where float x = something meant x is 32-bits. And tan( x ) meant pass 32-bit x to the double tan and return double, and float y = tan( x ) meant store the double returned from tan as 32-bits.
The program in the example that started the discussion is explicitly producing two double sums (that is, that's what the programmer wrote) so it should never compare one 80-bit and one 64-bit sum. That behavior is surely not something that you can find in Kahan's suggestions.
Please read once Kahan's document carefully to see if he writes what you believe he writes (because you keep referring to the definition he didn't make, and he explicitly knew that not "everything should be long double"). I claim he doesn't.
> An implementation of 80-bit floating point that doesn't allow you to store an 80-bit value in a variable is pretty much a toy implementation no matter what.
x87 instructions have it: FSTP m80fp and Kahan wanted that in the higher level languages too (pg. 77).
"Names for primitive floating-point types or for Borneo floating-point classes:"
"long double = 10+-byte IEEE 754 Double Extended with at least 64 sig. bits etc."
He also writes (pg. 43 of his PDF):
"Of course explicit casts and
assignments to a narrower precision must round superfluous digits away as the programmer directs"
And on page 80, regarding false arguments:
"At first sight, a frequently occurring assignment X = Y¤Z involving
floats X, Y, Z in just one algebraic operation ¤ appears to require that Y and Z be converted to double, and
that Y¤Z be computed and rounded to double, and then rounded again to float to be stored in X . The same
result X ( and the same exceptions if any ) are obtained sooner by rounding Y¤Z to float directly." That is, the standard and the x87 are already designed that it can be done directly.
> "calculate the partial results which aren't specified explicitly to higher precision"
I understand the paper. That is exactly what I'm arguing is the wrong behavior, and C compilers have stopped doing it on popular hardware too.
"Partial results which aren't specified explicitly" is really misleading terminology in a typed language. Given int x and int y, the type of "x/y" is specified explicitly given the typing rules of the language. And that's why programmers rightly expect that "int x = 5, y = 2; double z = x/y;" yields z=2.0, not z=2.5.
> "At first sight, a frequently occurring assignment X = Y¤Z involving floats X, Y, Z in just one algebraic operation ¤ appears to require that Y and Z be converted to double, and that Y¤Z be computed and rounded to double, and then rounded again to float to be stored in X . The same result X ( and the same exceptions if any ) are obtained sooner by rounding Y¤Z to float directly."
Yes, the unintuitive FLT_EVAL_METHOD==2 semantics can be optimized into the less confusing FLT_EVAL_METHOD==0 semantics in many cases. That doesn't change the fact that they're confusing semantics.
> Given int x and int y, the type of "x/y" is specified explicitly given the typing rules of the language. And that's why programmers rightly expect that "int x = 5, y = 2; double z = x/y;" yields z=2.0, not z=2.5
Red herring. Kahan advocated the approach of K&R C. K&R C didn't perform the division of two integers in floating
point. Your example would be 2 in K&R C too.
> the unintuitive FLT_EVAL_METHOD==2 semantics can be optimized into the less confusing FLT_EVAL_METHOD==0 semantics in many cases.
The example given is not "optimization" at all, it's required by the definition of IEEE 754, mostly designed by Kahan.
> That doesn't change the fact that they're confusing semantics.
I see that as confusion from you repeatedly misdirecting the discussion to the "FLT_EVAL_METHOD" which is neither what Kahan wrote in his Java paper, nor what's in IEEE 754 and even less in K&R C.
Disclaimer: I actually studied IEEE 754 shortly after it was standardized, and followed Kahan's work afterwards. The first C I've learned was K&R C, directly from their first book. I've implemented some hardware doing some IEEE 754 operations completely to the letter of the standard (to the last bit, rounding direction and trap, the control of rounding directions and "meaningful" trap handling is also what Kahan complained that Java didn't do) and implemented some compilers that used x87.
> Red herring. Kahan advocated the approach of K&R C. K&R C didn't perform the division of two integers in floating point. Your example would be 2 in K&R C too.
You missed the point. Kahan's proposed semantics are inconsistent with what programmers expect. The 5/2 example is intended to show why programmers expect Java-like semantics.
> The example given is not "optimization" at all, it's required by the definition of IEEE 754, mostly designed by Kahan.
How does an optimization being guaranteed to be correct (which all optimizations sure ought to be!) make it not an optimization?
> I see that as confusion from you repeatedly misdirecting the discussion to the "FLT_EVAL_METHOD" which is neither what Kahan wrote in his Java paper, nor what's in IEEE 754 and even less in K&R C.
We're talking past each other at this point. FLT_EVAL_METHOD is precisely what Kahan is talking about in the Java paper: it is the term introduced in the C99 spec for the precision that intermediate floating-point computations are performed in. The difference between Java and K&R C is that Java uses FLT_EVAL_METHOD==0, while K&R C uses FLT_EVAL_METHOD==1. I don't know how to make that clearer.
I can imagine that is "confusing" to you but "what programmers expect" is not what the people who talk about "spills" (like you did) expect. I understand where you come from when you mention them. But spills etc are all irrelevant to how we started this discussion:
As cpreciva stated that Kahan said that the behavior of the given example (the code where two sums of two some numbers are summed to doubles and compared to equality) should never give "false" you wrote "it seems to contradict his attitude ..." and your "proof" was Kahan's Java paper.
I showed that there's nowhere anything in Kahan's Java paper or K&R that would support such behavior. I quoted explicit lines form Kahan's paper which explicitly claim that the explicit evaluation to doubles should properly round away the bits that don't fit. And you continued to claim "but but FLT_EVAL_METHOD" whatever.
> We're talking past each other at this point.
I agree about that. You also gave false example 5/2 and you now explain it was your "intention" even if it's neither K&R nor what Kahan argued. You also use the name of the "optimization" for the basic design property of IEEE 754 operations, probably the best feature of IEEE 754 compared to all "common practices" of the times it was designed. I'd be glad to explain you what it is, you'd actually have to make the effort to understand the design principles of IEEE 754. If you're really interested, please post some place where we could start a new thread for that topic, I won't post in this thread more. There we can also discuss, if you are really interested, what Kahan actually says in the paper you posted.
The integer example is not a red herring - it's an argument to consistency by comparison with an isomorphic evaluation semantics.
If you look at it closely though, you might find material for your position in integer promotion. But I think you're missing the point of what is being argued via a vis refactoring of expression trees changing calculation results.
> the point of what is being argued via a vis refactoring of expression trees changing calculation results
That was not a topic why the discussion started and I never discussed that here. Check for yourself. It started with pcwalton's (I simplify) "the bug is because of Kahan, see his Java paper" and I showed the opposite: whoever reads Kahan's paper can see that following his suggestions in the paper the bug can only be a bug. I also cited the exact pages.
I know there are a lot of people who would like to handle FP arithmetic identically to the integer one, but that can't work by the definition of FP. And please see my other post here -- it's surely time to continue the discussion somewhere else.
His argument is pretty much "hey, you lose precision when you evaluate in fewer bits". Well, yeah, of course you do. The obvious counterargument is "if you need long double, write 'long double'; don't count on the compiler to automatically widen in expressions". The minor precision benefits in code that was fragile to begin with if written in C are by no means worth the giant refactoring hazard of "introducing temporaries can break your code".
> It's: if the programmer doesn't specify what the sizes of the calculations in the expressions are, do these calculations with the biggest sizes (but predictably!), and if he specifies, respect what he specified(1).
The programmer is specifying. When you say "tan(x)" with x as a float, then most programmers would expect to get a float out. In Haskell speak, people expect the type of "tan" to be "Floating a => a -> a" (which is, unsurprisingly, exactly what the type of "tan" is in Haskell [1]). Why? Because it's consistent with the rest of the language: everyone knows that x/y with x and y both ints yields an int, not a double. Likewise, x/y with x and y both floats intuitively yields a float, not a double (and with FLT_EVAL_METHOD==0, the norm in most cases nowadays, this is in fact how it works).
Saying that FLT_EVAL_METHOD==0 robs you of the ability to choose to evaluate in high precision makes no more sense than saying that the way that x/y truncates the remainder robs you of the ability to do floating point division. As all C programmers know, if you want a fractional result from integer division, you write (double)x/y. Likewise, if you want the double-precision tangent of the single-precision x, you would expect to write tan((double)x).
There is a slightly stronger argument that Java doesn't expose access to the 80-bit precision in the x86 FPU. That's true. But the right solution to that is to introduce "long double" in Java (which is what C did), not to complicate the semantics. An implementation of 80-bit floating point that doesn't allow you to store an 80-bit value in a variable is pretty much a toy implementation no matter what. You will need first-class "long double" in order to claim to have true 80-bit support, and once you have it, you have no need for FLT_EVAL_METHOD==2.
[1]: http://zvon.org/other/haskell/Outputprelude/tan_f.html