match.pd: Allow FNMA fold through conversions
The FMA folds in match.pd currently only matches (negate @0) directly.
When the negated operand is wrapped in a type conversion
(e.g. (convert (negate @0))), the simplification to IFN_FNMA does not
trigger.
This prevents folding of patterns such as:
*c = *c - (v8u)(*a * *b);
when the multiply operands undergo vector type conversions before being
passed to FMA. In such cases the expression lowers to neg + mla/mad
instead of the more optimal msb/mls on AArch64 SVE, because the current
fold cannot see through the casts.
Extend the match pattern to allow conversions on the negated
operand and the second multiplicand:
(fmas:c (nop_convert (negate @0)) @1 @2)
The fold rewrites the expression as:
convert (IFN_FNMA (convert:ut @0) (convert:ut @1) (convert:ut @2))
The match is restricted to nop_convert on the negated operand to avoid
folding through value-changing conversions. This enables recognition of
the subtraction-of-product form even when vector element type casts are
present.
The fold is only performed when signed overflow is unobservable for
both the outer FMA operation and the inner negation. This avoids
changing sanitized overflow behaviour when looking through the nop
conversion on the negated operand.
With this change, AArch64 SVE code generation is able to select msb/mls
instead of emitting a separate neg followed by mla/mad.
This patch was bootstrapped and regression tested on aarch64-linux-gnu.
PR target/123924
gcc/
* match.pd: Allow conversions in FMA-to-FNMA fold.
gcc/testsuite/
* gcc.target/aarch64/sve/fnma_match.c: New test.
* gcc.target/aarch64/sve/pr123897.c:
Update the test to scan for FNMA in the tree dump.
3 files changed