gnu/gcc/48425ec4ef2ad3697f07a3075ff52d2d9dea91f4 aarch64: implement ctz2 for the Advanced SIMD byte and halfword modes
Reversing the bits of an element turns its trailing zeros into leading
ones, so a count of trailing zeros is a bit reversal followed by a CLZ.
ctz<mode>2 only covered V2SI and V4SI, so the byte and halfword loops were
expanded by the middle end into the generic negate/and/clz/subtract
sequence, which needs two vector constants as well as four instructions.
RBIT reverses the bits within each byte, so a byte element needs nothing
else and a halfword element needs REV16 to put its two bytes in the
opposite order. That is the same shape the V2SI and V4SI expander already
had, so fold all of them into one expander over VDQ_BHSI and give
bitreverse<mode>2 the wider modes it builds on.
For a halfword loop the inner loop changes from
ldr q0, [x1]
orr v0.8h, #128, lsl #8
add v30.8h, v0.8h, v30.8h
bic v30.16b, v30.16b, v0.16b
clz v30.8h, v30.8h
sub v31.8h, v31.8h, v30.8h
str q31, [x0]
to
ldr q31, [x1]
orr v31.8h, #128, lsl #8
rev16 v31.16b, v31.16b
rbit v31.16b, v31.16b
clz v31.8h, v31.8h
str q31, [x0]
with the two constants no longer needed. A byte loop loses the REV16 as
well and counts in two instructions.
The 64-bit elements have no CLZ to pair with, so they keep the generic
expansion through popcount, which is what LLVM emits for them too.
A fixed-length loop keeps this Advanced SIMD form even when SVE is
available, while a variable-length one is vectorised with SVE and counts
there, so the two cases get a test each over a common source.
Bootstrapped and tested on aarch64-none-linux-gnu.
Ok for trunk?
Thanks,
Kyrill
gcc/ChangeLog:
* config/aarch64/aarch64-simd.md (bitreverse<mode>2): New expander
for VDQHS.
(ctz<mode>2): Replace the VB and VS expanders with one for
VDQ_BHSI.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/vect-ctz.h: New file.
* gcc.target/aarch64/vect-ctz-1.c: New test.
* gcc.target/aarch64/vect-ctz-2.c: New test.
* gcc.target/aarch64/vect-ctz-3.c: New test.
Signed-off-by: Kyrylo Tkachov <ktkachov@nvidia.com>
5 files changed