gnu/gcc/ffc5227ac2e198d1e88d7c686e18262a0bc06c61 libcpp: Optimize Arm search_line_fast
The existing Arm search_line_fast is ancient and not well optimized.
Optimize it for typical inputs (short lines) - what matters is to quickly
get the match result for the first few vectors with minimal initialization
overhead. We simply loop until a match is found even if the input pointer
is unaligned or close to the end. Since this may overread, it relies on
*end containing a match and CPP_BUFFER_PADDING >= 16.
Lookup the low 3 bits of each character using 2 64-bit TBLs and compare
the 128-bit result with the original input. The lookup table contains
the 4 search characters at entries MOD 8, thus if tab[ch % 8] == ch, we
have found a match. This works because the low 3 bits of the search
characters are unique in ASCII, allowing the use of TBL. Use a special
value in tab[0] to avoid matching NUL.
The optimized sequence is less than a third of the old code:
vmov.i8 q12, #7 @ v16qi
vldr d22, .L29
.L25:
vld1.8 {d18-d19}, [r0]!
vand q10, q9, q12
vtbl.8 d16, {d22}, d20
vtbl.8 d17, {d22}, d21
vceq.i8 q8, q8, q9
vaddhn.i16 d16, q8, q8
vmov r2, r3, d16 @ int
orrs r1, r2, r3
beq .L25
cmp r2, #0
rbiteq r2, r3
rbitne r2, r2
clzeq r2, r2
clzne r2, r2
addeq r2, r2, #32
sub r2, r2, #61
add r0, r0, r2, asr #2
bx lr
Performance is significantly better as a result: on a huge trace that replays
the calls from libcpp it is ~24% faster on Neoverse N1.
libcpp/ChangeLog:
* lex.cc (search_line_fast): Optimized Arm implementation.
1 file changed