gnu/gcc/b64fc298a3538c3d7d85ca05fef822acba23e4a1 x86-64: Expand bounded memset and memcpy like memmove
commit 401199377c50045ede560daf3f6e8b51749c2a87
Author: H.J. Lu <hjl.tools@gmail.com>
Date: Tue Jun 17 10:17:17 2025 +0800
x86: Improve vector_loop/unrolled_loop for memset/memcpy
uses move_by_pieces and store_by_pieces for memcpy and memset epilogues
with the fixed epilogue size. Since move_by_pieces and store_by_pieces
don't use the maximum size info, they generate extra branches and moves
for bounded memcpy and memset. Commit
commit b41f96465190751561f6909e858604ceab00595b
Author: H.J. Lu <hjl.tools@gmail.com>
Date: Mon Oct 20 16:14:34 2025 +0800
x86-64: Inline memmove with overlapping unaligned loads and stores.
inlines memmove with overlapping unaligned and stores which reduces the
numbers of branches as well as moves when the maximum size is known.
Rename ix86_expand_movmem to ix86_expand_set_or_movmem and extend it to
inline bounded memcpy and memset. Update ix86_expand_set_or_cpymem to
call ix86_expand_set_or_movmem for bounded memset and memcpy as memmove
if misaligned moves are preferred.
In addition to reduce 727.cppcheck_r O2 code size by ~9%, there're
another ~8 benchmarks whose code sizes are reduced >2% across spec2026
and spec2017 with march=x86-64-v3 O2, no big code size impact for Ofast,
performance impact is negligible(slightly better, but all under noise
range) for both O2 and Ofast.
gcc/
PR target/125856
PR target/125865
* config/i386/i386-expand.cc (ix86_expand_set_or_cpymem):
Call ix86_expand_set_or_movmem for
TARGET_MISALIGNED_MOVE_STRING_PRO_EPILOGUES.
(ix86_expand_memset_val): New.
(ix86_expand_n_move_movmem): Renamed to ...
(ix86_expand_n_move_set_or_movmem): This. Add a pointer to rtx
argument for memset. Also expand memset.
(ix86_expand_n_overlapping_move_movmem): Renamed to ...
(ix86_expand_n_overlapping_move_set_or_movmem): This. Add a
pointer to rtx argument for memset. Also expand memset.
(ix86_expand_less_move_movmem): Renamed to ...
(ix86_expand_less_move_set_or_movmem): This. Add a pointer to
rtx argument for memset. Also expand memset.
(ix86_expand_movmem): Renamed to ...
(ix86_expand_set_or_movmem): This. Add bool arguments for memcpy
and memset. Also expand memcpy and memset.
* config/i386/i386-protos.h (ix86_expand_movmem): Renamed to ...
(ix86_expand_set_or_movmem): This. Add bool arguments for memcpy
and memset.
* config/i386/i386.md (movmem<mode>): Replace ix86_expand_movmem
with ix86_expand_set_or_movmem.
gcc/testsuite/
PR target/125856
PR target/125865
* gcc.target/i386/builtin-memcpy-1a.c: New test.
* gcc.target/i386/builtin-memcpy-1b.c: Likewise.
* gcc.target/i386/builtin-memcpy-2a.c: Likewise.
* gcc.target/i386/builtin-memcpy-2b.c: Likewise.
* gcc.target/i386/builtin-memcpy-2c.c: Likewise.
* gcc.target/i386/builtin-memcpy-3a.c: Likewise.
* gcc.target/i386/builtin-memcpy-3b.c: Likewise.
* gcc.target/i386/builtin-memcpy-3c.c: Likewise.
* gcc.target/i386/builtin-memcpy-4a.c: Likewise.
* gcc.target/i386/builtin-memcpy-4b.c: Likewise.
* gcc.target/i386/builtin-memcpy-4c.c: Likewise.
* gcc.target/i386/builtin-memcpy-5a.c: Likewise.
* gcc.target/i386/builtin-memcpy-5b.c: Likewise.
* gcc.target/i386/builtin-memcpy-5c.c: Likewise.
* gcc.target/i386/builtin-memcpy-6.c: Likewise.
* gcc.target/i386/builtin-memcpy-bounded-1a.c: Likewise.
* gcc.target/i386/builtin-memcpy-bounded-1b.c: Likewise.
* gcc.target/i386/builtin-memset-1a.c: Likewise.
* gcc.target/i386/builtin-memset-1b.c: Likewise.
* gcc.target/i386/builtin-memset-2a.c: Likewise.
* gcc.target/i386/builtin-memset-2b.c: Likewise.
* gcc.target/i386/builtin-memset-2c.c: Likewise.
* gcc.target/i386/builtin-memset-3a.c: Likewise.
* gcc.target/i386/builtin-memset-3b.c: Likewise.
* gcc.target/i386/builtin-memset-3c.c: Likewise.
* gcc.target/i386/builtin-memset-4a.c: Likewise.
* gcc.target/i386/builtin-memset-4b.c: Likewise.
* gcc.target/i386/builtin-memset-4c.c: Likewise.
* gcc.target/i386/builtin-memset-5a.c: Likewise.
* gcc.target/i386/builtin-memset-5b.c: Likewise.
* gcc.target/i386/builtin-memset-5c.c: Likewise.
* gcc.target/i386/builtin-memset-6a.c: Likewise.
* gcc.target/i386/builtin-memset-6b.c: Likewise.
* gcc.target/i386/builtin-memset-6c.c: Likewise.
* gcc.target/i386/builtin-memset-7a.c: Likewise.
* gcc.target/i386/builtin-memset-7b.c: Likewise.
* gcc.target/i386/builtin-memset-7c.c: Likewise.
* gcc.target/i386/builtin-memset-8.c: Likewise.
* gcc.target/i386/builtin-memset-bounded-1a.c: Likewise.
* gcc.target/i386/builtin-memset-bounded-1b.c: Likewise.
Co-Authored-By: Hongtao Liu <hongtao.liu@intel.com>
Signed-off-by: H.J. Lu <hjl.tools@gmail.com>
43 files changed