gnu/gcc/bb1344be44e364da8377603697933a668b651c4e choose VF independent of externs/constants
The following decides on the VF based on assigned SLP_TREE_VECTYPE
rather than on the tracked max_nunits in the SLP graph which also
factors in external and constant nodes. Those get their vector
type assigned in vectorizable_* via vect_maybe_update_slp_op_vectype
and there's freedom to vectorizable_* to pick a suitable one.
Specifically conversions of externs/constants can cause a higher
than necessary VF which would be a missed optimization.
Changing the VF as outlined will run into both vect_prologue_cost_for_slp
and vect_create_constant_vectors ICEing when computing
vect_get_num_copies since that asserts it can exact_div the
number of lanes as in the unrolled loop by the number of lanes in the
(out of loop) vector type. But vectorizable_* can opt to just
use the lowpart of such vectors but require it in full due to target
constraints. So this RFC patch introduces vect_get_num_copies_for_invariant
which performs a ceil_div instead (I think we can maybe change the global
copy to do that). I added verification to vect_slp_analyze_node_operations
which should make the vect_maybe_update_slp_op_vectype change in
the predicated tails series unneeded.
This shifts the ICE to vectorizable_conversion which is mightly confused
by invariant vectors with too many lanes (in O3-vect-pr32243.c it is
unswitching that exposes a loop invariant conversion in the loop).
I have installed a narrow fix that is likely incomplete and I expect
similar issues to appear in multi-operand widening/narrowing operations.
The fundamental issue is that we expose constant/externals to
vectorizable_* that might have an excess number of lanes, and some
vectorizable_* might not be prepared (and most should be unaffected).
As vectorizable_* control the vector types themselves the fix lies
within them. Once fuzzers increase coverage.
* tree-vect-slp.cc (vect_update_slp_vf_for_node): Compute
VF based on SLP_TREE_VECTYPE only.
(vect_get_num_copies_for_invariant): New.
(vect_prologue_cost_for_slp): Take nvectors as argument.
(vect_slp_analyze_node_operations): Verify that we can
div_away_from_zero for vect_prologue_cost_for_slp.
(vect_create_constant_vectors): Use vect_get_num_copies_for_invariant.
* tree-vect-stmts.cc (vect_create_vectorized_promotion_stmts):
Support creating half of the promoted results.
(vectorizable_conversion): When we have excess input elements
request half of the promoted results.
2 files changed