)]}'
{
  "commit": "2e18f0c4f6c8f965e33ef9fb2e8bc7704d640a9f",
  "tree": "1e47d3795bbe80ee3f2ca743926105657686e2c6",
  "parents": [
    "5d061388db5a6cf057650777b43103f6dee0813b"
  ],
  "author": {
    "name": "Zack Weinberg",
    "email": "zack@owlfolio.org",
    "time": "Wed Jul 17 13:35:22 2024 -0400"
  },
  "committer": {
    "name": "Kamila Szewczyk",
    "email": "szewczyk@cs.uni-saarland.de",
    "time": "Wed Aug 26 20:25:48 2026 +0200"
  },
  "message": "m4sugar: clean up iteration impls and test them more thoroughly\n\nAll of the m4sugar macros that iterate over a macro argument list\nhave two implementations.  One is in m4sugar.m4 and operates by\nrecursion, processing some of the leading elements of $@ and then\nre-invoking itself on the remainder, using `m4_shift`.  This is the\nmost straightforward way to iterate over an argument list in M4, but\nit is quadratically expensive in GNU M4 1.4.x.\n\nThus, we have a second implementation of each such macro in\nm4sugar/foreach.m4.  Those work by generating a temporary helper macro\nthat references each argument by number, no matter how many there are,\nso that $@ only needs to be expanded once.  This is linear in the size\nof the argument list, but less portable (as it uses $10 and up) and\nmuch harder to understand.  According to the comments in foreach.m4,\nGNU M4 1.6.x is going to be cleverer about recursive macro expansion,\nand the straightforward implementations will be *faster* than the\nnon-recursive ones.  So we’ve kept the straightforward implementations\naround, and m4_init replaces them with the foreach.m4 implementations\nif it detects M4 1.4.x is in use.\n\nI say “according to” and “is going to be” because those comments were\nwritten in 2008 and, as of 2024, the latest released version of GNU M4\nis still from the 1.4.x series.  Development branches for versions\n1.6.x and 2.x exist (see \u003chttps://git.savannah.gnu.org/cgit/m4.git\u003e)\nbut have not seen any activity in the past fourteen and eight years,\nrespectively.\n\nThis is a *problem* for us because:\n\n - If you forget to call m4_init, you get the recursive implementations,\n   which might mean whatever weird one-off test you’re doing hangs.\n - Some of the recursive implementations have fallen out of sync with\n   the foreach implementations, as seen in the previous two patches.\n   (I noticed this because I forgot to call m4_init in some weird\n   one-off tests.)\n\nThis patch therefore makes the following changes:\n\n - All the recursive implementations are moved from m4sugar.m4 to a\n   new file m4sugar/recursive.m4.\n\n - m4sugar.m4 loads m4sugar/foreach.m4 at freeze time, so if you\n   forget to call m4_init, you get the foreach implementations.\n   This should also make Autoconf an eensy bit faster in the normal\n   case. m4_init loads m4sugar/recursive.m4 if it detects M4 1.6.x or later.\n\n - Since we now require M4 1.4.8 or later, some compatibility glue for\n   M4 1.4.[67] is removed from m4_init.\n\n - AT_CHECK_M4SUGAR and AT_CHECK_M4SUGAR_TEXT now, by default, run\n   their test twice, once with the recursive implementations forced and\n   once with the foreach implementations forced, and fails the overall\n   test if the results are not identical.  I was expecting this to\n   expose more mismatches between the two implementations but it did not.\n\n   Currently I don’t think it’s necessary to do this for any of the\n   other tests, but I could have missed something.\n\n   Only two subtests need to alter the new default: both are specifically\n   testing for linear performance with large argument lists.  Each of\n   these is split from its parent test and divided into two new tests,\n   one which compares the foreach implementation to the *default*\n   implementation and runs unconditionally, and a second which tests\n   only the recursive implementation and is skipped with M4 1.4.x.\n   Both pairs of tests have constant expected output, so we still get\n   a cross-check between recursive and foreach when M4 1.6.x is used.\n\n - To make it easy for AT_CHECK_M4SUGAR to force one implementation or\n   the other, there is now an internal override macro that can be\n   defined before invoking m4_init.  It only affects the choice of\n   recursive.m4 versus foreach.m4, not any of the other things m4_init\n   does if it detects M4 1.6.x.\n\n - autom4te encapsulates use of the override macro using --language:\n   --language m4sugar-recursive-iteration will always use the recursive\n   implementations and --language m4sugar-foreach-iteration will always\n   use the foreach implementations.  (There are no equivalents of these\n   for any of the other languages built on top of m4sugar.)\n\n - To make *that* possible, autom4te grows support for -D and -U on\n   its own command line, which are directly mapped to -D and -U on\n   m4’s command line.  This should be the only user visible change in\n   the patch.\n\nI did not test this patch with M4 1.6.x or 2.x, but I *did* locally\ncomment out the AT_SKIP_IFs on the tests that are expected to be slow\nwith M4 1.4.x.  They pass (providing more evidence that both\nimplementations are now in sync) but they take several minutes each to\nrun on my 2018-era workstation (verifying that foreach.m4 is still\nnecessary).\n\n* lib/m4sugar/m4sugar.m4 (m4_bmatch, _m4_bpatsubsts, m4_case, _m4_cond)\n  (m4_do, m4_dquote_elt, _m4_foreach, m4_join, _m4_list_cmp)\n  (_m4_list_cmp_1, _m4_list_cmp_2, _m4_list_cmp_raw, m4_map_args_pair)\n  (_m4_minmax, m4_reverse, _m4_set_add_all_check, _m4_set_add_all_clean)\n  (_m4_shiftn): Move definition to lib/m4sugar/recursive.m4.\n  (m4_cr_all): Use _m4_for instead of m4_for in the definition.\n  (m4_min): Definition is textually identical to m4_max, so m4_copy it\n  instead of repeating it.\n  (m4_init): Instead of including m4sugar/foreach.m4 if __m4_version__\n  is not defined, include m4sugar/recursive.m4 if that macro *is* defined.\n  However, if __m4sugar_use_iteration is defined to either ‘recursive’\n  or ‘foreach’, then use that implementation regardless of __m4_version__.\n  Remove backward compatibility code for M4 \u003c1.4.8.\n  (top level): Include m4sugar/foreach.m4 at freeze time.\n  (throughout): Update comments related to above changes.\n\n* lib/m4sugar/recursive.m4: New file containing the recursive\n  versions of each m4sugar macro that has a non-recursive version\n  in foreach.m4.  Each macro defined in this file should also be\n  defined in foreach.m4 and vice versa, except for subroutines used\n  only by one or the other.\n* lib/m4sugar/foreach.m4: Update comments throughout.  No code changes.\n* lib/local.mk: Install lib/m4sugar/recursive.m4.\n* lib/autom4te.in: Add new languages \u0027m4sugar-recursive-iteration\u0027\n  and \u0027m4sugar-foreach-iteration\u0027, marked as intended for testing only.\n  These are the same as the plain \u0027m4sugar\u0027 language except that\n  __m4sugar_use_iteration is predefined to \u0027recursive\u0027 or \u0027foreach\u0027\n  respectively.\n\n* bin/autom4te.in: Recognize command line options -D/--define and\n  -U/--undefine and pass them along to M4, preserving their order\n  relative to each other and relative to .m4(f) files.\n* doc/autoconf.texi: Document new autom4te options.\n\n* tests/local.at (AT_PREPARE): Set shell variable at_mfour_slow_recursion\n  to \u0027:\u0027 if M4 doesn\u0027t define __m4_version__, or to \u0027false\u0027 if it does.\n  Log the choice.\n  (_AT_CHECK_M4SUGAR, __AT_CHECK_M4SUGAR): New helper macros.\n  (AT_CHECK_M4SUGAR): New fifth argument, VARIANTS, expected to be a\n  space-separated list of keywords \u0027default\u0027, \u0027recursive\u0027, and/or\n  \u0027foreach\u0027; defaults to \u0027recursive foreach\u0027.  Run the entire test\n  repeatedly, once for each specified variant, with identical\n  expectations for each.  In addition, expect the generated script\n  to be identical for each variant tested.\n  (AT_CHECK_M4SUGAR_TEXT): Move here from m4sugar.at; reimplement\n  without using \u0027-o -\u0027; add same fifth argument as was added to\n  AT_CHECK_M4SUGAR.\n\n* tests/m4sugar.at (AT_CHECK_M4SUGAR_TEXT): Moved to local.at.\n  (m4st_long_iteration_script, m4st_long_iteration_output)\n  (m4st_set_stress_script, m4st_set_stress_output): New helper macros.\n  (recursion test): Split into two tests, now called\n  “iteration macros, long lists (foreach/default)” and\n  “iteration macros, long lists (recursive)”.\n  Test the recursive implementation only if M4 \u003e\u003d1.6 is detected.\n  (m4_set test): Similarly for the final subtest.\n  (throughout): Do not use \u0027-o -\u0027 in AT_CHECK_M4SUGAR tests.\n  Redo subtests using AT_CHECK_M4SUGAR_TEXT when possible.\n",
  "tree_diff": [
    {
      "type": "modify",
      "old_id": "564cd127ee47d457d133a5104337da533c1e67e6",
      "old_mode": 33188,
      "old_path": "NEWS",
      "new_id": "37b021e33e7e952d53ac8191e188b4aa7bac3a82",
      "new_mode": 33188,
      "new_path": "NEWS"
    },
    {
      "type": "modify",
      "old_id": "e4810a2cc5256d273d52ee95f6e2ca094211e2fb",
      "old_mode": 33188,
      "old_path": "bin/autom4te.in",
      "new_id": "fbf496606a11eb454cefa6a80cb059b2ebfececb",
      "new_mode": 33188,
      "new_path": "bin/autom4te.in"
    },
    {
      "type": "modify",
      "old_id": "3dbed5cc1138acbec93aa29ce8ce2a053090f4d2",
      "old_mode": 33188,
      "old_path": "bin/autoupdate.in",
      "new_id": "16f929543b734cb6f59c4315e600a2c5e2ce6359",
      "new_mode": 33188,
      "new_path": "bin/autoupdate.in"
    },
    {
      "type": "modify",
      "old_id": "b73cd5f632cecef91a12a0c140f293a940e91b3a",
      "old_mode": 33188,
      "old_path": "doc/autoconf.texi",
      "new_id": "5339a2d77b7ff7ab356d88928fcbf934c82bb217",
      "new_mode": 33188,
      "new_path": "doc/autoconf.texi"
    },
    {
      "type": "modify",
      "old_id": "11deb8b23ee6180062cbedadd64548b04ab575d0",
      "old_mode": 33188,
      "old_path": "lib/autom4te.in",
      "new_id": "2908717c2c5d5663a232aa98e56c322ffcb9917f",
      "new_mode": 33188,
      "new_path": "lib/autom4te.in"
    },
    {
      "type": "modify",
      "old_id": "f1ce8179cc6c7b8175b968e0b36e17d81a8cc4c9",
      "old_mode": 33188,
      "old_path": "lib/local.mk",
      "new_id": "d5a88981f67ad2d378e622db5f16953fb72df402",
      "new_mode": 33188,
      "new_path": "lib/local.mk"
    },
    {
      "type": "modify",
      "old_id": "aa42a852214ff1c83bb29343cd13fbc05ef4efe5",
      "old_mode": 33188,
      "old_path": "lib/m4sugar/foreach.m4",
      "new_id": "b16f547f9856235cb800b6db3dbc3bc90105c758",
      "new_mode": 33188,
      "new_path": "lib/m4sugar/foreach.m4"
    },
    {
      "type": "modify",
      "old_id": "54b60bde8fdf26f295b9fc945d30d210e12a0fdb",
      "old_mode": 33188,
      "old_path": "lib/m4sugar/m4sugar.m4",
      "new_id": "9a66077c55a293ef40cff6e1a159463963b8e36f",
      "new_mode": 33188,
      "new_path": "lib/m4sugar/m4sugar.m4"
    },
    {
      "type": "add",
      "old_id": "0000000000000000000000000000000000000000",
      "old_mode": 0,
      "old_path": "/dev/null",
      "new_id": "8cabb3f3ac8e0a7dead259cfba3795c9e1dccfea",
      "new_mode": 33188,
      "new_path": "lib/m4sugar/recursive.m4"
    },
    {
      "type": "modify",
      "old_id": "23d5472d0e6165f25308eb42c50dd12b8131e362",
      "old_mode": 33188,
      "old_path": "tests/local.at",
      "new_id": "c47769e1be574949965470e3bebdcfe4b61821d7",
      "new_mode": 33188,
      "new_path": "tests/local.at"
    },
    {
      "type": "modify",
      "old_id": "5c678d462400bc9ebd0404785a6c89e494b2b0c6",
      "old_mode": 33188,
      "old_path": "tests/m4sugar.at",
      "new_id": "56f85c5825d9aaef629d20c55362fd9566b2ccf0",
      "new_mode": 33188,
      "new_path": "tests/m4sugar.at"
    }
  ]
}