)]}' { "commit": "bfcb5da69a41f7a5e41faab39b763d9d7c8bd2ea", "tree": "acce6608d39a1303b616bd21ec0881a43cce2820", "parents": [ "f950bdbbe4be33e1b05d759c12be9fb64e6130cd" ], "author": { "name": "Jakub Jelinek", "email": "jakub@redhat.com", "time": "Wed May 07 17:25:42 2025 +0200" }, "committer": { "name": "Jakub Jelinek", "email": "jakub@gcc.gnu.org", "time": "Thu May 08 10:49:49 2025 +0200" }, "message": "libcpp: Further fixes for incorrect line numbers in large files [PR120061]\n\nThe backport of the PR108900 fix to 14 branch broke building chromium\nbecause static_assert (__LINE__ \u003d\u003d expected_line_number, \"\"); now triggers\nas the __LINE__ values are off by one.\nThis isn\u0027t the case on the trunk and 15 branch because we\u0027ve switched\nto 64-bit location_t and so one actually needs far longer header files\nto trigger it.\nhttps://gcc.gnu.org/bugzilla/show_bug.cgi?id\u003d120061#c11\nhttps://gcc.gnu.org/bugzilla/show_bug.cgi?id\u003d120061#c12\ncontain (large) testcases in patch form which show on the 14 branch\nthat the first one used to fail before the PR108900 backport and now\nworks correctly, while the second one attempts to match the chromium\nbehavior and it used to pass before the PR108900 backport and now it\nFAILs.\nThe two testcases show rare problematic cases, because\ndo_include_common -\u003e parse_include -\u003e check_eol -\u003e check_eol_1 -\u003e\ncpp_get_token_1 -\u003e _cpp_lex_token -\u003e _cpp_lex_direct -\u003e linemap_line_start\ntriggers there\n /* Allocate the new line_map. However, if the current map only has a\n single line we can sometimes just increase its column_bits instead. */\n if (line_delta \u003c 0\n || last_line !\u003d ORDINARY_MAP_STARTING_LINE_NUMBER (map)\n || SOURCE_COLUMN (map, highest) \u003e\u003d (1U \u003c\u003c (column_bits - range_bits))\n || ( /* We can\u0027t reuse the map if the line offset is sufficiently\n large to cause overflow when computing location_t values. */\n (to_line - ORDINARY_MAP_STARTING_LINE_NUMBER (map))\n \u003e\u003d (((uint64_t) 1)\n \u003c\u003c (CHAR_BIT * sizeof (linenum_type) - column_bits)))\n || range_bits \u003c map-\u003em_range_bits)\n map \u003d linemap_check_ordinary\n (const_cast \u003cline_map *\u003e\n (linemap_add (set, LC_RENAME,\n ORDINARY_MAP_IN_SYSTEM_HEADER_P (map),\n ORDINARY_MAP_FILE_NAME (map),\n to_line)));\nand so creates a new ordinary map on the line right after the\n(problematic) #include line.\nNow, in the spot that r14-11679-g8a884140c2bcb7 patched,\npfile-\u003eline_table-\u003ehighest_location in all 3 tests (also\nhttps://gcc.gnu.org/bugzilla/show_bug.cgi?id\u003d120061#c13\n) is before the decrement the start of the line after the #include line and so\nthe decrement is really desirable in that case to put highest_location\nsomewhere on the line where the #include actually is.\nBut at the same time it is also undesirable, because if we do decrement it,\nthen linemap_add LC_ENTER called from _cpp_do_file_change will then\n /* Generate a start_location above the current highest_location.\n If possible, make the low range bits be zero. */\n location_t start_location \u003d set-\u003ehighest_location + 1;\n unsigned range_bits \u003d 0;\n if (start_location \u003c LINE_MAP_MAX_LOCATION_WITH_COLS)\n range_bits \u003d set-\u003edefault_range_bits;\n start_location +\u003d (1 \u003c\u003c range_bits) - 1;\n start_location \u0026\u003d ~((1 \u003c\u003c range_bits) - 1);\n\n linemap_assert (!LINEMAPS_ORDINARY_USED (set)\n || (start_location\n \u003e\u003d MAP_START_LOCATION (LINEMAPS_LAST_ORDINARY_MAP (set))));\nand we can end up with the new LC_ENTER ordinary map having the same\nstart_location as the preceding LC_RENAME one.\nNext thing that happens is computation of included_from:\n if (reason \u003d\u003d LC_ENTER)\n {\n if (set-\u003edepth \u003d\u003d 0)\n map-\u003eincluded_from \u003d 0;\n else\n /* The location of the end of the just-closed map. */\n map-\u003eincluded_from\n \u003d (((map[0].start_location - 1 - map[-1].start_location)\n \u0026 ~((1 \u003c\u003c map[-1].m_column_and_range_bits) - 1))\n + map[-1].start_location);\nThe normal case (e.g. with the testcase included at the start of this comment) is\nthat map[-1] starts somewhere earlier and so map-\u003eincluded_from computation above\nnicely computes location_t which expands to the start of the #include line.\nWith r14-11679 reverted, for #c11 as well as #c12\nmap[0].start_location \u003d\u003d map[-1].start_location above, and so it is\n((location_t) -1 \u0026 ~((1 \u003c\u003c map[-1].m_column_and_range_bits) - 1)))\n+ map[-1].start_location,\nwhich happens to be start of the #include line.\nFor #c11 map[0].start_location is 0x500003a0 and map[-1] has\nm_column_and_range_bits 7 and map[-2] has m_column_and_range_bits 12 and\nmap[0].included_from is set to 0x50000320.\nFor #c12 map[0].start_location is 0x606c0402 and map[-2].start_location is\n0x606c0400 and m_column_and_range_bits is 0 for all 3 maps.\nmap[0].included_from is set to 0x606c0401.\nThe last important part is again in linemap_add when doing LC_LEAVE:\n /* (MAP - 1) points to the map we are leaving. The\n map from which (MAP - 1) got included should be the map\n that comes right before MAP in the same file. */\n from \u003d linemap_included_from_linemap (set, map - 1);\n\n /* A TO_FILE of NULL is special - we use the natural values. */\n if (to_file \u003d\u003d NULL)\n {\n to_file \u003d ORDINARY_MAP_FILE_NAME (from);\n to_line \u003d SOURCE_LINE (from, from[1].start_location);\n sysp \u003d ORDINARY_MAP_IN_SYSTEM_HEADER_P (from);\n }\nHere it wants to compute the right to_line which ought to be the line after\nthe #include directive.\nOn the #c11 testcase that doesn\u0027t work correctly though, because\nmap[-1].included_from is 0x50000320, from[0] for that is LC_ENTER with\nstart_location 0x4080 and m_column_and_range_bits 12 but note that we\u0027ve\nearlier computed map[-1].start_location + (-1 \u0026 0xffffff80) and so only\ndecreased by 7 bits, so to_line is still on the line with #include and not\nafter it. In the #c12 that doesn\u0027t happen, all the ordinary maps involved\nthere had 0 m_column_and_range_bits and so this computes correct line.\n\nBelow is a fix for the trunk including testcases using the\nlocation_overflow_plugin hack to simulate the bugs without needing huge\nfiles (in the 14 case it is just 330KB and almost 10MB, but in the 15\ncase it would need to be far bigger).\nThe pre- r15-9018 trunk has\nFAIL: gcc.dg/plugin/location-overflow-test-pr116047.c -fplugin\u003d./location_overflow_plugin.so scan-file static_assert[^\\n\\r]*6[^\\n\\r]*\u003d\u003d 6\nand current trunk\nFAIL: gcc.dg/plugin/location-overflow-test-pr116047.c -fplugin\u003d./location_overflow_plugin.so scan-file static_assert[^\\n\\r]*6[^\\n\\r]*\u003d\u003d 6\nFAIL: gcc.dg/plugin/location-overflow-test-pr120061.c -fplugin\u003d./location_overflow_plugin.so scan-file static_assert[^\\n\\r]*5[^\\n\\r]*\u003d\u003d 5\nand with the patch everything PASSes.\nI\u0027ll post afterwards a 14 version of the patch.\n\nThe patch reverts the r15-9018 change, because it is incorrect,\nwe really need to decrement it even when crossing ordinary map\nboundaries, so that the location is not on the line after the #include\nline but somewhere on the #include line. It also patches two spots\nin linemap_add mentioned above to make sure we get correct locations\nboth in the included_from location_t when doing LC_ENTER (second\nline-map.cc hunk) and when doing LC_LEAVE to compute the right to_line\n(first line-map.cc hunk), both in presence of an added LC_RENAME\nwith the same start_location as the following LC_ENTER (i.e. the\nproblematic cases).\nThe LC_ENTER hunk is mostly to ensure included_form location_t is\nat the start of the #include line (column 0), without it we can\ndecrease include_from not enough and end up at some random column\nin the middle of the line, because it is masking away\nmap[-1].m_column_and_range_bits bits even when in the end the resulting\ninclude_from location_t will be found in map[-2] map with perhaps\ndifferent m_column_and_range_bits. That alone doesn\u0027t fix the bug\nthough.\nThe more important is the LC_LEAVE hunk and the problem there is\ncaused by linemap_line_start not actually doing\n r \u003d set-\u003ehighest_line + (line_delta \u003c\u003c map-\u003em_column_and_range_bits);\nwhen adding a new map (the LC_RENAME one because we need to switch to\ndifferent number of directly encoded ranges, or columns, etc.).\nSo, in the original PR108900 case that\n to_line \u003d SOURCE_LINE (from, from[1].start_location);\ndoesn\u0027t do the right thing, from there is the last \u003c 0x50000000 map\nwith m_column_and_range_bits 12, from[1] is the first one above it\nand map[-1].included_from is the correct location of column 0 on\nthe #include line, but as the new LC_RENAME map has been created without\nactually increasing highest_location to be on the new line (we\u0027ve just\nset to_line of the new LC_RENAME map to the correct line),\n to_line \u003d SOURCE_LINE (from, from[1].start_location);\nstays on the same source line. I\u0027ve tried to just replace that with\n to_line \u003d SOURCE_LINE (from, linemap_included_from (map - 1)) + 1;\ni.e. just find out the #include line from map[-1].included_from and\nadd 1 to it, unfortunately that breaks the\nc-c++-common/cpp/line-4.c\ntest where we expect to stay on the same 0 line for LC_LEAVE from\n\u003ccommand line\u003e and gcc.dg/cpp/trad/Wunused.c, gcc.dg/cpp/trad/builtins.c\nand c-c++-common/analyzer/named-constants-via-macros-traditional.c tests\nall with -traditional-cpp preprocessing where to_line is also off-by-one\nfrom the expected one.\nSo, this patch instead conditionalizes it, uses the\n to_line \u003d SOURCE_LINE (from, linemap_included_from (map - 1)) + 1;\nway only if from[1] is a LC_RENAME map (rather than the usual\nLC_ENTER one), that should limit it to the problematic cases of when\nparse_include peeked after EOL and had to create LC_RENAME map with\nthe same start_location as the LC_ENTER after it.\n\nSome further justification for the LC_ENTER hunk, using the\nhttps://gcc.gnu.org/pipermail/gcc-patches/2025-May/682774.html testcase\n(old is 14 before r14-11679, vanilla current 14 and new with the 14 patch)\nI get\n$ /usr/src/gcc-14/obj/gcc/cc1.old -quiet -std\u003dc23 pr116047.c -nostdinc\nIn file included from pr116047-1.h:327677:21,\n from pr116047.c:4:\npr116047-2.h:1:1: error: unknown type name ‘a’\n 1 | a b c;\n | ^\npr116047-2.h:1:5: error: expected ‘\u003d’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘c’\n 1 | a b c;\n | ^\npr116047-1.h:327677:1: error: static assertion failed: \"\"\n327677 | #include \"pr116047-2.h\"\n | ^~~~~~~~~~~~~\n$ /usr/src/gcc-14/obj/gcc/cc1.vanilla -quiet -std\u003dc23 pr116047.c -nostdinc\nIn file included from pr116047-1.h:327678,\n from pr116047.c:4:\npr116047-2.h:1:1: error: unknown type name ‘a’\n 1 | a b c;\n | ^\npr116047-2.h:1:5: error: expected ‘\u003d’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘c’\n 1 | a b c;\n | ^\n$ /usr/src/gcc-14/obj/gcc/cc1.new -quiet -std\u003dc23 pr116047.c -nostdinc\nIn file included from pr116047-1.h:327677,\n from pr116047.c:4:\npr116047-2.h:1:1: error: unknown type name ‘a’\n 1 | a b c;\n | ^\npr116047-2.h:1:5: error: expected ‘\u003d’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘c’\n 1 | a b c;\n | ^\n\npr116047-1.h has on lines 327677+327678:\n #include \"pr116047-2.h\"\n static_assert (__LINE__ \u003d\u003d 327678, \"\");\nso the static_assert failure is something that was dealt mainly in the\nLC_LEAVE hunk and files.cc reversion, but please have a look at the\nIn file included from lines.\n14.2 emits correct line (#include \"pr116047-2.h\" is indeed on line\n327677) but some random column in there (which is not normally printed\nfor smaller headers; 21 is the . before extension in the filename).\nCurrent trunk emits incorrect line (327678 instead of 327677, clearly\nit didn\u0027t decrement).\nAnd the patched compiler emits the right line with no column, as would\nbe printed if I remove e.g. 300000 newlines from the file.\n\n2025-05-07 Jakub Jelinek \u003cjakub@redhat.com\u003e\n\n\tPR preprocessor/108900\n\tPR preprocessor/116047\n\tPR preprocessor/120061\n\t* files.cc (_cpp_stack_file): Revert 2025-03-28 change.\n\t* line-map.cc (linemap_add): Use\n\tSOURCE_LINE (from, linemap_included_from (map - 1)) + 1; instead of\n\tSOURCE_LINE (from, from[1].start_location); to compute to_line\n\tfor LC_LEAVE. For LC_ENTER included_from computation, look at\n\tmap[-2] or even lower if map[-1] has the same start_location as\n\tmap[0].\n\n\t* gcc.dg/plugin/plugin.exp: Add location-overflow-test-pr116047.c\n\tand location-overflow-test-pr120061.c.\n\t* gcc.dg/plugin/location_overflow_plugin.cc (plugin_init): Don\u0027t error\n\ton unknown values, instead just break. Handle 0x4fHHHHHH arguments\n\tdifferently.\n\t* gcc.dg/plugin/location-overflow-test-pr116047.c: New test.\n\t* gcc.dg/plugin/location-overflow-test-pr116047-1.h: New test.\n\t* gcc.dg/plugin/location-overflow-test-pr116047-2.h: New test.\n\t* gcc.dg/plugin/location-overflow-test-pr120061.c: New test.\n\t* gcc.dg/plugin/location-overflow-test-pr120061-1.h: New test.\n\t* gcc.dg/plugin/location-overflow-test-pr120061-2.h: New test.\n\n(cherry picked from commit edf745dc519ddbfef127e2789bf11bfbacd300b7)\n", "tree_diff": [ { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "3dd6434a938bff760cb40248e5c4dcd12edf3551", "new_mode": 33188, "new_path": "gcc/testsuite/gcc.dg/plugin/location-overflow-test-pr116047-1.h" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "048f715b4656f388eaeddede94b3808a59a3a95e", "new_mode": 33188, "new_path": "gcc/testsuite/gcc.dg/plugin/location-overflow-test-pr116047-2.h" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "75161fa5f055bb0eb04df77068b0b3ac099be5d3", "new_mode": 33188, "new_path": "gcc/testsuite/gcc.dg/plugin/location-overflow-test-pr116047.c" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "ebf7704f568e3b7d4d5755d0fb1b5944148f3b95", "new_mode": 33188, "new_path": "gcc/testsuite/gcc.dg/plugin/location-overflow-test-pr120061-1.h" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "048f715b4656f388eaeddede94b3808a59a3a95e", "new_mode": 33188, "new_path": "gcc/testsuite/gcc.dg/plugin/location-overflow-test-pr120061-2.h" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "e8e803898da3bd482eb10a4d267e29166e15ad6e", "new_mode": 33188, "new_path": "gcc/testsuite/gcc.dg/plugin/location-overflow-test-pr120061.c" }, { "type": "modify", "old_id": "f731b1421b0fd1e379f3dfda78ea557488cbf689", "old_mode": 33188, "old_path": "gcc/testsuite/gcc.dg/plugin/location_overflow_plugin.cc", "new_id": "f770d35ea518ec7aa247e62b84c40c012309b4bd", "new_mode": 33188, "new_path": "gcc/testsuite/gcc.dg/plugin/location_overflow_plugin.cc" }, { "type": "modify", "old_id": "90c91621d0aaa704a4cb2c253b65f08d6c43fb58", "old_mode": 33188, "old_path": "gcc/testsuite/gcc.dg/plugin/plugin.exp", "new_id": "96e76d2e0c362edc70900558a72e96eb13cf261f", "new_mode": 33188, "new_path": "gcc/testsuite/gcc.dg/plugin/plugin.exp" }, { "type": "modify", "old_id": "c1abde6639fe7a5ef3486987775d6c06c8f9fc37", "old_mode": 33188, "old_path": "libcpp/files.cc", "new_id": "d80c4bfd907755d9d9f630bfd5023614b6132ac4", "new_mode": 33188, "new_path": "libcpp/files.cc" }, { "type": "modify", "old_id": "17e7f12551c0335c4d71ee969681a1d054d95ff0", "old_mode": 33188, "old_path": "libcpp/line-map.cc", "new_id": "cf6557117c81cfe925bf38a916c5ac41fdb29f73", "new_mode": 33188, "new_path": "libcpp/line-map.cc" } ] }