gdb, amd64: extend the amd64 prologue analyzer to skip register pushes
A typical function's prologue can consist of setting up a frame pointer,
pushing registers onto the stack and allocating space on the stack.
Current amd64 prologue analyzer would stop after the frame setup.
This patch allows GDB to skip past register pushes, while also improving
unwinding pushed registers, for functions with a frame pointer, without
debug info and .cfi directives found in .eh_frame section that are used
for unwinding. Skipping register pushes was also present for i386
targets before - the proposed changes are based on i386 implementation.
It also improves the unwinding even if .cfi directives are present,
because GDB can only unwind a register if it has reached a corresponding
.cfi directive, which won't be there before the pushes.
Additionally, at least gcc 11.4 and later by default doesn't emit
necessary debug info, which GDB would try to use to find prologue's end.
In that case, extended prologue analyzer would take effect.
Using C source listed below as an example, compiled with gcc 11.4.0:
```
int __attribute__ ((noinline))
bar (int a)
{
return a + a;
}
int __attribute__ ((noinline))
foo (int a, int b, int c, int d, int e)
{
int x = bar (a) + bar (b) + bar (c) + bar (d) + bar (e);
return x;
}
int
main (int argc, char **argv)
{
return foo (1, 2, 3, 4, 5);
}
```
Compiling with "gcc -O1 -fno-omit-frame-pointer
-fno-asynchronous-unwind-tables", we get:
```
(gdb) b foo
Breakpoint 1 at 0x1139
(gdb) r
...
Breakpoint 1, 0x0000555555555139 in foo ()
(gdb) disassemble
Dump of assembler code for function foo:
0x0000555555555131 <+0>: endbr64
0x0000555555555135 <+4>: push %rbp
0x0000555555555136 <+5>: mov %rsp,%rbp
=> 0x0000555555555139 <+8>: push %r15
0x000055555555513b <+10>: push %r14
0x000055555555513d <+12>: push %r13
0x000055555555513f <+14>: push %r12
0x0000555555555141 <+16>: push %rbx
0x0000555555555142 <+17>: sub $0x8,%rsp
0x0000555555555146 <+21>: mov %esi,%r15d
...
(gdb) ni
0x000055555555513b in foo ()
(gdb) p $r15
$1 = 140737354125376
(gdb) p $r15=1234
$2 = 1234
(gdb) p $r15
$3 = 1234
(gdb) up
#1 0x00005555555551b7 in main ()
(gdb) p $r15
$4 = 1234
```
With the proposed changes, breakpoint gets past those register pushes:
```
(gdb) b foo
Breakpoint 1 at 0x1142
(gdb) r
...
Breakpoint 1, 0x0000555555555142 in foo ()
(gdb) disassemble
Dump of assembler code for function foo:
0x0000555555555131 <+0>: endbr64
0x0000555555555135 <+4>: push %rbp
0x0000555555555136 <+5>: mov %rsp,%rbp
0x0000555555555139 <+8>: push %r15
0x000055555555513b <+10>: push %r14
0x000055555555513d <+12>: push %r13
0x000055555555513f <+14>: push %r12
0x0000555555555141 <+16>: push %rbx
=> 0x0000555555555142 <+17>: sub $0x8,%rsp
0x0000555555555146 <+21>: mov %esi,%r15d
...
```
Also, unwinding pushed registers now works:
```
...
Breakpoint 1, 0x0000555555555142 in foo ()
(gdb) disassemble
Dump of assembler code for function foo:
0x0000555555555131 <+0>: endbr64
0x0000555555555135 <+4>: push %rbp
0x0000555555555136 <+5>: mov %rsp,%rbp
0x0000555555555139 <+8>: push %r15
0x0000555555555139 <+8>: push %r15
0x000055555555513b <+10>: push %r14
0x000055555555513d <+12>: push %r13
0x000055555555513f <+14>: push %r12
0x0000555555555141 <+16>: push %rbx
=> 0x0000555555555142 <+17>: sub $0x8,%rsp
0x0000555555555146 <+21>: mov %esi,%r15d
...
(gdb) p $r15
$1 = 140737354125376
(gdb) p $r15=1234
$2 = 1234
(gdb) p $r15
$3 = 1234
(gdb) up
#1 0x00005555555551b7 in main ()
(gdb) p $r15
$4 = 140737354125376
```
Additionally a new test was added to verify this behavior.
Reviewed-By: Guinevere Larsen <guinevere@redhat.com>
Approved-By: Andrew Burgess <aburgess@redhat.com>
6 files changed
