This is a loose collection of notes for people hacking on simulators.
If this document gets big enough it can be prettied up then.

Contents

- The "common" directory
- Common Makefile Support
- TAGS support
- Generating "configure" files
- C Language Assumptions
- "dump" commands under gdb

The "common" directory
======================

The common directory contains:

- common documentation files (e.g. run.1, and maybe in time .texi files)
- common source files (e.g. run.c)
- common Makefile fragment and configury (e.g. Make-common.in, aclocal.m4).

In addition "common" contains portions of the system call support
(e.g. callback.c, target-newlib-*.c).

Common Makefile Support
=======================

A common configuration framework is available for simulators that want
to use it.  The common framework exists to remove a lot of duplication
in configure.ac and Makefile.in, and it also provides a foundation for
enhancing the simulators uniformly (e.g. the more they share in common
the easier a feature added to one is added to all).

The configure.ac of a simulator using the common framework should look like:

--- snip ---
dnl Process this file with autoconf to produce a configure script.
AC_INIT(Makefile.in)
AC_CONFIG_MACRO_DIRS([../common ../.. ../../config])

... target specific additions ...

SIM_AC_OUTPUT
--- snip ---

SIM_AC_OUTPUT:

- creates the symbolic links defined in sim_link_{files,links}
- creates config.h
- creates the Makefile

The Makefile.in of a simulator using the common framework should look like:

--- snip ---
# Makefile for blah ...
# Copyright blah ...

## COMMON_PRE_CONFIG_FRAG

# These variables are given default values in COMMON_PRE_CONFIG_FRAG.
# We override the ones we need to here.
# Not all of these need to be mentioned, only the necessary ones.
# In fact it is better to *not* mention ones if the value is the default.

# List of object files, less common parts.
SIM_OBJS =
# List of extra dependencies.
# Generally this consists of simulator specific files included by sim-main.h.
SIM_EXTRA_DEPS =
# List of flags to always pass to $(CC).
SIM_EXTRA_CFLAGS =
# List of extra libraries to link with.
SIM_EXTRA_LIBS =
# Dependency of `install' to install any extra files.
SIM_EXTRA_INSTALL =
# Dependency of `clean' to clean any extra files.
SIM_EXTRA_CLEAN =

## COMMON_POST_CONFIG_FRAG

# Rules need to build $(SIM_OBJS), plus whatever else the target wants.

... target specific rules ...
--- snip ---

COMMON_{PRE,POST}_CONFIG_FRAG are markers for SIM_AC_OUTPUT to tell it
where to insert the two pieces of common/Make-common.in.
The resulting Makefile is created by doing autoconf substitions on
both the target's Makefile.in and Make-common.in, and inserting
the two pieces of Make-common.in into the target's Makefile.in at
COMMON_{PRE,POST}_CONFIG_FRAG.

Note that SIM_EXTRA_{INSTALL,CLEAN} could be removed and "::" targets
could be used instead.  However, it's not clear yet whether "::" targets
are portable enough.

TAGS support
============

Many files generate program symbols at compile time.
Such symbols can't be found with grep nor do they normally appear in
the TAGS file.  To get around this, source files can add the comment

/* TAGS: foo1 foo2 */

where foo1, foo2 are program symbols.  Symbols found in such comments
are greppable and appear in the TAGS file.

Generating "configure" files
============================

For targets using the common framework, "configure" can be generated
by running `autoconf'.

To regenerate the configure files for all targets using the common framework:

	$  cd devo/sim
	$  make -f Makefile.in SHELL=/bin/sh autoconf-common

To add a change-log entry to the ChangeLog file for each updated
directory (WARNING - check the modified new-ChangeLog files before
renaming):

	$  make -f Makefile.in SHELL=/bin/sh autoconf-changelog
	$  more */new-ChangeLog
	$  make -f Makefile.in SHELL=/bin/sh autoconf-install

In a similar vein, both the configure and config.in files can be
updated using the sequence:

	$  cd devo/sim
	$  make -f Makefile.in SHELL=/bin/sh autoheader-common
	$  make -f Makefile.in SHELL=/bin/sh autoheader-changelog
	$  more */new-ChangeLog
	$  make -f Makefile.in SHELL=/bin/sh autoheader-install

To add the entries to an alternative ChangeLog file, use:

	$  make ChangeLog=MyChangeLog ....


C Language Assumptions
======================

An ISO C11 compiler is required, as is an ISO C standard library.

"dump" commands under gdb
=========================

gdbinit.in contains the following

define dump
set sim_debug_dump ()
end

Simulators that define the sim_debug_dump function can then have their
internal state pretty printed from gdb.

FIXME: This can obviously be made more elaborate.  As needed it will be.

Rebuilding target-newlib-* files
================================

Checkout a copy of the SIM and LIBGLOSS modules (Unless you've already
got one to hand):

	$  mkdir /tmp/$$
	$  cd /tmp/$$
	$  cvs checkout sim-no-testsuite libgloss-no-testsuite newlib-no-testsuite

Configure things for an arbitrary simulator target (d10v is used here for
convenience):

	$  mkdir /tmp/$$/build
	$  cd /tmp/$$/build
	$  /tmp/$$/devo/configure --target=d10v-elf

In the sim/ directory rebuild the headers:

	$  cd sim/
	$  make nltvals

If the target uses the common syscall table (libgloss/syscall.h), then you're
all set!  If the target has a custom syscall table, you need to declare it:

	devo/sim/common/gennltvals.py

		Add your new processor target (you'll need to grub
		around to find where your syscall.h lives).

	devo/sim/<processor>/*.[ch]

		Include target-newlib-syscall.h instead of syscall.h.

Tracing
=======

For ports based on CGEN, tracing instrumentation should largely be for free,
so we will cover the basic non-CGEN setup here.  The assumption is that your
target is using the common autoconf macros and so the build system already
includes the sim-trace configure flag.

The full tracing API is covered in sim-trace.h, so this section is an overview.

Before calling any trace function, you should make a call to the trace_prefix()
function.  This is usually done in the main sim_engine_run() loop before
simulating the next instruction.  You should make this call before every
simulated insn.  You can probably copy & paste this:
  if (TRACE_ANY_P (cpu))
    trace_prefix (sd, cpu, NULL_CIA, oldpc, TRACE_LINENUM_P (cpu), NULL, 0, "");

You will then need to instrument your simulator code with calls to the
trace_generic() function with the appropriate trace index.  Typically, this
will take a form similar to the above snippet.  So to trace instructions, you
would use something like:
  if (TRACE_INSN_P (cpu))
    trace_generic (sd, cpu, TRACE_INSN_IDX, "NOP;");

The exact output format is up to you.  See the trace index enum in sim-trace.h
to see the different tracing info available.

To utilize the tracing features at runtime, simply use the --trace-xxx flags.
  run --trace-insn ./some-program

Profiling
=========

Similar to the tracing section, this is merely an overview for non-CGEN based
ports.  The full API may be found in sim-profile.h.  Its API is also similar
to the tracing API.

Note that unlike the tracing command line options, in addition to the profile
flags, you have to use the --verbose option to view the summary report after
execution.  Tracing output is displayed on the fly, but the profile output is
only summarized.

To profile core accesses (such as data reads/writes and insn fetches), add
calls to PROFILE_COUNT_CORE() to your read/write functions.  So in your data
fetch function, you'd use something like:
  PROFILE_COUNT_CORE (cpu, target_addr, size_in_bytes, map_read);
Then in your data write function:
  PROFILE_COUNT_CORE (cpu, target_addr, size_in_bytes, map_write);
And in your insn fetcher:
  PROFILE_COUNT_CORE (cpu, target_addr, size_in_bytes, map_exec);

To use the PC profiling code, you simply have to tell the system where to find
your simulator's PC.  So in your model initialization function:
  CPU_PC_FETCH (cpu) = function_that_fetches_the_pc;

To profile branches, in every location where a branch insn is executed, call
one of the related helpers:
  PROFILE_BRANCH_TAKEN (cpu);
  PROFILE_BRANCH_UNTAKEN (cpu);
If you have stall information, you can utilize the other helpers too.

Environment Simulation
======================

The simplest simulator doesn't include environment support -- it merely
simulates the Instruction Set Architecture (ISA).  Once you're ready to move
on to the next level, it's time to start handling the --env option.  It's
enabled by default for all ports already.

This will support for the user, virtual, and operating environments.  See the
sim-config.h header for a more detailed description of them.  The former are
pretty straight forward as things like exceptions (making system calls) are
handled in the simulator.  Which is to say, an exception does not trigger an
exception handler in the simulator target -- that is what the operating env
is about.  See the following userspace section for more information.

Userspace System Calls
======================

By default, the libgloss userspace is simulated.  That means the system call
numbers and calling convention matches that of libgloss.  Simulating other
userspaces (such as Linux) is pretty straightforward, but let's first focus
on the basics.  The basic API is covered in include/sim/callback.h.

When an instruction is simulated that invokes the system call method (such as
forcing a hardware trap or exception), your simulator code should set up the
CB_SYSCALL data structure before calling the common cb_syscall() function.
For example:
static int
syscall_read_mem (host_callback *cb, struct cb_syscall *sc,
		  unsigned long taddr, char *buf, int bytes)
{
  SIM_DESC sd = (SIM_DESC) sc->p1;
  SIM_CPU *cpu = (SIM_CPU *) sc->p2;
  return sim_core_read_buffer (sd, cpu, read_map, buf, taddr, bytes);
}
static int
syscall_write_mem (host_callback *cb, struct cb_syscall *sc,
		  unsigned long taddr, const char *buf, int bytes)
{
  SIM_DESC sd = (SIM_DESC) sc->p1;
  SIM_CPU *cpu = (SIM_CPU *) sc->p2;
  return sim_core_write_buffer (sd, cpu, write_map, buf, taddr, bytes);
}
void target_sim_syscall (SIM_CPU *cpu)
{
  SIM_DESC sd = CPU_STATE (cpu);
  host_callback *cb = STATE_CALLBACK (sd);
  CB_SYSCALL sc;

  CB_SYSCALL_INIT (&sc);

  sc.func = <fetch system call number>;
  sc.arg1 = <fetch first system call argument>;
  sc.arg2 = <fetch second system call argument>;
  sc.arg3 = <fetch third system call argument>;
  sc.arg4 = <fetch fourth system call argument>;
  sc.p1 = (PTR) sd;
  sc.p2 = (PTR) cpu;
  sc.read_mem = syscall_read_mem;
  sc.write_mem = syscall_write_mem;

  cb_syscall (cb, &sc);

  <store system call result from sc.result>;
  <store system call error from sc.errcode>;
}
Some targets store the result and error code in different places, while others
only store the error code when the result is an error.

Keep in mind that the CB_SYS_xxx defines are normalized values with no real
meaning with respect to the target.  They provide a unique map on the host so
that it can parse things sanely.  For libgloss, the common/target-newlib-syscall
file contains the target's system call numbers to the CB_SYS_xxx values.

To simulate other userspace targets, you really only need to update the maps
pointers that are part of the callback interface.  So create CB_TARGET_DEFS_MAP
arrays for each set (system calls, errnos, open bits, etc...) and in a place
you find useful, do something like:

...
static CB_TARGET_DEFS_MAP cb_linux_syscall_map[] = {
# define TARGET_LINUX_SYS_open 5
  { CB_SYS_open, TARGET_LINUX_SYS_open },
  ...
  { -1, -1 },
};
...
  host_callback *cb = STATE_CALLBACK (sd);
  cb->syscall_map = cb_linux_syscall_map;
  cb->errno_map = cb_linux_errno_map;
  cb->open_map = cb_linux_open_map;
  cb->signal_map = cb_linux_signal_map;
  cb->stat_map = cb_linux_stat_map;
...

Each of these cb_linux_*_map's are manually declared by the arch target.

The target_sim_syscall() example above will then work unchanged (ignoring the
system call convention) because all of the callback functions go through these
mapping arrays.

Events
======

Events are scheduled and executed on behalf of either a cpu or hardware devices.
The API is pretty much the same and can be found in common/sim-events.h and
common/hw-events.h.

For simulator targets, you really just have to worry about the schedule and
deschedule functions.

Device Trees
============

The device tree model is based on the OpenBoot specification.  Since this is
largely inherited from the psim code, consult the existing psim documentation
for some in-depth details.
	http://sourceware.org/psim/manual/

Hardware Devices
================

The simplest simulator doesn't include hardware device support.  Once you're
ready to move on to the next level, declare in your Makefile.in:
SIM_EXTRA_HW_DEVICES = devone devtwo devthree

The basic hardware API is documented in common/hw-device.h.

Each device has to have a matching file name with a "dv-" prefix.  So there has
to be a dv-devone.c, dv-devtwo.c, and dv-devthree.c files.  Further, each file
has to have a matching hw_descriptor structure.  So the dv-devone.c file has to
have something like:
  const struct hw_descriptor dv_devone_descriptor[] = {
    {"devone", devone_finish,},
    {NULL, NULL},
  };

The "devone" string as well as the "devone_finish" function are not hard
requirements, just common conventions.  The structure name is a hard
requirement.

The devone_finish() callback function is used to instantiate this device by
parsing the corresponding properties in the device tree.

Hardware devices typically attach address ranges to themselves.  Then when
accesses to those addresses are made, the hardware will have its callback
invoked.  The exact callback could be a normal I/O read/write access, as
well as a DMA access.  This makes it easy to simulate memory mapped registers.

Keep in mind that like a proper device driver, it may be instantiated many
times over.  So any device state it needs to be maintained should be allocated
during the finish callback and attached to the hardware device via set_hw_data.
Any hardware functions can access this private data via the hw_data function.

Ports (Interrupts / IRQs)
=========================

First, a note on terminology.  A "port" is an aspect of a hardware device that
accepts or generates interrupts.  So devices with input ports may be the target
of an interrupt (accept it), and/or they have output ports so that they may be
the source of an interrupt (generate it).

Each port has a symbolic name and a unique number.  These are used to identify
the port in different contexts.  The output port name has no hard relationship
to the input port name (same for the unique number).  The callback that accepts
the interrupt uses the name/id of its input port, while the generator function
uses the name/id of its output port.

The device tree is used to connect the output port of a device to the input
port of another device.  There are no limits on the number of inputs connected
to an output, or outputs to an input, or the devices attached to the ports.
In other words, the input port and output port could be the same device.

The basics are:
 - each hardware device declares an array of ports (hw_port_descriptor).
   any mix of input and output ports is allowed.
 - when setting up the device, attach the array (set_hw_ports).
 - if the device accepts interrupts, it will have to attach a port callback
   function (set_hw_port_event)
 - connect ports with the device tree
 - handle incoming interrupts with the callback
 - generate outgoing interrupts with hw_port_event