fixincludes/README - gcc - Git at Google



            GCC MAINTAINER INFORMATION

            
            ==========================


            If you are having some problem with a system header that is either

            
            broken by the manufacturer, or is broken by the fixinclude process,

            
            then you will need to alter or add information to the include fix

            
            definitions file, ``inclhack.def''.  Please also send relevant

            
            information according to https://gcc.gnu.org/bugs/, to

            
            gcc-patches@gcc.gnu.org and, please, to me:  bkorb@gnu.org.


            To make your fix, you will need to do several things:


            1.  Obtain access to the AutoGen program on some platform.  It does

            
                not have to be your build platform, but it is more convenient.


            2.  Edit "inclhack.def" to reflect the changes you need to make.

            
                See below for information on how to make those changes.


            3.  Run the "genfixes" shell script to produce a new copy of

            
                the "fixincl.x" file.


            4.  Rebuild the compiler and check the header causing the issue.

            
                Make sure it is now properly handled.  Add tests to the

            
                "test_text" entry(ies) that validate your fix.  This will

            
                help ensure that future fixes won't negate your work.

            
                Do *NOT* specify test text for "wrap" or "replacement" fixes.

            
                There is no real possibility that these fixes will fail.

            
                If they do, you will surely know straight away.


                NOTE:  "test_text" is interpreted by the shell as it gets

            
                copied into the test header.  THEREFORE you must quote

            
                dollar sign characters and back quotes -- unless you mean

            
                for them to be interpreted by the shell.


                e.g. the math_huge_val_from_dbl_max test_text needs to

            
                put text into both float.h and math.h, so it includes a

            
                back-quoted script to add text to float.h.


            5.  Go into the fixincludes build directory and type, "make check".

            
                You are guaranteed to have issues printed out as a result.

            
                Look at the diffs produced.  Make sure you have not clobbered

            
                the proper functioning of a different fix.  Make sure your

            
                fix is properly tested and it does what it is supposed to do.


            6.  Now that you have the right things happening, synchronize the

            
                $(srcdir)/tests/base directory with the $(builddir)/tests/res

            
                directory.  The output of "make check" will be some diffs that

            
                should give you some hints about what to do.


            7.  Rerun "make check" and verify that there are no issues left.


            MAKING CHANGES TO INCLHACK.DEF

            
            ==============================


            0.  If you are not the fixincludes maintainer, please send that

            
                person email about any changes you may want to make.  Thanks!


            1.  Every fix must have a "hackname" that is compatible with C syntax

            
                for variable names and is unique without regard to alphabetic case.

            
                Please keep them alphabetical by this name.  :-)


            2.  If the problem is known to exist only in certain files, then

            
                identify the files with "files = " entries.  If you use fnmatch(3C)

            
                wild card characters in a "files" entry, be certain that the first

            
                "files" entry has no such character.  Otherwise, the "make check"

            
                machinery will attempt to create files with those characters in the

            
                name.  That is inconvenient.


            3.  It is relatively expensive to fire off a process to fix a source

            
                file, therefore write apply tests to avoid unnecessary fix

            
                processes.  The preferred apply tests are "select", "bypass", "mach"

            
                "sum", and "c-test" because they are performed internally:


                * select - Run a regex on the contents of the file being considered.

            
                           All such regex-es must match.  Matching is done with

            
                           extended regular expressions.


                * bypass - Run a regex on the contents of the file being considered.

            
                           No such regex may match.


                * sum    - Select a specific version of a file that has a matching

            
                           check sum.  The BSD version of checksum ["sum(1BSD)"]

            
                           is used.  Each "sum" entry should contain exactly three

            
                           space separated tokens:  the sum, some number and the

            
                           basename of the file.  The "some number" is ignored.

            
                           If there are multiple "sum" entries, only one needs to

            
                           match in order to pass.  For example:


                               sum = '1234 3 foobar.h';


                           specifies that the "foobar.h" header in any directory

            
                           will match if it has the checksum 1234.


                * c_test - call a function in fixtests.c.  See that file.


                * files  - the "fnmatch" pattern of the file(s) to examine for

            
                           the issue.  There may be several copies of this attribute.

            
                           If the header lives in a /usr/include subdirectory, be

            
                           sure to include that subdirectory in the name. e.g. net/if.h


                * mach   - Match the output of config.guess against a series of fnmatch

            
                           patterns.  It must match at least one of the patterns, unless

            
                           "not-machine" has also been specified.  In that case, the

            
                           config.guess output must not match any of the patterns.


                The next test is relatively slow because it must be handled in a

            
                separate shell process.  Some platforms do not support server shells,

            
                so the whole process is even slower and more cumbersome there.


                * test   - These should be arguments to the program, "/bin/test".

            
                           You may perform multiple commands, if you enclose them

            
                           in backquotes and echo out valid test arguments.  For

            
                           example, you might echo out '0 -eq 1' if you want a false

            
                           result, or '0 -eq 0' for a true result.


                These tests are required to:


                1.  Be positive for all header files that require the fix.


                It is desirable to:


                2.  Be negative as often as possible whenever the fix is not

            
                    required, avoiding the process overhead.


                It is nice if:


                3.  The expression is as simple as possible to both

            
                    process and understand by people.  :-)


                    Please take advantage of the fact AutoGen will glue

            
                    together string fragments.  It helps.  Also take note

            
                    that double quote strings and single quote strings have

            
                    different formation rules.  Double quote strings are a

            
                    tiny superset of ANSI-C string syntax.  Single quote

            
                    strings follow shell single quote string formation

            
                    rules, except that the backslash is processed before

            
                    '\\', '\'' and '#' characters (using C character syntax).


                Each test must pass or the fix is not applied.  For example,

            
                all "select" expressions must be found and not one "bypass"

            
                selection may be found.


                Examples of test specifications:


                  hackname = broken_assert_stdio;

            
                  files    = assert.h;

            
                  select   = stderr;

            
                  bypass   = "include.*stdio.h";


                The ``broken_assert_stdio'' fix will be applied only to a file

            
                named "assert.h" if it contains the string "stderr" _and_ it

            
                does _not_ contain the expression "include.*stdio.h".


                  hackname = no_double_slash;

            
                  c_test   = "double_slash";


                The ``no_double_slash'' fix will be applied if the

            
                ``double_slash_test()'' function says to.  See ``fixtests.c''

            
                for documentation on how to include new functions into that

            
                module.


            4.  There are currently four methods of fixing a file:


                1.  a series of sed expressions.  Each will be an individual

            
                    "-e" argument to a single invocation of sed.  Unless you

            
                    need to use multiple or complex sed expressions, please

            
                    use the "replacement text" method instead.


                2.  a shell script.  These scripts are _required_ to read all

            
                    of stdin in order to avoid pipe stalls.  They may choose to

            
                    discard the input.


                3.  Replacement text.  If the replacement is empty, then no

            
                    fix is applied.  Otherwise, the replacement text is

            
                    written to the output file and no further fixes are

            
                    applied.  If you really want a no-op file, replace the

            
                    file with a comment.


                    Replacement text "fixes" must be first in this file!!


                4.  A C language subroutine method for both tests and fixes.

            
                    See ``fixtests.c'' for instructions on writing C-language

            
                    applicability tests and ``fixfixes.c'' for C-language fixing.

            
                    These files also contain tables that describe the currently

            
                    implemented fixes and tests.


                If at all possible, you should try to use one of the C language

            
                fixes as it is far more efficient.  There are currently five

            
                such fixes, three of which are very special purpose:


                i) char_macro_def - This function repairs the definition of an

            
                    ioctl macro that presumes CPP macro substitution within

            
                    pairs of single quote characters.


                ii) char_macro_use - This function repairs the usage of ioctl

            
                    macros that no longer can wrap an argument with single quotes.


                iii) machine_name - This function will look at "#if", "#ifdef",

            
                    "#ifndef" and "#elif" directive lines and replace the first

            
                    occurrence of a non-reserved name that is traditionally

            
                    pre-defined by the native compiler.


                The next two are for general use:


                iv) wrap - wraps the entire file with "#ifndef", "#define" and

            
                    "#endif" self-exclusionary text.  It also, optionally, inserts

            
                    a prolog after the "#define" and an epilog just before the

            
                    "#endif".  You can use this for a fix as follows:


                        c_fix     = wrap;

            
                        c_fix_arg = "/* prolog text */";

            
                        c_fix_arg = "/* epilog text */";


                    If you want an epilog without a prolog, set the first "c_fix_arg"

            
                    to the empty string.  Both or the second "c_fix_arg"s may be

            
                    omitted and the file will still be wrapped.


            	THERE IS A SPECIAL EXCEPTION TO THIS, HOWEVER:


            	If the regular expression '#if.*__need' is found, then it is

            
            	assumed that the file needs to be read and interpreted more

            
            	than once.  However, the prolog and epilog text (if any) will

            
            	be inserted.


                v) format - Replaces text selected with a regular expression with

            
                    a specialized formatting string.  The formatting works as follows:

            
                    The format text is copied to the output until a '%' character

            
                    is found.  If the character after the '%' is another '%', then

            
                    one '%' is output and processing continues.  If the following

            
                    character is not a digit, then the '%' and that character are

            
                    copied and processing continues.  Finally, if the '%' *is*

            
                    followed by a digit, that digit is used as an index into the

            
                    regmatch_t array to replace the two characters with the matched

            
                    text.  i.e.: "%0" is replaced by the full matching text, "%1"

            
                    is the first matching sub-expression, etc.


                    This is used as follows:


                        c_fix     = format;

            
                        c_fix_arg = "#ifndef %1\n%0\n#endif";

            
                        c_fix_arg = "#define[ \t]+([A-Z][A-Z0-9a-z_]*).*";


                    This would wrap a one line #define inside of a "#ifndef"/"#endif"

            
                    pair.  The second "c_fix_arg" may be omitted *IF* there is at least

            
                    one select clause and the first one identifies the text you wish to

            
                    reformat.  It will then be used as the second "c_fix_arg".  You may

            
                    delete the selected text by supplying an empty string for the

            
                    replacement format (the first "c_fix_arg").


            	Note: In general, a format c_fix may be used in place of one

            
            	sed expression.  However, it will need to be rewritten by

            
            	hand.  For example:


            	sed = 's@^#if __GNUC__ == 2 && __GNUC_MINOR__ >= 7$'

            
            	       '@& || __GNUC__ >= 3@';


            	may be rewritten using a format c_fix as:


            	c_fix     = format;

            
            	c_fix_arg = '%0 || __GNUC__ >= 3';

            
            	c_fix_arg = '^#if __GNUC__ == 2 && __GNUC_MINOR__ >= 7$';


            	Multiple sed substitution expressions probably ought to remain sed

            
            	expressions in order to maintain clarity.  Also note that if the

            
            	second sed expression is the same as the first select expression,

            
            	then you may omit the second c_fix_arg.  The select expression will

            
            	be picked up and used in its absence.


            EXAMPLES OF FIXES:

            
            ==================


                  hackname = AAA_ki_iface;

            
                  replace; /* empty replacement -> no fixing the file */


                When this ``fix'' is invoked, it will prevent any fixes

            
                from being applied.


                ------------------


                  hackname = AAB_svr4_no_varargs;

            
                  replace  = "/* This file was generated by fixincludes.  */\n"

            
                             "#ifndef _SYS_VARARGS_H\n"

            
                             "#define _SYS_VARARGS_H\n\n"


                             "#ifdef __STDC__\n"

            
                             "#include <stdarg.h>\n"

            
                             "#else\n"

            
                             "#include <varargs.h>\n"

            
                             "#endif\n\n"


                             "#endif  /* _SYS_VARARGS_H */\n";


                When this ``fix'' is invoked, the replacement text will be

            
                emitted into the replacement include file.  No further fixes

            
                will be applied.


                ------------------


                    hackname  = hpux11_fabsf;

            
                    files     = math.h;

            
                    select    = "^[ \t]*#[ \t]*define[ \t]+fabsf\\(.*";

            
                    bypass    = "__cplusplus";


                    c_fix     = format;

            
                    c_fix_arg = "#ifndef __cplusplus\n%0\n#endif";


                    test_text =

            
                    "#  define fabsf(x) ((float)fabs((double)(float)(x)))\n";


                This fix will ensure that the #define for fabs is wrapped

            
                with C++ protection, providing the header is not already

            
                C++ aware.


                ------------------


            5.  Testing fixes.


                The brute force method is, of course, to configure and build

            
                GCC.  But you can also:


                    cd ${top_builddir}/gcc

            
                    rm -rf include-fixed/ stmp-fixinc

            
                    make stmp-fixinc


                I would really recommend, however:


                    cd ${top_builddir}/fixincludes

            
                    make check


                To do this, you *must* have autogen installed on your system.

            
                The "check" step will proceed to construct a shell script that

            
                will exercise all the fixes, using the sample test_text

            
                provided with each fix.  Once done, the changes made will

            
                be compared against the changes saved in the source directory.

            
                If you are changing the tests or fixes, the change will likely

            
                be highlighted.

	GCC MAINTAINER INFORMATION
	==========================

	If you are having some problem with a system header that is either
	broken by the manufacturer, or is broken by the fixinclude process,
	then you will need to alter or add information to the include fix
	definitions file, ``inclhack.def''. Please also send relevant
	information according to https://gcc.gnu.org/bugs/, to
	gcc-patches@gcc.gnu.org and, please, to me: bkorb@gnu.org.

	To make your fix, you will need to do several things:

	1. Obtain access to the AutoGen program on some platform. It does
	not have to be your build platform, but it is more convenient.

	2. Edit "inclhack.def" to reflect the changes you need to make.
	See below for information on how to make those changes.

	3. Run the "genfixes" shell script to produce a new copy of
	the "fixincl.x" file.

	4. Rebuild the compiler and check the header causing the issue.
	Make sure it is now properly handled. Add tests to the
	"test_text" entry(ies) that validate your fix. This will
	help ensure that future fixes won't negate your work.
	Do NOT specify test text for "wrap" or "replacement" fixes.
	There is no real possibility that these fixes will fail.
	If they do, you will surely know straight away.

	NOTE: "test_text" is interpreted by the shell as it gets
	copied into the test header. THEREFORE you must quote
	dollar sign characters and back quotes -- unless you mean
	for them to be interpreted by the shell.

	e.g. the math_huge_val_from_dbl_max test_text needs to
	put text into both float.h and math.h, so it includes a
	back-quoted script to add text to float.h.

	5. Go into the fixincludes build directory and type, "make check".
	You are guaranteed to have issues printed out as a result.
	Look at the diffs produced. Make sure you have not clobbered
	the proper functioning of a different fix. Make sure your
	fix is properly tested and it does what it is supposed to do.

	6. Now that you have the right things happening, synchronize the
	$(srcdir)/tests/base directory with the $(builddir)/tests/res
	directory. The output of "make check" will be some diffs that
	should give you some hints about what to do.

	7. Rerun "make check" and verify that there are no issues left.


	MAKING CHANGES TO INCLHACK.DEF
	==============================

	0. If you are not the fixincludes maintainer, please send that
	person email about any changes you may want to make. Thanks!

	1. Every fix must have a "hackname" that is compatible with C syntax
	for variable names and is unique without regard to alphabetic case.
	Please keep them alphabetical by this name. :-)

	2. If the problem is known to exist only in certain files, then
	identify the files with "files = " entries. If you use fnmatch(3C)
	wild card characters in a "files" entry, be certain that the first
	"files" entry has no such character. Otherwise, the "make check"
	machinery will attempt to create files with those characters in the
	name. That is inconvenient.

	3. It is relatively expensive to fire off a process to fix a source
	file, therefore write apply tests to avoid unnecessary fix
	processes. The preferred apply tests are "select", "bypass", "mach"
	"sum", and "c-test" because they are performed internally:

	* select - Run a regex on the contents of the file being considered.
	All such regex-es must match. Matching is done with
	extended regular expressions.

	* bypass - Run a regex on the contents of the file being considered.
	No such regex may match.

	* sum - Select a specific version of a file that has a matching
	check sum. The BSD version of checksum ["sum(1BSD)"]
	is used. Each "sum" entry should contain exactly three
	space separated tokens: the sum, some number and the
	basename of the file. The "some number" is ignored.
	If there are multiple "sum" entries, only one needs to
	match in order to pass. For example:

	sum = '1234 3 foobar.h';

	specifies that the "foobar.h" header in any directory
	will match if it has the checksum 1234.

	* c_test - call a function in fixtests.c. See that file.

	* files - the "fnmatch" pattern of the file(s) to examine for
	the issue. There may be several copies of this attribute.
	If the header lives in a /usr/include subdirectory, be
	sure to include that subdirectory in the name. e.g. net/if.h

	* mach - Match the output of config.guess against a series of fnmatch
	patterns. It must match at least one of the patterns, unless
	"not-machine" has also been specified. In that case, the
	config.guess output must not match any of the patterns.

	The next test is relatively slow because it must be handled in a
	separate shell process. Some platforms do not support server shells,
	so the whole process is even slower and more cumbersome there.

	* test - These should be arguments to the program, "/bin/test".
	You may perform multiple commands, if you enclose them
	in backquotes and echo out valid test arguments. For
	example, you might echo out '0 -eq 1' if you want a false
	result, or '0 -eq 0' for a true result.

	These tests are required to:

	1. Be positive for all header files that require the fix.

	It is desirable to:

	2. Be negative as often as possible whenever the fix is not
	required, avoiding the process overhead.

	It is nice if:

	3. The expression is as simple as possible to both
	process and understand by people. :-)

	Please take advantage of the fact AutoGen will glue
	together string fragments. It helps. Also take note
	that double quote strings and single quote strings have
	different formation rules. Double quote strings are a
	tiny superset of ANSI-C string syntax. Single quote
	strings follow shell single quote string formation
	rules, except that the backslash is processed before
	'\\', '\'' and '#' characters (using C character syntax).

	Each test must pass or the fix is not applied. For example,
	all "select" expressions must be found and not one "bypass"
	selection may be found.

	Examples of test specifications:

	hackname = broken_assert_stdio;
	files = assert.h;
	select = stderr;
	bypass = "include.*stdio.h";

	The ``broken_assert_stdio'' fix will be applied only to a file
	named "assert.h" if it contains the string "stderr" _and_ it
	does _not_ contain the expression "include.*stdio.h".

	hackname = no_double_slash;
	c_test = "double_slash";

	The ``no_double_slash'' fix will be applied if the
	``double_slash_test()'' function says to. See ``fixtests.c''
	for documentation on how to include new functions into that
	module.

	4. There are currently four methods of fixing a file:

	1. a series of sed expressions. Each will be an individual
	"-e" argument to a single invocation of sed. Unless you
	need to use multiple or complex sed expressions, please
	use the "replacement text" method instead.

	2. a shell script. These scripts are _required_ to read all
	of stdin in order to avoid pipe stalls. They may choose to
	discard the input.

	3. Replacement text. If the replacement is empty, then no
	fix is applied. Otherwise, the replacement text is
	written to the output file and no further fixes are
	applied. If you really want a no-op file, replace the
	file with a comment.

	Replacement text "fixes" must be first in this file!!

	4. A C language subroutine method for both tests and fixes.
	See ``fixtests.c'' for instructions on writing C-language
	applicability tests and ``fixfixes.c'' for C-language fixing.
	These files also contain tables that describe the currently
	implemented fixes and tests.

	If at all possible, you should try to use one of the C language
	fixes as it is far more efficient. There are currently five
	such fixes, three of which are very special purpose:

	i) char_macro_def - This function repairs the definition of an
	ioctl macro that presumes CPP macro substitution within
	pairs of single quote characters.

	ii) char_macro_use - This function repairs the usage of ioctl
	macros that no longer can wrap an argument with single quotes.

	iii) machine_name - This function will look at "#if", "#ifdef",
	"#ifndef" and "#elif" directive lines and replace the first
	occurrence of a non-reserved name that is traditionally
	pre-defined by the native compiler.

	The next two are for general use:

	iv) wrap - wraps the entire file with "#ifndef", "#define" and
	"#endif" self-exclusionary text. It also, optionally, inserts
	a prolog after the "#define" and an epilog just before the
	"#endif". You can use this for a fix as follows:

	c_fix = wrap;
	c_fix_arg = "/* prolog text */";
	c_fix_arg = "/* epilog text */";

	If you want an epilog without a prolog, set the first "c_fix_arg"
	to the empty string. Both or the second "c_fix_arg"s may be
	omitted and the file will still be wrapped.

	THERE IS A SPECIAL EXCEPTION TO THIS, HOWEVER:

	If the regular expression '#if.*__need' is found, then it is
	assumed that the file needs to be read and interpreted more
	than once. However, the prolog and epilog text (if any) will
	be inserted.

	v) format - Replaces text selected with a regular expression with
	a specialized formatting string. The formatting works as follows:
	The format text is copied to the output until a '%' character
	is found. If the character after the '%' is another '%', then
	one '%' is output and processing continues. If the following
	character is not a digit, then the '%' and that character are
	copied and processing continues. Finally, if the '%' is
	followed by a digit, that digit is used as an index into the
	regmatch_t array to replace the two characters with the matched
	text. i.e.: "%0" is replaced by the full matching text, "%1"
	is the first matching sub-expression, etc.

	This is used as follows:

	c_fix = format;
	c_fix_arg = "#ifndef %1\n%0\n#endif";
	c_fix_arg = "#define[ \t]+([A-Z][A-Z0-9a-z_]).";

	This would wrap a one line #define inside of a "#ifndef"/"#endif"
	pair. The second "c_fix_arg" may be omitted IF there is at least
	one select clause and the first one identifies the text you wish to
	reformat. It will then be used as the second "c_fix_arg". You may
	delete the selected text by supplying an empty string for the
	replacement format (the first "c_fix_arg").

	Note: In general, a format c_fix may be used in place of one
	sed expression. However, it will need to be rewritten by
	hand. For example:

	sed = 's@^#if __GNUC__ == 2 && __GNUC_MINOR__ >= 7$'
	'@& \|\| __GNUC__ >= 3@';

	may be rewritten using a format c_fix as:

	c_fix = format;
	c_fix_arg = '%0 \|\| __GNUC__ >= 3';
	c_fix_arg = '^#if __GNUC__ == 2 && __GNUC_MINOR__ >= 7$';

	Multiple sed substitution expressions probably ought to remain sed
	expressions in order to maintain clarity. Also note that if the
	second sed expression is the same as the first select expression,
	then you may omit the second c_fix_arg. The select expression will
	be picked up and used in its absence.

	EXAMPLES OF FIXES:
	==================

	hackname = AAA_ki_iface;
	replace; /* empty replacement -> no fixing the file */

	When this ``fix'' is invoked, it will prevent any fixes
	from being applied.

	------------------

	hackname = AAB_svr4_no_varargs;
	replace = "/* This file was generated by fixincludes. */\n"
	"#ifndef _SYS_VARARGS_H\n"
	"#define _SYS_VARARGS_H\n\n"

	"#ifdef __STDC__\n"
	"#include <stdarg.h>\n"
	"#else\n"
	"#include <varargs.h>\n"
	"#endif\n\n"

	"#endif /* _SYS_VARARGS_H */\n";

	When this ``fix'' is invoked, the replacement text will be
	emitted into the replacement include file. No further fixes
	will be applied.

	------------------

	hackname = hpux11_fabsf;
	files = math.h;
	select = "^[ \t]#[ \t]define[ \t]+fabsf\\(.*";
	bypass = "__cplusplus";

	c_fix = format;
	c_fix_arg = "#ifndef __cplusplus\n%0\n#endif";

	test_text =
	"# define fabsf(x) ((float)fabs((double)(float)(x)))\n";

	This fix will ensure that the #define for fabs is wrapped
	with C++ protection, providing the header is not already
	C++ aware.

	------------------

	5. Testing fixes.

	The brute force method is, of course, to configure and build
	GCC. But you can also:

	cd ${top_builddir}/gcc
	rm -rf include-fixed/ stmp-fixinc
	make stmp-fixinc

	I would really recommend, however:

	cd ${top_builddir}/fixincludes
	make check

	To do this, you must have autogen installed on your system.
	The "check" step will proceed to construct a shell script that
	will exercise all the fixes, using the sample test_text
	provided with each fix. Once done, the changes made will
	be compared against the changes saved in the source directory.
	If you are changing the tests or fixes, the change will likely
	be highlighted.