Nick Bowler [Fri, 7 Jul 2023 03:56:29 +0000 (23:56 -0400)]
libcdecl: Re-use strings from parser in spec_string.
With the fixups applied by fix-yytname, we now have two distinct string
tables containing the same strings. To avoid this duplication, add a
new internal function to allow spec_string to access the parser's token
name table.
Nick Bowler [Fri, 7 Jul 2023 02:46:20 +0000 (22:46 -0400)]
libcdecl: Replace yytname array in the Bison parser.
Bison generates a list of symbol names as a static array of char pointers
initialized with string literals, which is simply horrible. These pointers
are two to four times larger than necessary, but with position-independent
code this also forces them into an unshareable, writeable segment since
they are not compile-time constants and must be initialized by the dynamic
loader at runtime.
Furthermore, the names of nonterminal symbols are always included but they
are not always needed; they should only be output by tracing code which
is disabled by default at compile time.
Fix this by adding a new build script which postprocesses the output,
replacing the yytname array with a function implementing the same lookup
with truly constant tables. Inclusion of nonterminals is now conditional
on YYDEBUG.
This is a big win, reducing the overall library size about 5 to 10 percent
(64-bit hosts see the most improvement).
Nick Bowler [Fri, 7 Jul 2023 00:57:41 +0000 (20:57 -0400)]
tests: Don't pull output routines into normalize testcase.
This program isn't about testing the output routines, and it's simple
enough to just have the test program do its own printing, so linking
in the output routines to this testcase is just causing more trouble
than it's worth.
Nick Bowler [Fri, 7 Jul 2023 00:50:22 +0000 (20:50 -0400)]
libcdecl: Combine error and i18n init.
We can just do the gettext initialization at the same time as error
initialization, which lets us use a single "glthread_once" invocation
for everything.
Nick Bowler [Thu, 6 Jul 2023 06:15:35 +0000 (02:15 -0400)]
libcdecl: Fix regression in function parsing.
Merging the reduce_parentheses and simplify_functions step missed a case
where the function simplification could pull a fake function declarator
up to the current tree position. Continuing the tree traversal will miss
this, leaving the fake parameter in the tree (which is then rejected as
a syntax error later).
The fix is easy enough, just repeat the whole reduction step until no
changes are needed at the current position before continuing traversal.
Nick Bowler [Thu, 6 Jul 2023 05:59:43 +0000 (01:59 -0400)]
libcdecl: Remove "too many parentheses" error check.
It is simply not the case that "int ((int))" is invalid.
Since "int (x(int))" declares x as a function returning int, it follows
that "int ((int))" is simply a type name for function returning int.
Likewise, "int (((int)))" is equivalent, etc.
Rejecting these type names is longstanding broken behaviour. I think
the entire error check can simply be deleted outright, along with its
associated error message. The bogus negative test cases are replaced
with new positive tests.
Furthermore, rewrite the big comment that suggests this was reasonable
behaviour, replacing it with a (hopefully) better explanation that
covers how this particular case should be interpreted.
Nick Bowler [Wed, 5 Jul 2023 01:27:47 +0000 (21:27 -0400)]
cdecl99: Don't check blank lines if no history support.
Move the whole "if not blank, then add history" logic under the #ifdef,
so it can be eliminated completely if add_history is not available at
compile time.
This probably makes no major difference on GNU/Linux, since GCC knows
that strspn has no side effects and can drop the call as dead code, but
this may help with less sophisticated toolchains.
Nick Bowler [Wed, 5 Jul 2023 01:15:39 +0000 (21:15 -0400)]
cdecl99: Fall back to getline, instead of Gnulib's readline.
When building without readline, instead of using Gnulib's readline
replacement, use getline directly. We can combine most of the batch
and interactive mode processing loops which saves a bunch of pointless
extra code in the readline-disabled case.
To actually do this, we prepare a Gnulib "local dir" to implement
a cut-down readline module that includes the configure tests only.
Simplify the reduce_parentheses implementation so it operates in a
much more straightforward way. We can also just call simplify_functions
directly during the same tree traversal, instead of trying to predict
what simplify_functions will do to a node on a later traversal.
Nick Bowler [Fri, 30 Jun 2023 01:22:44 +0000 (21:22 -0400)]
libcdecl: Combine several postprocessing steps into one.
We currently have 5 postprocessing steps common to both parse variants,
four of which don't do any modification at all (just syntax checks).
This is overkill. We can easily combine all five of these operations
into one function which does everything in a single tree traversal.
Nick Bowler [Wed, 28 Jun 2023 05:07:59 +0000 (01:07 -0400)]
libcdecl: Simplify declarator traversal logic.
It seems unnecessary to handle the case where the traversal function
modifies the subtree differently from the case where the function does
not modify the subtree.
So we only have to deal with two cases for the traversal function's
return value, which allows us to simplify a bit.
Nick Bowler [Wed, 28 Jun 2023 02:48:11 +0000 (22:48 -0400)]
libcdecl: Merge both parser function implementations.
The only remaining meaningful difference between the cdecl_parse_decl
and cdecl_parse_english implementations is the function-simplification
passes are only done for cdecl_parse_decl.
We can easily just make those conditional in a common parser function,
to reduce code duplication.
Nick Bowler [Fri, 23 Jun 2023 05:11:15 +0000 (01:11 -0400)]
libcdecl: Avoid vsnprintf for error reporting.
Using vsnprintf is overkill here. We only need to handle a single %s
conversion, which can be done by direct call to snprintf.
As this is the only caller of vsnprintf, dropping it means we can drop
the vsnprintf gnulib module. Also, at least with gcc, using va_start
and such is fairly expensive. The direct-snprintf version is quite
a bit more compact.
Nick Bowler [Fri, 23 Jun 2023 04:39:45 +0000 (00:39 -0400)]
libcdecl: Make cdecl__emit_specs return value usable directly.
Since there is now only one caller of cdecl__emit_specs that cares
about its return value, let's adjust this function to return exactly
what that caller wants, so it doesn't have to do any extra calculation.
Nick Bowler [Fri, 23 Jun 2023 04:05:41 +0000 (00:05 -0400)]
libcdecl: Accumulate output length in structure.
Instead of cascading the lengths back via function return values, since
we now have a state structure we can just track the total length in one
place. This is quite a lot conceptually simpler and cuts out a good
chunk of code.
Nick Bowler [Fri, 23 Jun 2023 03:25:50 +0000 (23:25 -0400)]
libcdecl: Use a structure for dst/dstlen in output routines.
Since pretty much all the output functions now just directly pass the
dst and dstlen pointers around without touching them in any other way,
it is more efficient to use a structure so there is only one pointer.
Nick Bowler [Fri, 23 Jun 2023 02:00:34 +0000 (22:00 -0400)]
libcdecl: Rework specifier output logic.
With both the explain and declare code paths using cdecl__emit, we
can give the same treatment to cdecl__explain_specs (now called
cdecl__emit_specs) to simplify things a bit.
This removes the last caller of cdecl__advance, so we can remove
that function (and use this better name for cdecl__advance_).
Nick Bowler [Fri, 23 Jun 2023 01:24:19 +0000 (21:24 -0400)]
libcdecl: Rework cdecl_declare output logic.
Instead of adjusting the output pointer/length values after each
internal call, tweak all the functions in declare.c to take an extra
level of indirection so they can just directly adjust the destination
as they go.
Nick Bowler [Fri, 23 Jun 2023 01:21:46 +0000 (21:21 -0400)]
libcdecl: Rework cdecl_explain output logic.
Instead of adjusting the output pointer/length values after each
internal call, tweak all the functions in explain.c to take an
extra level of indirection so they can just directly adjust the
destination as they go.
A new helper, cdecl__emit, is provided to simplify the common pattern
of "print a single C string" followed by "advance pointer/length."
Nick Bowler [Thu, 22 Jun 2023 05:20:12 +0000 (01:20 -0400)]
libcdecl: Simplify cdecl__explain_specs.
A bunch of conditions in the specifier printing loop are meaningless,
since all valid specifier types take idenical execution paths through
this function. So much of this can just be deleted.
Nick Bowler [Thu, 22 Jun 2023 05:01:46 +0000 (01:01 -0400)]
libcdecl: Improve specifier to string conversions.
Instead of a big switch statement, we can generate some compact lookup
tables to convert specifiers to strings, which appears to produce much
better results with gcc at least.
Nick Bowler [Thu, 22 Jun 2023 02:11:07 +0000 (22:11 -0400)]
tests: Adjust positive tests to verify both parse directions.
We don't actually have any canned tests of the "declare" or "type"
commands to produce C syntax from pseudo-English. While the randomized
crossparse test does provide some coverage of this, failures here
are difficult to understand compared to more simple, standalone test
cases.
So to start, let's just expand the existing "positive" tests to check
in both directions.
Rather than threading a flag through this function just to print
"declare" or "type" at the toplevel, we can just directly print
that in the one place where it is needed, which simplifies the
implementation a bit.
Since reworking how specifiers are printed, the "pre"/"post" specifier
printing functions are pretty much pointless, and can (mostly) be
deleted, which reduces the library size somewhat.
Nick Bowler [Fri, 16 Jun 2023 04:34:41 +0000 (00:34 -0400)]
cdecl99: Avoid passing uninitialized value to help_print_option.
For options that do not take arguments, the "arg" member of the help
structure is not assigned by lopt_get_help. Since the structure
is not otherwise initialized, it is technically undefined to even
evaluate this member in order to pass it to the help_print_option
function.
In this case, the argument is not actually used, and I'm not aware
of any actual failures as a result, but it easy enough to avoid.
Nick Bowler [Fri, 16 Jun 2023 04:12:42 +0000 (00:12 -0400)]
tests: Eliminate random floating-point generation.
The only use of test_rng_uniform in the test suite is to do 50/50 coin
toss type checks, add a simple helper based on test_rng_uniform_int to
do this, instead of bringing in all this floating-point machinery for
no real reason.
Nick Bowler [Fri, 16 Jun 2023 04:01:07 +0000 (00:01 -0400)]
libcdecl: Avoid stray semicolon after gl_once_define.
For some weird reason the gnulib gl_once_define function-like macro
expansion includes the semicolon. Thus, the extra semicolon after
the macro invocation is technically a syntax error, although most
compilers seem to not care too much.
Nick Bowler [Thu, 15 Jun 2023 00:34:56 +0000 (20:34 -0400)]
libcdecl: Tweak invalid character error from scanner.
By adjusting how we format the error message, the format string is
changed to be completely identical to a format string used by the
parser error reporting, which avoids some wasted code space in
the library.
Furthermore, make some tweaks to the invalid character pretty-printing
which seems to let GCC generate a bit more compact code.
Nick Bowler [Fri, 9 Jun 2023 14:43:36 +0000 (10:43 -0400)]
libcdecl: Consolidate most error messages.
Almost every error message returned by the library is a fixed
string describing some particular syntax problem. Keep this
list of strings in one place, and add a new internal helper
to report one of these errors.
Furthermore, reduce the amount of distinct error codes returned
to the user to (once again) just two, as the current plethora of
codes seems completely pointless.
Nick Bowler [Tue, 13 Jun 2023 06:14:44 +0000 (02:14 -0400)]
Use gnulib's vsnprintf module.
The snprintf module provides only snprintf, not vsnprintf. As we
currently depend on both functions in the library, it is necessary
to use both modules.
This fixes failures in the new cdeclerr test on HP-UX 11, which has
vsnprintf but it is not entirely C99-like (wrong return value).
Nick Bowler [Thu, 1 Jun 2023 00:29:41 +0000 (20:29 -0400)]
Factor out common parser invocation.
The two main parsing functions have nearly identical parser invocation
sequences, with the only difference being the flag passed to the scanner
init. Split that off into a separate function, to simplify the code.
Additionally, if the code is compiled with YYDEBUG, enable parser
debugging.
Nick Bowler [Tue, 30 May 2023 02:10:54 +0000 (22:10 -0400)]
Restore gperf-related definitions to makefile.
Previously, gnulib was providing definitions for GPERF and other
variables in the makefile indirectly via the striconv module. But
since we removed that module, the definitions disappeared, leading
to build failures after maintainer-clean.
Easy enough to just do the same thing explicitly. In the future
we might want to use configure to locate gperf but for now this
will do.
Nick Bowler [Mon, 29 May 2023 02:05:17 +0000 (22:05 -0400)]
cdecl99: Use packed option format from gen-options.awk.
This feature uses a compact, fully constant array to generate the
real struct option array at runtime. This allows the full-sized array
to be dropped after command-line processing is finished, and with
position-independent executables, reduces the amount of relocation
processing needed.
Nick Bowler [Sat, 27 May 2023 01:25:04 +0000 (21:25 -0400)]
Replace Gnulib striconv with copyright_symbol from dxcommon.
The only use of the striconv module is to produce the copyright symbol
for --version output and at the start of an interactive session. Using
the new, single-purpose function reduces code size quite a bit.
Nick Bowler [Sat, 27 May 2023 00:38:11 +0000 (20:38 -0400)]
Stop using gnulib's flexmember module.
The only thing we're actually using from this module is provided
directly by Autoconf, via AC_C_FLEXIBLE_ARRAY_MEMBER, so we can
just use that macro instead.
Nick Bowler [Thu, 25 May 2023 23:38:59 +0000 (19:38 -0400)]
cdecl99: Fix some improper error message formatting.
Errors opening the file specified by a --file option are printed with
two newlines, and errors generated by actual commands are not properly
prefixed with the program name.
Add test cases to catch these specific problems and fix them.
Nick Bowler [Tue, 24 Jan 2023 07:37:00 +0000 (02:37 -0500)]
Avoid POSIX character classes in the test suite.
Instead of [[:alnum:]] and friends, expand to an explicit list of
characters, which is a bit more portable usage (and also avoids
unneeded locale dependency). We can use macros to make this
just as convenient to write.
Nick Bowler [Tue, 24 Jan 2023 07:14:53 +0000 (02:14 -0500)]
Fix configuration on Solaris 8.
There is an quoting error in Gnulib's threadlib.m4 which causes
incorrect configuration settings on old Solaris. We can repeat
the simple check with correct quoting to workaround the problem.
Additionally, pull in DX_LINGUAS fixes from dxcommon to avoid
tripping on Solaris' pre-POSIX /bin/awk.
Nick Bowler [Tue, 24 Jan 2023 04:58:55 +0000 (23:58 -0500)]
cdecl99: Drop locale-sensitive isblank usage.
The intention of this function is to avoid recording no-op command lines
in the history. This is exactly the set of commands containing just
regular tabs and spaces.
It is inappropriate to use the locale-sensitive isblank for this, as
this may be a little bit different. In practice there is probably no
meaningful difference, but as isblank is a C99 feature losing this call
also helps when building against older C libraries that lack it.
Nick Bowler [Tue, 24 Jan 2023 04:47:18 +0000 (23:47 -0500)]
Provide strtoumax fallback in the scanner.
We already do something similar in the test suite. We don't really care
about the full range of uintmax_t, we just prefer the widest type that
is available to us. It is no real problem to fall back to a narrower
conversion function.
Nick Bowler [Tue, 10 Jan 2023 01:20:19 +0000 (20:20 -0500)]
Don't parse command-line options more than once.
Instead of parsing the options a second time to collect the --execute
option arguments, we can easily permute the argv array on the first
pass through, which simplifies the subsequent evaluation.
Nick Bowler [Fri, 18 Nov 2022 03:45:07 +0000 (22:45 -0500)]
Avoid Gnulib std-gnu11 module.
It has come to my attention that this module rewrites AC_PROG_CC in
a way that actually breaks Automake's AM_PROG_CC_C_O functionality.
This results in the "compile" script not being included or used in
packages bootstrapped with Autoconf 2.69.
With later versions of Autoconf things work because this module
doesn't touch things and thus disables itself.
I don't care about building with C11 one way or the other. Let's
just skip the module.
Nick Bowler [Fri, 22 Apr 2022 02:01:57 +0000 (22:01 -0400)]
Explicitly require gnulib getline module.
Currently, the gnulib getline module is only pulled in indirectly via
the readline module. As a result, when configuring for a system that
has GNU readline installed (and readline is enabled), the readline
replacement is not used and therefore the getline replacement is
never included.
But on systems that lack getline in the C library, without the gnulib
replacement the build will fail. Simply listing getline as a needed
module suffices to allow such configurations to build successfully.
Nick Bowler [Sun, 3 Apr 2022 01:56:22 +0000 (21:56 -0400)]
tests: Correct RNG implementation.
Due to a mistake in the adaptation, the output of the generator was
not done correctly. Correct that, and add a little test program that
would have caught this mistake by directly comparing against the
reference implementation.
Nick Bowler [Sat, 26 Mar 2022 04:00:56 +0000 (00:00 -0400)]
Bump dxcommon to fix builds using --disable-dependency-tracking.
Some build rules were inadvertently depending on directory creation that
happens as a side effect of depfiles generation. This does not happen
when configuring with --disable-dependency-tracking, leading to issues
with both VPATH builds and the gnulib symfiles machinery.
Failures are easily observed with a command like
make DISTCHECK_CONFIGURE_FLAGS=--disable-dependency-tracking distcheck
Using parallel make can hide problems since it seems Automake also
generates make rules that incidentally create these directories.
Nick Bowler [Sat, 19 Feb 2022 23:07:59 +0000 (18:07 -0500)]
Fix regressions caused by symbol renaming changes.
A bug in dxcommon was causing VPATH builds to continue to apply symbol
renaming to all gnulib sources. Fixing this revealed that dependency
tracking was broken for the non-renamed objects (distcheck noticed that
the dependency products were not being cleaned).
To fix dependency tracking Automake needs to be aware of all the source
files going into the statically-linked version of the library.
Unfortunately, Automake complains if we just add the same sources to a
non-libtool library, even though there is no real conflict since only
one will actually be built. The issue can be worked around by using
Automake's object renaming facilities, which complicates things slightly
but is straightforward enough to implement.
Nick Bowler [Fri, 18 Feb 2022 22:15:34 +0000 (17:15 -0500)]
Fix gen-typegen.awk incompatibility with busybox awk.
It seems that busybox awk does not support * in printf conversions.
There is only one use of this feature in the scripts and we can use
the substr function instead.
Nick Bowler [Fri, 18 Feb 2022 04:08:19 +0000 (23:08 -0500)]
Improve gnulib build times.
Use the new dxcommon features in an attempt to avoid the expensive
symbol renaming and PIC build steps for the portions of gnulib that
are not actually needed by the libcdecl library.
Nick Bowler [Thu, 10 Feb 2022 01:37:50 +0000 (20:37 -0500)]
Plug memory leak in declgen.
When a void typespec is generated in a context where it is invalid,
gen_typespecs just rolls the dice again. Unfortunately, the typespec
is not freed in this case, leaking memory. Easily fixed.
Nick Bowler [Wed, 9 Feb 2022 08:06:02 +0000 (03:06 -0500)]
Portability improvements for new random number generator.
On HP-UX 11, the ldexp function requires linking against libm.
Moreover, instead of strtoull declared in <stdlib.h> we have
__strtoull declared in <inttypes.h>. Add configure tests to
find these.
Nick Bowler [Wed, 9 Feb 2022 04:37:53 +0000 (23:37 -0500)]
Clean up declgen a bit.
Use wrapper functions to perform the "allocate a structure and
initialize its members" sequence which is a common sequence here,
and avoid the use of compound literals for this which improves
portability to older compilers.
Nick Bowler [Wed, 9 Feb 2022 04:34:17 +0000 (23:34 -0500)]
Simplify and improve randomdecl sanity test.
We don't need to do any weird shell variable stuff here, we can just
directly compute the expected output and verify against that. As a
bonus, when the test fails this gives a much better description of
which expected forms are missing in the testsuite log.
Nick Bowler [Wed, 9 Feb 2022 03:30:48 +0000 (22:30 -0500)]
Remove randomdecl test dependency on GSL.
It's a bit silly for a test application to depend on this huge library
just for random number generation. We can just directly incorporate
a simple RNG implementation which should be plenty good enough for
this purpose.
Nick Bowler [Thu, 14 Oct 2021 03:28:37 +0000 (23:28 -0400)]
Fix formatting error in libcdecl(3) man page.
The use of "e.g." at the end of a line confuses troff into thinking this
is the end of the sentence. That is not correct, so adjust the syntax
to avoid such interpretation.
Nick Bowler [Fri, 13 Aug 2021 02:41:14 +0000 (22:41 -0400)]
Make library i18n init conditional on NLS support.
If the package is configured with --disable-nls, the library's i18n
initialization does nothing useful but the internal functions are
not fully removed. Looks like a simple opportunity for improvement.
Nick Bowler [Fri, 13 Aug 2021 00:43:00 +0000 (20:43 -0400)]
Include glthread headers late.
It seems that in some configurations, the glthread headers can
include Windows headers which define macros that can conflict
with the libcdecl headers.
As the damage appears to be isolated to the headers, re-ordering
the includes appears to be sufficient to avoid any problems, at
least within the library.
Nick Bowler [Sat, 13 Mar 2021 22:23:46 +0000 (17:23 -0500)]
Rework library error reporting.
This removes (hopefully) all cases where the libcdecl prints
error messages directly to stderr, and reports these messages
via cdecl_get_error instead.
Nick Bowler [Fri, 12 Mar 2021 05:34:27 +0000 (00:34 -0500)]
Generate specifier strings directly from cdecl.h
Since the specs.lst file is no longer used for output ordering, it is
now mostly redundant information. There is another list of specifiers:
the enumeration constants in cdecl.h. We can fairly easily reconstruct
the strings from this instead.
Nick Bowler [Fri, 12 Mar 2021 04:54:24 +0000 (23:54 -0500)]
Restructure the type specifier check.
As the typemap.c functionality is only used for one thing and is quite
simple, let's integrate that more closely with the declaration specifier
checks. This further reduces the code size of the library somewhat.
The old sed line noise is replaced by a similar awk script which should
be much easier to adjust as needed.
Nick Bowler [Fri, 12 Mar 2021 04:58:52 +0000 (23:58 -0500)]
Hand-code the normalized specifier ordering.
For the most part, the enumerated values for specifier types in the
library are already in an acceptable order for normalization. This
list is part of the API and is expected to be stable.
We can achieve the desired ordering by simply tweaking these values
as needed. There is also no need to totally order the specifiers
because some combinations are never valid. This change not only
eliminates a pile of comparatively complex code generation, but
also reduces the overall size of the library somewhat.
Add a new testcase to validate the resulting ordering.
Nick Bowler [Wed, 10 Mar 2021 07:31:59 +0000 (02:31 -0500)]
Replace typegen.sh with a new and improved script.
The old typegen.sh has a bunch of portability problems; let's write
a new script in awk which is a better tool for this sort of code
generation task anyway.
Nick Bowler [Wed, 10 Mar 2021 01:34:05 +0000 (20:34 -0500)]
Work around designated initializer bug on HP-UX cc.
Work around a designated initializer bug encountered on old versions
of HP-UX. It seems there are some glitches leading to compilation
failures when the initializers are out of order (or omitted), and
the initializer for such a member is non-constant, and there is a
type conversion involved.
This affects several of the parser actions that initialize declspec
values. In this case, it is super easy to just change the type and
avoid type conversion, which easily avoids the bug.
Nick Bowler [Fri, 5 Mar 2021 00:15:37 +0000 (19:15 -0500)]
Add configure option to disable readline.
Apparently the Gnulib readline module does not provide any obvious way
for users to disable it. Setting the gl_cv_lib_readline cache variable
works but probably only Autoconfers will know how to discover that.
Add an explicit --with-readline (and --without-readline) option which
internally sets this. Since this option shows up in configure's help
output it is hopefully more obvious.
Nick Bowler [Wed, 3 Mar 2021 02:21:50 +0000 (21:21 -0500)]
Fix testcase compilation with --disable-shared.
Since the libtest library depends on functions in libcdecl,
we must list libcdecl after libtest on the linker command line,
otherwise required objects from static libcdecl will not be
pulled in to the link. A similar problem also occurs when
building tests with LDFLAGS=-Wl,--as-needed.
Nick Bowler [Tue, 2 Mar 2021 05:13:56 +0000 (00:13 -0500)]
Consolidate header files.
There is no need for this quantity of tiny header files. We can simply
use a single internal header file for libcdecl, and a single internal
header file for cdecl99, including all necessary declarations.