Nick Bowler [Wed, 28 Jun 2023 02:48:11 +0000 (22:48 -0400)]
libcdecl: Merge both parser function implementations.
The only remaining meaningful difference between the cdecl_parse_decl
and cdecl_parse_english implementations is the function-simplification
passes are only done for cdecl_parse_decl.
We can easily just make those conditional in a common parser function,
to reduce code duplication.
Nick Bowler [Fri, 23 Jun 2023 05:11:15 +0000 (01:11 -0400)]
libcdecl: Avoid vsnprintf for error reporting.
Using vsnprintf is overkill here. We only need to handle a single %s
conversion, which can be done by direct call to snprintf.
As this is the only caller of vsnprintf, dropping it means we can drop
the vsnprintf gnulib module. Also, at least with gcc, using va_start
and such is fairly expensive. The direct-snprintf version is quite
a bit more compact.
Nick Bowler [Fri, 23 Jun 2023 04:39:45 +0000 (00:39 -0400)]
libcdecl: Make cdecl__emit_specs return value usable directly.
Since there is now only one caller of cdecl__emit_specs that cares
about its return value, let's adjust this function to return exactly
what that caller wants, so it doesn't have to do any extra calculation.
Nick Bowler [Fri, 23 Jun 2023 04:05:41 +0000 (00:05 -0400)]
libcdecl: Accumulate output length in structure.
Instead of cascading the lengths back via function return values, since
we now have a state structure we can just track the total length in one
place. This is quite a lot conceptually simpler and cuts out a good
chunk of code.
Nick Bowler [Fri, 23 Jun 2023 03:25:50 +0000 (23:25 -0400)]
libcdecl: Use a structure for dst/dstlen in output routines.
Since pretty much all the output functions now just directly pass the
dst and dstlen pointers around without touching them in any other way,
it is more efficient to use a structure so there is only one pointer.
Nick Bowler [Fri, 23 Jun 2023 02:00:34 +0000 (22:00 -0400)]
libcdecl: Rework specifier output logic.
With both the explain and declare code paths using cdecl__emit, we
can give the same treatment to cdecl__explain_specs (now called
cdecl__emit_specs) to simplify things a bit.
This removes the last caller of cdecl__advance, so we can remove
that function (and use this better name for cdecl__advance_).
Nick Bowler [Fri, 23 Jun 2023 01:24:19 +0000 (21:24 -0400)]
libcdecl: Rework cdecl_declare output logic.
Instead of adjusting the output pointer/length values after each
internal call, tweak all the functions in declare.c to take an extra
level of indirection so they can just directly adjust the destination
as they go.
Nick Bowler [Fri, 23 Jun 2023 01:21:46 +0000 (21:21 -0400)]
libcdecl: Rework cdecl_explain output logic.
Instead of adjusting the output pointer/length values after each
internal call, tweak all the functions in explain.c to take an
extra level of indirection so they can just directly adjust the
destination as they go.
A new helper, cdecl__emit, is provided to simplify the common pattern
of "print a single C string" followed by "advance pointer/length."
Nick Bowler [Thu, 22 Jun 2023 05:20:12 +0000 (01:20 -0400)]
libcdecl: Simplify cdecl__explain_specs.
A bunch of conditions in the specifier printing loop are meaningless,
since all valid specifier types take idenical execution paths through
this function. So much of this can just be deleted.
Nick Bowler [Thu, 22 Jun 2023 05:01:46 +0000 (01:01 -0400)]
libcdecl: Improve specifier to string conversions.
Instead of a big switch statement, we can generate some compact lookup
tables to convert specifiers to strings, which appears to produce much
better results with gcc at least.
Nick Bowler [Thu, 22 Jun 2023 02:11:07 +0000 (22:11 -0400)]
tests: Adjust positive tests to verify both parse directions.
We don't actually have any canned tests of the "declare" or "type"
commands to produce C syntax from pseudo-English. While the randomized
crossparse test does provide some coverage of this, failures here
are difficult to understand compared to more simple, standalone test
cases.
So to start, let's just expand the existing "positive" tests to check
in both directions.
Rather than threading a flag through this function just to print
"declare" or "type" at the toplevel, we can just directly print
that in the one place where it is needed, which simplifies the
implementation a bit.
Since reworking how specifiers are printed, the "pre"/"post" specifier
printing functions are pretty much pointless, and can (mostly) be
deleted, which reduces the library size somewhat.
Nick Bowler [Fri, 16 Jun 2023 04:34:41 +0000 (00:34 -0400)]
cdecl99: Avoid passing uninitialized value to help_print_option.
For options that do not take arguments, the "arg" member of the help
structure is not assigned by lopt_get_help. Since the structure
is not otherwise initialized, it is technically undefined to even
evaluate this member in order to pass it to the help_print_option
function.
In this case, the argument is not actually used, and I'm not aware
of any actual failures as a result, but it easy enough to avoid.
Nick Bowler [Fri, 16 Jun 2023 04:12:42 +0000 (00:12 -0400)]
tests: Eliminate random floating-point generation.
The only use of test_rng_uniform in the test suite is to do 50/50 coin
toss type checks, add a simple helper based on test_rng_uniform_int to
do this, instead of bringing in all this floating-point machinery for
no real reason.
Nick Bowler [Fri, 16 Jun 2023 04:01:07 +0000 (00:01 -0400)]
libcdecl: Avoid stray semicolon after gl_once_define.
For some weird reason the gnulib gl_once_define function-like macro
expansion includes the semicolon. Thus, the extra semicolon after
the macro invocation is technically a syntax error, although most
compilers seem to not care too much.
Nick Bowler [Thu, 15 Jun 2023 00:34:56 +0000 (20:34 -0400)]
libcdecl: Tweak invalid character error from scanner.
By adjusting how we format the error message, the format string is
changed to be completely identical to a format string used by the
parser error reporting, which avoids some wasted code space in
the library.
Furthermore, make some tweaks to the invalid character pretty-printing
which seems to let GCC generate a bit more compact code.
Nick Bowler [Fri, 9 Jun 2023 14:43:36 +0000 (10:43 -0400)]
libcdecl: Consolidate most error messages.
Almost every error message returned by the library is a fixed
string describing some particular syntax problem. Keep this
list of strings in one place, and add a new internal helper
to report one of these errors.
Furthermore, reduce the amount of distinct error codes returned
to the user to (once again) just two, as the current plethora of
codes seems completely pointless.
Nick Bowler [Tue, 13 Jun 2023 06:14:44 +0000 (02:14 -0400)]
Use gnulib's vsnprintf module.
The snprintf module provides only snprintf, not vsnprintf. As we
currently depend on both functions in the library, it is necessary
to use both modules.
This fixes failures in the new cdeclerr test on HP-UX 11, which has
vsnprintf but it is not entirely C99-like (wrong return value).
Nick Bowler [Thu, 1 Jun 2023 00:29:41 +0000 (20:29 -0400)]
Factor out common parser invocation.
The two main parsing functions have nearly identical parser invocation
sequences, with the only difference being the flag passed to the scanner
init. Split that off into a separate function, to simplify the code.
Additionally, if the code is compiled with YYDEBUG, enable parser
debugging.
Nick Bowler [Tue, 30 May 2023 02:10:54 +0000 (22:10 -0400)]
Restore gperf-related definitions to makefile.
Previously, gnulib was providing definitions for GPERF and other
variables in the makefile indirectly via the striconv module. But
since we removed that module, the definitions disappeared, leading
to build failures after maintainer-clean.
Easy enough to just do the same thing explicitly. In the future
we might want to use configure to locate gperf but for now this
will do.
Nick Bowler [Mon, 29 May 2023 02:05:17 +0000 (22:05 -0400)]
cdecl99: Use packed option format from gen-options.awk.
This feature uses a compact, fully constant array to generate the
real struct option array at runtime. This allows the full-sized array
to be dropped after command-line processing is finished, and with
position-independent executables, reduces the amount of relocation
processing needed.
Nick Bowler [Sat, 27 May 2023 01:25:04 +0000 (21:25 -0400)]
Replace Gnulib striconv with copyright_symbol from dxcommon.
The only use of the striconv module is to produce the copyright symbol
for --version output and at the start of an interactive session. Using
the new, single-purpose function reduces code size quite a bit.
Nick Bowler [Sat, 27 May 2023 00:38:11 +0000 (20:38 -0400)]
Stop using gnulib's flexmember module.
The only thing we're actually using from this module is provided
directly by Autoconf, via AC_C_FLEXIBLE_ARRAY_MEMBER, so we can
just use that macro instead.
Nick Bowler [Thu, 25 May 2023 23:38:59 +0000 (19:38 -0400)]
cdecl99: Fix some improper error message formatting.
Errors opening the file specified by a --file option are printed with
two newlines, and errors generated by actual commands are not properly
prefixed with the program name.
Add test cases to catch these specific problems and fix them.
Nick Bowler [Tue, 24 Jan 2023 07:37:00 +0000 (02:37 -0500)]
Avoid POSIX character classes in the test suite.
Instead of [[:alnum:]] and friends, expand to an explicit list of
characters, which is a bit more portable usage (and also avoids
unneeded locale dependency). We can use macros to make this
just as convenient to write.
Nick Bowler [Tue, 24 Jan 2023 07:14:53 +0000 (02:14 -0500)]
Fix configuration on Solaris 8.
There is an quoting error in Gnulib's threadlib.m4 which causes
incorrect configuration settings on old Solaris. We can repeat
the simple check with correct quoting to workaround the problem.
Additionally, pull in DX_LINGUAS fixes from dxcommon to avoid
tripping on Solaris' pre-POSIX /bin/awk.
Nick Bowler [Tue, 24 Jan 2023 04:58:55 +0000 (23:58 -0500)]
cdecl99: Drop locale-sensitive isblank usage.
The intention of this function is to avoid recording no-op command lines
in the history. This is exactly the set of commands containing just
regular tabs and spaces.
It is inappropriate to use the locale-sensitive isblank for this, as
this may be a little bit different. In practice there is probably no
meaningful difference, but as isblank is a C99 feature losing this call
also helps when building against older C libraries that lack it.
Nick Bowler [Tue, 24 Jan 2023 04:47:18 +0000 (23:47 -0500)]
Provide strtoumax fallback in the scanner.
We already do something similar in the test suite. We don't really care
about the full range of uintmax_t, we just prefer the widest type that
is available to us. It is no real problem to fall back to a narrower
conversion function.
Nick Bowler [Tue, 10 Jan 2023 01:20:19 +0000 (20:20 -0500)]
Don't parse command-line options more than once.
Instead of parsing the options a second time to collect the --execute
option arguments, we can easily permute the argv array on the first
pass through, which simplifies the subsequent evaluation.
Nick Bowler [Fri, 18 Nov 2022 03:45:07 +0000 (22:45 -0500)]
Avoid Gnulib std-gnu11 module.
It has come to my attention that this module rewrites AC_PROG_CC in
a way that actually breaks Automake's AM_PROG_CC_C_O functionality.
This results in the "compile" script not being included or used in
packages bootstrapped with Autoconf 2.69.
With later versions of Autoconf things work because this module
doesn't touch things and thus disables itself.
I don't care about building with C11 one way or the other. Let's
just skip the module.
Nick Bowler [Fri, 22 Apr 2022 02:01:57 +0000 (22:01 -0400)]
Explicitly require gnulib getline module.
Currently, the gnulib getline module is only pulled in indirectly via
the readline module. As a result, when configuring for a system that
has GNU readline installed (and readline is enabled), the readline
replacement is not used and therefore the getline replacement is
never included.
But on systems that lack getline in the C library, without the gnulib
replacement the build will fail. Simply listing getline as a needed
module suffices to allow such configurations to build successfully.
Nick Bowler [Sun, 3 Apr 2022 01:56:22 +0000 (21:56 -0400)]
tests: Correct RNG implementation.
Due to a mistake in the adaptation, the output of the generator was
not done correctly. Correct that, and add a little test program that
would have caught this mistake by directly comparing against the
reference implementation.
Nick Bowler [Sat, 26 Mar 2022 04:00:56 +0000 (00:00 -0400)]
Bump dxcommon to fix builds using --disable-dependency-tracking.
Some build rules were inadvertently depending on directory creation that
happens as a side effect of depfiles generation. This does not happen
when configuring with --disable-dependency-tracking, leading to issues
with both VPATH builds and the gnulib symfiles machinery.
Failures are easily observed with a command like
make DISTCHECK_CONFIGURE_FLAGS=--disable-dependency-tracking distcheck
Using parallel make can hide problems since it seems Automake also
generates make rules that incidentally create these directories.
Nick Bowler [Sat, 19 Feb 2022 23:07:59 +0000 (18:07 -0500)]
Fix regressions caused by symbol renaming changes.
A bug in dxcommon was causing VPATH builds to continue to apply symbol
renaming to all gnulib sources. Fixing this revealed that dependency
tracking was broken for the non-renamed objects (distcheck noticed that
the dependency products were not being cleaned).
To fix dependency tracking Automake needs to be aware of all the source
files going into the statically-linked version of the library.
Unfortunately, Automake complains if we just add the same sources to a
non-libtool library, even though there is no real conflict since only
one will actually be built. The issue can be worked around by using
Automake's object renaming facilities, which complicates things slightly
but is straightforward enough to implement.
Nick Bowler [Fri, 18 Feb 2022 22:15:34 +0000 (17:15 -0500)]
Fix gen-typegen.awk incompatibility with busybox awk.
It seems that busybox awk does not support * in printf conversions.
There is only one use of this feature in the scripts and we can use
the substr function instead.
Nick Bowler [Fri, 18 Feb 2022 04:08:19 +0000 (23:08 -0500)]
Improve gnulib build times.
Use the new dxcommon features in an attempt to avoid the expensive
symbol renaming and PIC build steps for the portions of gnulib that
are not actually needed by the libcdecl library.
Nick Bowler [Thu, 10 Feb 2022 01:37:50 +0000 (20:37 -0500)]
Plug memory leak in declgen.
When a void typespec is generated in a context where it is invalid,
gen_typespecs just rolls the dice again. Unfortunately, the typespec
is not freed in this case, leaking memory. Easily fixed.
Nick Bowler [Wed, 9 Feb 2022 08:06:02 +0000 (03:06 -0500)]
Portability improvements for new random number generator.
On HP-UX 11, the ldexp function requires linking against libm.
Moreover, instead of strtoull declared in <stdlib.h> we have
__strtoull declared in <inttypes.h>. Add configure tests to
find these.
Nick Bowler [Wed, 9 Feb 2022 04:37:53 +0000 (23:37 -0500)]
Clean up declgen a bit.
Use wrapper functions to perform the "allocate a structure and
initialize its members" sequence which is a common sequence here,
and avoid the use of compound literals for this which improves
portability to older compilers.
Nick Bowler [Wed, 9 Feb 2022 04:34:17 +0000 (23:34 -0500)]
Simplify and improve randomdecl sanity test.
We don't need to do any weird shell variable stuff here, we can just
directly compute the expected output and verify against that. As a
bonus, when the test fails this gives a much better description of
which expected forms are missing in the testsuite log.
Nick Bowler [Wed, 9 Feb 2022 03:30:48 +0000 (22:30 -0500)]
Remove randomdecl test dependency on GSL.
It's a bit silly for a test application to depend on this huge library
just for random number generation. We can just directly incorporate
a simple RNG implementation which should be plenty good enough for
this purpose.
Nick Bowler [Thu, 14 Oct 2021 03:28:37 +0000 (23:28 -0400)]
Fix formatting error in libcdecl(3) man page.
The use of "e.g." at the end of a line confuses troff into thinking this
is the end of the sentence. That is not correct, so adjust the syntax
to avoid such interpretation.
Nick Bowler [Fri, 13 Aug 2021 02:41:14 +0000 (22:41 -0400)]
Make library i18n init conditional on NLS support.
If the package is configured with --disable-nls, the library's i18n
initialization does nothing useful but the internal functions are
not fully removed. Looks like a simple opportunity for improvement.
Nick Bowler [Fri, 13 Aug 2021 00:43:00 +0000 (20:43 -0400)]
Include glthread headers late.
It seems that in some configurations, the glthread headers can
include Windows headers which define macros that can conflict
with the libcdecl headers.
As the damage appears to be isolated to the headers, re-ordering
the includes appears to be sufficient to avoid any problems, at
least within the library.
Nick Bowler [Sat, 13 Mar 2021 22:23:46 +0000 (17:23 -0500)]
Rework library error reporting.
This removes (hopefully) all cases where the libcdecl prints
error messages directly to stderr, and reports these messages
via cdecl_get_error instead.
Nick Bowler [Fri, 12 Mar 2021 05:34:27 +0000 (00:34 -0500)]
Generate specifier strings directly from cdecl.h
Since the specs.lst file is no longer used for output ordering, it is
now mostly redundant information. There is another list of specifiers:
the enumeration constants in cdecl.h. We can fairly easily reconstruct
the strings from this instead.
Nick Bowler [Fri, 12 Mar 2021 04:54:24 +0000 (23:54 -0500)]
Restructure the type specifier check.
As the typemap.c functionality is only used for one thing and is quite
simple, let's integrate that more closely with the declaration specifier
checks. This further reduces the code size of the library somewhat.
The old sed line noise is replaced by a similar awk script which should
be much easier to adjust as needed.
Nick Bowler [Fri, 12 Mar 2021 04:58:52 +0000 (23:58 -0500)]
Hand-code the normalized specifier ordering.
For the most part, the enumerated values for specifier types in the
library are already in an acceptable order for normalization. This
list is part of the API and is expected to be stable.
We can achieve the desired ordering by simply tweaking these values
as needed. There is also no need to totally order the specifiers
because some combinations are never valid. This change not only
eliminates a pile of comparatively complex code generation, but
also reduces the overall size of the library somewhat.
Add a new testcase to validate the resulting ordering.
Nick Bowler [Wed, 10 Mar 2021 07:31:59 +0000 (02:31 -0500)]
Replace typegen.sh with a new and improved script.
The old typegen.sh has a bunch of portability problems; let's write
a new script in awk which is a better tool for this sort of code
generation task anyway.
Nick Bowler [Wed, 10 Mar 2021 01:34:05 +0000 (20:34 -0500)]
Work around designated initializer bug on HP-UX cc.
Work around a designated initializer bug encountered on old versions
of HP-UX. It seems there are some glitches leading to compilation
failures when the initializers are out of order (or omitted), and
the initializer for such a member is non-constant, and there is a
type conversion involved.
This affects several of the parser actions that initialize declspec
values. In this case, it is super easy to just change the type and
avoid type conversion, which easily avoids the bug.
Nick Bowler [Fri, 5 Mar 2021 00:15:37 +0000 (19:15 -0500)]
Add configure option to disable readline.
Apparently the Gnulib readline module does not provide any obvious way
for users to disable it. Setting the gl_cv_lib_readline cache variable
works but probably only Autoconfers will know how to discover that.
Add an explicit --with-readline (and --without-readline) option which
internally sets this. Since this option shows up in configure's help
output it is hopefully more obvious.
Nick Bowler [Wed, 3 Mar 2021 02:21:50 +0000 (21:21 -0500)]
Fix testcase compilation with --disable-shared.
Since the libtest library depends on functions in libcdecl,
we must list libcdecl after libtest on the linker command line,
otherwise required objects from static libcdecl will not be
pulled in to the link. A similar problem also occurs when
building tests with LDFLAGS=-Wl,--as-needed.
Nick Bowler [Tue, 2 Mar 2021 05:13:56 +0000 (00:13 -0500)]
Consolidate header files.
There is no need for this quantity of tiny header files. We can simply
use a single internal header file for libcdecl, and a single internal
header file for cdecl99, including all necessary declarations.
Nick Bowler [Fri, 26 Feb 2021 05:37:17 +0000 (00:37 -0500)]
Use the newly-minted option generator script from dxcommon.
Instead of maintaining the relation between long options and help text
dircetly in C code, let's use this new script to generate C code from
a simple description file.
This ditches support for translated option names. I'm a bit unsure
about the practical use of this functionality as I do not personally
use it and locale-specific interpretation of command-line arguments
just seems like it would cause more problems than it would solve.
Nick Bowler [Wed, 24 Feb 2021 01:46:55 +0000 (20:46 -0500)]
Bundle scripts to help re-bootstrap the package.
We can include the main bootstrapping scripts easily enough in the
package. If the user has a (possibly updated) Gnulib available, it
is now possible to regenerate the build system just by running the
included bootstrap script.
Nick Bowler [Wed, 24 Feb 2021 00:44:48 +0000 (19:44 -0500)]
Use AC_CONFIG_HEADERS rather than AC_CONFIG_HEADER.
The former name has been supported since approximately forever ago and
the latter form is now formally deprecated (with a warning) in recent
versions of Autoconf.
Nick Bowler [Wed, 24 Feb 2021 00:41:44 +0000 (19:41 -0500)]
Ensure INSTALL is packaged.
When Automake is run in foreign mode, the standard INSTALL file is not
copied by automake --add-missing. As this file provides general usage
instructions for the GNU build system, it is useful to have included
in the package.
Just copy the file manually when bootstrapping to make that happen.
Nick Bowler [Wed, 24 Feb 2021 00:33:16 +0000 (19:33 -0500)]
Rework the README.
This consolidates the text of this README with the text on my website,
and gets rid of a bunch of bootstrapping-related suggestions which
are not particularly interesting and probably obsolete anyway as none
of the tool versions listed have been updated in about a decade.
Nick Bowler [Tue, 23 Feb 2021 05:39:30 +0000 (00:39 -0500)]
Generate ChangeLog from git at packaging time.
Import the gitlog-to-changelog script from gnulib and add rules to
generate an up-to-date changelog from the git history, if available,
when running 'make dist'.
The ChangeLog is otherwise taken from srcdir as usual, so that
modified versions can be prepared from a release tarball without
requiring the full git history. In this scenario, the ChangeLog
would have to be manually edited.
In case the ChangeLog generation fails, a distcheck-hook is added
to hopefully catch issues before releasing tarballs with a broken
ChangeLog.
Nick Bowler [Tue, 23 Feb 2021 04:50:06 +0000 (23:50 -0500)]
Bump gnulib to latest.
Switch to using the gettext-h module, which I believe should work
exactly the same as before, just that the Autoconf macros get pulled
in from gettext rather than gnulib when bootstrapping.
Nick Bowler [Sat, 4 Jul 2020 17:28:53 +0000 (13:28 -0400)]
Fix use-after-free during parser error recovery.
When parsing a declaration containing more than one full declarator,
each such declarator references the same list of declaration specifiers.
While processing the declarators the specifier list is normalized and
each declarator needs to be updated to the new list.
However, if a syntax error is detected we break out of the processing
loop and end up with only some of these updates occurring. When the
partially-updated declaration list is subsequently freed, this can
in some cases lead to a use after free when the stale pointers are
encountered.
Fix this by updating all the specifier references before doing any
further processing to avoid dealing with partially-updated lists.
Nick Bowler [Fri, 3 Jul 2020 03:16:51 +0000 (23:16 -0400)]
Port random crossparse test to Autotest.
I don't really know why the existing crossparse testcase is so
complicated. Sure running the test generation and execution in
parallel may be an interesting approach, but it seems to be total
overkill for this use case.
By enhancing the crossparse application to take a list of test cases
from a file, we can just generate the stimuli in one step and execute
the tests in another which is simple and works fine.
As this is the final test to port to Autotest, we can now retire the
use of the Automake test harness.