EWONTFIX https://ewontfix.com A blog about bugs en The worst of time64 breakage https://ewontfix.com/19 https://ewontfix.com/19 15 Feb 2020 19:25:16 GMT In preparing to release musl 1.2.0, I worked with distro maintainers from Adélie Linux and Yoe to find serious application compatibility problems users would hit when upgrading, so that we could have patches ready and reduce user frustration with the upgrade. Here are some of the findings.

By far the most dangerous type of app compatibility issue we found was in Berkeley DB 5.x, which defines its own wrong version of the timespec struct to pass to clock_gettime:

typedef struct {
    time_t  tv_sec;             /* seconds */
#ifdef HAVE_MIXED_SIZE_ADDRESSING

...

]]>
32-bit x86 Position Independent Code - It's that bad https://ewontfix.com/18 https://ewontfix.com/18 15 Apr 2015 03:23:06 GMT Let's start by looking at a simple C function to be compiled as position-independent code (i.e. -fPIC, for use in a shared library):

void bar(void);

void foo(void)
{
    bar();
}

And now, what GCC compiles it to (listing 2):

foo:
    pushl   %ebx

...

]]>
Multi-threaded setxid on Linux https://ewontfix.com/17 https://ewontfix.com/17 15 Jan 2015 16:12:00 GMT Background

Linux has a legacy of treating threads like processes that share memory. The situation was a lot worse about 15 years ago, but it's still far from perfect. Despite lots of fixes to the way signals, process termination and replacement via execve, etc. are handled to make threads behave like threads, plenty of ugly remnants of the idea that "threads are just processes sharing memory" remain; the big areas are:

  • Scheduling properties
  • Resource limits
  • Permissions

...

]]>
Open race-condition bugs in glibc https://ewontfix.com/16 https://ewontfix.com/16 06 Mar 2014 19:10:09 GMT As part of the freeze announcement for musl 1.0, we mentioned longstanding open race condition bugs in glibc. Shortly after the announcement went out on Phoronix, I got a request for details on which bugs we were referring to, so I put together a list.

The most critical (in my opinion) open race bugs are the ones I described on this blog back in 2012; they are reported in glibc issue 12683:

]]>
Systemd has 6 service startup notification types, and they're all wrong https://ewontfix.com/15 https://ewontfix.com/15 27 Feb 2014 04:14:26 GMT In my last post, Broken by design: systemd, I covered technical aspects of systemd outside its domain of specialization that make it a poor choice for the future of the Linux userspace's init system. Since then, it's come to my attention as a result of a thread on the glibc development list that systemd can't even get things right in its own problem domain: service supervision.

Per the manual, systemd has the following 6 "types" that can be used in a service file to control how systemd will supervise the service (daemon):

...

]]>
Broken by design: systemd https://ewontfix.com/14 https://ewontfix.com/14 09 Feb 2014 19:56:09 GMT Recently the topic of systemd has come up quite a bit in various communities in which I'm involved, including the musl IRC channel and on the Busybox mailing list.

While the attitude towards systemd in these communities is largely negative, much of what I've seen has been either dismissable by folks in different circles as mere conservatism, or tempered by an idea that despite its flaws, "the design is sound". This latter view comes with the notion that systemd's flaws are fixable without scrapping it or otherwise incurring major costs, and therefore not a major obstacle to adopting systemd.

...

]]>
Incorrect configure checks for availability of functions https://ewontfix.com/13 https://ewontfix.com/13 14 Aug 2013 00:35:06 GMT The short version: using functions without prototypes is dangerous, and bad configure script recipes directly encourage this practice.

One of the most basic and important checks configure scripts perform is checking for the availability of library functions (either in the standard library or third-party libraries) that are optional, present only in certain versions, wrongly missing on some systems, and so on. In a sense, this is the main purpose of having a configure script, and one would think this kind of check would be hard to get wrong. Unfortunately, these checks are not only possible to get wrong, they're usually wrong, especially in GNU software or other software using gnulib.

The basic problem is that most configure scripts check only for the ...

]]>
Breakincludes https://ewontfix.com/12 https://ewontfix.com/12 04 Jul 2013 16:46:23 GMT A little-known part of GCC's build process is a script called "fixincludes", or fixinc.sh. Purportedly, the purpose of this script is to fix "non-ANSI system header files" which GCC "cannot compile". This description seems to correspond roughly to the original intended purpose of fixincludes, but the scope of what it does has since ballooned into all sorts of unrelated changes. Let's look at the first few rules in fixincludes' inclhack.def:

  • Changing AIX's _LARGE_FILES redirection of open to open64, etc. to use GCC's __asm__ keyword rather than #define, as the latter breaks C++.

  • Exposing the long double math functions in math.h on Mac OS 10.3.9, which inexplicably omitted declarations for them. ...

]]>
NULL considered harmful https://ewontfix.com/11 https://ewontfix.com/11 04 Jul 2013 03:25:02 GMT The C and C++ languages define the macro NULL, widely taught as the correct way to write a literal null pointer. The motivations for using NULL are well-meaning; largely they come down to fact that it documents the intent, much like how a macro named FALSE might better document boolean intent than a literal 0 would do. Unfortunately, use of the NULL macro without fully understanding it can lead to subtle bugs and portability issues, some of which are difficult for compilers and static analysis tools to diagnose.

Despite it being superceded by the 2011 standard, I'm going to quote C99 because it's what I'm most familar with, and I suspect most readers are in the same situation. 7.17 specifies the NULL macro as:

NULL ...

]]>
Non-invasive printf debugging https://ewontfix.com/10 https://ewontfix.com/10 12 Dec 2012 16:09:05 GMT This post is not about any particular bug or bad programming practice, just a new “printf debugging” technique I came up with.

Often when tracking down a bug, it’s useful to add extra output to track the state of the program in the moments leading up to the crash or incorrect behavior, aka “printf debugging”. However, this technique is “invasive” in the sense that it interleaves unwanted data into the program output. Using stderr instead of stdout can alleviate the problem to some extent, but when you’re inserting the debugging code into a widely-used library (in my case, libc.so) or a shell, even having unwanted output on stderr can be a problem.

A previous approach I had used was sending the output to an arbitrary high file descriptor instead of stdout or stderr. For example, ...

]]>
Stubborn and ignorant use of int where size_t is needed https://ewontfix.com/9 https://ewontfix.com/9 25 Oct 2012 23:48:02 GMT What’s wrong with this C function?

char *my_strchr(char *s, int c)
{
    int i;
    for (i=0; s[i]!=c; i++)
        if (!s[i]) return 0;
    return &s[i];
}

Unless its interface contract requires that the caller pass a string no longer than INT_MAX, it can invoke undefined behavior due to integer overflow, most likely resulting in a crash. Even if you change the type to unsigned instead of int to avoid the signed overflow ...

]]>
Unexpected observability of lock states https://ewontfix.com/8 https://ewontfix.com/8 25 Oct 2012 03:20:38 GMT This post is going to be the first that’s about one of my own bugs, in musl. For a long time, I’ve had certain stdio functions such as feof and ferror forgoing any locking, and simply relying on the fact that, per the memory model that’s assumed, reading the associated flags is safe without any locks. The issue with doing this is that, while it’s safe, it’s not correct; it leads to observably incorrect behavior in some cases.

Per POSIX,

All functions that reference ( FILE *) objects shall behave as if they use flockfile() and funlockfile() internally to obtain ownership of these ( FILE *) objects.

...

]]>
vfork considered dangerous https://ewontfix.com/7 https://ewontfix.com/7 21 Oct 2012 21:20:22 GMT Traditional unix systems had a vfork function, which works like fork, but without creating a new virtual address space; the parent and child run in the same address space. Unlike with pthread_create, where the new thread runs on its own stack, vfork behaves like fork and “returns twice”, once in the child and once in the parent. This seems impossible, since the parent and child would clobber one another’s stacks, but a clever trick saves the day: the parent process is suspended until the child performs exec or _exit, breaking the shared-memory-space relation between the two processes.

vfork was omitted from POSIX and modern standards because it’s difficult to use; the original specification for the function left it undefined to do basically anything except exec or _exit after vfork in the child. However, many systems (including Linux) still ...

]]>
AS + DC = AC https://ewontfix.com/6 https://ewontfix.com/6 15 Oct 2012 00:24:57 GMT Where A.S. are asynchronous signals, D.C. is deferred cancellation, and A.C. is asynchronous cancellation. In the previous post, I discussed asychronous versus deferred cancellation in POSIX threads, and issues that make it hard to use asynchronous cancellation well. I also mentioned that there are almost no functions which are async-cancel-safe. What if you want to cheat and get the behavior of asynchronous cancellation, but without having to follow the rules?

Enter asynchronous signals. Particularly, I’m thinking of signals sent to a specific thread using pthread_kill, but really the signal could be coming from another source like pressing the interrupt or quit key on a terminal.

Suppose the main flow of excution in a thread uses only ...

]]>
Asynchronous cancellation pitfalls https://ewontfix.com/5 https://ewontfix.com/5 07 Oct 2012 04:24 GMT In the past few posts, I’ve introduced thread cancellation and some of the implementation and application usage difficulties in making cancellation robust. One topic I haven’t yet touched on is asynchronous cancellation. POSIX threads support two cancellation types: asynchronous and deferred. The latter, deferred cancellation, is the default, whereby a cancellation request is only acted upon immediately if the thread to be cancelled is suspended at a cancellation point, and otherwise remains pending until the next call to a cancellation point.

The other option, asynchronous cancellation, allows (but does not require) the implementation to act on cancellation requests at any time. This obviously has the potential, to leave data in a horribly inconsistent state, so rules are imposed; the application cannot call ...

]]>
Updates on close, EINTR, & cancellation https://ewontfix.com/4 https://ewontfix.com/4 03 Oct 2012 00:05 GMT Last week I took the time to file a report with the Austin Group (responsible for POSIX) about the close issue. It is Issue #614, and it turns out the problem was already solved by fixing the specification of close when interrupted by a signal. Whew. I thought that latter would be a lot more controversial and harder to get fixed

Some basic history on the issue: Apparently, there was a historical disagreement over the behavior of close when interrupted by a signal. Some implementations (e.g. HPUX) had it leave the file descriptor open when returning with EINTR; others (Linux, AIX) closed it unconditionally, but returned with EINTR if a signal arrived while close() was interrupted before returning. This ambiguity was acceptable for single-threaded applications, which could just ...

]]>
To overcommit or not to overcommit https://ewontfix.com/3 https://ewontfix.com/3 23 Sep 2012 01:43 GMT I’ve written in the past on the topic of overcommit, which depending on your perspective, is either a feature of Linux and some other kernels, or a bug left over from a time when folks didn’t know how to do virtual memory accounting properly. I’m a serious proponent of strict commit accounting (opposite of overcommit), but for this article, I want to look at the state of the software ecosystem and how it often leaves us overcommit-enabled Linux systems being more failproof than their strict-accounting brothers and sisters.

The idea of strict commit accounting is that malloc never reports success only to let your program crash when you actually try to use the memory. If the kernel cannot ensure that there’s no possible sequence of paging events that would cause it to run out of physical ...

]]>
Thread cancellation and resource leaks https://ewontfix.com/2 https://ewontfix.com/2 21 Sep 2012 02:00 GMT In a multi-threaded C program where threads share address space and may be operating on shared objects as long as they use the proper synchronization tools, it’s unsafe to asynchronously kill an individual thread without killing the whole process. Stale locks may be left behind and data being modified under those locks may be in an inconsistent state. This includes even internal heap management structures used by malloc.

As such, the POSIX threads standard does not even offer a mechanism for forcible termination of individual threads. Instead, it offers thread cancellation, a mechanism by which early termination of a thread whose work is no longer needed can be negotiated in such a way that the thread to be cancelled cleans up any shared state and/or private resources it may be using before it terminates. ...

]]>
Introducing EWONTFIX https://ewontfix.com/1 https://ewontfix.com/1 22 Sep 2012 22:47 GMT Welcome to EWONTFIX, a blog about, well, bugs. Especially longstanding unfixed ones in C code for Linux or Unix-like systems. The idea for this blog grew out of conversations during the development of musl libc. Aside from the fact that longstanding bugs in glibc were one of the original motivations for musl, it turns out that developing a libc leads to spending a lot of time building and testing applications. And in the process of testing, one ends up reading a lot of source. And a lot of source is appallingly bad.

Most low-quality source code just isn’t that interesting to write about. It’s more just a matter of identifying the problems, submitting them to bug trackers, and following up until somebody fixes things. However there are also a good deal of cases where buggy code is interesting to discuss. These fall mostly under two major categories: ...

]]>