Freestanding Library: Easy [utilities], [ranges], and [iterators]

Document number: D1642R11 TODO
Date: 2022-06-18 TODO
Reply-to: Ben Craig <ben dot craig at gmail dot com>
Audience: Library Working Group, Core Working Group

Change history

R11

R10

R9

R8

R7

R6

R5

R4

R3

R2

R1

R0

Introduction

This paper proposes adding many of the facilities in the [utilities], [ranges], and [iterators] clause to the freestanding subset of C++. The paper will be adding only complete entities, and will not tackle partial classes. For example, classes like pair and tuple are being added, but trickier classes like optional, variant, and bitset will come in another paper.

The <memory> header depends on facilities in <ranges> and <iterator>, so those headers (and clauses) are addressed as well.

This paper also updates the editorial techniques for declaring facilities as freestanding. This update will make it easier to mark headers as partially freestanding. This wording change will also make it easier to mark in-flight proposals as freestanding, without causing a major blocking point in the standardization process.

The paper also includes a drive-by fix to add _Exit to freestanding. quick_exit is already part of C++20's freestanding library, and quick_exit is specified to call _Exit [support.start.term#13]. This has been a bug since C++11 (introduced with LWG1264 / UK-172).

Changes to the design of feature test macros are discussed in P2198. This paper maintains the status quo for feature test macros on freestanding.

Motivation and Design

Many existing facilities in the C++ standard library could be used without trouble in freestanding environments. This series of papers will specify the maximal subset of the C++ standard library that does not require an OS or space overhead.

For a more in-depth rationale, see P0829.

This paper adds blanket wording to make it easier to mark things freestanding, and it retrofits the existing freestanding facilities to use the new editorial approach. It also includes a non-normative note to allow implementers to use existing no-exception builds of their libraries as an implementation-defined extensions, with a fair number of qualifications. Here's that note, reproduced so that it can be discussed more effectively:

[ Note: Throwing a standard library provided exception is not observably different from terminate() if the implementation doesn't unwind the stack during exception handling ([except.handle#9]) and the user's program contains no catch blocks. -end note]

<optional>, <variant>, and <bitset> are not discussed in this paper, as all have non-essential functions that can throw an exception. <charconv> is also not discussed in this paper as it will require us to deal with the thorny issue of overload sets involving floating point and non-floating point types. I plan on addressing all four of these headers in later papers, so that the topics in question can be discussed in relative isolation.

Splitting overload sets

Modifying an overload set needs to be treated with great care. Ideally, libraries built in freestanding environments will have the same semantics (including selected overloads) when built in a hosted environment. I doubt that goal is fully attainable, as sufficiently clever programmers will be able to detect the difference and modify behavior accordingly.

My approach will be to avoid splitting overload sets that could cause accidental freestanding / hosted differences. (In future papers, I may need to lean on = delete techniques to avoid silent behavior changes, but this paper hasn't needed that approach.)

swap has shared_ptr, weak_ptr, and unique_ptr overloads, but this paper marks only the unique_ptr overload as freestanding. <memory> has many algorithms with ExecutionPolicy overloads. This paper does not mark the ExectuionPolicy overloads as freestanding. I was unable to come up with any compelling "accidental" way to select one swap or algorithm overload in freestanding, but a different one in hosted.

Also note that the swap overload set visible to a given translation unit is already indeterminate. A user may include <memory> which guarantees the smart pointer overloads, but an implementation could expose any (or none!) of the other swap overloads from other STL headers.

Memory allocation and allocators

P2013 makes the default allocating ::operator news optional in freestanding. This would make std::allocator::allocate ill-formed on implementations without an ::operator new definition.

This paper (and most of the freestanding papers) will attempt to avoid adding facilities to freestanding that throw exceptions.

The Cpp17Allocator requirements permit only one and a half ways of signaling failures. The first way to signal a failure is by throwing an exception; this is the only way to signal a failure that is explicitly described in [allocator.requirements]. The "half" of a way to signal a failure from an allocator is to terminate. This can happen if the implementation has exceeded its (potentially tiny) implementation limits. Note that returning nullptr is not an option here, as making nullptr a standard permitted error handling strategy for std::allocator::allocate would be a massively breaking change.

Future papers will attempt to add constexpr facilities to freestanding, so long as they are evaluated at compile time only. The intent is for these future papers to allow a compile-time std::vector and std::allocator, but without requiring a runtime std::vector or std::allocator.

This paper shouldn't make any of those efforts more difficult to address in a distinct paper. WG21 should not wait for those problems to be solved before adding the facilities in this paper to freestanding.

Justification for omissions

The following functions and classes rely on dynamic memory allocation and exceptions:

The following classes rely on iostreams facilities. iostreams facilities use dynamic memory allocations and rely on the operating system.

The ExecutionPolicy overloads of algorithms are of minimal utility on systems that do not support C++ threads.

out_ptr is fine to add to freestanding in principle. However, it was added to the working draft after this paper series was forwarded to LWG. The wording was able to tolerate out_ptr's addition without a change in design, so this paper won't add it. I encourage a future paper to add this facility to freestanding, once P1642 lands.

Feature test macros

Feature test macros are non-trivial for this new design space, and are now discussed in paper P2198. For now, the present paper maintains the status quo with regards to the contents of <version>.

Dependency on optional-like thing in ranges

[range.semi.wrap] describes an exposition-only type with behavior specified in terms of std::optional. This exposition-only type is used to specify single_view, filter_view, transform_view, take_while, and drop_while_view. std::optional is not added in this paper.

I claim that this is not a problem. First, the type std::optional is not required to be used here. Second, [compliance] paragraph 2 permits implementations to provide extra headers, and an implementation could provide std::optional if it so desired. Third, I plan on getting std::optional added to freestanding in a future paper that tackles the issue of omitting potentially throwing members, like optional::value.

Implementation and field experience

NI has been using a modified form of STLPort in the Microsoft Windows, Linux, and Apple OSX kernel for more than 10 years. That only covers C++03 facilities though.

Paul Bendixen has implemented P0829 in a libstdc++ fork. P0829 is (almost) a superset of the facilities in this paper. This implementation is available on Conan.

I have run a 2018 era of libc++'s test suite against the Microsoft Visual Studio 15.8 (released Aug 14, 2018) STL implementation, run in the Microsoft Windows kernel. Running in that environment allowed me to audit whether exceptions, RTTI, memory allocation, or floating point was used in appropriately. This largely tested the C++14 parts of the STL. The C++17 and C++20 parts were either missing implementations, or tests.

Post-LEWG merge conflicts

This paper was last discussed in LEWG in December 2020, and forwarded to LWG in March 2021. The paper touches a lot of headers, and the standard is a moving target.

P0228: move_only_function (was any_invocable) was adopted in October 2021. It adds move_only_function to <functional>. Prior revisions of this paper added all of <functional> to freestanding, with a small list of entities to not include. That would inappropriately add move_only_function to freestanding. This revision of the paper is excluding move_only_function. This issue was raised in a November 2021 reflector discussion, and it received five votes in favor, with no opposition (the author of this paper did not vote).

P2136R3: invoke_r was adopted in June 2021. It adds invoke_r to <functional>. invoke_r is just as suitable for freestanding as invoke is. This revision will include invoke_r implicitly in the wording. This issue was raised in a November 2021 reflector discussion, and it received five votes in favor, with no opposition (the author of this paper did not vote).

P2321R2: zip was adopted in October 2021. It adds zip, zip_transform, adjacent, adjacent_transform, and closely related entities to <ranges>. This revision will include the above facilities implicitly in the wording. This issue was raised in a November 2021 reflector discussion, and it received five votes in favor, with no opposition (the author of this paper did not vote).

P1682R3: std::to_underlying was adopted in February 2021. It adds to_underlying to <utility>. This revision will include to_underlying implicitly in the wording. This issue was raised in a November 2021 reflector discussion, and it received five votes in favor, with no opposition (the author of this paper did not vote).

P0627R6: Function to mark unreachable code was adopted in February 2022. It adds std::unreachable to <utility>. This revision will include to_underlying implicitly in the wording. This issue was raised in a September 2021 reflector discussion, and it received five votes in favor, with no opposition (the author of this paper did not vote).

The following papers all add facilities to <ranges>. All were adopted in February 2022. This revision will include those facilities implicitly in the wording.

P2387R3 also adds bind_back to <functional>. This issue was raised in an April 2022 reflector discussion, and it received six votes in favor, with no opposition (the author of this paper did not vote).

Potential Post-LEWG merge conflicts

The following papers are all in the LWG queue as of 2022-04-09: The above papers add facilities to <ranges>, <utility>, and <functional>. The wording in this revision will include those facilities implicitly.

The following papers are also in the LWG queue as of 2022-04-09:

P2278 adds facilities to <span>, <iterator>, and <ranges>. <span> is out of scope for this paper, but the <iterator>, and <ranges> facilities will be handled by this paper's wording implicitly.

P2165 adds facilities to <tuple> and <memory>. These facilities will be handled by this paper's wording implicitly. The <memory> change is adding an overload of a template function that was already marked for inclusion.

This post-LEWG merge conflicts were raised as an issue in an April 2022 reflector discussion, and it received six votes in favor of allowing these papers into freestanding with the existing wording, with no opposition (the author of this paper did not vote).

Editorial alternatives

This paper uses an opt-in editorial syntax like the following: 

#define NULL see below // freestanding

This gives normative meaning to a comment in the synopsis. There is precedent for this, particularly for // exposition only comments.

An alternative used by the POSIX spec would be to use an extra-lingual annotation to mark entities as freestanding or not. Here, I'll use an italicized [FS] to indicate freestanding membership. 

[FS] #define NULL see below

The choices of square brackets and the "FS" tag are arbitrary . Other separators or tags could be used instead. This would be a substantially new editorial technique for the C++ standard.

We could continue with the current approach, and use prose to indicate freestanding membership, similar to what we do with abort and friends in [compliance]. This would be a substantial amount of specification work for some headers. It also hides the choice of freestanding, such that future paper authors are even less likely to keep it in mind.

When picking an editorial alternative, we should keep in mind likely future needs. P0829 makes use of //freestanding, partial and //freestanding, omit annotations. P1641R3 makes use of //freestanding only annotations. I can also see uses for a //freestanding delete annotation, for maintaining overload sets while removing functionality. These could be represented in the POSIX-like annotation as follows: [FS partial], [FS omit], [FS only], and [FS delete].

Status Quo

Please see [compliance] to see what is freestanding in C++20. Note that NULL and size_t are declared in both <cstdlib> and <cstddef>. The wording in [compliance] does not make it clear that <cstdlib> should continue to provide those declarations in a freestanding implementation.

Wording

The following wording is relative to N4910.

Change in [intro.compliance.general]

Change in [intro.compliance.general] paragraph 7:
7
Two kinds of implementations are defined: a hosted implementation and a freestanding implementation.
For a hosted implementation, this document defines the set of available libraries.
A freestanding implementation is one in which execution may take place without the benefit of an operating system , and has an implementation-defined set of libraries that includes certain language-support libraries ([compliance]). A hosted implementation supports all the facilities described in this document, while a freestanding implementation supports the entire C++ language described in clauses 5 through 15 and the subset of the library facilities described in [compliance].

Change in [conventions]

Add a new subclause [freestanding.entity], under [conventions] and after [objects.within.classes]:

?.?.?.? Freestanding entities [freestanding.entity]

1
A freestanding entity is a declaration or macro definition that is present in a freestanding implementation and a hosted implementation.
2
Unless otherwise specified, the requirements on freestanding entities on a freestanding implementation are the same as the corresponding requirements in a hosted implementation.
3
In a header synopsis, entities followed with a comment that includes freestanding are freestanding entities.
[ Example: 
#define NULL see below // freestanding
-end example]
4
If a header synopsis begins with a comment that includes all freestanding, then all of the declarations and macro definitions in the header synopsis are freestanding entities.
[ Example: 
// all freestanding
namespace std {
-end example]
5
Deduction guides for freestanding entity class templates are freestanding entities.
6
Enclosing namespaces of freestanding entities are freestanding entities.
7
Friends of freestanding entities are freestanding entities.
8
Entities denoted by freestanding entity typedef-names and freestanding entity alias templates are freestanding entities.
Wording involving comment placement is inspired by [expos.only.func]#1.

Changes in [compliance]

Add new rows to Table 24:

Table 24: C++ headers for freestanding implementations [tab:headers.cpp.fs]
SubclauseHeader(s)
[…] […] […]
?.? [utility] Utility components <utility>
?.? [tuple] Tuples <tuple>
?.? [memory] Memory <memory>
?.? [function.objects] Function objects <functional>
?.? [ratio] Compile-time rational arithmetic <ratio>
?.? [iterators] Iterators library <iterator>
?.? [ranges] Ranges library <ranges>
[…] […] […]
Change in [compliance] paragraph 3:
The supplied version of the header <cstdlib> shall declare at least the functions abort, atexit, at_­quick_­exit, exit, and quick_­exit ([support.start.term]).
The supplied version of the header <atomic> shall meet the same requirements as for a hosted implementation except that support for always lock-free integral atomic types ([atomics.lockfree]) is implementation-defined, and whether or not the type aliases atomic_­signed_­lock_­free and atomic_­unsigned_­lock_­free are defined ([atomics.alias]) is implementation-defined.
The other headers listed in this table shall meet the same requirements as for a hosted implementation.
For each of the headers listed in [tab:headers.cpp.fs], a freestanding implementation provides at least the freestanding entities ([freestanding.entity]) declared in the header.
Add a new paragraph to [compliance]:
4
[ Note: Throwing a standard library provided exception is not observably different from terminate() if the implementation does not unwind the stack during exception handling ([except.handle#9]) and the user's program contains no catch blocks. -end note]

Full headers

Drafting note:

P2445R1: std::forward_like will be voted on in plenary at the same meeting as this paper. P2445R1 adds facilities to <utility>. LWG has reviewed those changes in the context of this paper, and approves of those contents being made freestanding.

Instructions to the editor:

Please insert a // all freestanding comment at the beginning of the following synopses. These headers are entirely freestanding.

Change in [cstdlib.syn]

Instructions to the editor:
Please append a // freestanding comment to the following entities:

Change in [memory.syn]

Instructions to the editor:
Please append a // freestanding comment to the following entities: Please append a // freestanding comment to all overloads of the following function templates, except for the ExecutionPolicy overloads. This includes the function templates in the ranges namespace. Note: the following portions of <memory> are omitted. No change to the working draft should be made for these entities:

Change in [functional.syn]

Instructions to the editor:
Please append a // freestanding comment to every entity in <functional> except for the following entities:

Change in [func.bind.place]

Instructions to the editor:
Add a new paragraph to [func.bind.place]:
3
Placeholders are freestanding entities ([freestanding.entity]).

Change in [iterator.synopsis]

Drafting note:

P2278R3: cbegin should always return a constant iterator will be voted on in plenary at the same meeting as this paper. P2278R3 adds facilities to <iterator>. LWG has reviewed those changes in the context of this paper, and approves of those contents being made freestanding.

Instructions to the editor:

Please append a // freestanding comment to every entity in <iterator> except for the following entities:

Change in [ranges.syn]

Drafting note:

P2278R3: cbegin should always return a constant iterator and P2446R2: views::as_rvaluewill be voted on in plenary at the same meeting as this paper. Those papers add facilities to <ranges>. LWG has reviewed those changes in the context of this paper, and approves of those contents being made freestanding.

Instructions to the editor:

Please append a // freestanding comment to every entity in <ranges> except for the following entities:

Change in [atomics.syn]

Instructions to the editor:
Please append a // freestanding comment to every entity in <atomic> except for the following entities:

Change in [atomics.lockfree], paragraph 2

On a hosted implementation ([compliance]), aAt least one signed integral specialization of the atomic template, along with the specialization for the corresponding unsigned type ([basic.fundamental]), is always lock-free.
[ Note
:
This requirement is optional in freestanding implementations ([compliance]).
— end note
 ]

Acknowledgements

Thanks to Brandon Streiff, Joshua Cannon, Phil Hindman, and Irwan Djajadi for reviewing P0829.

Thanks to Odin Holmes for providing feedback and helping publicize P0829.

Thanks to Paul Bendixen for providing feedback while prototyping P0829.

Thanks to Walter Brown for providing wording feedback.

Similar work was done in the C++11 timeframe by Lawrence Crowl and Alberto Ganesh Barbati in N3256.