DUDS Description

Distributed Update of Data from Something

A silly name, because I'm not good at making up names, that attempts to describe the ultimate goal for this family of libraries. At present, DUDS only has a single library with functionality for time and interfacing with hardware through the Linux kernel. It aspires to be a set of libraries that include functionality for representing measurements taken from hardware, sending that data over a network, and doing so in a multi-platform environment. At the moment, it is more about using low-level hardware with a high-level interface.

While only a single library results from the build at present, it will be broken up into several libraries as it grows to allow use of only part of DUDS functionality.

Notable missing features:

• Asynchronous serial support
• Interrupt or event like response to changes in an input pin's state
• An alternative to the gcc specific restrict keyword (has underscores, but that makes Doxygen produce bold text)
• Some unit tests

Author

Jeff Jackowski

Note

If you are viewing the documentation from Codedocs.xyz, a number of diagrams will be missing. It looks like the site doesn't like my configuration for the dot tool, but I'm not sure how to fix it.

License

The license is the two-clause simplified BSD license. See the License page for the text.

A few files in duds/general have code I did not write. They are marked with the correct author and license. It is my intent that any other licenses work well with the BSD license. So far, the only code not under a BSD license is in the public domain.

Goals

• Be a well designed C++ happy library that is multi-platform where applicable.
• Take advantage of C++17.
• Support multi-threaded programs.
• Support dynamic resource usage.
• Favor correctness over performance.
• Favor flexibility and capability over ease of use.
• Consist of namespaces with well defined purposes that are as independent as sensible.
• Offer robust error reporting and handling.
• Have unambiguous time stamps.
• Track accuracy, precision, resolution, and origin of sample data.
• Have good reference documentation.
• Allow for internationalization support.

Audience

The audience, users of the library, include:

• Intermediate and experienced C++ developers
• Developers who want to mix high-level application code with use of low-level gizmos

The audience is not intended to include:

• Beginning developers
• Developers unfamiliar with C++
• Developers making high performance applications
• Developers who avoid error handling

While it is not intentional to exclude people from using this library, the library isn't intended to be a perfect fit for everyone or every use. For instance, it isn't made with the goal of being super easy to use as a top priority, and the documentation assumes a familiarity with C++.

Documentation

The documentation is generated with the help of Doxygen and Graphviz. If you are looking at something else, like a Markdown page on Github, it may not be the most recent. Documentation comments are used throughout all header files and I consider writing them to be part of the development process. Currently, the documentation is limited to a reference with some sample source code thrown in.

Supported Platforms

Operating Systems

At present, DUDS only supports Linux, including kernel versions 3.6 through 6.6, and likely newer. The kernel's user-space interfaces for hardware may change and break support, but that doesn't happen on a regular basis. The hardware interface libraries are written to allow multiple implementations, but I have no other operating systems to test with, nor the inclination to make an implementation for another system.

I plan on supporting data representations and network communications on Windows as well as Linux, and hopefully this will work on other systems, too. The intention is to allow front-ends on a variety of platforms. However, since my only Windows running computer is an old thing with XP and Visual Studio 2005, I'm not yet sure how I'm going to build DUDS on Windows. Also, I'm not giving up C++17 to build on Windows and I'm not sure if VS2015 supports all the C++17 features used in the source, but it should at least have almost all of the support needed. Earlier versions of Visual Studio all lack required C++14 features.

All development will be on Linux with gcc for a while longer, and I'm in no rush to change that.

Target Hardware

The target hardware is the system that runs the operating system, DUDS, and the programs. Currently DUDS uses the user-space interfaces of the Linux kernel for hardware support. As a result, it requires the kernel to be built with support for the hardware. This limits hardware speciifc code in DUDS to drivers that communicate with gizmos through kernel interfaces like i2c-dev.

The library is structured to allow support to be written for specific hardware, although the build system doesn't yet have the ability to build a subset of the code.

Device Hardware

The devices include all hardware separate from the target hardware (although potentially on the same board) that the software will communicate with. Some devices are supported by drivers in the Linux kernel. Others will have support in duds::hardware::devices.

Supported devices are listed on the Supported Devices page.

Build Setup

Long list

• SCons 2.3 to 4.5 or so (the build isn't very picky about the version)
  • Python 2.7 or 3.x
• Build configuration options: run "scons -h"
  • The configuration can be changed for one build by specifying an option as an argument to scons.
  • To keep a configuration saved, set variables with the same name of the configuration options in Python syntax in localbuildconfig.py in the same directory as SConstruct.
• gcc 13.2.x, 12.3.x, 11.3.x
  • Other versions are untested. Older ones will likely fail.
• Boost version 1.62 or newer, tested to 1.84
  • Boost libraries used:
    • Date_time
    • Exception (header only)
    • Filesystem (optional)
    • Multiprecision (header only)
    • Program options (sample programs only)
    • Property tree (header only)
    • Serialization
    • Signals2 (header only)
    • Stacktrace (optional; header + dl library; Boost 1.65.1 or newer)
    • System
    • Unit test framework (optional; used for test programs, not DUDS library)
    • Variant (header only)
  • Build configuration options include Boost directories and variations on the names of the Boost library binaries.
  • Default configuration is for a regular Boost build that is installed to default paths. This works on Gentoo Linux when Boost is installed by Portage.
  • Windows will need several build options because there is no standard install that I know about. DUDS isn't useful on Windows at the moment, so don't worry about it.
• Build host
  • Linux
    • Raspberry Pi OS, Raspbian, Debian, Ubuntu, and similar:
      • The libraries and build system can be installed from root using this command:
        • apt-get install libboost-date-time-dev libboost-exception-dev libboost-filesystem-dev libboost-program-options-dev libboost-serialization-dev libboost-stacktrace-dev libboost-system-dev libboost-test-dev libeigen3-dev libevdev-dev scons
    • Gentoo works with the default configuration on a normal Gentoo install.
    • Parallel builds work best with 1GB RAM per job and some swap space.
      • Using "-j4" on a Raspberry Pi 2 or 3 isn't helpful. Using "-j2" needs swap space, but may be faster than "-j1", the default.
    • Builds tested mostly on armv6 and amd64.
    • Time for a complete debug build:
      • Raspberry Pi Zero: about two and a half hours
      • Raspberry Pi 3 Model B: getting close to half an hour with -j1 (default).
  • Windows
    • Not yet
    • VS2015 might be lacking some C++17 support needed by DUDS, and older versions will be lacking.

More Words!

DUDS uses SCons 3.x as its build system, which is written in Python. Compiling and linking is done with gcc 7.3.x, 9.2.x, or something similar.

The so-called C++17 standard, officially ISO/IEC 14882:2017, is used by DUDS. Use of features in newer C++ specifications is being avoided for now.

The Boost C++ library is required. Version 1.62 or newer must be used. Version 1.65.1 or newer is best as it allows exceptions to include a stack trace. Boost's multiprecision library is used for 128-bit integer support when gcc's __int128 is not available, and for serializing 128-bit integers even when __int128 is available. This results in some notable differences in 128-bit integer support on 32 and 64 bit platforms, but maintains a serialization format that is identical across platforms when things like endianness and padding do not affect the results.

The Boost unit test framework is used by a test program. This program does not require any special hardware to run. Some of the tests require leap second information from a zoneinfo (is that the proper name?) file. If the file isn't specified as the first argument to the program, those tests will be skipped.

Build targets

The default build target is the DUDS library made for debugging. Sample programs are built with the "samples" target, and the tests program (one program right now) with the "tests" target. All those targets make debug builds. There are also targets with names that end in either "-dbg", or "-opt", for debug or optimized builds respectively. Build from the root directory of DUDS by running scons. The binaries will be placed in a directory under bin named for the target and build. For example:

• duds
  • bin
    • linux-armv6l-dbg
      • samples (built with target "samples" or "samples-dbg")
    • linux-armv6l-opt
      • samples (built with target "samples-opt")
    • linux-x86_64-dbg
      • tests (built with target "tests" or "tests-dbg")
    • images (image archives built with target "images")

There is currently no target for installation. Despite using Linux since 2001, I'm not really sure how software is typically installed.

Multiple target support

There is some support in the build for handling building for multiple targets with the same source files. I intended this to make it easier to go back and forth between my deskunder computer (AMD64) and a Raspberry Pi using the same directory with help from sshfs. It is my first attempt at such support with SCons, and has a few limitations and issues.

• Currently, the target platform must be the host platform. This prevents building for slower systems, like the Raspberry Pi, on a different faster host, like AMD64. Fixing this also requires some non-trivial setup, like having the Boost libraries for the target available on the host. I won't be spending time on solving this; I'd rather improve and use DUDS.
• The use of distcc is not supported. I do use it to build the software Gentoo installs on my Raspberry Pis, and configured it so that my AMD64 system can assist. Supporting this may be easier than supporting a target that is different from the host while getting some of the benefit.
• Cleaning the build only cleans the it for the host platform. I think this is good, but may be unexpected to some users.
• When building from the same filesystem on two different hosts of the same architecture that needs a different build configuration, the build must be cleaned after building on one host and before building on the other. Failure to clean the build will leave the build partly configured for the host that last built it after cleaning.
• The file duds/BuildConfig.h is used by all targets, but may have different contents for each target. It will change between 32 and 64-bit targets, so I'd like a better solution. The dependency checking of SCons is smart enough to limit rebuilding in this case because it uses an MD5 hash of the file contents rather than a modification time, so I don't consider this a high priority yet.
• Simultaneously building for multiple targets using a shared filesystem and multiple hosts is not supported and will not work correctly. If each host has its own copy of the source and build files, then they can all build together.
• Any programs that use the DUDS library that are not built by the same build script and use the header and library files in the DUDS development tree will need to be built after DUDS has been built for the same target and before it is built for another target. This is to ensure that duds/BuildConfig.h is correct for the target.

Timezone configuration

The time support in the DUDS library requires knowledge of leap seconds to support unambiguous times, but it can be used without this data. The data is read from the zoneinfo files that are common on Linux installations. Not all of these files include leap second data. The files in /usr/share/zoneinfo usually lack the data. The files in /usr/share/zoneinfo-leaps, or /usr/share/zoneinfo/right (older) should have the data.

On Gentoo Linux, the files with leap second data are provided by the package sys-libs/timezone-data when the use flag leaps_timezone is given.

The system should not be configured with a timezone from a zoneinfo file with leap second data. As of 2018 May 10, Gentoo's zoneinfo files with leap second data causes local timezones to differ from UTC by the regular amount minus 27 seconds. This matches no system of time I can find. It looks like the zonefile was intended to configure the system such that what it claimed was UTC was really TAI, but it only accounted for the 27 leap seconds and not the 10 second difference that existed when UTC officially started.

Internationalization

Support for internationalization is mostly an attempt at keeping string literals out of the library or Human language neutral. For instance, the exception classes do not define error strings returned by std::exception::what().

All std::string objects are presumed to contain UTF-8 strings, but nothing actually cares right now.