Cloe empowers developers of automated-driving software components by providing a unified interface to closed-loop simulation. It achieves this by abstracting over environment and vehicle simulators and building upon these.
Cloe isn't a single tool. It is a set of components written in different languages employing different tools that work together:
- Cloe runtime & engine (C++)
- Cloe command line interface (Python)
- Cloe web user interface (Javascript/React)
Contributing to the CLI and the UI is pretty straightforward, as these are tools that live more-or-less 100% in their respective subdirectories. The runtime and engine are a different story though.
Cloe is meant to be extended through plugins that build on the Cloe runtime and are integrated at runtime by the Cloe engine. These are written in C++, which doesn't have a defacto method of packaging. Finding a solution for integrating C++ packages is always going to be a trade-off between ease-of-use for developers and users. In our experience, it is better to require more from developers than from users, as demanding too much from users usually ends up backfiring with increased support issues.
For in-depth documentation, see our website.
For building, deploying, and running the runtime and engine we use Conan, a modern C++ package manager. We currently have not published any Conan packages that can be downloaded directly. Building them yourself is pretty straightforward.
Currently, we only support Linux or WSL.
We provide automatic dependency installation for Ubuntu from 18.04 to 22.04
via the Makefile.setup Makefile. You should inspect it before
running the targets, as these will modify your system.
Other distributions may work, if the packages are available.
git clone https://github.com/eclipse/cloe.git
cd cloe
sudo make install-system-deps
make install-python-deps
You may need to setup your Conan profile before continuing on to the next point. In a pinch, the following steps should suffice:
- Install Conan with Python.
pip3 install --user --upgrade "conan<2.0.0" - Define a Conan profile, which defines the machine configuration.
If everything works out, your Conan profile should look something like this.
conan profile new --detect default conan profile update settings.compiler.libcxx=libstdc++11 default conan profile update settings.build_type=RelWithDebInfo default$ conan profile show default Configuration for profile default: [settings] os = Linux os_build = Linux arch = x86_64 arch_build = x86_64 compiler = gcc compiler.version = 9 compiler.libcxx = libstdc++11 build_type = RelWithDebInfo
See the Conan documentation for more information on how to do this.
To build all packages, you should run the following:
make export-vendor export-all
make -C meta package
This will export all Conan recipes from this repository and create the cloe package. Conan will download and build all necessary dependencies. Should any errors occur during the build, you may have to force Conan to build all packages instead of re-using packages it finds:
make -C meta package CONAN_OPTIONS="--build"
Run make help to get an overview of the available targets we expect you to
use. For more details on how this is done, have a look at the Makefiles in the
repository root.
If you experience timeout issues waiting for Conan Center, you can slightly increase Conan's timeout configuration like so:
export CONAN_REQUEST_TIMEOUT=360
Since Cloe is made up of many packages, running the Cloe engine directly is
somewhat tricky. Conan provides the virtualrunenv generator, which creates
shell scripts that you can source, similar to Python's virtualenv. Or, you can
use the cloe-launch tool, in the cli directory, which wraps all this
functionality for you in one convenient place.
You can install cloe-launch with pipx (or pip), but we recommend you just
use the make target:
make deploy-cli
If you want to install it in editable mode, you can use:
make -C cli editable
This has the advantage that any updates to the repository will be transparently used.
Once the cloe-launch tool is available, you can do one of the following:
- Launch a shell with the environment adjusted:
$ cloe-launch -v shell tests/conanfile_default.py Source profile: tests/conanfile_default.py Profile name: 7745ffb0e036192c8e29a8b8cc2b9571e7a72c8c Configuration: ... Runtime directory: /home/captain/.cache/cloe/launcher/7745ffb0e036192c8e29a8b8cc2b9571e7a72c8c Exec: conan install --install-folder /home/captain/.cache/cloe/launcher/7745ffb0e036192c8e29a8b8cc2b9571e7a72c8c -g virtualenv -g virtualrunenv tests/conanfile_default.py Exec: /bin/bash -c source /home/captain/.cache/cloe/launcher/7745ffb0e036192c8e29a8b8cc2b9571e7a72c8c/activate_run.sh &>/dev/null && env Exec: /bin/bash -c source /home/captain/.cache/cloe/launcher/7745ffb0e036192c8e29a8b8cc2b9571e7a72c8c/activate_all.sh &>/dev/null && env [cloe-shell] $ cloe-engine usage Cloe 0.19.0 (2022-11-21) ...
- Launch
cloe-enginedirectly:$ cloe-launch -v exec conanfile.py -- usage Source profile: tests/conanfile_default.py Profile name: 7745ffb0e036192c8e29a8b8cc2b9571e7a72c8c Configuration: ... Runtime directory: /home/captain/.cache/cloe/launcher/7745ffb0e036192c8e29a8b8cc2b9571e7a72c8c Exec: conan install --install-folder /home/captain/.cache/cloe/launcher/7745ffb0e036192c8e29a8b8cc2b9571e7a72c8c -g virtualenv -g virtualrunenv tests/conanfile_default.py Exec: /bin/bash -c source /home/captain/.cache/cloe/launcher/7745ffb0e036192c8e29a8b8cc2b9571e7a72c8c/activate_run.sh &>/dev/null && env Exec: /bin/bash -c source /home/captain/.cache/cloe/launcher/7745ffb0e036192c8e29a8b8cc2b9571e7a72c8c/activate_all.sh &>/dev/null && env Exec: /home/captain/.conan/data/cloe-engine/0.19.0/cloe/develop/package/218a01b0a23497c52f95fbb0967b2d6b1d9ac1e4/bin/cloe-engine usage --- Cloe 0.19.0 (2022-11-21) ...
Note that the above examples show the verbose output of the cloe-launch tool.
Integration and system tests can be run to ensure that all the packages built are working together as expected:
make export-all smoketest-deps
make smoketest
This will build packages in the required configurations as defined by the
profiles in the tests directories in the various packages. Then it will
run all available BATS tests with each of the profiles.
Unit tests are compiled and run during the build step of each package, so this shouldn't need to be performed manually.
See the documentation on testing Cloe for more details.
You can disable running unit tests by setting CONAN_RUN_TESTS=0,
as documented in the Conan Manual.
Alternatively, you can also prevent the unit tests from being built
in the first place, by setting tools.build:skip_test=1 in the
Conan configuration, as documented
here
and here.
When using make to build the project, add this to the command line:
CONAN_OPTIONS="-c tools.build:skip_test=1"
Bazel tooling is currently experimental, and is meant to support users who already are using Bazel in their projects and are not afraid of providing their own registries and modules for Cloe's dependencies.
(That allows us to not have to vendor massive amounts of Bazel modules in this repository, of which Boost is the main problematic point. Once Boost makes it into the Bazel Central Registry, it may be worthwhile to vendor the remaining libraries in this repository.)
You will need to create a .bazelrc.user file in the repository root
with the following contents:
common --registry=file:///path/to/your/bazel/registry
This file is ignored by Git to prevent you from exposing your secrets. The registry should contain the following modules:
boost
cli11
esmini
incbin
inja
luajit (optional)
oatpp
open-simulation-interface
sol
The rest of the dependencies are taken from the Bazel Central Registry.
See MODULE.bazel for the full list.
The Dockerfile provided in the repository root can be used to create one
or more Docker images. These can be used as devcontainers or simply to
create Conan packages in a reproducible environment.
The Makefile.docker contains multiple targets to build, test, and
release Docker images for the current Cloe version in the repository.
Run make -f Makefile.docker help to get an impression of what is possible:
$ make -f Makefile.docker help
Usage: make target
Available Docker targets:
all build and test all Ubuntu versions
ubuntu-VERSION build and test the Ubuntu VERSION image
build-all build all Ubuntu versions
build-ubuntu-VERSION build the Ubuntu VERSION image
test-all build all Ubuntu versions
test-ubuntu-VERSION test the Ubuntu VERSION image
run-ubuntu-VERSION run the Ubuntu VERSION image
release-all release all Ubuntu versions
release-ubuntu-VERSION release the Ubuntu VERSION image
remove-current-images remove and prune all cloe/cloe-engine:0.19.0 Docker images
remove-all-images remove and prune all cloe/cloe-engine Docker images
User configuration:
CONAN_PROFILE=
PACKAGE_TARGET=
KEEP_SOURCES=
DOCKER_USER_ARGS=
Docker configuration:
UBUNTU_NAME=ubuntu
UBUNTU_VERSIONS=\
18.04
20.04
22.04
DOCKER=DOCKER_BUILDKIT=1 docker
DOCKER_CONTEXT=/home/captain/cloe
DOCKER_NETWORK=host
DOCKER_IMAGE=cloe/cloe-engine:0.19.0
DOCKER_BUILD_ARGS=\
--build-arg PROJECT_VERSION=0.19.0
DOCKER_RUN_ARGS=\
-it
--rm
Because Docker images may be built in environments that have a proxy running, the Makefile will automatically add the proxy variables if they are are detected in the host environment.
The section User configuration shows variables that can be set to modify
the resulting Docker image; if they are empty here, then the defaults in
the Dockerfile are used, which should suffice for most use-cases.
If you want to use a different Conan remote from the default, you need to
copy setup.sh.example to setup.sh and modify the values to match your
environment.
Note that this build requires the use of docker buildx, which has been available for some time. This allows us to mount secrets in at build time and also speeds up the build by the strategic use of caches.
When using build-all, cloe-osi fails to build with:
ERROR: Build-require 'protobuf' cannot be found in lockfile
See the upstream issue. In Conan 1, a dependency in a lockfile is only in host or build context, not both. Unfortunately, this is only fixed in Conan 2.
The use-case for build-all is to build everything in editable mode. For that, use the new super-build instead of building individual packages.
When using the cloe super-build package in tests, you may run into the
following error:
ERROR: At least two recipes provides the same functionality:
- 'fable' provided by 'cloe/0.25.0@cloe/develop', 'fable/0.25.0@cloe/develop'
- 'cloe-runtime' provided by 'cloe/0.25.0@cloe/develop', 'cloe-runtime/0.25.0@cloe/develop'
- 'cloe-models' provided by 'cloe/0.25.0@cloe/develop', 'cloe-models/0.25.0@cloe/develop'
This problem occurs when both cloe and one of the packages that cloe
provides are pulled in as dependencies. Ideally, Conan should then
use cloe instead, but it would appear that Conan, at least in v1,
is unable to do this.
The cloe super-build package is primarily used for development purposes,
and until these kinks can be worked out, you should use the set of
individual packages for production.