Set up your local machine manually¶
These instructions are maintained on a best-effort basis. Prefer using the developer environment instead.
Prerequisites¶
Be sure that you already set up the development requirements.
Warning
Building the Agent for Windows requires using the build image. Setting up a local Windows environment is not officially supported.
Build tools¶
CMake version 3.15 or later and a C++ compiler are required for building the Agent.
Python¶
The Agent embeds a full-fledged CPython interpreter so it requires the development files to be available in the dev env. The Agent can embed Python 3, you will need development files for the version you want to support.
If you're on OSX/macOS, install 3.12 with Homebrew:
On Linux, depending on the distribution, you might need to explicitly install the development files, for example on Ubuntu:
On Windows, install 3.12 via the official installer brings along all the development files needed:
Warning
If you don't use one of the Python versions that are explicitly supported, you may have problems running the built Agent's Python checks, especially if using a virtualenv. At this time, only Python 3.12 is confirmed to work as expected in the development environment.
Python Dependencies¶
To protect and isolate your system-wide python installation, a python virtual environment is highly recommended (though optional). It will help keep a self-contained development environment and ensure a clean system Python.
Note
Due to the way some virtual environments handle executable paths (e.g. python -m venv), not all virtual environment options will be able to run the built Agent correctly. At this time, the only confirmed virtual environment creator that is known for sure to work is virtualenv.
- Install the virtualenv module:
- Create the virtual environment:
- Activate the virtualenv (OS-dependent). This must be done for every new terminal before you start.
If using virtual environments when running the built Agent, you may need to override the built Agent's search path for Python check packages using the PYTHONPATH variable (your target path must have the pre-requisite core integration packages installed though).
See also some notes in ./checks about running custom python checks.
Golang¶
You must install Golang version 1.24.6 or later. Make sure that $GOPATH/bin is in your $PATH, otherwise tooling cannot use any additional tool it might need.
Note
Versions of Golang that aren't an exact match to the version specified in our build images (see e.g. here) may not be able to build the agent and/or the rtloader binary properly.
Installing tools¶
From the root of datadog-agent, run dda inv install-tools to install go tooling. This uses go to install the necessary dependencies.
System or Embedded?¶
When working on the Agent codebase you can choose among two different ways to build the binary, informally named System and Embedded builds. For most contribution scenarios you should rely on the System build (the default) and use the Embedded one only for specific use cases. Let's explore the differences.
System build¶
System builds use your operating system's standard system libraries to satisfy the Agent's external dependencies. Since, for example, macOS 10.11 may provide a different version of Python than macOS 10.12, system builds on each of these platforms may produce different Agent binaries. If this doesn't matter to you—perhaps you just want to contribute a quick bugfix—do a System build; it's easier and faster than an Embedded build. System build is the default for all build and test tasks, so you don't need to configure anything there. But to make sure you have system copies of all the Agent's dependencies, skip the Embedded build section below and read on to see how to install them via your usual package manager (apt, yum, brew, etc).
Embedded build¶
Embedded builds download specifically-versioned dependencies and compile them locally from sources. We run Embedded builds to create Datadog's official Agent releases (i.e. RPMs, debs, etc), and while you can run the same builds while developing locally, the process is as slow as it sounds. Hence, you should only use them when you care about reproducible builds. For example:
- you want to build an agent binary that can be used as-is to replace the binary of an existing agent installation
- some dependencies are not available on your system
- you're working or debugging at a very low level: let's say you're adding a function to the Python bindings, you want to make sure you're using the exact same versions of Python as the official Agent packages
Embedded builds rely on Omnibus to download and build dependencies, so you need a recent ruby environment with bundler installed. See how to build Agent packages with Omnibus for more details.
Systemd¶
The agent is able to collect systemd journal logs using a wrapper on the systemd utility library.
On Ubuntu/Debian:
On Redhat/CentOS:
Doxygen¶
We use Doxygen to generate the documentation for the rtloader part of the Agent.
To generate it (using the dda inv rtloader.generate-doc command), you'll need to have Doxygen installed on your system and available in your $PATH. You can compile and install Doxygen from source with the instructions available here. Alternatively, you can use already-compiled Doxygen binaries from here.
To get the dependency graphs, you may also need to install the dot executable from graphviz and add it to your $PATH.
Pre-commit hooks¶
It is optional but recommended to install pre-commit to run a number of checks done by the CI locally.
Installation¶
To install it, run:
python3 -m pip install pre-commit
GOFLAGS=-buildvcs=false pre-commit install # buildvcs avoids errors when getting go dependencies
The shellcheck pre-commit hook requires having the shellcheck binary installed and in your $PATH. To install it, run:
(by default, the shellcheck binary is installed in /usr/local/bin).
Skipping pre-commit¶
If you want to skip pre-commit for a specific commit you can add --no-verify to the git commit command.
Running pre-commit manually¶
If you want to run one of the checks manually, you can run pre-commit run <check name>.
You can run it on all files with the --all-files flag.
See pre-commit run --help for further options.
Setting up Visual Studio Code Dev Container¶
Tip
Using the developer environment approach is recommended.
Microsoft Visual Studio Code with the devcontainer plugin allow to use a container as remote development environment in vscode. It simplify and isolate the dependencies needed to develop in this repository.
To configure the vscode editor to use a container as remote development environment you need to:
- Install the devcontainer plugin and the golang language plugin.
- Run the following command
dda inv vscode.setup-devcontainer --image "<image name>". This command will create the devcontainer configuration file./devcontainer/devcontainer.json. - Start or restart your vscode editor.
- A pop-up should show-up to propose to "reopen in container" your workspace.
- The first start, it might propose you to install the golang plugin dependencies/tooling.