It’s hard to deny the simplified dependency management and deployment that Docker brings to the software industry. But some things don’t play well with containerized environments, such as working with Android phones. Whether you’re trying to automate device provisioning or execute a suite of tests, you might find a need to interact with the phone from inside of a Docker container. This post will outline the steps we took to make that work in our project. If you’d like to jump right into the code, see our example repo.
Why would you want to integrate Docker and adb together?
Imagine you’re on a development team that has invested heavily into automated integration testing of your product. You’re using state of the art test orchestration tools. You’ve abstracted away all the tedious and error-prone steps to run and interact with your software. But all that automation comes with a price–namely that your test orchestration tools need to be installed, configured, and run in a certain way for consistent, reliable results. Imagine a new developer joining your team and the list of tools they’ll need to install and correctly configure. To make matters worse, let’s assume that you’re not all using the same exact OS either. Now you have to worry about different installation procedures for MacOS and potentially multiple Linux distributions. What happens when you’ve got specific dependency requirements that aren’t satisfied by your Linux distro’s package manager? You have better things to spend your time on than your development and test environments.
Enter docker. Docker containers provide standardized, isolated–and most importantly–consistent runtime environments. You can pre-install and pre-configure all your orchestration tools into a Docker image. Now you’re only requirement to onboard someone onto the team is that they can run docker. You can even use the same Docker images in the cloud to scale up your automated testing! This is great! Now you can get back to developing your…
…Android application?!
You’re going to need to interact with an Android device in order to perform all the automated tests your team has worked so hard to create. And chances are, you’ll need adb (the Android Debug Bridge) to communicate with the Android device. So you install the Android platform tools into your Docker image… then remember that you’re in a container that’s isolated from the host hardware. That raises an issue because adb communicates with Android devices through USB.
Aha! You can bind your host’s USB device into the Docker container. Now adb works again! Except that this approach will only work when your host OS is Linux. This won’t work when the host is MacOS. An open ticket against Docker (still open as of April 2023) describes the technical limitation as to why USB passthrough is unsupported on MacOS.
If we look a little more closely at how adb is implemented, it consists of three components: a client, a daemon, and a server. When you run adb commands, the executable that runs is the client. Other applications (such as Android Studio) or libraries (such as the adbutils Python package) may also be adb clients. The daemon runs on each connected Android device. And finally the server runs in the background on your machine (usually started automatically when you first invoke adb) and handles all the communication between clients and device daemons. Communication between the server and device daemons is typically through USB, but clients and servers communicate over TCP.
Knowing this, we can keep the adb server on the host and run the adb client commands (and other adb client tools) inside our Docker container if we enable a TCP connection back to the host. Because this does not rely on running privileged containers nor binding of USB devices, this works for both Linux and MacOS hosts!
In theory, this approach may also work on Windows hosts that are capable of running docker. However as of the time of this writing, no one on our team uses Windows and thus we haven’t been able to confirm that it would work on Windows hosts.
How To Use ADB In Docker
Set Up the Host
In order for adb clients in a docker container to communicate with an adb server on the host, the adb server has to be manually started on the host since it will not be started automatically. Additionally, by default when you start the adb server (or when it is automatically started), it is only listening on 127.0.0.1, which will not accept connections coming from a container.
We have to instruct the adb server to listen on all interfaces with the -a option. Unfortunately, this option is not forwarded to the server when it is started automatically in the background. Therefore we have to start the server using adb -a nodaemon server start (note the use of server start instead of start-server), which will run the server in the foreground and respect the all-interface option. We then redirect this to /dev/null and fork it to send it to the background.
Set Up the Container
Communicating with the docker host
In order for adb clients in a docker container to communicate with an adb server on the host, we have to be able to make TCP connections from the container back to the host. According to Docker’s docs, the host.docker.internal hostname will resolve to the host from inside a container. What is not explained well by their docs is that this only works out-of-the-box for Windows or MacOS (i.e., when using Docker Desktop). For compatibility between Linux and MacOS, we have to add the host.docker.internal:host-gateway host explicitly when starting the container. This will now allow the adb client inside the container to establish a connection back to the adb server on the host using host.docker.internal as the address for the server.
docker run --add-host=host.docker.internal:host-gatewayOr if using docker-compose,
services:
service-name:
extra_hosts:
- "host.docker.internal:host-gateway"And lastly, we have to instruct the adb clients in the container to connect to the host’s adb server address by setting the ANDROID_ADB_SERVER_ADDRESS environment variable to host.docker.internal.
Putting it all together:
A note about ADB versions
It will be important that the version of adb installed into the Docker image is compatible with the version of adb installed on the host machine. While the exact versions of the adb executable do not need to match, the version of the Android Debug Bridge protocol supported by both need to match exactly. Given the following:
Even though the host and container versions of adb differ (30.0.5 vs 34.0.1), they both use the 1.0.41 version of the Android Debug Bridge protocol.
When the protocol versions differ, the use of the adb client will result in a restart of the adb server from inside the container, which will fail.
An Example
To showcase all the pieces put together, we’ve created a small example project to demonstrate the use of adb inside a Docker container, which can be found here (https://github.com/twosixtech/demo-integrating-docker-and-adb). This repo contains a Python script that uses the adbutils package to communicate with Android devices. In a real-world project, this might be a script to automate the setup of an Android device for testing. The repo also contains a Robot Framework test case file that invokes adb commands. In a real-world project, this might be a suite of automated tests.
The Dockerfile defines the Docker image used by this example project. The key takeaway here is the installation of the Android SDK platform-tools inside of the Docker image. For our specific example, we also install Python 3.8 and the packages we desire, including Robot Framework.
The docker-compose.yml is how we’ve chosen to start the container, but this could just as easily have been a docker run... command. The important container parameters defined in this file are the ANDROID_ADB_SERVER_ADDRESS environment variable and the host.docker.internal extra host. We also use the run_docker_image.sh script to invoke docker-compose because we have to kill and re-start the adb server before running the docker compose command.
Once the Docker container has been started, we use docker exec to execute the Python script or run the Robot tests. In a real-world project these might be configured as the ENTRYPOINT of the Docker image.
To run the Python script, we run docker exec adb-docker-demo ./python_adb_example.py. The script will iterate over all connected Android devices and print out some of the device properties for each.
To run the Robot tests, we run docker exec adb-docker-demo robot robot_adb_example.robot. In our example, this is only confirming that the adb devices command runs successfully and does not result in any error.
If we did not set up the container correctly or start the host adb server, we would see very different results.
Integrating with Android Studio
By default Android Studio will manage the adb server on the host. Even if you’ve run the adb server manually with the necessary all-interface option, it will be restarted with the default interface when you launch Android Studio.
Android Studio has a preference setting to use an externally-managed adb server, however it won’t work with the default port 5037. It will only work with adb servers running on port 5038 or above.
We can start the adb server on port 5038 and set the ANDROID_ADB_SERVER_PORT environment variable to match. This will allow for adb clients in the container and Android Studio to both communicate with the host adb server.
In Conclusion
The ability to use adb from inside a Docker container has enabled us to perform complicated Android device orchestration and execution of tests against Android devices in a consistent, cross-platform manner. Docker is a great solution to managing a complicated tech stack but every now and then the abstraction of the host system breaks. Android Debug Bridge (adb) which typically relies on USB connectivity, is one of these cases. By configuring the containerized adb client to work with the host’s adb server, we enable adb inside of a Docker container to interact with Android devices outside of the Docker container.
While the final solution is not very complicated, we found that all the pieces necessary to make it work were not always well documented nor did we find any good end-to-end examples. Our example repo (https://github.com/twosixtech/demo-integrating-docker-and-adb) includes a complete end-to-end example including a Dockerfile and docker-compose.yml to build and run a Docker container, as well as Python and Robot Framework examples of using adb from within the container.
Best of luck to all in your Android adventures.
Bridging the gap between the impossible and the practical with innovative technology solutions in cyber, data science, mobile, microelectronics and information operations and a full spectrum of products and capabilities to advance national security missions.