- If you want to use the Docker executor, make sure to install Docker before using GitLab Runner. Read how to install Docker for your distribution. Installing GitLab Runner.
- Sep 16, 2020 Make sure you have installed the NVIDIA driver and Docker engine for your Linux distribution Note that you do not need to install the CUDA Toolkit on the host system, but the NVIDIA driver needs to be installed. For instructions on getting started with the NVIDIA Container Toolkit, refer to the installation guide.
- Install and register GitLab Runner for autoscaling with Docker Machine The auto scale feature was introduced in GitLab Runner 1.1.0. For an overview of the autoscale architecture, take a look at the comprehensive documentation on autoscaling.
- GitLab CE is an open source, cloud-based Git repository and version control system used by thousands of organizations worldwide. Written in Ruby, GitLab CE includes a host of features that enable software development teams to consolidate source code, track and manage releases, increase code quality, deploy code changes, and track the evolution of software over time.
To install Docker on Kali you need to remember that there is already a package named “docker”, therefore Docker has to be installed under a different name. If you install docker you will not end up with the container version. The version we will be installing is named docker.io. All commands are the same however, so running docker on the command line will be the appropriate command.
- Using the forked version of Docker Machine
- Using GPUs on Google Compute Engine
The auto scale feature was introduced in GitLab Runner 1.1.0.
For an overview of the autoscale architecture, take a look at thecomprehensive documentation on autoscaling.
Forked version of Docker machine
Because docker-machine
is in maintenancemode, GitLab isproviding it’s own fork ofdocker-machine
,which is based on the latest master
branch of docker-machine
withsome additional patches for the following bugs:
The intent of this fork is to fix critical and bugs affecting runningcosts only. No new features will be added.
Preparing the environment
To use the autoscale feature, Docker and GitLab Runner must beinstalled in the same machine:
- Log in to a new Linux-based machine that will serve as a bastion serverwhere Docker will spawn new machines from
- Optionally but recommended, prepare aproxy container registry and a cache serverto be used with the autoscaled runners
If you need to use any virtualization/cloud providers that aren’t handled byDocker Machine’s internal drivers, the appropriate driver plugin must beinstalled. The Docker Machine driver plugin installation and configuration isout of the scope of this documentation. For more details please read theDocker Machine documentation
Configuring GitLab Runner
- Register a runner and select the
docker+machine
executor when asked. - Edit
config.toml
and configurethe runner to use Docker machine. Visit the dedicated page covering detailedinformation about GitLab Runner Autoscaling. The first time you’re using Docker Machine, it’s best to execute manually
docker-machine create ..
with your chosen driver and all options from theMachineOptions
section. This will set up the Docker Machine environmentproperly and will also be a good validation of the specified options.After this, you can destroy the machine withdocker-machine rm [machine_name]
and start the runner.Multiple concurrent requests todocker-machine create
that are doneat first usage are not good. When thedocker+machine
executor is used,the runner may spin up few concurrentdocker-machine create
commands. IfDocker Machine was not used before in this environment, each started processtries to prepare SSH keys and SSL certificates (for Docker API authenticationbetween GitLab Runner and Docker Engine on the autoscaled spawned machine), and theseconcurrent processes are disturbing each other. This can end with a non-workingenvironment. That’s why it’s important to create a test machine manually thevery first time you set up GitLab Runner with Docker Machine.- Now, you can try and start a new pipeline in your project. In a few seconds,if you run
docker-machine ls
you should see a new machine being created.
Upgrading GitLab Runner
- Check if your operating system is configured to automatically restart GitLabRunner (for example, by checking its service file):
if yes, ensure that service manager is configured to use
SIGQUIT
and use the service’s tools to stop the process:if no, you may stop the process manually:
Sending theSIGQUIT
signal will make theprocess stop gracefully. The process will stop accepting new jobs, and will exitas soon as the current jobs are finished. Wait until GitLab Runner exits. You can check its status with
gitlab-runner status
or await a graceful shutdown for up to 30 minutes with:- You can now safely install the new version of GitLab Runner without interrupting any jobs.
Using the forked version of Docker Machine
Install
Download the appropriate
docker-machine
binary.Copy the binary to a location accessible toPATH
and make itexecutable. For example, to download and installv0.16.2-gitlab.11
:
Using GPUs on Google Compute Engine
Introduced in GitLab Docker Machine 0.16.2-gitlab.10
and GitLab Runner 13.9.
You can use the Docker Machine fork to create Google Compute Engine instanceswith graphics processing units (GPUs).GitLab Runner 13.9 is required for GPUs to work in a Docker executor.
Docker Machine GPU options
To create an instance with GPUs, use these Docker Machine options:Option | Example | Description |
---|---|---|
--google-accelerator | type=nvidia-tesla-p4,count=1 | Specifies the type and number of GPU accelerators to attach to the instance (type=TYPE,count=N format) |
--google-maintenance-policy | TERMINATE | Always use TERMINATE because Google Cloud does not allow GPU instances to be live migrated. |
--google-machine-image | https://www.googleapis.com/compute/v1/projects/deeplearning-platform-release/global/images/family/tf2-ent-2-3-cu110 | The URL of a GPU-enabled operating system. See the list of available images. |
--google-metadata | install-nvidia-driver=True | This flag tells the image to install the NVIDIA GPU driver. |
These arguments map to command-line arguments for gcloud compute
.See the Google documentation on creating VMs with attached GPUsfor more details.
Verifying Docker Machine options
To prepare your system and test that GPUs can be created with Google Compute Engine:
Set up the Google Compute Engine driver credentialsfor Docker Machine. You may need to export environment variables to therunner if your VM does not have a default service account. Howthis is done depends on how the runner is launched. For example:
- Via
systemd
orupstart
: See the documentation on setting custom environment variables. - Via Kubernetes with the Helm Chart: Update the
values.yaml
entry. - Via Docker: Use the
-e
option (for example,docker run -e GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials.json gitlab/gitlab-runner
).
- Via
Verify that
docker-machine
can create a virtual machine with yourdesired options. For example, to create ann1-standard-1
machinewith a single NVIDIA Telsa P4 accelerator, substitutetest-gpu
with a name and run:To verify the GPU is active, SSH into the machine and run
nvidia-smi
:Remove this test instance to save money:
Configuring GitLab Runner
Once you have verified these options, configure the Docker executorto use all available GPUs in the
runners.docker
configuration.Then add the Docker Machine options to yourMachineOptions
settings in the GitLab Runnerrunners.machine
configuration. For example:
Help & feedback
Docs
Edit this pageto fix an error or add an improvement in a merge request.Create an issueto suggest an improvement to this page.
Show and post commentsto review and give feedback about this page.
Docker Install Windows
Product
Create an issueif there's something you don't like about this feature.Propose functionalityby submitting a feature request.
Join First Lookto help shape new features.
Feature availability and product trials
View pricingto see all GitLab tiers and features, or to upgrade.Try GitLab for freewith access to all features for 30 days.
Get Help
If you didn't find what you were looking for,search the docs.
If you want help with something specific and could use community support,post on the GitLab forum.
Catalina install on unsupported mac. Attempting to install Catalina on an unsupported Mac is in no way supported by Apple, and you are doing so at your own risk. MacRumors cannot be held liable for any data loss or damage as a result.
For problems setting up or using this feature (depending on your GitLabsubscription).