Jobs based on custom runners: docs and gitlab-runner setup playbook

To have the jobs dispatched to custom runners, gitlab-runner must
be installed, active as a service and properly configured.  The
variables file and playbook introduced here should help with those
steps.

The playbook introduced here covers the Linux distributions and
has been primarily tested on OS/machines that the QEMU project
has available to act as runners, namely:

 * Ubuntu 20.04 on aarch64
 * Ubuntu 18.04 on s390x

But, it should work on all other Linux distributions.  Earlier
versions were tested on FreeBSD too, so chances of success are
high.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210630012619.115262-4-crosa@redhat.com>
Message-Id: <20210709143005.1554-4-alex.bennee@linaro.org>
This commit is contained in:
Cleber Rosa 2021-07-09 15:29:28 +01:00 committed by Alex Bennée
parent 159c5d177b
commit 40de78c284
4 changed files with 139 additions and 1 deletions

View File

@ -70,3 +70,58 @@ privileges, such as those from the ``root`` account or those obtained
by ``sudo``. If necessary, please refer to ``ansible-playbook``
options such as ``--become``, ``--become-method``, ``--become-user``
and ``--ask-become-pass``.
gitlab-runner setup and registration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The gitlab-runner agent needs to be installed on each machine that
will run jobs. The association between a machine and a GitLab project
happens with a registration token. To find the registration token for
your repository/project, navigate on GitLab's web UI to:
* Settings (the gears-like icon at the bottom of the left hand side
vertical toolbar), then
* CI/CD, then
* Runners, and click on the "Expand" button, then
* Under "Set up a specific Runner manually", look for the value under
"And this registration token:"
Copy the ``scripts/ci/setup/vars.yml.template`` file to
``scripts/ci/setup/vars.yml``. Then, set the
``gitlab_runner_registration_token`` variable to the value obtained
earlier.
To run the playbook, execute::
cd scripts/ci/setup
ansible-playbook -i inventory gitlab-runner.yml
Following the registration, it's necessary to configure the runner tags,
and optionally other configurations on the GitLab UI. Navigate to:
* Settings (the gears like icon), then
* CI/CD, then
* Runners, and click on the "Expand" button, then
* "Runners activated for this project", then
* Click on the "Edit" icon (next to the "Lock" Icon)
Tags are very important as they are used to route specific jobs to
specific types of runners, so it's a good idea to double check that
the automatically created tags are consistent with the OS and
architecture. For instance, an Ubuntu 20.04 aarch64 system should
have tags set as::
ubuntu_20.04,aarch64
Because the job definition at ``.gitlab-ci.d/custom-runners.yml``
would contain::
ubuntu-20.04-aarch64-all:
tags:
- ubuntu_20.04
- aarch64
It's also recommended to:
* increase the "Maximum job timeout" to something like ``2h``
* give it a better Description

View File

@ -1,2 +1,2 @@
inventory
vars.yml

View File

@ -0,0 +1,71 @@
# Copyright (c) 2021 Red Hat, Inc.
#
# Author:
# Cleber Rosa <crosa@redhat.com>
#
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
#
# This is an ansible playbook file. Run it to set up systems with the
# gitlab-runner agent.
---
- name: Installation of gitlab-runner
hosts: all
vars_files:
- vars.yml
tasks:
- debug:
msg: 'Checking for a valid GitLab registration token'
failed_when: "gitlab_runner_registration_token == 'PLEASE_PROVIDE_A_VALID_TOKEN'"
- name: Create a group for the gitlab-runner service
group:
name: gitlab-runner
- name: Create a user for the gitlab-runner service
user:
user: gitlab-runner
group: gitlab-runner
comment: GitLab Runner
home: /home/gitlab-runner
shell: /bin/bash
- name: Remove the .bash_logout file when on Ubuntu systems
file:
path: /home/gitlab-runner/.bash_logout
state: absent
when: "ansible_facts['distribution'] == 'Ubuntu'"
- name: Set the Operating System for gitlab-runner
set_fact:
gitlab_runner_os: "{{ ansible_facts[\"system\"]|lower }}"
- debug:
msg: gitlab-runner OS is {{ gitlab_runner_os }}
- name: Set the architecture for gitlab-runner
set_fact:
gitlab_runner_arch: "{{ ansible_to_gitlab_arch[ansible_facts[\"architecture\"]] }}"
- debug:
msg: gitlab-runner arch is {{ gitlab_runner_arch }}
- name: Download the matching gitlab-runner
get_url:
dest: /usr/local/bin/gitlab-runner
url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}"
owner: gitlab-runner
group: gitlab-runner
mode: u=rwx,g=rwx,o=rx
- name: Register the gitlab-runner
command: "/usr/local/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --tag-list {{ ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'"
- name: Install the gitlab-runner service using its own functionality
command: /usr/local/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner
register: gitlab_runner_install_service_result
failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr"
- name: Enable the gitlab-runner service
service:
name: gitlab-runner
state: started
enabled: yes

View File

@ -0,0 +1,12 @@
# The version of the gitlab-runner to use
gitlab_runner_version: 13.12.0
# The URL of the gitlab server to use, usually https://gitlab.com unless you're
# using a private GitLab instance
gitlab_runner_server_url: https://gitlab.com
# A mapping of the ansible to gitlab architecture nomenclature
ansible_to_gitlab_arch:
x86_64: amd64
aarch64: arm64
s390x: s390x
# A unique token made available by GitLab to your project for registering runners
gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN