跳到主要内容

CI/CD层

此部分是DevOps敏捷开发理念的核心实现。

3.1 镜像仓库harbor​

harbor资源

https://github.com/vmware/harbor/releases https://github.com/goharbor/harbor/releases

很多公司提供了它们公开的容器 Registr 服务，比如

  1. Docker 官方的 Registry
  2. 亚马逊 ECR（Elastic Container Registry）
  3. Google云Registry
  4. Project Atomic
  5. JFrog Artifactory
  6. dockerhub
  7. harbor
  8. quay.io

其中harbor是VMware公司开源的DockerRegistry项目，其目标是帮助用户迅速搭建一个企业级的DockerRegistry服务，提供图形界面和权限控制。它包括权限管理(RBAC)、LDAP、日志审核、管理界面、自我注册、镜像复制和中文支持等功能.

3.1.1 安装前准备​

主机基本信息

# hostname
image

# cat /etc/redhat-release 
Rocky Linux release 9.1 (Blue Onyx)

# ip addr | grep ens
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    inet 10.2.20.70/24 brd 10.2.20.255 scope global noprefixroute ens33
3: ens36: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    inet 192.168.3.20/24 brd 192.168.3.255 scope global dynamic noprefixroute ens36

安装docker

yum -y install yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum -y install docker-ce docker-ce-cli containerd.io
systemctl enable docker containerd
systemctl start docker containerd

测试

# docker version
Client: Docker Engine - Community
 Version:           23.0.2
 API version:       1.42
 Go version:        go1.19.7
 Git commit:        569dd73
 Built:             Mon Mar 27 16:19:13 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.2
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.7
  Git commit:       219f21b
  Built:            Mon Mar 27 16:16:18 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.20
  GitCommit:        2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc:
  Version:          1.1.5
  GitCommit:        v1.1.5-0-gf19387a
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

安装Docker-compose

# ver=v2.17.3
# curl -L https://github.com/docker/compose/releases/download/$ver/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
# chmod +x /usr/local/bin/docker-compose
# docker-compose version
Docker Compose version v2.17.3

3.1.2 安装harbor​

下载

# wget https://github.com/goharbor/harbor/releases/download/v2.8.1/harbor-offline-installer-v2.8.1.tgz
# tar zxvf harbor-offline-installer-v2.8.1.tgz 
# tree harbor
harbor
├── common.sh
├── harbor.v2.8.1.tar.gz
├── harbor.yml.tmpl       //配置文件模板，需“cp harbor.yml.tmpl harbor.yml”
├── install.sh            //安装脚本
├── LICENSE
└── prepare

配置harbor.yml

# cp harbor.yml.tmpl harbor.yml
# vi harbor.yml
hostname: harbor.demo.com		//配置访问域名和证书。
http:
  port: 80
https:
  port: 443
  certificate: /etc/ssl/test-ssl/web.pem
  private_key: /etc/ssl/test-ssl/web-key.pem
harbor_admin_password: 123qweasd+pp
database:
  password: root123
data_volume: /data/harbor

提示，需查将harbor.demo.com的证书复制到/etc/ssl/test-ssl/

安装harbor

# ./install.sh 
Note: docker version: 23.0.2
Note: Docker Compose version v2.17.2
Note: stopping existing Harbor instance ...
[+] Running 5/5
 ✔ Container harbor-portal  Removed                                                                                                                                0.1s 
 ✔ Container redis          Removed                                                                                                                                0.1s 
 ✔ Container harbor-db      Removed                                                                                                                                0.1s 
 ✔ Container harbor-log     Removed                                                                                                                                0.1s 
 ✔ Network harbor_harbor    Removed                                                                                                                                0.3s 


[Step 5]: starting Harbor ...
[+] Running 10/10
 ✔ Network harbor_harbor        Created                                                                                                                            0.3s 
 ✔ Container harbor-log         Started                                                                                                                            2.1s 
 ✔ Container registryctl        Started                                                                                                                            4.6s 
 ✔ Container harbor-db          Started                                                                                                                            4.1s 
 ✔ Container redis              Started                                                                                                                            4.1s 
 ✔ Container harbor-portal      Started                                                                                                                            3.6s 
 ✔ Container registry           Started                                                                                                                            4.4s 
 ✔ Container harbor-core        Started                                                                                                                            5.2s 
 ✔ Container harbor-jobservice  Started                                                                                                                            6.4s 
 ✔ Container nginx              Started                                                                                                                            6.8s 
✔ ----Harbor has been installed and started successfully.----

查验

# docker ps
CONTAINER ID   IMAGE                                COMMAND                  CREATED       STATUS                 PORTS                                                                            NAMES
ea6ff7de2bd3   goharbor/harbor-jobservice:v2.8.1    "/harbor/entrypoint.…"   9 days ago    Up 6 hours (healthy)                                                                                    harbor-jobservice
51c3d360f8f7   goharbor/nginx-photon:v2.8.1         "nginx -g 'daemon of…"   9 days ago    Up 6 hours (healthy)   0.0.0.0:80->8080/tcp, :::80->8080/tcp, 0.0.0.0:443->8443/tcp, :::443->8443/tcp   nginx
1d777e5c999c   goharbor/harbor-core:v2.8.1          "/harbor/entrypoint.…"   9 days ago    Up 6 hours (healthy)                                                                                    harbor-core
f37900962e2c   goharbor/harbor-registryctl:v2.8.1   "/home/harbor/start.…"   9 days ago    Up 6 hours (healthy)                                                                                    registryctl
64bf28a7ee91   goharbor/registry-photon:v2.8.1      "/home/harbor/entryp…"   9 days ago    Up 6 hours (healthy)                                                                                    registry
86f26071fac1   goharbor/harbor-db:v2.8.1            "/docker-entrypoint.…"   9 days ago    Up 6 hours (healthy)                                                                                    harbor-db
2988ed0c418f   goharbor/redis-photon:v2.8.1         "redis-server /etc/r…"   9 days ago    Up 6 hours (healthy)                                                                                    redis
f898c0d10656   goharbor/harbor-portal:v2.8.1        "nginx -g 'daemon of…"   9 days ago    Up 6 hours (healthy)                                                                                    harbor-portal
f99caa642448   goharbor/harbor-log:v2.8.1           "/bin/sh -c /usr/loc…"   9 days ago    Up 6 hours (healthy)   127.0.0.1:1514->10514/tcp                                                        harbor-log

# ss -lnt
State       Recv-Q      Send-Q           Local Address:Port             Peer Address:Port      Process    
LISTEN      0           4096                 127.0.0.1:1514                  0.0.0.0:*                    
LISTEN      0           4096                   0.0.0.0:80                    0.0.0.0:*                    
LISTEN      0           32                     0.0.0.0:53                    0.0.0.0:*                    
LISTEN      0           128                    0.0.0.0:22                    0.0.0.0:*                    
LISTEN      0           4096                   0.0.0.0:443                   0.0.0.0:*                    
LISTEN      0           4096                      [::]:80                       [::]:*                    
LISTEN      0           128                       [::]:22                       [::]:*                    
LISTEN      0           4096                      [::]:443                      [::]:*                    
LISTEN      0           4096                         *:2375                        *:*

重启harbor

# docker-compose start | stop | restart    //此命令的运行，依赖于安装目录中的docker-compose.yml文件。

3.1.3 访问测试​

提示: 在访问需要将私有CA的根证书添加到浏览器的根证书可信任区域。 harbor登录

harbor私仓

3.1.4 镜像push和pull测试​

可采用crictl/podman/docker等客户端工作都可以。

本测试临时安装一台测试机验，采用docker命令。

登录私有仓

下载证书
# wget http://10.2.20.59/ssl/ca.pem
加入根CA证书链
# cat ca.pem >> /etc/pki/tls/certs/ca-bundle.crt
登录私有仓
# docker login harbor.demo.com
Username: admin
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
#

push镜像到私有仓


下载一个镜像
# docker pull busybox:latest
latest: Pulling from library/busybox
325d69979d33: Pull complete 
Digest: sha256:560af6915bfc8d7630e50e212e08242d37b63bd5c1ccf9bd4acccf116e262d5b
Status: Downloaded newer image for busybox:latest
docker.io/library/busybox:latest

修改tag
# docker tag busybox:latest harbor.demo.com/temp/busybox:latest

上传到私有仓库
# docker push harbor.demo.com/temp/busybox:latest
The push refers to repository [harbor.demo.com/temp/busybox]
9547b4c33213: Pushed 
latest: digest: sha256:5cd3db04b8be5773388576a83177aff4f40a03457a63855f4b9cbe30542b9a43 size: 528

进入 https://harbor.demo.com 查看下

从私有仓拉取镜像到本地

查看镜像
# docker images
REPOSITORY                     TAG       IMAGE ID       CREATED       SIZE
harbor.demo.com/temp/busybox   latest    8135583d97fe   12 days ago   4.86MB
busybox                        latest    8135583d97fe   12 days ago   4.86MB

删除现有镜像
# docker image rm -f 8135583d97fe
Untagged: busybox:latest
Untagged: busybox@sha256:560af6915bfc8d7630e50e212e08242d37b63bd5c1ccf9bd4acccf116e262d5b
Untagged: harbor.demo.com/temp/busybox:latest
Untagged: harbor.demo.com/temp/busybox@sha256:5cd3db04b8be5773388576a83177aff4f40a03457a63855f4b9cbe30542b9a43
Deleted: sha256:8135583d97feb82398909c9c97607159e6db2c4ca2c885c0b8f590ee0f9fe90d
Deleted: sha256:9547b4c33213e630a0ca602a989ecc094e042146ae8afa502e1e65af6473db03

查看镜像，为空
# docker images
REPOSITORY   TAG       IMAGE ID   CREATED   SIZE

从私有仓拉取镜像
# docker pull harbor.demo.com/temp/busybox:latest
latest: Pulling from temp/busybox
325d69979d33: Pull complete 
Digest: sha256:5cd3db04b8be5773388576a83177aff4f40a03457a63855f4b9cbe30542b9a43
Status: Downloaded newer image for harbor.demo.com/temp/busybox:latest
harbor.demo.com/temp/busybox:latest

查看镜像
# docker images
REPOSITORY                     TAG       IMAGE ID       CREATED       SIZE
harbor.demo.com/temp/busybox   latest    8135583d97fe   12 days ago   4.86MB

3.2 代码仓库Gitlab​

gitlab(极狐)是一个利用 Ruby on Rails 开发的开源应用程序，实现一个自托管的Git项目仓库，可通过Web界面进行访问公开的或者私人项目。Ruby on Rails 是一个可以使你开发、部署、维护 web 应用程序变得简单的框架。

GitLab拥有与Github类似的功能，能够浏览源代码，管理缺陷和注释。可以管理团队对仓库的访问，它非常易于浏览提交过的版本并提供一个文件历史库。它还提供一个代码片段收集功能可以轻松实现代码复用，便于日后有需要的时候进行查找。

GitLab 分为如下两个版本 1.GitLab Community Edition(CE) 社区版。 社区版免费 2.GitLab Enterprise Edition(EE) 专业版。专业版收费

提示： 若在公网搭建gitlab服务，需开放http80、https443、ssh22三个端口。

3.2.1 安装及配置gitlab​

gitlab安装方式有源码方式、yum方式、docker方式。 本测试采用yum方式安装。

安装gitlab

yum -y install epel-release  curl policycoreutils openssh-server openssh-clients
systemctl disable firewalld
systemctl stop firewalld
curl -s https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | sudo bash
yum -y install gitlab-ce
systemctl enable gitlab-runsvdir.service
systemctl start gitlab-runsvdir.service

配置https(/etc/gitlab/gitlab.rb)，把证书复制到/etc/gitlab/ssl/(需新创建此目录)

# vi /etc/gitlab/gitlab.rb
external_url 'https://git.demo.com'
letsencrypt['enable'] = false
nginx['redirect_http_to_https'] = true
nginx['ssl_certificate'] = "/etc/gitlab/ssl/web.pem"
nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/web-key.pem"
# gitlab-ctl reconfigure

查看gitlab版本
# cat /opt/gitlab/embedded/service/gitlab-rails/VERSION
16.0.1

查看gitlab管理员root初始密码
# cat /etc/gitlab/initial_root_password

访问 https://git.demo.com

提示: 第一次进入后建议改密码

GitLab头像无法正常显示，可以配置如下

# vi /etc/gitlab/gitlab.rb
### Gravatar Settings
gitlab_rails['gravatar_plain_url'] = 'https://sdn.geekzu.org/avatar/%{hash}?s=%{size}&d=identicon'
gitlab_rails['gravatar_ssl_url'] = 'https://sdn.geekzu.org/avatar/%{hash}?s=%{size}&d=identicon'
# gitlab-ctl reconfigure

通知信息配置(邮件方式)

# vi /etc/gitlab/gitlab.rb
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.qq.com"
gitlab_rails['smtp_port'] = 465
gitlab_rails['smtp_user_name'] = "xxxxxxx@qq.com"
gitlab_rails['smtp_password'] = "oherqwzatxxxxxxxxj"
gitlab_rails['smtp_domain'] = "qq.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = false
gitlab_rails['smtp_tls'] = true
gitlab_rails['smtp_pool'] = true
gitlab_rails['gitlab_email_from'] = 'xxxxxxxxx@qq.com'
gitlab_rails['gitlab_email_display_name'] = 'Administrator'
gitlab_rails['gitlab_email_reply_to'] = 'xxxxxxxxxx@qq.com'
# gitlab-ctl reconfigure

测试

进入console
# gitlab-rails console
--------------------------------------------------------------------------------
 Ruby:         ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
 GitLab:       16.0.1 (34d6370bacd) FOSS
 GitLab Shell: 14.20.0
 PostgreSQL:   13.8
------------------------------------------------------------[ booted in 96.22s ]
Loading production environment (Rails 6.1.7.2)

发测试邮件：如Notify.test_email('xxxxxxx@163.com','this is title','hello gitlab').deliver_now

irb(main):001:0> Notify.test_email('xxxxxxx@163.com','this is title','hello gitlab').deliver_now
Delivered mail 647965bb85fe7_21e4317463563@test.mail (959.0ms)
=> #<Mail::Message:291920, Multipart: false, Headers: <Date: Fri, 02 Jun 2023 11:44:59 +0800>, <From: Administrator <xxxxxxxxxxxxxxx@qq.com>>, <Reply-To: Administrator <xxxxxxxxxxxx@qq.com>>, <To: xxxxxxxxxxxx@163.com>, <Message-ID: <647965bb85fe7_21e4317463563@test.mail>>, <Subject: this is title>, <Mime-Version: 1.0>, <Content-Type: text/html; charset=UTF-8>, <Content-Transfer-Encoding: 7bit>, <Auto-Submitted: auto-generated>, <X-Auto-Response-Suppress: All>>
irb(main):002:0> 
若能成功收到，说明配置是正常的。

创建新用户 创建新用户

3.2.2 测试​

假如已创建用户guofs，并以guofs帐户创建了项目web.git

3.2.2.1 ssh方式​

在用户机器上生成公钥和私钥（用于gitlab）

# ssh-keygen -t rsa -C "guofs@163.com"
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:AP1rvv6ySqYAwZeXQb7AXCvOFx2riAwToptIV++DixI guofs@163.com
The key's randomart image is:
+---[RSA 2048]----+
|o   o* .         |
|+.o =.B o        |
|=o O *.=         |
|==* = *..        |
|=E + = oS.       |
| .. o . +        |
| ... .oo         |
|  .. +  o        |
|    . .o+=.      |
+----[SHA256]-----+
# cat /root/.ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDAQSNJDMRjORJ599Ez6qdYpKh8V7L+TWv3kqkqxTmJf0ijEvdG/NqPAuC1QqswMlRVb8Zlu1hYawCYfF2FTQUnxW7dvgUXkbxaUvYBacarG/3ewVoa60+9w/kQFNyQsndt4BCYy8G0XsZfB1OmqFlErgQogHAGyau+CF3Fa8yY5j8b5dbHwtR9Yhrs3wyQlNuluU4TAAHTBMDQ6XkAagc53lAbz8VOF7NUbcDMXQ3EdZ74gYHh/RygS003gE+pNSoON+QX9y2uDmPWZQyB0ouRlqRpQx7taxq/nFva3bq55gCIzLAD52CotKeEPnHjEBnhUOAqMo8BIoMVs4Wl8mk5 guofs@163.com

将id_rsa.pub公钥复制到gitlab

clone项目

# git clone -b main git@git.demo.com:guofs/web.git
Cloning into 'web'...
The authenticity of host 'git.demo.com (10.2.20.36)' can't be established.
ED25519 key fingerprint is SHA256:KDdQTbTJm1fCmC0n3RrNmCJXGBBzXehOQbm4j31tYNg.
This host key is known by the following other names/addresses:
    ~/.ssh/known_hosts:65: 10.2.20.36
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'git.demo.com' (ED25519) to the list of known hosts.
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), done.

# tree web -a -L 2
web
|-- .git
|   |-- HEAD
|   |-- branches
|   |-- config
|   |-- description
|   |-- hooks
|   |-- index
|   |-- info
|   |-- logs
|   |-- objects
|   |-- packed-refs
|   `-- refs
`-- README.md

push代码

# cd web
# git config user.name "Darry.Guo"
# git config user.email "guofs@139.com"
# date >> 1.txt
# git add *
# git commit -m "11"
[main d4a8520] 11
 1 file changed, 1 insertion(+)
# git push git@git.demo.com:guofs/web.git main
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 2 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 301 bytes | 301.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To git.demo.com:guofs/web.git
   b492ec7..d4a8520  main -> main

3.2.2.2 http/https方式​

使用guofs进入 https://git.demo.com 创建token值。 例如：glpat-nC7mYxxfTdJQEJLuGpsR 创建token值

clone项目

# git clone -b main https://git.demo.com/guofs/web.git
Cloning into 'web'...
Username for 'https://git.demo.com': guofs
Password for 'https://guofs@git.demo.com': 	//此处录入token值
remote: Enumerating objects: 12, done.
remote: Counting objects: 100% (12/12), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 12 (delta 1), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (12/12), done.
Resolving deltas: 100% (1/1), done.

push代码

# cd web
# git config user.name "Darry.Guo"
# git config user.email "guofs@139.com"
# date >> 1.txt
# git add *
# git commit -m "11"
[main d4a8520] 11
 1 file changed, 1 insertion(+)
# git push https://git.demo.com/guofs/web.git main
Username for 'https://git.demo.com': guofs
Password for 'https://guofs@git.demo.com': 	//此处录入token值
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 2 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 308 bytes | 308.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0
To https://git.demo.com/guofs/web.git
   10e3dec..347fa15  main -> main

3.3 配置Gitlab CI/CD、Gitlab Runner​

持续集成（Continuous Integration）、持续交付（Continuous Delivery）是敏捷开发体现。 可以采用jenkins实现，也可以使用gitlab提供的CI/CD来实现。采用gitlab CI/CD时，主要分为四步

其一：安装gitlab runner

其二：通过gitlab runner向CI/CD注册各类executor。executor体具执行pipeline动作。executor不同，其执行的动作不相同。某个executor只完成特定动作。

其三：编写.gitlab-ci.yml

其四：Gitlab CI/CD依据.gitlab-ci.yml开展工作。CI/CD通过过push/merge/trigger等多种动作执行pipeline.

3.3.1 安装 Gitlab Runner​

官网：https://docs.gitlab.com/runner

runner安装在独立机器上或采用docker方式安装。 本文以docker方式安装为例。

# docker pull gitlab/gitlab-runner

# docker images
REPOSITORY             TAG       IMAGE ID       CREATED         SIZE
gitlab/gitlab-runner   latest    4575ef0329b0   2 weeks ago     749MB

# docker run -itd --restart=always  \
--name gitlab-runner \
-v /etc/gitlab-runner:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock  \
gitlab/gitlab-runner:latest

说明：
-v $HOME/gitlab-runner/config:/etc/gitlab-runner：
这个挂载是将gitlab-runner的配置文件挂载到宿主机上，这样我们可以通过修改宿主机上的这个配置文件对gitlab-runner进行配置
-v /var/run/docker.sock:/var/run/docker.sock:
这个挂载是将宿主机上的docker socket挂载到了容器内，这样容器内执行的docker命令会被宿主机docker daemon最终执行。
此操作是为Executor Docker时做准备工作。
这两个挂载很关健

进入容器
# docker exec -it gitlab-runner /bin/bash		//可以在容器内部做注册工作。
root@7a8453ddab09:/# gitlab-runner -h

3.3.2 注册runner executor/docker-in-docker​

executor是gitlab CI的重要概念，不同的executor完成不同的动作，比如：不同开发语言的源码编译需不同的executor，不同的 executor可以让应用部署在物理节点、虚拟机、docker容器、或k8s中的pod等。

executor被runner注册到gitlab CI中，每个注册成功的executor都有一个tag，配置.gitlab-ci.yml凭tag对executor进行调用。

runner有如下3种情况

类型说明
共享级runner供所有项目使用
群组级runner供群组成员中的项目使用
项目级runner仅供本项目使用

本文采用docker-in-docker方式。 目标：创建一个executor，用于编译go源码，并将编译后的应用打包成镜像上传到私有仓库harbor.demo.com中。

提示

在此docker-in-docker时采用了一个自制镜像存于私仓harbor.demo.com/cicd/centos8.5-tool:v0.2，它是基于centos8.5制作，其中安装也测试所需的软件，如docker客户端、go、curl等等。

进入容器gitlab-runner

# docker exec -it gitlab-runner /bin/bash

注册runner到Gitlab CI

root@7a8453ddab09:/# gitlab-runner register \
--url http://git.demo.com \
--registration-token GR13489413ocr9Hnhx-eTHAbcNCXx \
--tag-list docker-in-docker-test-1 \
--description "docker-in-docker-test-1" \
--maintenance-note "docker-in-docker-test-1" \
--executor docker \
--docker-pull-policy if-not-present \
--docker-helper-image "harbor.demo.com/cicd/gitlab-runner-helper:x86_64-dcfb4b66" \
--docker-image "harbor.demo.com/cicd/centos8.5-tool:v0.2" \
--docker-volumes /var/run/docker.sock:/var/run/docker.sock \
--env 'DOCKER_AUTH_CONFIG={"auths":{"harbor.demo.com":{"auth":"YWRtaW46MTJxd2FzenhvcGtsbm0="}}}'

说明： 其中--docker-helper-image和--docker-image这两个镜像，都是由gitlab服务器从私有仓库上面下载，因此需配置如下：

1。私有仓库采用https时，需将证书追回到gitlab服务器所在的/etc/ssl/certs/ca-certificates.crt文件中。

2。有密码时，需配置--env参数，如下。
     --env 'DOCKER_AUTH_CONFIG={"auths":{"harbor.demo.com":{"auth":"YWRtaW46MTJxd2FzenhvcGtsbm0="}}}'
   其中用户名称和密码需采用如下办法输出：
     # printf "admin:12qwaszxopklnm" | openssl base64 -A
   此处配置的变量DOCKER_AUTH_CONFIG值本质是一个json字串，如下：
		{
		  "auths": {
		    "harbor.demo.com": {
		      "auth": "YWRtaW46MTJxd2FzenhvcGtsbm0="
		    }
		  }
		}

3。这两个镜像的pull规则：
       --docker-pull-policy {never, if-not-present, always}
   默认是always

查看注册情况

root@7a8453ddab09:/# gitlab-runner list
Runtime platform                                    arch=amd64 os=linux pid=51 revision=dcfb4b66 version=15.10.1
docker-in-docker-test-1                             Executor=docker Token=uyyzkGqaTayfRsJ8yJxg URL=http://git.demo.com

此runner可以编译go并可访问 harbor.demo.com 私有仓库。

3.3.3 编写.gitlab-ci.yml​

采用GoLand编写一段测试代码，如下： 测试代码 创建CI/CD流水文件.gitlab-ci.yml，如下

#全局变量赋值
variables:
  image_name: "busybox"
  #image_name: "centos"
  image_ver: "v2.1"

#定义stages
stages:
  - build
  - push_image
  - test
  - deploy

#job1:编译go源码为二进制文件
#局部变量Is_Run默认值为yes，则在push时会执行此任务。
job1:
  variables:
    Is_Run: "yes"
  stage: build
  script:
    - echo "build the code..."
    - export GOROOT=/usr/local/go
    - export PATH=$PATH:/usr/local/go/bin
    - export GOPATH=/opt
    - export GO115MODULE=on
    - export GOOS=linux
    - export GOARCH=amd64
    - export GOPROXY="https://goproxy.cn,direct"
    - go version
    - go mod tidy
    - go build -o app .
    - mkdir build
    - mv app build/
    - mv Dockerfile_nobuild build/Dockerfile
  artifacts:
    paths:
      - build
  tags:
    - docker-in-docker-test-1
  rules:
    - if: $Is_Run == "yes"

#job2的工作是将job1生成的应用打包上镜像，并push到私有仓库。
#局部变量Is_Run默认值为yes，则在push时会执行此任务。
#提示：$UserName和$PassWord是在gitlab项目定义的项目级变量，用于存放私有仓库的用户名和密码
job2:
  variables:
    Is_Run: "yes"
  stage: push_image
  needs:
    - job: job1
      artifacts: true
  script:
    - echo "build image and push harbor register ..."
    - cd build/
    - ls -l
    - docker build -t harbor.demo.com/web/$image_name:$image_ver .
    - docker logout harbor.demo.com
    - echo $PassWord | base64 -d | docker login --username $UserName  --password-stdin harbor.demo.com
    - docker push harbor.demo.com/web/$image_name:$image_ver
    - docker rmi harbor.demo.com/web/$image_name:$image_ver
  tags:
    - docker-in-docker-test-1
  rules:
    - if: $Is_Run == "yes"

#job3的任务是测试应用。
#局部变量Is_Run默认值为yes，则在push时会执行此任务。通常开发过程中测试。
job3:
  variables:
    Is_Run: "yes"
    deploy_svc_name: "app-test"
  stage: test
  script:
    - echo "deploy_to_k8s, $deploy_svc_name, http://www.test.com ..."
  tags:
    - docker-in-docker-test-1
  rules:
    - if: $Is_Run == "yes"

#job4的任务用于发布
#局部变量Is_Run默认值为no，则在push时不会执行此任务，执行条件为：$Is_Run == "deploy"。
#需通过webhook方式执行此任务。通常用于在OA工作流中供领导审批是否正式发布此应用。
job4:
  variables:
    Is_Run: "no"
    deploy_svc_name: "app-demo-io"
  stage: deploy
  script:
    - echo "deploy_to_k8s, $deploy_svc_name, http://www.demo.io ..."
  tags:
    - docker-in-docker-test-1
  rules:
    - if: $Is_Run == "deploy"

此.gitlab-ci.yml有4个job，每个job定义有runner executor，采用tag来指定runner的tag标识。每个job可以有不同的runner，本例中采用相同的runner.

runner executor可以处理java、node.js、python、c、c++、php、go等多种前后端开发语言，同时可以部署应用于物理服务、云主机、容器、k8s/istio中。

此步测试仅用于流程测试，在最后测试将应用发布在k8s/istio中。

3.3.4 CI/CD测试​

Gitlab CI/CD默认是启动，只要有“.gitlab-ci.yml”文件，会自动触发pipeline。 通常有两种方式来触发CI/CD流。

  1. push/merge动作
  2. 采用webhook方式。

3.3.4.1 配置变量及webhok token​

定义变量UserName和PassWord用于存储私有仓库的用户名称和密码。 定义webhook使用的token值。

3.3.4.2 push测试​

push源码

git add *
git commit -m "test-1"
git push http://git.demo.com/guofs/cicdtest.git main

查看CI/CD变过程 可以查看每一个job的详细过程。例如job2. 查看镜像私有仓库

3.3.4.3 webhook测试​

从配置上看，job4可以采用webhook方式赋值Is_Run=deploy来执行。 此步可通过OA流来调用，让领导通过工作流方式来审批此应用是否正式发布。

curl -X POST \
     --fail \
     -F token=glptt-938d9966afdc10180540a775d6e5e399fcd2cea0 \
     -F ref=main \
     -F "variables[Is_Run]=deploy" \
     -F "variables[deploy_svc_name]=demo-io-test" \
     http://git.demo.com/api/v4/projects/8/trigger/pipeline

查看 查看job详细

$ echo "deploy_to_k8s, $deploy_svc_name, http://www.demo.io ..."
deploy_to_k8s, demo-io-test, http://www.demo.io ...