如果你讨厌一个人,你实际讨厌的是你自己的某些部分。我们自身没有的东西,是不会干扰到我们的。仁者见仁,智者见智。——德尔曼 黑塞《德米安》
写在前面
聊聊CICD
的环境搭建
以及一个基于Hexo
的博客系统在CICD
流程中的配置Demo
很早就想着写这样一篇博文,但是没有时间,之前写了一半,正好春节假期把剩下的一般写完.
本文属于Devpos
实战类文章,基本没有理论,所有,小伙伴需要对devops
有些基本的了解,
博文中讲的CICD
属于Devops的一部分
当然小伙伴们也可以找一些云服务商提供的商业的Depops平台去学习,一般有免费的体验,比如华为云的软开云平台。
博文涉及内容:
Gitlab
+Jenkins
+Docker
+Harbor
+K8S集群
的CICD
搭建教程
在搭建好的CICD
平台上持续集成部署hexo博客系统
,Demo有些简陋,仅用于学习。
其中Gitlab
+Jenkins
+Harbor
都是通过容器化
部署
篇幅有限,关于CD环境k8s集群
这里用之前部署好的,并且已经做了kubeconfig
证书,关于这方便感兴趣小伙伴可以看看我之前的文章
下面为涉及到的机器:
用到的机器
ip
域名
客户机
192.168.26.1
本地物理机:
Gitlab+Jenkins+Docker
192.168.26.55
虚机:liruilongs.github.io
docker镜像仓库:harbor
192.168.26.56
虚机:vms56.liruilongs.github.io
k8s集群-master节点
192.168.26.81
虚机:vms81.liruilongs.github.io
k8s集群-node节点
192.168.26.82
虚机:vms82.liruilongs.github.io
k8s集群-node节点
192.168.26.83
虚机:vms83.liruilongs.github.io
拓扑图
这里客户机
用本地的IDE持续编码
,然后push
代码到gitlab
,gitlab
中的web钩子
触发jenkins
中配置好的构建触发器
,通过shell命令
拉取gitlab仓库中的代码
,然后通过拉取的应用源码
和Dockerfile
文件来构建应用镜像
,构建完成后将应用镜像push到harbor私有镜像仓库
,然后通过shell
命令的方式在jenkins
中用kubelet客户端
将镜像
从私有仓库拉取到k8s集群
并更新其deploy
中的镜像,默认deploy
更新副本的方式为滚动更新
,整个流程中,只有客户机push代码是手手动的方式,其他全是自动
上帝借由各种途径使人变得孤独,好让我们可以走向自己。 ——赫尔曼·黑塞《德米安》
一、CICD服务器环境搭建 CI即为持续集成(Continue Integration,简称CI)
,用通俗的话讲,就是持续的整合版本库代码编译后制作应用镜像
。建立有效的持续集成环境可以减少开发过程中一些不必要的问题、提高代码质量、快速迭代
等,
常用的工具和平台有:
Jenkins
:基于Java开发的一种持续集成工具,用于监控持续重复的工作,旨在提供一个开放易用的软件平台,使软件的持续集成变成可能。Bamboo
: 是一个企业级商用软件,可以部署在大规模生产环境中。
CD即持续交付Continuous Delivery和持续部署Continuous Deployment,用通俗的话说,即可以持续的部署到生产环境给客户使用,这里分为两个阶段,持续交付我理解为满足上线条件的过程,但是没有上线,持续部署,即为上线应用的过程
关于CD环境
,我们使用以前搭建好的K8s集群
,K8s集群可以实现应用的健康检测,动态扩容,滚动更新
等优点,关于K8s集群的搭建,小伙伴可以看看我的其他文章
我们来搭建CI服务器:操作服务器: liruilongs.github.io:192.168.26.55
docker 环境安装 拉取镜像,启动并设置开机自启
1 2 3 4 ┌──[root@liruilongs.github.io]-[~] └─$ yum -y install docker-ce ┌──[root@liruilongs.github.io]-[~] └─$ systemctl enable docker --now
配置docker加速器
1 2 3 4 5 6 7 8 sudo mkdir -p /etc/docker sudo tee /etc/docker/daemon.json <<-'EOF' { "registry-mirrors" : ["https://2tefyfv7.mirror.aliyuncs.com" ] } EOF sudo systemctl daemon-reload sudo systemctl restart docker
1.安装GitLab 并配置 GitLab 不多介绍。一个基于Git的版本控制平台,,提供了Git仓库管理、代码审查、问题跟踪、活动反馈和wiki,当然同时也提供了
1 2 ┌──[root@liruilongs.github.io]-[~] └─$ docker pull beginor/gitlab-ce
–
创建共享卷目录 1 2 3 4 ┌──[root@liruilongs.github.io]-[~] └─$ mkdir -p /data/gitlab/etc/ /data/gitlab/log /data/gitlab/data ┌──[root@liruilongs.github.io]-[~] └─$ chmod 777 /data/gitlab/etc/ /data/gitlab/log /data/gitlab/data
创建 gitlab 容器 1 2 3 4 5 6 7 8 9 10 11 ┌──[root@liruilongs.github.io]-[~] └─$ docker run -itd --name=gitlab --restart=always --privileged=true -p 8443:443 -p 80:80 -p 222:22 -v /data/gitlab/etc:/etc/gitlab -v /data/gitlab/log :/var/log /gitlab -v /data/gitlab/data:/var/opt/gitlab beginor/gitlab-ce acc95b2896e8475915275d5eb77c7e63f63c31536432b68508f2f216d4fec634 ┌──[root@liruilongs.github.io]-[~] └─$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES acc95b2896e8 beginor/gitlab-ce "/assets/wrapper" 53 seconds ago Up 51 seconds (health: starting) 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:222->22/tcp, :::222->22/tcp, 0.0.0.0:8443->443/tcp, :::8443->443/tcp gitlab ┌──[root@liruilongs.github.io]-[~] └─$ ┌──[root@liruilongs.github.io]-[~] └─$#
切记:这里的端口要设置成80,要不push项目会提示没有报错,如果宿主机端口被占用,需要把这个端口腾出来
关闭容器修改配置文件 1 2 3 ┌──[root@liruilongs.github.io]-[~] └─$ docker stop gitlab gitlab
external_url ‘http://192.168.26.55'
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 ┌──[root@liruilongs.github.io]-[~] └─$ cat /data/gitlab/etc/gitlab.rb | grep external_url ┌──[root@liruilongs.github.io]-[~] └─$ sed -i "/external_url 'GENERATED_EXTERNAL_URL'/a external_url\t'http://192.168.26.55' " /data/gitlab/etc/gitlab.rb ┌──[root@liruilongs.github.io]-[~] └─$ cat /data/gitlab/etc/gitlab.rb | grep external_url external_url 'http://192.168.26.55' ┌──[root@liruilongs.github.io]-[~] └─$
gitlab_rails[‘gitlab_ssh_host’] = ‘192.168.26.55’
1 2 3 4 5 6 7 8 9 10 11 ┌──[root@liruilongs.github.io]-[~] └─$ cat /data/gitlab/etc/gitlab.rb | grep gitlab_ssh_host ┌──[root@liruilongs.github.io]-[~] └─$ sed -i "/gitlab_ssh_host/a gitlab_rails['gitlab_ssh_host'] = '192.168.26.55' " /data/gitlab/etc/gitlab.rb ┌──[root@liruilongs.github.io]-[~] └─$ cat /data/gitlab/etc/gitlab.rb | grep gitlab_ssh_host gitlab_rails['gitlab_ssh_host' ] = '192.168.26.55' ┌──[root@liruilongs.github.io]-[~] └─$
gitlab_rails[gitlab_shell_ssh_port] = 222
1 2 3 4 5 6 7 8 9 10 11 ┌──[root@liruilongs.github.io]-[~] └─$ cat /data/gitlab/etc/gitlab.rb | grep gitlab_shell_ssh ┌──[root@liruilongs.github.io]-[~] └─$ sed -i "/gitlab_shell_ssh_port/a gitlab_rails['gitlab_shell_ssh_port'] = 222" /data/gitlab/etc/gitlab.rb ┌──[root@liruilongs.github.io]-[~] └─$ cat /data/gitlab/etc/gitlab.rb | grep gitlab_shell_ssh gitlab_rails[gitlab_shell_ssh_port] = 222 ┌──[root@liruilongs.github.io]-[~] └─$
1 2 3 4 5 6 7 8 9 10 ┌──[root@liruilongs.github.io]-[~] └─$ vim /data/gitlab/data/gitlab-rails/etc/gitlab.yml ┌──[root@liruilongs.github.io]-[~] └─$ gitlab: host: 192.168.26.55 port: 80 https: false
修改完配置文件之后。直接启动容器
1 2 ┌──[root@liruilongs.github.io]-[~] └─$ docker start gitlab
–
在宿主机所在的物理机访问,http://192.168.26.55/
,会自动跳转到修改密码(root用户),如果密码设置的没有满足一定的复杂性,则会报500,需要从新设置
登录进入仪表盘
然后我们简单测试一下,push一个项目上去,这里的项目是一个基于hexo的博客系统
项目成功上传Gitlab
相关的git命名
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 PS F:\blogger> git init Initialized empty Git repository in F:/blogger/.git/ PS F:\blogger> git config --global user.name "Administrator" PS F:\blogger> git config --global user.email "admin@example.com" PS F:\blogger> git remote add origin http://192.168.26.55/root/blog.git PS F:\blogger> git add . PS F:\blogger> git commit -m "Initial commit" PS F:\blogger> git push -u origin master Enumerating objects: 322, done . Counting objects: 100% (322/322), done . Delta compression using up to 8 threads Compressing objects: 100% (302/302), done . Writing objects: 100% (322/322), 11.31 MiB | 9.22 MiB/s, done . Total 322 (delta 24), reused 0 (delta 0) remote: Resolving deltas: 100% (24/24), done . To http://192.168.26.55/root/blog.git * [new branch] master -> master Branch 'master' set up to track remote branch 'master' from 'origin' . PS F:\blogger>
2.安装配置远程镜像仓库harbor 下面我们要配置私有的docker镜像仓库,用到的机器为:
操作服务器:vms56.liruilongs.github.io:192.168.26.56
这里仓库我们选择harbor
,因为有web页面,当然也可以使用 registry
harbor的配置
harbor的安装使用步骤
安装并启动docker并安装docker-compose
上传harbor的离线包
导入harbor的镜像
编辑harbor.yml
修改hostname 为自己的主机名,不用证书需要注释掉https
harbor_admin_password 登录密码
安装compose
运行脚本 ./install.sh
在浏览器里输入IP访问
docker login IP –家目录下会有一个.docker文件夹
下面我们开始安装
首先需要设置selinux、防火墙
1 2 3 ┌──[root@vms56.liruilongs.github.io]-[~] └─ Disabled
1 2 3 4 ┌──[root@vms56.liruilongs.github.io]-[~] └─ Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
安装并启动docker并安装docker-compose,关于docker-compose,这里不用了解太多,一个轻量的docker编排工具
1 2 3 4 ┌──[root@vms56.liruilongs.github.io]-[~] └─ ┌──[root@vms56.liruilongs.github.io]-[~] └─
解压harbor 安装包:harbor-offline-installer-v2.0.6.tgz,导入相关镜像
1 2 3 4 5 6 7 8 9 10 11 12 13 14 ┌──[root@vms56.liruilongs.github.io]-[/] └─ bin dev harbor-offline-installer-v2.0.6.tgz lib machine-id mnt proc run srv tmp var boot etc home lib64 media opt root sbin sys usr ┌──[root@vms56.liruilongs.github.io]-[/] └─ harbor/harbor.v2.0.6.tar.gz harbor/prepare harbor/LICENSE harbor/install.sh harbor/common.sh harbor/harbor.yml.tmpl ┌──[root@vms56.liruilongs.github.io]-[/] └─
修改配置文件
1 2 3 4 5 6 7 8 9 10 11 12 13 14 ┌──[root@vms56.liruilongs.github.io]-[/] └─ ┌──[root@vms56.liruilongs.github.io]-[/harbor] └─ common.sh harbor.v2.0.6.tar.gz harbor.yml.tmpl install.sh LICENSE prepare ┌──[root@vms56.liruilongs.github.io]-[/harbor] └─ ┌──[root@vms56.liruilongs.github.io]-[/harbor] └─ common.sh harbor.v2.0.6.tar.gz harbor.yml harbor.yml.tmpl install.sh LICENSE prepare ┌──[root@vms56.liruilongs.github.io]-[/harbor] └─ ┌──[root@vms56.liruilongs.github.io]-[/harbor] └─
harbor.yml:设置IP和用户名密码
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 4 5 hostname: 192.168.26.56 6 7 ....... 12 13 14 15 16 17 18 .... 33 34 harbor_admin_password: Harbor12345 35 36
./prepare && ./install.sh
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ┌──[root@vms56.liruilongs.github.io]-[/harbor] └─ prepare base dir is set to /harbor WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https Generated configuration file: /config/log /logrotate.conf Generated configuration file: /config/log /rsyslog_docker.conf Generated configuration file: /config/nginx/nginx.conf Generated configuration file: /config/core/env Generated configuration file: /config/core/app.conf Generated configuration file: /config/registry/config.yml Generated configuration file: /config/registryctl/env Generated configuration file: /config/registryctl/config.yml Generated configuration file: /config/db/env Generated configuration file: /config/jobservice/env Generated configuration file: /config/jobservice/config.yml Generated and saved secret to file: /data/secret/keys/secretkey Successfully called func: create_root_cert Generated configuration file: /compose_location/docker-compose.yml Clean up the input dir
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 ┌──[root@vms56.liruilongs.github.io]-[/harbor] └─ [Step 0]: checking if docker is installed ... Note: docker version: 20.10.9 [Step 1]: checking docker-compose is installed ... Note: stopping existing Harbor instance ... Removing harbor-jobservice ... done Removing nginx ... done Removing harbor-core ... done Removing registry ... done Creating harbor-log ... done Removing harbor-portal ... done Removing redis ... done Removing network harbor_harbor Creating registry ... done Creating harbor-core ... done Creating network "harbor_harbor" with the default driver Creating nginx ... done Creating redis ... Creating registry ... Creating harbor-portal ... Creating registryctl ... Creating harbor-db ... Creating harbor-core ... Creating harbor-jobservice ... Creating nginx ... ✔ ----Harbor has been installed and started successfully.---- ┌──[root@vms56.liruilongs.github.io]-[/harbor] └─
查看相关的镜像
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ┌──[root@vms56.liruilongs.github.io]-[/harbor] └─ CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0efcf7b83dcf goharbor/nginx-photon:v2.0.6 "nginx -g 'daemon of…" 16 minutes ago Up 16 minutes (healthy) 0.0.0.0:80->8080/tcp, :::80->8080/tcp nginx ee9d418c7cee goharbor/harbor-jobservice:v2.0.6 "/harbor/entrypoint.…" 16 minutes ago Up 16 minutes (healthy) harbor-jobservice 6052c481dbd0 goharbor/harbor-core:v2.0.6 "/harbor/entrypoint.…" 16 minutes ago Up 16 minutes (healthy) harbor-core 001ff83b037d goharbor/harbor-db:v2.0.6 "/docker-entrypoint.…" 17 minutes ago Up 16 minutes (healthy) 5432/tcp harbor-db 2ebc81356ef1 goharbor/harbor-registryctl:v2.0.6 "/home/harbor/start.…" 17 minutes ago Up 16 minutes (healthy) registryctl 6ca721c0fa75 goharbor/harbor-portal:v2.0.6 "nginx -g 'daemon of…" 17 minutes ago Up 16 minutes (healthy) 8080/tcp harbor-portal 2b06e2cf91ab goharbor/registry-photon:v2.0.6 "/home/harbor/entryp…" 17 minutes ago Up 16 minutes (healthy) 5000/tcp registry 2292a20780e2 goharbor/redis-photon:v2.0.6 "redis-server /etc/r…" 17 minutes ago Up 16 minutes (healthy) 6379/tcp redis a0e3e49cf9db goharbor/harbor-log:v2.0.6 "/bin/sh -c /usr/loc…" 17 minutes ago Up 17 minutes (healthy) 127.0.0.1:1514->10514/tcp harbor-log ┌──[root@vms56.liruilongs.github.io]-[/harbor] └─
访问测试
harbor
CI服务器的docker配置 这里因为我们要在192.168.26.55(CI服务器)上push镜像到192.168.26.56(私仓),所有需要修改CI服务器上的Docker配置。添加仓库地址
操作服务器: liruilongs.github.io:192.168.26.55
1 2 3 4 5 6 7 ┌──[root@liruilongs.github.io]-[~] └─$ cat /etc/docker/daemon.json { "registry-mirrors" : ["https://2tefyfv7.mirror.aliyuncs.com" ] } ┌──[root@liruilongs.github.io]-[~] └─$ vim /etc/docker/daemon.json
修改后的配置文件
1 2 3 4 5 6 7 ┌──[root@liruilongs.github.io]-[~] └─$ cat /etc/docker/daemon.json { "registry-mirrors" : ["https://2tefyfv7.mirror.aliyuncs.com" ], "insecure-registries" : ["192.168.26.56" ] }
加载使其生效
1 2 3 4 5 ┌──[root@liruilongs.github.io]-[~] └─$ systemctl daemon-reload ┌──[root@liruilongs.github.io]-[~] └─$ systemctl restart docker ┌──[root@liruilongs.github.io]-[~]
CI机器简单测试一下
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 ┌──[root@liruilongs.github.io]-[~] └─$ docker login 192.168.26.56 Authenticating with existing credentials... WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/ Login Succeeded ┌──[root@liruilongs.github.io]-[~] └─$ docker tag busybox 192.168.26.56/demo/busybox ┌──[root@liruilongs.github.io]-[~] └─$ docker push 192.168.26.56/demo/busybox Using default tag: latest The push refers to repository [192.168.26.56/demo/busybox] cfd97936a580: Pushed latest: digest: sha256:febcf61cd6e1ac9628f6ac14fa40836d16f3c6ddef3b303ff0321606e55ddd0b size: 527 ┌──[root@liruilongs.github.io]-[~] └─$
push一个镜像,可以在私仓的web页面查看
harbor
到这里。我们配置了镜像仓库
3.安装配置jenkins
操作服务器: liruilongs.github.io:192.168.26.55
镜像jenkins拉取
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 ┌──[root@liruilongs.github.io]-[~] └─$ docker pull jenkins/jenkins:centos7-jdk8 。。。。。。 ┌──[root@liruilongs.github.io]-[~] └─$ docker history jenkins/jenkins:centos7-jdk8 IMAGE CREATED CREATED BY SIZE COMMENT de64a05279ba 5 days ago LABEL org.opencontainers.image.vendor=Jenkin… 0B buildkit.dockerfile.v0 <missing> 5 days ago COPY install-plugins.sh /usr/local /bin/insta… 10.6kB buildkit.dockerfile.v0 。。。。。。。。。。 <missing> 5 days ago ARG JENKINS_HOME=/var/jenkins_home 0B buildkit.dockerfile.v0 <missing> 5 days ago ARG agent_port=50000 0B buildkit.dockerfile.v0 <missing> 5 days ago ARG http_port=8080 0B buildkit.dockerfile.v0 <missing> 5 days ago ARG gid=1000 0B buildkit.dockerfile.v0 <missing> 5 days ago ARG uid=1000 0B buildkit.dockerfile.v0 <missing> 5 days ago ARG group=jenkins 0B buildkit.dockerfile.v0 <missing> 5 days ago ARG user=jenkins 0B buildkit.dockerfile.v0 <missing> 5 days ago RUN |5 TARGETARCH=amd64 COMMIT_SHA=0b797f024… 10.4MB buildkit.dockerfile.v0 <missing> 5 days ago COPY git_lfs_pub.gpg /tmp/git_lfs_pub.gpg <missing> 5 days ago ARG GIT_LFS_VERSION=3.0.1 0B buildkit.dockerfile.v0 <missing> 5 days ago ARG COMMIT_SHA 0B buildkit.dockerfile.v0 <missing> 5 days ago ARG TARGETARCH 0B buildkit.dockerfile.v0 <missing> 5 days ago RUN |2 TARGETARCH=amd64 COMMIT_SHA=0b797f024… 426MB buildkit.dockerfile.v0 <missing> 5 days ago ARG COMMIT_SHA 0B buildkit.dockerfile.v0 <missing> 5 days ago ARG TARGETARCH 0B buildkit.dockerfile.v0 <missing> 5 days ago ENV LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_AL… 0B buildkit.dockerfile.v0 <missing> 7 weeks ago /bin/sh -c <missing> 7 weeks ago /bin/sh -c <missing> 7 weeks ago /bin/sh -c ┌──[root@liruilongs.github.io]-[~] └─$
–
创建共享卷,修改所属组和用户,和容器里相同 这里为什么要改成 1000,是因为容器里是以 jenkins 用户的身份去读写数据,而在容器里jenkins 的 uid 是 1000,
1 2 3 4 ┌──[root@liruilongs.github.io]-[~] └─$ mkdir /jenkins && chown 1000:1000 /jenkins ┌──[root@liruilongs.github.io]-[~] └─$
创建创建 jenkins 容器 1 2 3 4 5 6 7 8 9 ┌──[root@liruilongs.github.io]-[~] └─$ cat jenkins.docker.sh docker run -dit -p 8080:8080 -p 50000:50000 --name jenkins --privileged=true --restart=always -v /jenkins:/var/jenkins_home jenkins/jenkins:centos7-jdk8 ┌──[root@liruilongs.github.io]-[~] └─$ docker run -dit -p 8080:8080 -p 50000:50000 --name jenkins --privileged=true --restart=always -v /jenkins:/var/jenkins_home jenkins/jenkins:centos7-jdk8 39afa098c8a56973ce1559d374b058b8e6091175b5b783d613a9f2e356827684 ┌──[root@liruilongs.github.io]-[~] └─$ docker ps | grep jenkins 39afa098c8a5 jenkins/jenkins:centos7-jdk8 "/sbin/tini -- /usr/…" 3 minutes ago Up 2 minutes 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 0.0.0.0:50000->50000/tcp, :::50000->50000/tcp jenkins
访问jenkins
先打开浏览器打开这个页面,让其初始化一下,直到看到界面
因为要修改 jenkins 的配置,所以此时关闭 jenkins 容器
1 2 3 ┌──[root@liruilongs.github.io]-[~] └─$ docker stop jenkins jenkins
更换国内清华大学镜像,Jenkins下载插件特别慢,更换国内的清华源的镜像地址会快不少
1 2 3 4 5 6 7 8 9 10 11 ┌──[root@liruilongs.github.io]-[~] └─$ cat /jenkins/hudson.model.UpdateCenter.xml <?xml version='1.1' encoding='UTF-8' ?> <sites> <site> <id>default</id> <url>https://updates.jenkins.io/update-center.json</url> </site> </sites> ┌──[root@liruilongs.github.io]-[~] └─$
1 2 3 4 5 6 7 8 9 10 11 12 ┌──[root@liruilongs.github.io]-[~] └─$ sed -i 's#updates.jenkins.io/update-center.json#mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json#g ' /jenkins/hudson.model.UpdateCenter.xml ┌──[root@liruilongs.github.io]-[~] └─$ cat /jenkins/hudson.model.UpdateCenter.xml <?xml version='1.1' encoding='UTF-8' ?> <sites> <site> <id>default</id> <url>https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json</url> </site> </sites>┌──[root@liruilongs.github.io]-[~] └─$
“http://www.google.com/" 替换为 “http://www.baidu.com/"
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 yum -y install jq ┌──[root@liruilongs.github.io]-[~] └─$ cat /jenkins/updates/default.json | jq '.connectionCheckUrl' "http://www.google.com/" ┌──[root@liruilongs.github.io]-[~] └─$ cat /jenkins/updates/default.json | jq 'keys' [ "connectionCheckUrl" , "core" , "deprecations" , "generationTimestamp" , "id" , "plugins" , "signature" , "updateCenterVersion" , "warnings" ] ┌──[root@liruilongs.github.io]-[~] └─$ sed -i s
替换后查看
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 ┌──[root@liruilongs.github.io]-[~] └─$ cat /jenkins/updates/default.json | jq '.connectionCheckUrl' "http://www.baidu.com/" ┌──[root@liruilongs.github.io]-[~] └─$ cat /jenkins/updates/default.json | jq 'keys' [ "connectionCheckUrl" , "core" , "deprecations" , "generationTimestamp" , "id" , "plugins" , "signature" , "updateCenterVersion" , "warnings" ] ┌──[root@liruilongs.github.io]-[~] └─$
重启docker,获取登录密匙
1 2 3 4 5 6 7 8 ┌──[root@liruilongs.github.io]-[~] └─$ docker start jenkins jenkins ┌──[root@liruilongs.github.io]-[~] └─$ ┌──[root@liruilongs.github.io]-[~] └─$ cat /jenkins/secrets/initialAdminPassword be15eaabc4c946de913dd5af8636cae9
**需要修改jenkins绑定的docker的启动参数 **,ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H fd:// --containerd=/run/containerd/containerd.sock
修改镜像库启动参数后需要重启docker
1 2 3 4 5 ┌──[root@vms56.liruilongs.github.io]-[~] └─ ┌──[root@vms56.liruilongs.github.io]-[~] └─
安装 docker 插件
jenkins相关配置,这里的配置照着图片就好,需要配置一个docker集群供jenkins来根据Dockerfile构建镜像并push到私仓,这里docker集群即为CI服务器的docker
修改镜像库启动参数,ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H fd:// --containerd=/run/containerd/containerd.sock
关联docker和jenkins
jenkins 安全设置 后面 gitlab 要和 jenkins 进行联动,所以必须要需要对 jenkins 的安全做一些设置,依次点击 系统管理-全局安全配置-授权策略,勾选”匿名用户具有可读权限”
添加 JVM 运行参数 -Dhudson.security.csrf.GlobalCrumbIssuerConfiguration.DISABLE_CSRF_PROTECTION=true
运行跨站请求访问
1 2 3 4 5 6 7 8 9 ┌──[root@liruilongs.github.io]-[~] └─$ docker exec -u root -it jenkins bash [root@39afa098c8a5 /] anaconda-post.log bin dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var [root@39afa098c8a5 /] [root@39afa098c8a5 /] exit ┌──[root@liruilongs.github.io]-[~] └─$
下载kubectl客户端工具 这里的话我们要通过jenkins上的kubectl客户端连接k8s,所以我们需要安装一个k8s的客户端kubectl,下载k8s客户端
1 wget https://storage.googleapis.com/kubernetes-release/release/v1.22.2/bin/linux/amd64/kubectl
1 2 ┌──[root@liruilongs.github.io]-[~] └─$ yum install -y kubectl-1.22.2-0 --disableexcludes=kubernetes
拷贝 kubeconfig 文件 然后拷贝kubeconfig 证书,k8s集群中查看证书位置,这里的证书是之前创建好的,小伙伴可以看看我之前的文章
1 2 3 4 5 ┌──[root@liruilongs.github.io]-[~] └─$ scp root@192.168.26.81:/root/ansible/k8s-rbac-create/kc1 . Warning: Permanently added '192.168.26.81' (ECDSA) to the list of known hosts. root@192.168.26.81's password: kc1 100% 5566 108.7KB/s 00:00
拷贝证书和k8s集群客户端工具到jenkins容器内 1 2 3 4 5 6 ┌──[root@liruilongs.github.io]-[~] └─$ docker cp kc1 jenkins:/ ┌──[root@liruilongs.github.io]-[~] └─$ docker cp kubectl jenkins:/ ┌──[root@liruilongs.github.io]-[~] └─$
kubectl命令测试 1 2 3 4 5 6 7 8 9 ┌──[root@liruilongs.github.io]-[~] └─$ docker exec -u root -it jenkins bash [root@39afa098c8a5 /] anaconda-post.log bin dev etc home kc1 kubectl lib lib64 media mnt opt proc root run sbin srv sys tmp usr var [root@39afa098c8a5 /] Error: unknown flag: --kubeconfi See 'kubectl get --help' for usage. [root@39afa098c8a5 /] Error from server (Forbidden): pods is forbidden: User "liruilong" cannot list resource "pods" in API group "" at the cluster scope
发现没有权限,这里我们为了方便,直接赋予集群中的cluster-admin
角色
1 2 3 4 5 ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create] └─$kubectl create clusterrolebinding test --clusterrole=cluster-admin --user=liruilong clusterrolebinding.rbac.authorization.k8s.io/test created ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-rbac-create] └─$
命令测试没有问题
1 2 3 4 5 6 [root@39afa098c8a5 /] NAME STATUS ROLES AGE VERSION vms81.liruilongs.github.io Ready control-plane,master 51d v1.22.2 vms82.liruilongs.github.io NotReady <none> 51d v1.22.2 vms83.liruilongs.github.io NotReady <none> 51d v1.22.2 [root@39afa098c8a5 /]
二 、hexo博客系统CICD实战 4. k8s集群中配置hexo生产环境高可用 我们要部署Nginx
来运行hexo
博客系统,hexo
编译完后为一堆静态文件,所以我们需要创建一个svc
和一个deploy
,使用SVC
提供服务,使用deploy
提供服务能力,使用Nginx+hexo的静态文件
构成的镜像
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 apiVersion: apps/v1 kind: Deployment metadata: creationTimestamp: null labels: app: nginx name: nginxdep spec: replicas: 2 selector: matchLabels: app: nginx strategy: {} template: metadata: creationTimestamp: null labels: app: nginx spec: containers: - image: blog name: web resources: requests: cpu: 100m restartPolicy: Always
deployments创建 这里我们先用一个Nginx镜像来代替hexo博客的镜像
1 2 3 ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$kubectl apply -f nginx.yaml deployment.apps/nginxdep created
查看deployments和pod
1 2 3 4 5 6 7 8 9 10 11 ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$kubectl get deployments.apps | grep nginxdep nginxdep 2/2 2 2 109s ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$kubectl get pods -o wide | grep web ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$kubectl get pods -o wide | grep nginxdep nginxdep-645bf755b9-2w8jv 1/1 Running 0 2m22s 10.244.171.164 vms82.liruilongs.github.io <none> <none> nginxdep-645bf755b9-jfqxj 1/1 Running 0 2m22s 10.244.171.157 vms82.liruilongs.github.io <none> <none> ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$
service创建 1 2 3 4 5 6 ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$kubectl expose deploy nginxdep --port=8888 --target-port=80 --type =NodePort service/nginxdep exposed ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$kubectl get svc -o wide | grep nginxdep nginxdep NodePort 10.106.217.50 <none> 8888:31964/TCP 16s app=nginx
访问测试没有问题,之后我们配置好jenkins上的触发器,直接替换就OK
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$curl 127.0.0.1:31964 <!DOCTYPE html > <html > <head > <title > Welcome to nginx!</title > <style > html { color -scheme: light dark; }body { width : 35em ; margin : 0 auto;font-family : Tahoma, Verdana, Arial, sans-serif; }</style > </head > <body > <h1 > Welcome to nginx!</h1 > <p > If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p > <p > For online documentation and support please refer to<a href ="http://nginx.org/" > nginx.org</a > .<br /> Commercial support is available at <a href ="http://nginx.com/" > nginx.com</a > .</p > <p > <em > Thank you for using nginx.</em > </p > </body > </html > ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$
5.k8s集群配置私仓地址 我们通过kubectl set
命令更新deploy
的镜像时,获取的镜像是通过私仓获取的,所以需要在启动参数添加私仓地址
ExecStart=/usr/bin/dockerd –insecure-registry 192.168.26.56 -H fd:// –containerd=/run/containerd/containerd.sock
这里所有的节点都需要设置后重启docker
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$vim /usr/lib/systemd/system/docker.service ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$systemctl daemon-reload ;systemctl restart docker & [1] 23273 ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$ssh root@192.168.26.82 Last login: Sun Jan 16 06:09:07 2022 from 192.168.26.1 ┌──[root@vms82.liruilongs.github.io]-[~] └─$vim /usr/lib/systemd/system/docker.service ┌──[root@vms82.liruilongs.github.io]-[~] └─$systemctl daemon-reload ;systemctl restart docker & [1] 26843 ┌──[root@vms82.liruilongs.github.io]-[~] └─$exit 登出 Connection to 192.168.26.82 closed.
6.jenkins配置CICD流程 访问jenkins,接下来才是重点,我们要的jenkins上配置整个CICD流程,从而实现自动化
访问jenkins,接下来才是重点,我们要的jenkins上配置整个CICD流程,从而实现自动化
这里的Token
我们设置为:4bf636c8214b7ff0a0fb,同时需要记住访问方式:JENKINS_URL/job/liruilong-cicd/build?token=TOKEN_NAME
构建触发器选择shell构建:克隆代码
选择镜像构建
构建镜像并push私仓
这里切记需要添加私仓的认证信息,即上面设置的用户名和密码
选择shell构建,更新镜像
相关的文本信息
1 2 3 cd ~rm -rf blog git clone http://192.168.26.55/root/blog.git
1 2 3 /var/jenkins_home/blog/ 192.168.26.56/library/blog:${BUILD_NUMBER}
1 2 export KUBECONFIG=/kc1;/kubectl set image deployment/nginxdep *="192.168.26.56/library/blog:${BUILD_NUMBER} " -n kube-system
7.配置 gitlab 和 jenkins 的联动
访问gitlab配置联动
点击增加web钩子
/view/all/job/liruilong-cicd/build?token=
到这里,联动已经配置完成
8.编写Dockerfile文件,更新代码测试 下面我们编译一下hexo,生成public的一个文件夹,然后上传gitlab
1 2 3 4 5 6 7 8 9 10 11 12 13 PS F:\blogger> hexo g ..... PS F:\blogger> git add .\public\ PS F:\blogger> git commit -m "编译代码" PS F:\blogger> git push ``` **<font color=seagreen>同时需要编写Dockerfile文件来创建镜像</font>** ```bash FROM docker.io/library/nginx:latest MAINTAINER liruilong ADD ./public/ /usr/share/nginx/html/ EXPOSE 80 CMD ["nginx" , "-g" ,"daemon off;" ]
1 2 3 4 5 6 7 8 9 10 11 12 13 14 PS F:\blogger> git add . PS F:\blogger> git commit -m "Dockcerfile文件编写" [master 217e0ed] Dockcerfile文件编写 1 file changed, 1 deletion(-) PS F:\blogger> git push Enumerating objects: 5, done . Counting objects: 100% (5/5), done . Delta compression using up to 8 threads Compressing objects: 100% (3/3), done . Writing objects: 100% (3/3), 307 bytes | 307.00 KiB/s, done . Total 3 (delta 2), reused 0 (delta 0) To http://192.168.26.55/root/blog.git 6690612..217e0ed master -> master PS F:\blogger>
jenkins输出
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 Started by remote host 192.168.26.1 Running as SYSTEM Building in workspace /var/jenkins_home/workspace/liruilong-cicd [liruilong-cicd] $ /bin/sh -xe /tmp/jenkins6108687102523328796.sh + cd /var/jenkins_home + rm -rf blog + git clone http://192.168.26.55/root/blog.git Cloning into 'blog' ... Docker Build Docker Build: building image at path /var/jenkins_home/blog Step 1/5 : FROM docker.io/library/nginx:latest ---> f8f4ffc8092c Step 2/5 : MAINTAINER liruilong ---> Running in e341b5562b64 Removing intermediate container e341b5562b64 ---> 4e9f5aa47ab5 Step 3/5 : ADD ./public/ /usr/share/nginx/html/ ---> 3956cff32507 Step 4/5 : EXPOSE 80 ---> Running in b4c27124989d Removing intermediate container b4c27124989d ---> ba9d1764d764 Step 5/5 : CMD ["nginx" , "-g" ,"daemon off;" ] ---> Running in 61dca01a4883 Removing intermediate container 61dca01a4883 ---> 2aadc5732a60 Successfully built 2aadc5732a60 Tagging built image with 192.168.26.56/library/blog:41 Docker Build Response : 2aadc5732a60 Pushing [192.168.26.56/library/blog:41] The push refers to repository [192.168.26.56/library/blog] 89570901cdea: Preparing 65e1ea1dc98c: Preparing 88891187bdd7: Preparing 6e109f6c2f99: Preparing 0772cb25d5ca: Preparing 525950111558: Preparing 476baebdfbf7: Preparing 525950111558: Waiting 476baebdfbf7: Waiting 88891187bdd7: Layer already exists 6e109f6c2f99: Layer already exists 65e1ea1dc98c: Layer already exists 0772cb25d5ca: Layer already exists 89570901cdea: Pushing [> ] 301.6kB/28.75MB 89570901cdea: Pushing [==> ] 1.193MB/28.75MB 476baebdfbf7: Layer already exists 525950111558: Layer already exists 89570901cdea: Pushing [======> ] 3.917MB/28.75MB 89570901cdea: Pushing [==========> ] 5.996MB/28.75MB 89570901cdea: Pushing [==============> ] 8.097MB/28.75MB 89570901cdea: Pushing [==================> ] 10.76MB/28.75MB 89570901cdea: Pushing [=====================> ] 12.57MB/28.75MB 89570901cdea: Pushing [========================> ] 13.8MB/28.75MB 89570901cdea: Pushing [=========================> ] 14.71MB/28.75MB 89570901cdea: Pushing [===========================> ] 15.59MB/28.75MB 89570901cdea: Pushing [=============================> ] 16.79MB/28.75MB 89570901cdea: Pushing [===============================> ] 18.27MB/28.75MB 89570901cdea: Pushing [=================================> ] 19.45MB/28.75MB 89570901cdea: Pushing [===================================> ] 20.34MB/28.75MB 89570901cdea: Pushing [=====================================> ] 21.55MB/28.75MB 89570901cdea: Pushing [=======================================> ] 22.44MB/28.75MB 89570901cdea: Pushing [=========================================> ] 23.64MB/28.75MB 89570901cdea: Pushing [==========================================> ] 24.52MB/28.75MB 89570901cdea: Pushing [============================================> ] 25.42MB/28.75MB 89570901cdea: Pushing [==============================================> ] 26.61MB/28.75MB 89570901cdea: Pushing [===============================================> ] 27.19MB/28.75MB 89570901cdea: Pushing [=================================================> ] 28.69MB/28.75MB 89570901cdea: Pushing [==================================================>] 29.32MB 89570901cdea: Pushed 41: digest: sha256:c90b64945a8d063f7bcdcc39f00f91b6d83acafcd6b2ec6aba5b070474bafc37 size: 1782 Cleaning local images [2aadc5732a60] Docker Build Done [liruilong-cicd] $ /bin/sh -xe /tmp/jenkins246013519648603221.sh + export KUBECONFIG=/kc1 + KUBECONFIG=/kc1 + /kubectl set image deployment/nginxdep '*=192.168.26.56/library/blog:41' -n kube-system deployment.apps/nginxdep image updated Finished: SUCCESS
9.访问hexo博客系统 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$kubectl get deployments.apps | grep nginxdep nginxdep 2/2 2 2 30h ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$kubectl get pods -o wide | grep nginxdep nginxdep-bddfd9b5f-94d88 1/1 Running 0 110s 10.244.171.142 vms82.liruilongs.github.io <none> <none> nginxdep-bddfd9b5f-z57qc 1/1 Running 0 35m 10.244.171.177 vms82.liruilongs.github.io <none> <none> ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$kubectl get svc -o wide | grep nginxdep nginxdep NodePort 10.106.217.50 <none> 8888:31964/TCP 30h app=nginx ┌──[root@vms81.liruilongs.github.io]-[~/ansible/k8s-deploy-create] └─$kubectl describe pods nginxdep-bddfd9b5f-94d88 Name: nginxdep-bddfd9b5f-94d88 Namespace: kube-system Priority: 0 Node: vms82.liruilongs.github.io/192.168.26.82 Start Time: Fri, 04 Feb 2022 03:11:14 +0800 Labels: app=nginx pod-template-hash=bddfd9b5f Annotations: cni.projectcalico.org/podIP: 10.244.171.142/32 cni.projectcalico.org/podIPs: 10.244.171.142/32 Status: Running IP: 10.244.171.142 IPs: IP: 10.244.171.142 Controlled By: ReplicaSet/nginxdep-bddfd9b5f Containers: web: Container ID: docker://669f48cb626d5067f40bb1aaa378268a7ee9879488b0b298a86271957c162316 Image: 192.168.26.56/library/blog:41 Image ID: docker-pullable://192.168.26.56/library/blog@sha256:c90b64945a8d063f7bcdcc39f00f91b6d83acafcd6b2ec6aba5b070474bafc37 Port: <none> Host Port: <none> State: Running Started: Fri, 04 Feb 2022 03:11:15 +0800 Ready: True Restart Count: 0 Requests: cpu: 100m Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-trn5n (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: kube-api-access-trn5n: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true QoS Class: Burstable Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 4m10s default-scheduler Successfully assigned kube-system/nginxdep-bddfd9b5f-94d88 to vms82.liruilongs.github.io Normal Pulling 4m9s kubelet Pulling image "192.168.26.56/library/blog:41" Normal Pulled 4m9s kubelet Successfully pulled image "192.168.26.56/library/blog:41" in 67.814838ms Normal Created 4m9s kubelet Created container web Normal Started 4m9s kubelet Started container web
访问hexo博客系统