Lunski's Clutter

This is a place to put my clutters, no matter you like it or not, welcome here.

0%

Spring Boot on Kubernetes

Posted on Edited on In
Symbols count in article: 14k Reading time ≈ 13 mins.

Hi Spring Boot。

Spring mvc > Spring Boot
Tensorflow > Keras

O

Docker 與 Kubernetes。

KRs

  1. Spring Boot 100%
  2. 建置Spring Boot在Docker Image 100%
  3. 部署Spring Boot 應用在Kubernetes 100%
  4. 部署Spring Boot 微服務專案在Kubernetes 80%

Requirements

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ java -version
openjdk version "19.0.1" 2022-10-18
OpenJDK Runtime Environment (build 19.0.1+10-21)
OpenJDK 64-Bit Server VM (build 19.0.1+10-21, mixed mode, sharing)

$ javac -version # JDK
javac 19.0.1

$ ~/apache-maven-3.9.0/bin/mvn -v
Apache Maven 3.9.0 (9b58d2bad23a66be161c4664ef21ce219c2c8584)
Maven home: /Users/eka/apache-maven-3.9.0
Java version: 19.0.1, vendor: Oracle Corporation, runtime: /Users/eka/Library/Java/JavaVirtualMachines/openjdk-19.0.1/Contents/Home
Default locale: zh_TW_#Hant, platform encoding: UTF-8
OS name: "mac os x", version: "13.1", arch: "x86_64", family: "mac"

$ docker -v
Docker version 20.10.22, build 3a2c30b

Docker

img

Docker是著名的CD工具，建置在作業系統上，優勢：快速管理，遷移，擴展

Path in Project

1
2
Jar: ../../../target/spring5webapp-0.0.1-SNAPSHOT.jar  
DockerFile: src>main>dockerBase>DockerFile

DockerFile

1
2
3
4
5
FROM openjdk:19-jdk-alpine
ENV JAVA_OPTS "-Xmx512m -Xms512m"
WORKDIR application
COPY ../../../target/spring5webapp-0.0.1-SNAPSHOT.jar ./
ENTRYPOINT ["java", "-jar", "spring5webapp-0.0.1-SNAPSHOT.jar"]

Command

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
docker build --tag node-docker . 生成鏡像
docker ps 顯示了啟動了哪些的 Docker Container
-a 列出所有容器
-l 列出最後一次執行容器
docker images 顯示出在 local 下載了哪些 Docker Image
docker rmi [OPTIONS] IMAGE [IMAGE...] 删除本地一个或多少镜像
docker rm <277e632692e2>: Error response from daemon: conflict: unable to delete 7bc4c38be46a (must be forced) - image is being used by stopped container 277e632692e2 時使用
docker exec -it bash: runs a new command in a running container 如果要使用 foreground，我們則要使用 -it 這個參數 沒有加 -it ，基本上，只 run 一下，馬上就跳回來了
docker kill
docker search 搜尋想裝的 Docker Image
docker pull 指 Docker Image 下載回 local
docker run 使用 Docker Image 啟動執行 Docker Container
docker container stop 會優雅停止 (gracefully stop) 容器，若要強制關閉 (force shutdown) 容器，可用 docker container kill 指令
docker container ls 或 docker ps 確認目前執行中的容器
docker --version:查看版本
docker info:查看狀態
docker start ：啟動容器
docker COMMAND --help:查看所有指令
docker run --help:查看可使用操作

Bulid & Push to DockerHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
到專案目錄的DockerFile

$ docker build -f ./src/main/dockerBase/DockerFile -t docker-test .

$ docker run -p 8080:8080 docker-test 

$   docker ps
CONTAINER ID   IMAGE         COMMAND                  CREATED          STATUS          PORTS     NAMES
745159676ccd   docker-test   "java -jar spring5we…"   30 seconds ago   Up 28 seconds             stoic_mendeleev

$ docker logs 745159676ccd
2023-02-18 09:19:38.674  INFO 1 --- [           main] g.s.s.Spring5webappApplication           : Starting Spring5webappApplication v0.0.1-SNAPSHOT on 47d036941a06 with PID 1 (/application/spring5webapp-0.0.1-SNAPSHOT.jar started by root in /application)
2023-02-18 09:19:38.681  INFO 1 --- [           main] g.s.s.Spring5webappApplication           : No active profile set, falling back to default profiles: default
...

$ docker login

$ docker tag docker-test lunski/docker-test:latest

$ docker push lunski/docker-test:latest

Docker-compose

啟動的過程中，產生新的image再啟動多個 Docker Container

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
# 啟動
$ docker compose -f compose-local.yaml up -d

$ docker ps
CONTAINER ID   IMAGE                                                  COMMAND                  CREATED         STATUS         PORTS                                                                                                 NAMES
939e5b6d5069   springframeworkguru/kbe-brewery-gateway                "java -Djava.securit…"   7 minutes ago   Up 5 minutes   8080/tcp, 0.0.0.0:9090->9090/tcp                                                                      kbe-sb-microservices-gateway-1
8e83a8ebfcce   springframeworkguru/kbe-brewery-order-service          "java -Djava.securit…"   7 minutes ago   Up 5 minutes   8080/tcp, 0.0.0.0:8081->8081/tcp                                                                      kbe-sb-microservices-order-service-1
397a3aa28db5   springframeworkguru/kbe-brewery-beer-service           "java -Djava.securit…"   7 minutes ago   Up 5 minutes   0.0.0.0:8080->8080/tcp                                                                                kbe-sb-microservices-beer-service-1
c0fd84ac07cc   springframeworkguru/kbe-brewery-inventory-service      "java -Djava.securit…"   7 minutes ago   Up 5 minutes   8080/tcp, 0.0.0.0:8082->8082/tcp                                                                      kbe-sb-microservices-inventory-service-1
fe888556718e   docker.elastic.co/kibana/kibana:7.12.1                 "/bin/tini -- /usr/l…"   7 minutes ago   Up 5 minutes   0.0.0.0:5601->5601/tcp                                                                                kbe-sb-microservices-kibana-1
63c580abe7df   docker.elastic.co/beats/filebeat:7.12.1                "/usr/bin/tini -- /u…"   7 minutes ago   Up 6 minutes                                                                                                         kbe-sb-microservices-filebeat-1
114892532cef   docker.elastic.co/elasticsearch/elasticsearch:7.12.1   "/bin/tini -- /usr/l…"   7 minutes ago   Up 6 minutes   0.0.0.0:9200->9200/tcp, 9300/tcp                                                                      kbe-sb-microservices-elasticsearch-1
8c82873f197f   mysql:5.7                                              "docker-entrypoint.s…"   7 minutes ago   Up 6 minutes   3306/tcp, 33060/tcp                                                                                   kbe-sb-microservices-mysql-1
30b5d87f882f   springframeworkguru/kbe-brewery-inventory-failover     "java -Djava.securit…"   7 minutes ago   Up 6 minutes   8080/tcp, 0.0.0.0:8083->8083/tcp                                                                      kbe-sb-microservices-inventory-failover-1
4710a51dcdd9   vromero/activemq-artemis                               "/docker-entrypoint.…"   7 minutes ago   Up 6 minutes   1883/tcp, 5445/tcp, 5672/tcp, 9404/tcp, 0.0.0.0:8161->8161/tcp, 61613/tcp, 0.0.0.0:61616->61616/tcp   kbe-sb-microservices-jms-1

# 終止
$ docker compose -f compose-local.yaml down
[+] Running 8/9
 ⠿ Container kbe-sb-microservices-filebeat-1            Removed                                                                57.2s
 ⠿ Container kbe-sb-microservices-kibana-1              Removed                                                                63.7s
 ⠿ Container kbe-sb-microservices-mysql-1               Removed                                                                57.2s
 ⠿ Container kbe-sb-microservices-gateway-1             Removed                                                                57.8s
 ⠿ Container kbe-sb-microservices-inventory-failover-1  Removed                                                                24.4s
 ⠿ Container kbe-sb-microservices-order-service-1       Removed                                                                25.2s
 ⠿ Container kbe-sb-microservices-elasticsearch-1       Removed                                                                19.6s
 ⠿ Container kbe-sb-microservices-beer-service-1        Removed                                                                20.7s
 ⠿ Container kbe-sb-microservices-inventory-service-1   Error while Sto...                                                     22.8s
Error response from daemon: cannot stop container: c0fd84ac07ccc32040510f0bfb6ecb25a6f36cc75674d76f55015211dc3bf3a4: container c0fd84ac07cc PID 37030 is zombie and can not be killed. Use the --init option when creating containers to run an init inside the container that forwards signals and reaps processes

$  docker ps
CONTAINER ID   IMAGE                      COMMAND                  CREATED          STATUS          PORTS                                                                                                 NAMES
4710a51dcdd9   vromero/activemq-artemis   "/docker-entrypoint.…"   35 minutes ago   Up 34 minutes   1883/tcp, 5445/tcp, 5672/tcp, 9404/tcp, 0.0.0.0:8161->8161/tcp, 61613/tcp, 0.0.0.0:61616->61616/tcp   kbe-sb-microservices-jms-1

$ docker stop 4710a51dcdd9
4710a51dcdd9

$ docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

Command

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
docker-compose 常用指令
(一次全部 services 一起執行)

docker-compose up
docker-compose up -d       (在背後執行)
docker-compose up --build  (Dockerfile 重新 build)
docker-compose down
docker-compose down -v     (連 volume 一起刪除)
docker-compose ps          (看目前所有 services 狀態)
 

(僅用於單一 serivce)

docker-compose start
docker-compose stop
docker-compose pause
docker-compose unpause
docker-compose logs -f     (看該 service 的 logs)

Podman v.s. Docker

  • Architecture: Docker requires a daemon process that runs with root privileges. Podman does not require a daemon and the podman commands are run as a regular user. This removes the need for root privileges that Docker requires.

  • Image registry: Docker uses a centralized image registry/repository by default. Podman can use any OCI compliant registry, including Docker’s registry as well as decentralized registries.

  • Container engine: Docker uses its own APIs and CLI. Podman is compliant with OCI (Open Container Initiative) specs and can use any OCI compliant container engine. This makes it more compatible with Kubernetes.

  • Rootless containers: Podman can run rootless containers without requiring root access. This improves security. Docker now allows rootless containers as well, but the setup is more complex compared to Podman.

  • Networking: Docker has its own networking stack. Podman uses CNI plugins for networking. This makes it easier for Podman to leverage Kubernetes networking.

  • Storage drivers: Docker has its own storage driver architecture. Podman builds on top of existing solutions like device mapper, overlayfs, VFS, etc. This allows more flexibility in leveraging existing tools.

Podman is more secure, compatible and flexible.
Docker is more popular and widely-used in production.

Kubernetes(K8S)

基於容器技術的開源平臺，用於自動部署、維護和縮放容器化的應用程序。

img

Check status

1
2
3
4
5
6
7
8
9
10
11
12
13
14
$ kubectl config get-contexts
CURRENT   NAME             CLUSTER          AUTHINFO         NAMESPACE
*         docker-desktop   docker-desktop   docker-desktop

$ kubectl config use-context docker-desktop
Switched to context "docker-desktop"

$ kubectl get nodes
NAME             STATUS   ROLES           AGE   VERSION
docker-desktop   Ready    control-plane   10d   v1.25.4

$ kubectl get all
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   10d

Deployment & Service

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
$ kubectl create deployment kbe-test --image /Users/eka/Desktop/git/spring5webapp --dry-run=client -o=yaml > deployment.yml

# 注意image欄位必須與docker hub一致
- image: lunski/docker-test 
   name: docker-test
   resources: {}
   env: # 環境設定<視情況添加以下段落>
   - name: var1 # 自訂參數
     value: "info"
     # 健康監控
   - name: MANAGEMENT_ENDPOINT_HEALTH_PROBES_ENABLED
     value: "true"
   - name: MANAGEMENT_HEALTH_READINESSTATE_ENABLED
     value: "true"   
   - name: MANAGEMENT_HEALTH_LIVENESSSTATE_ENABLED
     value: "true"
   - name: SERVER_SHUTDOWN
     value: "graceful" # 等到所有程序結束才關機
   lifecycle:
       preStop:
           exec:
               command: ["sh", "-c", "sleep 10"]
   livenessProbe:
      httpGet:
          port: 8080
          path: /actuator/health/livenes   
   readinessProbe:
      httpGet:
          port: 8080
          path: /actuator/health/readines

$ kubectl apply -f deployment.yml
deployment.apps/kbe-test created

$ kubectl create service clusterip kbe-test --tcp=8080:8080 --dry-run=client -
o=yaml >> service.yml

如果type: NodePort，打API時要帶port

$ kubectl apply -f service.yml
service/kbe-test created

Example

  • Inventory Service
1
2
3
4
5
6
7
$ kubectl create deployment inventory-service --image springframeworkguru/kbe-brewery-inventory-service --dry-run=client -o=yaml > inventory-deployment.yml

$ kubectl apply -f inventory-deployment.yml

$ kubectl create service clusterip inventory-service --tcp=8082:8082 --dry-run=client -o=yaml > inventory-service.yml

$ kubectl apply -f inventory-service.yml
  • Inventory Failover Service
1
2
3
4
5
6
7
$ kubectl create deployment inventory-failover --image springframeworkguru/kbe-brewery-inventory-failover --dry-run=client -o=yaml > inventory-failover-deployment.yml

$ kubectl apply -f inventory-failover-deployment.yml

$ kubectl create service clusterip inventory-service --tcp=8083:8083 --dry-run=client -o=yaml > inventory-failover-service.yml

$ kubectl apply -f inventory-failover-service.yml
  • Beer Service
1
2
3
4
5
6
7
$ kubectl create deployment beer-service --image springframeworkguru/kbe-beer-service --dry-run=client -o=yaml > beer-service-deployment.yml

$ kubectl apply -f beer-service-deployment.yml

$ kubectl create service clusterip beer-service  --tcp=8080:8080 --dry-run=client -o=yaml > beer-service.yml

$ kubectl apply -f beer-service.yml
  • Order Service
1
2
3
4
5
6
7
$ kubectl create deployment order-service --image springframeworkguru/kbe-brewery-order-service --dry-run=client -o=yaml > order-service-deployment.yml

$ kubectl apply -f order-service-deployment.yml

$ kubectl create service clusterip order-service  --tcp=8081:8081--dry-run=client -o=yaml > order-service.yml

$ kubectl apply -f order-service.yml

Check services status

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
$ kubectl get all
NAME                            READY   STATUS    RESTARTS   AGE
pod/kbe-test-66bc4c5859-ctcq6   1/1     Running   0          53s

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/kbe-test     ClusterIP   10.97.167.213   <none>        8080/TCP   21h
service/kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP    10d

NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/kbe-test   1/1     1            1           22h

NAME                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/kbe-test-559cc4776d   0         0         0       22h
replicaset.apps/kbe-test-66bc4c5859   1         1         1       54s

$ kubectl logs --tail=20 kbe-test-66bc4c5859-ctcq6
...

$ docker ps
CONTAINER ID   IMAGE                COMMAND                  CREATED         STATUS         PORTS     NAMES
b2006f0bcc16   lunski/docker-test   "java -jar spring5we…"   2 minutes ago   Up 2 minutes             k8s_docker-test_kbe-test-66bc4c5859-ctcq6_default_44e1d282-ea5b-4809-89d0-9995139b80ba_0

Port Forwarding

forward local ip 192.168.0.139 to cluster-ip 10.97.167.213

1
2
3
4
5
6
$ ipconfig getifaddr en0
192.168.0.139

$ kubectl port-forward service/kbe-test 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

Delete

1
2
3
4
5
$ kubectl delete service kbe-test
service "kbe-test" deleted

$ kubectl delete deployment kbe-test
deployment.apps "kbe-test" deleted

Mysql

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
$ kubectl create deployment mysql --image=mysql:5.7 --dry-run=client -o=yaml > mysql-deployment.ym
l

$ kubectl apply -f mysql-deployment.yml
deployment.apps/mysql created

$ kubectl create service clusterip mysql --tcp=3306:3306 --dry-run=client -o=yaml > mysql-service.yml

$ kubectl apply -f mysql-service.yml
service/mysql created

$ kubectl get all
NAME                         READY   STATUS             RESTARTS        AGE
pod/mysql-74bd96b965-6qs4w   0/1     CrashLoopBackOff   6 (2m20s ago)   8m10s

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP    16d
service/mysql        ClusterIP   10.100.88.211   <none>        3306/TCP   78s

NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/mysql   0/1     1            0           8m10s

NAME                               DESIRED   CURRENT   READY   AGE
replicaset.apps/mysql-74bd96b965   1         1         0       8m10s

JMS

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
$ kubectl create deployment jms --image=vromero/activemq-artemis --dry-run=client -o=yaml > jms-de
ployment.yml

$ kubectl apply -f jms-deployment.yml
deployment.apps/jms created

$  kubectl create service clusterip jms --tcp=8161:8161 --tcp=6161:6161 --dry-run=client -o=yaml > jms-service.yml

$  kubectl apply -f jms-service.yml
service/jms created

$ kubectl get all
 kubectl get all
NAME                         READY   STATUS             RESTARTS        AGE
pod/jms-5cd9686f76-vjrl2     1/1     Running            0               7m37s
pod/mysql-74bd96b965-6qs4w   0/1     CrashLoopBackOff   9 (3m18s ago)   24m

NAME                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
service/jms          ClusterIP   10.102.217.165   <none>        8161/TCP,6161/TCP   47s
service/kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP             16d
service/mysql        ClusterIP   10.100.88.211    <none>        3306/TCP            17m

NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/jms     1/1     1            1           7m37s
deployment.apps/mysql   0/1     1            0           24m

NAME                               DESIRED   CURRENT   READY   AGE
replicaset.apps/jms-5cd9686f76     1         1         1       7m37s
replicaset.apps/mysql-74bd96b965   1         1         0       24m

$ docker ps
CONTAINER ID   IMAGE                      COMMAND                  CREATED         STATUS         PORTS     NAMES
6999802c8c0b   vromero/activemq-artemis   "/docker-entrypoint.…"   8 minutes ago   Up 8 minutes             k8s_activemq-artemis_jms-5cd9686f76-vjrl2_default_ba1c34e2-ff52-41f4-bc63-35022965b6fa_0

OpenShift

img

PlayGround

時程: 2023/2/1- 2023/3/1

RabbitMQ

H2 Database

如果你覺得這篇文章很棒，請你不吝點讚 (ﾟ∀ﾟ)

Welcome to my other publishing channels