쿠버네티스 설치하기 #3 - Container Runtime(containerd) 설치 및 설정

1. 필수 커널 모듈 및 sysctl 설정 (모든 노드)

1.1. 커널 모듈 로드

basscraft@master:~$ sudo modprobe overlay
basscraft@master:~$ sudo modprobe br_netfilter

 

부팅시 자동 로드 되도록 설정

basscraft@master:~$ sudo vi /etc/modules-load.d/containerd.conf
overlay
br_netfilter
~
~
~
basscraft@master:~$

 

1.2. Kubernetes 네트워크용 sysctl 설정

basscraft@master:~$ sudo vi /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
~
~
~
basscraft@master:~$

 

적용

basscraft@master:~$ sudo sysctl --system
* Applying /usr/lib/sysctl.d/10-apparmor.conf ...
* Applying /etc/sysctl.d/10-bufferbloat.conf ...
* Applying /etc/sysctl.d/10-console-messages.conf ...
* Applying /etc/sysctl.d/10-ipv6-privacy.conf ...
* Applying /etc/sysctl.d/10-kernel-hardening.conf ...
...생략 ...
fs.protected_hardlinks = 1
fs.protected_regular = 2
fs.protected_symlinks = 1
basscraft@master:~$

 

2. containerd 설치 (모든 노드)

basscraft@master:~$ sudo apt update
... 생략 ...
basscraft@master:~$ sudo apt install -y containerd
... 생략 ...

 

버전 확인

basscraft@master:~$ containerd --version
containerd github.com/containerd/containerd 1.7.28
basscraft@master:~$

containerd 1.7.28 설치 확인

 

3. containerd 기본 설정 파일 생성 (모든 노드)

디렉토리 생성

basscraft@master:~$ sudo mkdir /etc/containerd

 

basscraft@master:~$ sudo mkdir -p /etc/containerd
basscraft@master:~$ containerd config default | sudo tee /etc/containerd/config.toml
... 생략 ...

 

4. SystemdCgroup 활성화 (모든 노드)

/etc/containerd/config.toml 파일에서 [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] 아래에 부분 근처에 SystemdCgroup = false 를 찾아서  true로 변경 (systemd_group 아님)

... 생략 ...
          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
            BinaryName = ""
            CriuImagePath = ""
            CriuPath = ""
            CriuWorkPath = ""
            IoGid = 0
            IoUid = 0
            NoNewKeyring = false
            NoPivotRoot = false
            Root = ""
            ShimCgroup = ""
            SystemdCgroup = true
... 생략 ...

5. containerd 서비스 재시작 및 활성화

basscraft@master:~$ sudo systemctl restart containerd
basscraft@master:~$ sudo systemctl enable containerd

 

상태 확인

basscraft@master:~$ sudo systemctl status containerd
● containerd.service - containerd container runtime
     Loaded: loaded (/usr/lib/systemd/system/containerd.service; enabled; preset: enabled)
     Active: active (running) since Thu 2025-12-18 15:27:31 UTC; 7min ago
       Docs: https://containerd.io
   Main PID: 35979 (containerd)
      Tasks: 18
     Memory: 15.0M (peak: 16.3M)
        CPU: 359ms
     CGroup: /system.slice/containerd.service
             └─35979 /usr/bin/containerd

... 생략 ...

 

Active: active (running) 상태 확인

6. Kubernetes 패키지 설치 (모든 노드)

6.1. 필수 패키지 설치

basscraft@master:~$ sudo apt update
... 생략 ...
basscraft@master:~$ sudo apt install -y apt-transport-https ca-certificates curl gpg
... 생략 ...

 

6.2. Kubernetes 공식 저장소 추가 (Ubuntu 24.04 권장)

basscraft@master:~$ sudo mkdir -p /etc/apt/keyrings
basscraft@master:~$ curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes.gpg
basscraft@master:~$ echo "deb [signed-by=/etc/apt/keyrings/kubernetes.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list

 

6.3. kubelet / kubeadm / kubectl 설치

basscraft@master:~$ sudo apt update
basscraft@master:~$ apt install -y kubelet kubeadm kubectl

 

설치 확인

basscraft@master:~$ kubelet --version
Kubernetes v1.30.14

basscraft@master:~$ kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"30", GitVersion:"v1.30.14", GitCommit:"9e18483918821121abdf9aa82bc14d66df5d68cd", GitTreeState:"clean", BuildDate:"2025-06-17T18:34:53Z", GoVersion:"go1.23.10", Compiler:"gc", Platform:"linux/amd64"}

basscraft@master:~$ kubectl version --client
Client Version: v1.30.14
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3

 

자동 업그레이드 방지 (필수)

클러스터 구성 중 버전이 바뀌는 사고 방지

basscraft@master:~$ sudo apt-mark hold kubelet kubeadm kubectl
[sudo] password for basscraft:
kubelet set on hold.
kubeadm set on hold.
kubectl set on hold.
basscraft@master:~$

 

6.4. kubelet 서비스 상태 확인

basscraft@master:~$ systemctl status kubelet
○ kubelet.service - kubelet: The Kubernetes Node Agent
     Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; preset: enabled)
    Drop-In: /usr/lib/systemd/system/kubelet.service.d
             └─10-kubeadm.conf
     Active: inactive (dead)
       Docs: https://kubernetes.io/docs/
basscraft@master:~$

kubeadm init 전이기 때문에 active (running) 이 아니어도 정상입니다.

7. kubeadm init (마스터 노드 만 실행)

7.1. kubeadm init

basscraft@master:~$ sudo kubeadm init \
  --apiserver-advertise-address=192.168.2.100 \
  --pod-network-cidr=10.244.0.0/16

Pod 네트워크는 각 노드의 물리적 네트워크와 곂치면 안됩니다.

이를 위해 사설 IP로 pod 네트워크 범위를 지정해 줍니다.

7.2. kubectl 설정

basscraft@master:~$ mkdir -p $HOME/.kube
basscraft@master:~$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
basscraft@master:~$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

7.3 Flannel 설치

basscraft@master:~$ kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
namespace/kube-flannel created
serviceaccount/flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
basscraft@master:~$

 

8. 방화벽 설정

8.1 전체 노드

방화벽이 활성화 되어 있지 않으면 활성화 한다.

basscraft@master:~$ sudo ufw status
Status: inactive
basscraft@master:~$ sudo ufw enable
Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup
basscraft@master:~$ sudo ufw status
Status: active
basscraft@master:~$

 

8.1. 마스터 노드

• 6443/tcp : Kubernetes API Server (Kubernetes의 중앙 제어 API)
• 10250/tcp : Kubelet HTTPS API (각 노드의 로컬 에이전트(kubelet) 관리 인터페이스)
• 8472/udp : Flannel VXLAN Overlay Network (노드 간 Pod-to-Pod 오버레이 네트워크 트래픽 전달)
• 30000:32767/tcp : Kubernetes NodePort Service Range (클러스터 외부에서 Service에 직접 접근)
• 2379/tcp : etcd Client API (Kubernetes 상태 데이터 저장소 접근)
• 2380/tcp : etcd Peer Port (etcd 노드 간 복제/동기화 통신)

basscraft@master:~$ sudo ufw allow from 192.168.2.0/24 to any port 22 proto tcp comment 'Local Network'
Rule added
basscraft@master:~$ sudo ufw allow from 192.168.2.0/24 to any port 6443 proto tcp comment 'K8s kube-apiserver'
Rule added
basscraft@master:~$ sudo ufw allow from 192.168.2.0/24 to any port 10250 proto tcp comment 'K8s kubelet API'
Rule added
basscraft@master:~$ sudo ufw allow from 192.168.2.0/24 to any port 8472 proto udp comment 'K8s Flannel VXLAN'
Rule added
basscraft@master:~$ sudo ufw allow from 192.168.2.0/24 to any port 30000:32767 proto tcp comment 'K8s NodePort range'
Rule added
basscraft@master:~$ sudo ufw allow from 192.168.2.100 to any port 2379 proto tcp comment 'K8s etcd client (local)'
Rule added
basscraft@master:~$ sudo ufw allow from 192.168.2.100 to any port 2380 proto tcp comment 'K8s etcd peer (local)'
Rule added
basscraft@master:~$ sudo ufw reload
Firewall reloaded
basscraft@master:~$ sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       192.168.2.0/24             # Local Network
6443/tcp                   ALLOW       192.168.2.0/24             # K8s kube-apiserver
10250/tcp                  ALLOW       192.168.2.0/24             # K8s kubelet API
8472/udp                   ALLOW       192.168.2.0/24             # K8s Flannel VXLAN
30000:32767/tcp            ALLOW       192.168.2.0/24             # K8s NodePort range
2379/tcp                   ALLOW       192.168.2.100              # K8s etcd client (local)
2380/tcp                   ALLOW       192.168.2.100              # K8s etcd peer (local)

basscraft@master:~$

 

8.2. 워커 노드

• 10250/tcp : Kubelet HTTPS API (마스터가 워커 노드의 kubelet을 관리 하기 위한 포트, Master -> Worker)
• 8472/udp : Flannel VXLAN Overlay Network ( 노드 간 Pod-to-Pod 오버레이 네트워크, Worker <-> Worker, Worker <-> Master)
• 30000–32767/tcp : Kubernetes NodePort Service Range (외부(로컬 네트워크)에서 서비스 접근, 외부 -> Worker)

basscraft@node1:~$ sudo ufw allow from 192.168.2.0/24 to any port 22 proto tcp comment 'SSH'
Rule added
basscraft@node1:~$ sudo ufw allow from 192.168.2.100 to any port 10250 proto tcp comment 'K8s kubelet API'
Rule added
basscraft@node1:~$ sudo ufw allow from 192.168.2.0/24 to any port 8472 proto udp comment 'K8s Flannel VXLAN'
Rule added
basscraft@node1:~$ sudo ufw allow from 192.168.2.0/24 to any port 30000:32767 proto tcp comment 'K8s NodePort range'
Rule added
basscraft@node1:~$ sudo ufw reload
Firewall reloaded
basscraft@node1:~$ sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       192.168.2.0/24             # SSH
10250/tcp                  ALLOW       192.168.2.100              # K8s kubelet API
8472/udp                   ALLOW       192.168.2.0/24             # K8s Flannel VXLAN
30000:32767/tcp            ALLOW       192.168.2.0/24             # K8s NodePort range

basscraft@node1:~$

 

9. 토큰 생성 (마스터 노드에서 실행)

basscraft@master:~$ kubeadm token create --print-join-command
kubeadm join 192.168.2.100:6443 --token 보안삭제 --discovery-token-ca-cert-hash sha256:보안삭제
basscraft@master:~$

10. 워커노드 Join (워커노드에서 실행)

10.1. 포트 활성화 확인

마스터 노드의 kube-apiserver 포트로 접속이 가능한지 확인

basscraft@node1:~$ nc -zv 192.168.2.100 6443
Connection to 192.168.2.100 6443 port [tcp/*] succeeded!
basscraft@node1:~$

정상 적으로 접속 가능

 

10.2 각 워커노드에서 Join 실행

basscraft@node1:~$ sudo kubeadm join 192.168.2.100:6443 \
  --token 보안삭제 \
  --discovery-token-ca-cert-hash sha256:보안삭제
... 생략 ...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

basscraft@node1:~$

This node has joined the cluster 로 정상 Join 확인

 

11. 노드 상태 확인 (마스터 노드에서 실행)

11.1. 노드 상태 확인

워커노드 Join 후 최초 상태 NotReady -> 30초 ~ 1분후 Ready 상태로 바뀜

basscraft@master:~$ kubectl get nodes -o wide
NAME     STATUS     ROLES           AGE     VERSION    INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                                                                                                                                     KERNEL-VERSION     CONTAINER-RUNTIME
master   Ready      control-plane   88m     v1.30.14   192.168.2.100   <none>        Ubuntu 24.04.3 L                                                                                                                        TS   6.8.0-90-generic   containerd://1.7.28
node1    NotReady   <none>          7m25s   v1.30.14   192.168.2.101   <none>        Ubuntu 24.04.3 L                                                                                                                        TS   6.8.0-1043-raspi   containerd://1.7.28
node2    NotReady   <none>          7m25s   v1.30.14   192.168.2.102   <none>        Ubuntu 24.04.3 L                                                                                                                        TS   6.8.0-1043-raspi   containerd://1.7.28
node3    NotReady   <none>          7m24s   v1.30.14   192.168.2.103   <none>        Ubuntu 24.04.3 L                                                                                                                        TS   6.8.0-1043-raspi   containerd://1.7.28
basscraft@master:~$ kubectl get pods -n kube-flannel -o wide
NAME                    READY   STATUS     RESTARTS   AGE     IP              NODE     NOMINATED NODE                                                                                                                           READINESS GATES
kube-flannel-ds-86ntt   1/1     Running    0          79m     192.168.2.100   master   <none>                                                                                                                                   <none>
kube-flannel-ds-kqf8s   0/1     Init:1/2   0          8m55s   192.168.2.102   node2    <none>                                                                                                                                   <none>
kube-flannel-ds-llb5r   0/1     Init:1/2   0          8m54s   192.168.2.103   node3    <none>                                                                                                                                   <none>
kube-flannel-ds-mrxfx   0/1     Init:1/2   0          8m55s   192.168.2.101   node1    <none>                                                                                                                                   <none>
basscraft@master:~$

 

나의 경우 시간이 지나도 Ready 상태로 바뀌지 않음 -> 각 워커노드에서 서비스를 재시작 함

basscraft@node1:~$ sudo systemctl restart containerd
basscraft@node1:~$ sudo systemctl restart kubelet

 

이후 마스터 노드에서 정상 확인함

basscraft@master:~$ kubectl get nodes -o wide
NAME     STATUS   ROLES           AGE    VERSION    INTERNAL-IP     EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
master   Ready    control-plane   106m   v1.30.14   192.168.2.100   <none>        Ubuntu 24.04.3 LTS   6.8.0-90-generic   containerd://1.7.28
node1    Ready    <none>          25m    v1.30.14   192.168.2.101   <none>        Ubuntu 24.04.3 LTS   6.8.0-1043-raspi   containerd://1.7.28
node2    Ready    <none>          25m    v1.30.14   192.168.2.102   <none>        Ubuntu 24.04.3 LTS   6.8.0-1043-raspi   containerd://1.7.28
node3    Ready    <none>          25m    v1.30.14   192.168.2.103   <none>        Ubuntu 24.04.3 LTS   6.8.0-1043-raspi   containerd://1.7.28
basscraft@master:~$

STATUS Ready 면 정상

 

11.2. Flannel 정상 동작 확인

basscraft@master:~$ kubectl get pods -n kube-flannel
NAME                    READY   STATUS    RESTARTS   AGE
kube-flannel-ds-86ntt   1/1     Running   0          101m
kube-flannel-ds-kqf8s   1/1     Running   0          30m
kube-flannel-ds-llb5r   1/1     Running   0          30m
kube-flannel-ds-mrxfx   1/1     Running   0          30m
basscraft@master:~$

STATUS 가 Running 면 정상

 

12. 클러스터 동작 검증 (마스터 노드)

12.1. 테스트 Deployment 배포

basscraft@master:~$ kubectl create deployment nginx --image=nginx
deployment.apps/nginx created
basscraft@master:~$

 

12.2. Pod 분산 확인

basscraft@master:~$ kubectl get pods -o wide
NAME                    READY   STATUS              RESTARTS   AGE   IP       NODE    NOMINATED NODE   READINESS GATES
nginx-bf5d5cf98-ccswm   0/1     ContainerCreating   0          19s   <none>   node1   <none>           <none>
basscraft@master:~$

 

12.3. NodePort로 외부 접근 테스트

basscraft@master:~$ kubectl expose deployment nginx \
  --type=NodePort \
  --port=80
service/nginx exposed
basscraft@master:~$

basscraft@master:~$ kubectl get svc nginx
NAME    TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
nginx   NodePort   10.96.14.96   <none>        80:30625/TCP   24s
basscraft@master:~$

 

12.4. 브라우저에서 접근 확인

http://192.168.2.101:30625

http://192.168.2.102:30625

http://192.168.2.103:30625

 

13. 안전한 종료 (마스터 노드에서 실행)

항상 워커 노드 → 마지막에 마스터 노드 순서로 종료

마스터를 먼저 끄면 etcd 비정상 종료, 클러스터 상태 손상 위험이 있습니다.

모든 노드는 마스터에서 제어/상태 관리 합니다.

 

13.1. 종료 순서

node1 -> node2 -> node3 -> master

basscraft@master:~$ kubectl drain node1 --ignore-daemonsets --delete-emptydir-data
node/node1 cordoned
Warning: ignoring DaemonSet-managed Pods: kube-flannel/kube-flannel-ds-mrxfx, kube-system/kube-proxy-qsqf6
node/node1 drained
basscraft@master:~$ kubectl drain node2 --ignore-daemonsets --delete-emptydir-data
node/node2 cordoned
Warning: ignoring DaemonSet-managed Pods: kube-flannel/kube-flannel-ds-kqf8s, kube-system/kube-proxy-mh887
node/node2 drained
basscraft@master:~$ kubectl drain node3 --ignore-daemonsets --delete-emptydir-data
node/node3 cordoned
Warning: ignoring DaemonSet-managed Pods: kube-flannel/kube-flannel-ds-llb5r, kube-system/kube-proxy-drf7w
node/node3 drained
basscraft@master:~$

 

13.2. 종료 상태 확인

basscraft@master:~$ kubectl get nodes
NAME     STATUS                     ROLES           AGE    VERSION
master   Ready                      control-plane   133m   v1.30.14
node1    Ready,SchedulingDisabled   <none>          52m    v1.30.14
node2    Ready,SchedulingDisabled   <none>          52m    v1.30.14
node3    Ready,SchedulingDisabled   <none>          52m    v1.30.14
basscraft@master:~$

STATUS : Ready,SchedulingDisabled
ROLES : <none> 

정상 적용된 상태

 

13.3. 개별 노드 상세 확인

basscraft@master:~$ kubectl describe node node1
... 생략 ...
Unschedulable:      true
... 생략 ...

 

13.4. Pod 이동 여부 확인

basscraft@master:~$ kubectl get pods -A -o wide
NAMESPACE      NAME                             READY   STATUS    RESTARTS   AGE     IP              NODE     NOMINATED NODE   READINESS GATES
default        nginx-bf5d5cf98-bjv8r            1/1     Running   0          8m38s   10.244.0.4      master   <none>           <none>
kube-flannel   kube-flannel-ds-86ntt            1/1     Running   0          130m    192.168.2.100   master   <none>           <none>
kube-flannel   kube-flannel-ds-kqf8s            1/1     Running   0          59m     192.168.2.102   node2    <none>           <none>
kube-flannel   kube-flannel-ds-llb5r            1/1     Running   0          59m     192.168.2.103   node3    <none>           <none>
kube-flannel   kube-flannel-ds-mrxfx            1/1     Running   0          59m     192.168.2.101   node1    <none>           <none>
kube-system    coredns-55cb58b774-68ccv         1/1     Running   0          140m    10.244.0.3      master   <none>           <none>
kube-system    coredns-55cb58b774-8vbcd         1/1     Running   0          140m    10.244.0.2      master   <none>           <none>
kube-system    etcd-master                      1/1     Running   0          140m    192.168.2.100   master   <none>           <none>
kube-system    kube-apiserver-master            1/1     Running   0          140m    192.168.2.100   master   <none>           <none>
kube-system    kube-controller-manager-master   1/1     Running   0          140m    192.168.2.100   master   <none>           <none>
kube-system    kube-proxy-drf7w                 1/1     Running   0          59m     192.168.2.103   node3    <none>           <none>
kube-system    kube-proxy-krvpq                 1/1     Running   0          140m    192.168.2.100   master   <none>           <none>
kube-system    kube-proxy-mh887                 1/1     Running   0          59m     192.168.2.102   node2    <none>           <none>
kube-system    kube-proxy-qsqf6                 1/1     Running   0          59m     192.168.2.101   node1    <none>           <none>
kube-system    kube-scheduler-master            1/1     Running   0          140m    192.168.2.100   master   <none>           <none>
basscraft@master:~$

 

14. 워커노드 종료 (각 워커노드에서 실행)

basscraft@node1:~$ sudo shutdown -h now
[sudo] password for basscraft:

Broadcast message from root@node1 on pts/1 (Fri 2025-12-19 03:46:32 KST):

The system will power off now!

basscraft@node1:~$

15. 마스터 노드 종료

15.1. 테스트용 이미지 삭제 (선택)

basscraft@master:~$ kubectl delete deployment nginx
basscraft@master:~$ kubectl delete svc nginx 2>/dev/null

 

15.2. 마스터 노드 전원 종료

basscraft@master:~$ sudo shutdown -h now
[sudo] password for basscraft:

Broadcast message from root@master on pts/1 (Fri 2025-12-19 03:46:32 KST):

The system will power off now!

basscraft@master:~$

 

 

16. 재기동시

16.1. 마스터 부팅

basscraft@master:~$ kubectl get nodes

16.2. 워커 부팅

basscraft@master:~$ kubectl uncordon node1
basscraft@master:~$ kubectl uncordon node2
basscraft@master:~$ kubectl uncordon node3

16.3. 확인

basscraft@master:~$ kubectl get nodes

끝.