Focus

CN-Series

Test Case: CN-MGMT Failure Handling

Table of Contents

Scale Out Firewalls Based on Custom Metrics Supported

Test Case: CN-MGMT Failure Handling

Where Can I Use This?	What Do I Need?
CN-Series HSF Firewall deployment	CN-Series 11.0.x or above Container Images Panorama running PAN-OS 11.0.x or above version

This test evaluates CN-NGMT failure handling.

The desired minimum number of CN-MGMT pods for a CN-Series HSF deployment is two to ensure failure handling. After deployment, the CN-MGMT pod that becomes active first becomes the Leader and the second CN-MGMT becomes the Follower. Both CN-MGMT pods have the same configuration. At any instance, one CN-MGMT pod is in READY state. CN-DB, CN-GW, and CN-NGFW pods connect to the CN-MGMT pod in READY state through Traffic Interconnect (TI) links.

The two CN-MGMT pods are not in HA Active-Passive or HA Active-Active mode. Both pods have the same configuration, and is configured using Panorama.

The CN-MGMT pod failure happens due to one of the following conditions.

Liveness check fails
- If slotd is down
- If ipsec or strongswan is down
CN-MGMT pod crashes and restarts

From the Panorama CLI, enter show clusters name <cluster-name> to view the Leader and Follower CN-MGMT pod.

The following output shows that the pan-mgmt-sts-1 pod is active.

Cluster: cluster-001
Creation time: 2022/11/30 03:23:50
CN-MGMT pods:
88C00D31E1FC86B (pan-mgmt-sts-0.cluster-001, connected, In Sync)
84CC9A394B3E196 (active, pan-mgmt-sts-1.cluster-001, connected, In Sync)


Slot-ID         PodName                             Type                      Version
----------------------------------------------------------------------------------------
5               pan-db-dep-6774cd774d-k49cm         CN-DB                11.0.1-c183.dev_e_rel
1               pan-gw-dep-d849c7df8-4sk54          CN-GW                11.0.1-c183.dev_e_rel
6               pan-ngfw-dep-668965d598-pnthb       CN-NGFW              11.0.1-c183.dev_e_rel
8               pan-ngfw-dep-668965d598-s2zcc       CN-NGFW              11.0.1-c183.dev_e_rel
7               pan-ngfw-dep-668965d598-vf9l4       CN-NGFW              11.0.1-c183.dev_e_rel
9               pan-ngfw-dep-668965d598-pmmjd       CN-NGFW              11.0.1-c183.dev_e_rel
10              pan-db-dep-6774cd774d-gjpkr         CN-DB                11.0.1-c183.dev_e_rel
2               pan-gw-dep-d849c7df8-ct6wk          CN-GW                11.0.1-c183.dev_e_rel

View cluster membership for the pan-mgmt-sts-1 pod and state of CN-DB, CN-GW, and CN-NGFW pods from the Kubernetes controller CLI.

Enter kubectl get pods -n kube-system to view the state of all pods.

Output:

The pan-mgmt-sts-1 is Active. All CN-DB, CN-GW, and CN-NGFW pods are connected to pan-mgmt-sts-1.

NAME                            READY   STATUS    RESTARTS   AGE
pan-db-dep-6774cd774d-gjpkr     1/1     Running   0          69m
pan-db-dep-6774cd774d-k49cm     1/1     Running   0          69m
pan-gw-dep-d849c7df8-4sk54      1/1     Running   0          69m
pan-gw-dep-d849c7df8-ct6wk      1/1     Running   0          69m
pan-mgmt-sts-0                  0/1     Running   0          83m
pan-mgmt-sts-1                  1/1     Running   0          83m
pan-ngfw-dep-668965d598-pmmjd   1/1     Running   0          69m
pan-ngfw-dep-668965d598-pnthb   1/1     Running   0          69m
pan-ngfw-dep-668965d598-s2zcc   1/1     Running   0          69m
pan-ngfw-dep-668965d598-vf9l4   1/1     Running   0          69m

Check cluster membership from pan-mgmt-sts-1.

Get in to the pan-mgmt-sts-1 pod.

kubectl -n kube-system exec -it pan-mgmt-sts-1 -- bash

su - admin

Check if all CN-DB, CN-GW, and CN-NGFW pods are connected to the Leader CN-MGMT pod using the following command.

show cluster-membership show-slot-info slot all

Output:

MP leader status: Leader
        Slot-id Type            CI-IP           TI-IP           State   CI-State        TI-State
        ========================================================================================
        1       CN-GW           192.168.23.101  192.168.24.100  UP      UP              UP
        10      CN-DB           192.168.23.104  ::              UP      UP              NA
        2       CN-GW           192.168.23.100  192.168.24.98   UP      UP              UP
        5       CN-DB           192.168.23.102  ::              UP      UP              NA
        6       CN-NGFW         192.168.23.89   192.168.24.83   UP      UP              UP
        7       CN-NGFW         192.168.23.105  192.168.24.86   UP      UP              UP
        8       CN-NGFW         192.168.23.103  192.168.24.84   UP      UP              UP
        9       CN-NGFW         192.168.23.82   192.168.24.81   UP      UP              UP

Check cluster membership from pan-mgmt-sts-0.
Get in to the pan-mgmt-sts-0 pod.
```
kubectl -n kube-system exec -it pan-mgmt-sts-0 -- bash
```
su - admin

Check if any CN-DB, CN-GW, and CN-NGFW pods are connected to the Follower CN-MGMT pod using the following command.

show cluster-membership show-slot-info slot all

Output:
No members info present

Test CN-MGMT pod failure handling.

From the Kubernetes controller CLI, enter the following command to delete the Leader pan-mgmt-sts-1 pod.
```
kubectl -n kube-system delete pod pan-mgmt-sts-1
```

From the Panorama CLI, enter show clusters name <cluster-name> to view the new Leader and Follower CN-MGMT pod.

The following output shows that the pan-mgmt-sts-0 pod is now active.

Cluster: cluster-001
Creation time: 2022/11/30 03:23:50
CN-MGMT pods:
88C00D31E1FC86B (active, pan-mgmt-sts-0.cluster-001, connected, In Sync)
84CC9A394B3E196 (pan-mgmt-sts-1.cluster-001, connected, In Sync)


Slot-ID         PodName                             Type                      Version
----------------------------------------------------------------------------------------
5               pan-db-dep-6774cd774d-k49cm         CN-DB                11.0.1-c183.dev_e_rel
1               pan-gw-dep-d849c7df8-4sk54          CN-GW                11.0.1-c183.dev_e_rel
6               pan-ngfw-dep-668965d598-pnthb       CN-NGFW              11.0.1-c183.dev_e_rel
8               pan-ngfw-dep-668965d598-s2zcc       CN-NGFW              11.0.1-c183.dev_e_rel
7               pan-ngfw-dep-668965d598-vf9l4       CN-NGFW              11.0.1-c183.dev_e_rel
9               pan-ngfw-dep-668965d598-pmmjd       CN-NGFW              11.0.1-c183.dev_e_rel
10              pan-db-dep-6774cd774d-gjpkr         CN-DB                11.0.1-c183.dev_e_rel
2               pan-gw-dep-d849c7df8-ct6wk          CN-GW                11.0.1-c183.dev_e_rel

View cluster membership for the pan-mgmt-sts-0 pod and state of CN-DB, CN-GW, and CN-NGFW pods from the Kubernetes controller CLI.

Enter kubectl get pods -n kube-system to view the state of all pods.

Output:

The pan-mgmt-sts-0 is Active. All CN-DB, CN-GW, and CN-NGFW pods are connected to pan-mgmt-sts-1.

NAME                            READY   STATUS    RESTARTS   AGE
pan-db-dep-6774cd774d-gjpkr     1/1     Running   0          76m
pan-db-dep-6774cd774d-k49cm     1/1     Running   0          76m
pan-gw-dep-d849c7df8-4sk54      1/1     Running   0          76m
pan-gw-dep-d849c7df8-ct6wk      1/1     Running   0          76m
pan-mgmt-sts-0                  1/1     Running   0          90m
pan-mgmt-sts-1                  0/1     Running   0          90m
pan-ngfw-dep-668965d598-pmmjd   1/1     Running   0          76m
pan-ngfw-dep-668965d598-pnthb   1/1     Running   0          76m
pan-ngfw-dep-668965d598-s2zcc   1/1     Running   0          76m
pan-ngfw-dep-668965d598-vf9l4   1/1     Running   0          76m

Check cluster membership from pan-mgmt-sts-0.

Get in to the pan-mgmt-sts-0 pod.

kubectl -n kube-system exec -it pan-mgmt-sts-0 -- bash

su - admin

Check if all CN-DB, CN-GW, and CN-NGFW pods are connected to the Leader CN-MGMT pod using the following command.

show cluster-membership show-slot-info slot all

Output:

MP leader status: Leader
        Slot-id Type            CI-IP           TI-IP           State   CI-State        TI-State
        ========================================================================================
        1       CN-GW           192.168.23.101  192.168.24.100  UP      UP              UP
        10      CN-DB           192.168.23.104  ::              UP      UP              NA
        2       CN-GW           192.168.23.100  192.168.24.98   UP      UP              UP
        5       CN-DB           192.168.23.102  ::              UP      UP              NA
        6       CN-NGFW         192.168.23.89   192.168.24.83   UP      UP              UP
        7       CN-NGFW         192.168.23.105  192.168.24.86   UP      UP              UP
        8       CN-NGFW         192.168.23.103  192.168.24.84   UP      UP              UP
        9       CN-NGFW         192.168.23.82   192.168.24.81   UP      UP              UP

Check cluster membership from pan-mgmt-sts-1.
Get in to the pan-mgmt-sts-1 pod.
```
kubectl -n kube-system exec -it pan-mgmt-sts-1 -- bash
```
su - admin

Check if any CN-DB, CN-GW, and CN-NGFW pods are connected to the Follower CN-MGMT pod using the following command.

show cluster-membership show-slot-info slot all

Output:
No members info present

Test Result: When the Leader pod pan-mgmt-sts-1 fails, the Follower pod pan-mgmt-sts-0 becomes the new Leader. This CN-MGMT failure handling mechanism ensures that traffic flow is uninterrupted. No impact to existing or new sessions.

Scale Out Firewalls Based on Custom Metrics Supported

Test Case: CN-NGFW Failure Handling

CN-Series Firewall Docs

Test Case: CN-MGMT Failure Handling

CN-Series Firewall Docs

Test Case: CN-MGMT Failure Handling

Activation & Onboarding

Next-Generation Firewalls

Cloud-Delivered Security Services

Best Practices

Experts Corner