CN-Series
Enable Horizontal Pod Autoscaling on the CN-Series
Table of Contents
Expand All
|
Collapse All
CN-Series Firewall Docs
-
-
- Deployment Modes
- HSF
- In-Cloud and On-Prem
-
-
-
Enable Horizontal Pod Autoscaling on the CN-Series
Where Can I Use This? | What Do I Need? |
---|---|
|
|
The horizontal pod autoscaler (HPA) is a Kubernetes resource available in all cloud environments
that automatically scales the number of CN-MGMT and CN-NGFW pods in a deployment based on
monitored metrics. HPA uses two standard metrics across all cloud environments—CPU and
memory utilization—as well as custom metrics specific to each cloud environment. As such,
each cloud requires specific yaml files to enable HPA in AKS, EKS, and GKE.
HPA uses a cloud-specific
metric adapter to retrieve metrics data from a monitoring adapter
in the cloud environment, such as CloudWatch in EKS, to determine when
to scale up or down based on the thresholds you define. You must
modify the necessary yaml files to set the minimum and maximum number
of replicas, the thresholds for each metric, and which metric are
used in autoscaling your firewalls.
In PAN OS 10.1, if you use CN-MGMT pods HPA scaling, you may scale many CN-MGMT pods with
no DP pods connected to them. It is recommended to create maximum replicas of CN-MGMT pods
to prevent unnecessary scaling.
Cloud Environment | Metrics | Average Value | |
---|---|---|---|
AKS, EKS, and GKE | CN-MGMT | panloggingrate | Log count |
pandataplaneslots | Dataplane Slot count | ||
CN-NFGW | dataplanecpuutilizationpct | Percent of CN-NGFW CPU utilization | |
dataplanepacketbufferutilization | Percent of CN-NGFW packet buffer utilization | ||
pansessionactive | Number of sessions active on the CN-NGFW | ||
pansessionutilization | Percent of session utilization | ||
pansessionsslproxyutilization | Percent of session SSL proxy utilization | ||
panthroughput | Throughput in kbps | ||
panpacketrate | Packet rate in packets per second (pps) | ||
panconnectionspersecond | Connections per second |
In the example below is the pan-cn-hpa-dp.yaml
file for EKS. This example is using the dataplane CPU utilization
percentage to autoscale the CN-NGFW pods. At 25 percent, the cluster
will scale up. If the CPU utilization reaches 50 percent, the cluster
will deploy one additional pod. If the CPU utilization were to reach
75 percent, the cluster would deploy two additional pods. This is
determined by dividing the total metric by the metric threshold
and then deploying enough pods to bring the metric down to the configure
threshold across all CN-NGFW pods in the cluster. However, the cluster
will not deploy more CN-NGFW pods than the maxReplicas. If more
than one metric exceed the threshold at the same time, the cluster
will deploy the necessary number of pods to address the higher metric.
By
default, the HPA adapter polls the metrics adapter every 15 seconds.
If the metrics you have specified exceed the configured threshold
for 60 seconds, the cluster will deploy an additional CN-NGFW pod.
The cluster then waits for 300 seconds (five minutes) before deciding
whether or not additional CN-NGFW pods are required. By default,
one pod is deployed at a time. The cluster then checks the metric
(in this case, CPU utilization) after 300 seconds. If the utilization
has dropped down to a level where a pod is no longer needed, the
cluster will delete a pod. The cluster will then wait an additional
60 seconds before deciding if another pod can be removed.
All
the values shown below and for any metric can be modified to best
fit your deployment.
kind: HorizontalPodAutoscaler apiVersion: autoscaling/v2beta2 metadata: name: hpa-dp-eks namespace: kube-system spec: scaleTargetRef: apiVersion: apps/v1beta1 kind: Deployment name: pan-ngfw-dep minReplicas: 1 maxReplicas: 10 behavior: scaleDown: stabilizationWindowSeconds: 300 policies: - type: Pods value: 1 periodSeconds: 60 - type: Percent value: 1 periodSeconds: 60 selectPolicy: Max scaleUp: stabilizationWindowSeconds: 60 policies: - type: Pods value: 1 periodSeconds: 300 # assuming 5 mins for dp to be ready - type: Percent value: 1 periodSeconds: 300 # assuming 5 mins for dp to be ready selectPolicy: Max metrics: - type: External external: metric: name: dataplaneCpuUtilizationPct target: type: Value value: 25
AKS
- Deploy an Azure Application Insights instance in your the cluster. You must provide the required Azure Application Insights Instrumentation Key and Azure Application Insight APP ID API key as a K8s secret.Download the AKS-specific HPA yaml files from the Palo Alto Networks GitHub repository.If your CN-MGMT is deployed in a custom namespace, update pan-cn-adapater.yaml with the custom namespace. The default namespace is kube-system.If you have not already done so, update the HPA parameters in the AKS-specific pan-cn-mgmt-configmap.yaml.
#PAN_CLOUD: "AKS" #HPA_NAME: "<name>" #unique name to identify hpa resource per namespace or per tenant #PAN_INSTRUMENTATION_KEY: "<>" #Azure APP Insight Instrumentation Key #PUSH_INTERVAL: "15" #time interval to publish metrics to azure app insight
Edit the pan-cn-hpa-secret.yaml.appinsights-appid: "<Azure App Insight Application ID obtained from API Access>" appinsights-key: "<Azure App Insight API Key created under API Access>" azure-client-id: "<Azure SP APP ID associated with corresponding resource group with monitoring reader access>" azure-client-secret: "<Azure SP Password associated with corresponding resource group with monitoring reader access>" azure-tenant-id: "<Azure SP tenant ID associated with corresponding resource group with monitoring reader access>"
Add the HPA name you created above to the appropriate places in pan-cn-custommetrics.yaml.Modify pan-cn-hpa-dp.yaml and pan-cn-hpa-mp.yaml.- Enter the minimum and maximum number of replicas.(Optional) Change the scale down and scale up frequency values to suit your deployment. If you do not change these values, the default values are used.Copy the following section for each metric you want to use for scaling.
- type: Pods pods: metric: name: pansessionactive target: type: AverageValue averageValue: 30
Change the name the metric you want to use and set the averageValue to the threshold described in the table above. If you do not change these values, the default values are used.Save you changes.Deploy the HPA yaml files. The files must be deployed in the order described below.- Use Kubectl to run the pan-cn-hpa-secret.yamlkubectl apply -f pan-cn-hpa-secret.yamlUse Kubectl to run the pan-cn-adapter.yamlkubectl apply -f pan-cn-adapter.yamlUse Kubectl to run the pan-cn-custommetrics.yamlkubectl apply -f pan-cn-custommetrics.yamlUse Kubectl to run the pan-cn-hpa-dp.yamlkubectl apply -f pan-cn-hpa-dp.yamlUse Kubectl to run the pan-cn-hpa-mp.yamlkubectl apply -f pan-cn-hpa-mp.yamlVerify your deployment.
- Use kubectl to verify that the custom metrics adapter pod in the custom metrics namespace.kubectl get pods -n custom-metrics
- Use kubectl to check for the HPA resource.kubectl get hpa -n kube-systemkubectl describe hpa <hpa-name> -n kube-system
EKS
- Deploy the Amazon CloudWatch Metrics Adapter for Kubernetes in your CN-Series as a Service cluster. You must allow CloudWatch complete access to both IAM roles associated with your Kubernetes pods and clusters. To publish the custom metrics to CloudWatch, the worker nodes’ role must have the AWS managed policy CloudWatchAgentServerPolicy so that the HPA can retrieve them.Download the EKS-specific HPA yaml files from the Palo Alto Networks GitHub repository.If your CN-MGMT is deployed in a custom namespace, update pan-cn-adapater.yaml with the custom namespace. The default namespace is kube-system.Modify pan-cn-hpa-dp.yaml and pan-cn-hpa-mp.yaml.
- Enter the minimum and maximum number of replicas.(Optional) Change the scale down and scale up frequency values to suit your deployment. If you do not change these values, the default values are used.Copy the following section for each metric you want to use for scaling.
- type: Pods pods: metric: name: pansessionactive target: type: AverageValue averageValue: 30
Change the name the metric you want to use and set the averageValue to the threshold described in the table above. If you do not change these values, the default values are used.Save you changes.Deploy the HPA yaml files. The files must be deployed in the order described below.- Use Kubectl to run the pan-cn-adapter.yamlkubectl apply -f pan-cn-adapter.yamlUse Kubectl to run the pan-cn-externalmetrics.yamlkubectl apply -f pan-cn-externalmetrics.yamlUse Kubectl to run the pan-cn-hpa-dp.yamlkubectl apply -f pan-cn-hpa-dp.yamlUse Kubectl to run the pan-cn-hpa-mp.yamlkubectl apply -f pan-cn-hpa-mp.yamlVerify your deployment.
- Use kubectl to verify that the custom metrics adapter pod in the custom metrics namespace.kubectl get pods -n custom-metrics
- Use kubectl to check for the HPA resource.kubectl get hpa -n kube-systemkubectl describe hpa <hpa-name> -n kube-system