: Recover from Split Brain in Panorama HA Deployments
Focus
Focus

Recover from Split Brain in Panorama HA Deployments

Table of Contents
End-of-Life (EoL)

Recover from Split Brain in Panorama HA Deployments

When Panorama is configured in a high availability (HA) setup, the managed firewalls are connected to both the active and passive Panorama HA peers. When the connection between the active and the passive Panorama peers fails, before the passive Panorama takes over as the active peer it checks whether any firewall is connected to both the active and the passive peer. If even one firewall is connected to both peers, the failover is not triggered.
In the rare event that a failover is triggered when a set of firewalls are connected to the active peer and a set of firewalls are connected to the passive peer, but none of the firewalls are connected to both peers, it is called a split brain. When a split brain occurs, the following conditions occur:
  • Neither Panorama peer is aware of the state nor the HA role of the other peer.
  • Both Panorama peers become active and manage a unique set of firewalls.
To resolve a split brain, debug your network issues and restore connectivity between the Panorama HA peers.
However, if you need to make configuration changes to your firewalls without restoring the connection between the peers, here are a couple of options:
  • Manually add the same configuration changes on both Panorama peers. This ensures that when the link is reestablished the configuration is synchronized.
  • If you need to add/change the configuration at only one Panorama location, make the changes and synchronize the configuration (make sure that you initiate the synchronization from the peer on which you made the changes) when the link between the Panorama peers is re-established. To synchronize the peers, select the Dashboard tab and click the Sync to peer link in the High Availability widget.
  • If you need to add/change the configuration for only the connected firewalls at each location, you can make configuration changes independently on each Panorama peer. Because the peers are disconnected, there is no replication and each peer now has a completely different configuration file (they are out of sync). Therefore, to ensure that the configuration changes on each peer are not lost when the connection is restored, you cannot allow the configuration to be automatically re-synchronized. To solve this problem, export the configuration from each Panorama peer and manually merge the changes using an external diff and merge tool. After the changes are integrated, you can import the unified configuration file on the primary Panorama and then synchronize the imported configuration file with the peer.