>
    Forward Logs to an HTTPS Server
    Focus
    Download PDF
    Focus
    1. Home
    2. Strata Logging Service
    3. Strata Logging Service Administration
    4. Forward Logs from Strata Logging Service
    5. Forward Logs to an HTTPS Server
    Download PDF
    Strata Logging Service

    Forward Logs to an HTTPS Server

    Table of Contents

    Forward Logs to an HTTPS Server

    Learn how to forward logs from Strata Logging Service to an HTTPS server.
    Where Can I Use This?What Do I Need?
    One of these:
    To meet your long-term storage, reporting and monitoring, or legal and compliance needs, you can configure Strata Logging Service to forward logs to an HTTPS server or to the following SIEMs:
    • Exabeam
    • Google Chronicle
    • Microsoft Sentinel
    • Splunk HTTP Event Collector (HEC)
    For successful log transmission, ensure that your HTTPS receiver:
    • Accepts and parses the correct log format. The format in which Strata Logging Service forwards logs depends on the HTTPS receiver:
      Google Chronicle
      (JSON Array)
      		 
      {
  "customer_id": "xxxxx-xxxxx-xxxxx-xxxxx-xxxxx",
  "log_type": "ARCSIGHT_CEF",
  "entries": [
    {
      "log_text": "CEF:0|palo alto networks|LF|2.0|DNS|realtime_dns_query|3|ProfileToken=hello-cef dtz=UTC deviceExternalID=xxxxx rt=Jun 09 2022 18:08:31 startTime=Dec 09 1654  PanOSRecordType=null PanOSCloudDNSClientIP= PanOSDNSResolverIP=9.9.9.9 PanOSThreatID=109010001 PanOSDNSCategory=phishing cat=Phishing:blockchain.ppckite.com src= cs4= act= cs5= duser= DestinationDNSDomain="
    },
    {
      "log_text": "CEF:0|palo alto networks|LF|2.0|DNS|realtime_dns_query|3|ProfileToken=hello-cef dtz=UTC deviceExternalID=xxxxx rt=Jun 09 2022 18:08:31 startTime=Dec 09 1654  PanOSRecordType=null PanOSCloudDNSClientIP= PanOSDNSResolverIP=9.9.9.9 PanOSThreatID=109010001 PanOSDNSCategory=phishing cat=Phishing:blockchain.ppckite.com src= cs4= act= cs5= duser= DestinationDNSDomain="
    },
    {
      "log_text": "CEF:0|palo alto networks|LF|2.0|DNS|realtime_dns_query|3|ProfileToken=hello-cef dtz=UTC deviceExternalID=xxxxx rt=Jun 09 2022 18:08:31 startTime=Dec 09 1654  PanOSRecordType=null PanOSCloudDNSClientIP= PanOSDNSResolverIP=9.9.9.9 PanOSThreatID=109010001 PanOSDNSCategory=phishing cat=Phishing:blockchain.ppckite.com src= cs4= act= cs5= duser= DestinationDNSDomain="
    }
  ]
}
      Microsoft Sentinel
      (JSON Array)
      		 
      [
  {
    "time": 1437522387,
    "host": "stream-logfwd20",
    "source": "Palo Alto Networks FLS LF",
    "event": {
      "message": "Something happened",
      "severity": "INFO"
    }
  },
  {
    "time": 1437522387,
    "host": "stream-logfwd20",
    "source": "testapp",
    "event": {
      "message": "Something happened",
      "severity": "INFO"
    }
  }
]
      Splunk HEC
      (Stacked JSON)
      		 
      {
  "time": 1437522387,
  "host": "stream-logfwd20",
  "source": "Palo Alto Networks FLS LF",
  "event": {
    "LogType": "THREAT",
    "Severity": "Critical",
    ...
  }
}
{
  "time": 1437522387,
  "host": "stream-logfwd20",
  "source": "Palo Alto Networks FLS LF",
  "event": {
    "LogType": "THREAT",
    "Severity": "Info",
    ...
  }
}
      Exabeam
      (JSON Array)
      		 
      {
  "DestinationDeviceCategory": "N-Phone",
  "DestinationDeviceOSFamily": "H1511",
  "DestinationDeviceOSVersion": "Android v7",
  "SourceDeviceOS": null,
  "NATDestination": "xxx.xx.x.xxx",
  "ApplicationSubcategory": "database",
  "VendorName": "Palo Alto Networks",
  "Protocol": "tcp",
  "IsServertoClient": false,
  "PacketID": 0,
  "NSSAINetworkSliceType": "fc",
  "DirectionOfAttack": "client to server",
  "EndpointSerialNumber": "SG0000001",
  "ApplicationTechnology": "network-protocol",
  "VendorSeverity": "Critical",
  "SubType": "data",
  "DeviceSN": "xxxxxxxxxx",
  "ConfigVersion": "10.2",
  "IsMptcpOn": false,
  "SourceDeviceModel": "Nexus",
  "InboundInterface": "ethernet1/1",
  "LogExported": false,
  "ParentSessionID": 0,
  "CloudHostname": "PA-VM-E2E-PCL-TEST",
  "SourcePort": 25195,
  "CaptivePortal": false,
  "LogSource": "firewall",
  "LogType": "THREAT",
  "InboundInterfaceDetailsType": "ethernet",
  "Severity": "Critical",
  "TimeReceived": "2023-07-22T08:49:30.000000Z",
  "SourceUserDomain": "paloaltonetwork",
  "IsDuplicateLog": false,
}
      For more information about HTTPS log format, see the Log Forwarding Schema Reference.
    • Accepts and decompresses GZIP HTTPS payloads. Strata Logging Service compresses the JSON data using GZIP when forwarding through HTTPS.
    • (For Microsoft Sentinel Only) - Microsoft Sentinel does not accept GZIP, so you must deploy a web app to decompress the data.
    • Accepts a batch size of 500 logs, or 2.25 MB (500 logs x 4500 B = 2.25 MB). This does not apply to Google Chronicle, which accepts only 1 MB of data at a time, or a batch size of 250 logs. For each instance of Strata Logging Service, you can forward logs to up to 200 destinations.
    • Can connect to Strata Logging Service -
      • Allow an inbound TLS feed to your HTTPS receiver from the IP address range for your Strata Logging Service region.
      • Obtain a certificate from a well-known, public CA.
        Because Strata Logging Service validates the server certificate to establish a connection, you must verify that you have configured the receiver to properly send the TLS certificate chain to Strata Logging Service. If the app can't verify that the certificate of the receiver and all CAs in the chain are trustworthy, it can't establish a connection. See the list of trusted certificates.
      If the connection with the HTTPS server fails, Strata Logging Service will wait 60 seconds and retry. If the connection times out, Strata Logging Service waits 20 seconds and drops the connection with the server before establishing a new one.
    1. Sign In to the hub.
    2. Select the Strata Logging Service instance that you want to configure for HTTPS forwarding.
      If you have multiple Strata Logging Service instances, click the Strata Logging Service tile and select an instance from the list of those available.
      If you are using Strata Cloud Manager to manage Strata Logging Service, click SettingsStrata Logging ServiceLog Forwarding to manage devices onboarded to your Strata Logging Service instance.
    3. Select Log ForwardingAdd to add a new HTTPS forwarding profile.
    4. Enter a descriptive Name for the profile.
    5. Enter the URL that you copied in step 13 for the HTTPS receiver.
      The URL entered must begin with https. At the end of the top-level domain, you can specify a port number. The default port is 443.
      For Splunk HEC, ensure that the URL ends with /event.
      If you're forwarding logs to Google Chronicle, you must enter the URL that corresponds to your Strata Logging Service region:
      UShttps://malachiteingestion-pa.googleapis.com/v2/unstructuredlogentries:batchCreate
      EUhttps://europe-malachiteingestion-pa.googleapis.com/v2/unstructuredlogentries:batchCreate
      Asiahttps://asia-southeast1-malachiteingestion-pa.googleapis.com/v2/unstructuredlogentries:batchCreate
    6. (Optional) Authenticate your https server. Click + to upload a certificate. You can upload:
      • a self-signed certificate if you do not want to use a publicly signed certificate.
      • the private Root CA and intermediate CAs (If an intermediate CA exists). Do not upload the certificate issued for the syslog server—only CA certificates are needed to verify the chain from the syslog server.
      Only do this if you installed a private CA-signed, self-signed certificate on your receiver, or the public CA is not in the list of trusted CAs. The file containing the certificates must be in PEM format.
    7. (Optional) Enable client authentication.
      Do this if company or regulatory policy requires client authentication when forwarding logs to your server.
      1. Download the certificate chain.
      2. Upload the certificate chain to your server.
        Refer to the documentation for your server management software to find out how to do this.
    8. Configure Client Authorization.
      Do this if you're forwarding logs to a cloud HTTPS receiver, or if your HTTPS server requires a username and password.
      1. Select the Type of client authorization and enter the necessary credentials.
        HTTPS ReceiverTypeCredentials
        Splunk HECSplunk AuthorizationHEC Token
        HTTPS Server (Password-protected)Basic Authorization
        • Username
        • Password
        ExabeamExabeam Authorization
        Microsoft SentinelSentinel Authorization
        • Workspace ID
        • Primary Key
        Find your workspace ID and key
        Google ChronicleChronicle Authorization
        • Customer ID
        • Service Account
          This should be a properly formatted JSON document. If you don’t know your Service Account credentials, contact Google Chronicle support.
        HTTPS ServerNoneNone
    9. Acknowledge to reach out to your Palo Alto Networks team to enable log forwarding from Strata Logging Service in China to an external log server. Be aware that configuring log forwarding profiles to send logs to servers outside China can result in personally identifiable information leaving China.
    10. Test Connection to ensure that Strata Logging Service can communicate with the receiver.
      This sends an empty log to the configured destination to verify that transmission is possible.
      If the test fails, you won't be able to proceed.
    11. Click Next.
    12. (Optional) To receive a STATUS NOTIFICATION when Strata Logging Service is unable to connect to the HTTPS receiver, enter the email address at which you’d like to receive the notification.
      You will continue to receive these notifications at least once every 60 minutes until the service restores connectivity. If the connectivity issue resolves within 72 hours, the service won't lose logs. However, the service could lose any log older than 72 hours following the service disconnection.
    13. Enter a unique PROFILE TOKEN if your receiver needs to distinguish logs coming from different tenants.
    14. Select the logs you want to forward.
      1. Add a new log filter.
      2. Select the log type.
        The Threat log type does not include URL logs or Data logs. If you wish to forward these log types, you must add them individually.
      3. (Optional) Create a log filter to forward only the logs that are most critical to you.
        You can either write your own queries from scratch or use the query builder. You can also select the query field to choose from among a set of common predefined queries.
        • No double quotes (“”).
        • No subnet masks. To return IP addresses with subnets, use the LIKE operator. Example: src_ip.value LIKE “192.1.1.%”.
        If you want to forward all logs of the type you selected, don't enter a query. Instead, proceed to the next step.
      4. Save your changes.
    15. Save your changes.
    16. Verify that the Status of your HTTPS forwarding profile is Running (
      ).
      Immediately after creating or editing your profile, the Status might be Provisioning for up to 10 minutes.
    17. Verify that you can view logs on the HTTPS receiver.
      For Splunk Common Information Model (CIM) fields and Enterprise Security, follow the guide at https://splunk.paloaltonetworks.com to install the Palo Alto Networks Splunk add-on and create a Splunk HEC input.
    18. (Optional) You can use the running HTTPS forwarding profile to forward past logs spanning up to 3 days.

    Create and Deploy a Agent Web Application

    Microsoft Sentinel does not accept GZIP to compress the data when forwarding through HTTPS receiver. To enable this, you must deploy a web application.
    1. Log in to your Microsoft Azure account, and create a log analytics workspace in your Sentinel.
    2. Install Visual Studio Code version 1.64.1 or a later version.
    3. Install the Azure Tools and Azure App Service extensions in Visual Studio Code.
    4. Obtain the agent web application’s code from GitHub - git clone https://github.com/PaloAltoNetworks/cdl-decompress-proxy-sentinel-ingest.git
      This is a sample application code and is not maintained by Palo Alto Networks. Don't use the code as-is but we recommend you to develop your own agent or customize this base version to align with your specific needs and requirements.
      Download and extract the ZIP folder if you did not install Git. https://github.com/PaloAltoNetworks/cdl-decompress-proxy-sentinel-ingest/archive/refs/heads/master.zip
    5. Open the cdl-decompress-proxy-sentinel-ingest folder in Visual Studio Code.
    6. Sign in to Azure and click Resourcesyour subscriptionApp Services.
    7. Right click and select Create New Web App. Select the advanced option if you want to make use of previously created Azure resources.
    8. Enter a name, choose the Python 3.9 runtime stack, and select an appropriate pricing tier.
    9. Right click the new agent web app and select Deploy to Web App.
    10. Connect the web app to the Log Analytics workspace - In Azure, navigate to the desired Log Analytics workspace, and select Agents management Linux servers. Copy the Workspace ID and Primary Key values. These details are used while adding a new setting in your agent web application.
    11. (Optional) In Azure, navigate to agent web app and select SettingsIdentitySystem assigned, change Status to On.
    12. (Optional) Access or create an Azure Key Vault to store the workspace ID and primary key values as secrets in the key vault. If you don’t have an Azure key vault, you can add the values as settings directly in the agent web app.
      • To create or import a key vault:
        1. Click Settings > Secrets > and click Generate or Import.
        2. Enter the name for the secret where you will store the Workspace ID value you collected earlier, enter the value, and click Create.
        3. Copy the secret URI you just created. For example, https://<key vault name>.vault.azure.net/secrets/<secret name>
        4. Click Settings > Access policies and click Add Access Policy.
        5. Select Get from the Secret permissions drop-down.
        6. Click None selected, select your agent web application and click Add.
        7. Save the access policy.
      • Add Value in agent web application:
        1. In Visual Studio, open the agent web application, right-click Application Settings and select Add New Settings.
        2. Enter WORKSPACE_ID and press enter.
          If you have stored the values as secrets in a Key Vault, enter the Secret URI, that represents the Workspace ID you collected earlier, in this format - @Microsoft.KeyVault(SecretUri=https://<key vault name>.vault.azure.net/secrets/<secret name>)
          If you did not store the value as a secret in a Key Vault, enter in the WORKSPACE_ID value you copied down earlier ((step 10) while connecting to Log Analytics workspace).
        3. Follow the same process to add another new setting named SHARED_KEY with the Primary Key value collected earlier, or enter the SecretURI if you have this value stored in a Key Vault.
        4. Right click the web app and select Restart.
    13. In Azure, copy the URL from your web app (App Services). This URL is used in the URL field while adding a HTTPS forwarding profile.

    © 2024 Palo Alto Networks, Inc. All rights reserved.