.. |br| raw:: html
.. _csa.callengine.userguide: User guide ========== Upload and analyze log file(s) ------------------------------ On the main screen, select the **Log analysis** tool. .. image:: images/csa/tool_list.jpg :align: center .. only:: internal The files available on CSA for analysis are the files which are present in your BDB sandbox, there are several ways to get files there: 1. Upload new files by dragging them over to the dropzone or by clicking the dropzone and selecting the file in the dialog window 2. Fetch them from a service request by entering the SR number and clicking *Fetch from SR*; this will fetch all attachments marked as important 3. Use the silver robot icon next to an attachment in CSOne 4. Use the CSA BJB plugin which will add a CSA logo next to attachments in CSOne and will transfer and analyze the file with one click .. image:: images/csa/callengine/upload_files_internal.jpg :align: center CSA tries to automatically detect the product type, but lets you also manually change it with a dropdown list. For the list of the supported products and log file formats please refer to this document: :ref:`csa.callengine.supportedproducts`. .. image:: images/csa/callengine/available_files_internal.jpg :align: center If you have one archive that contains multiple products logs, the automatic detection will result in identifying one of the products, or none. The intended way to work with such archives is to click on the *blue plus* icon next to the file to expand the archive. Then select each file or folder you want to analyze individually and manually specify the product type for that file or folder. .. image:: images/csa/callengine/available_files_internal_manual_expand.png :align: center .. note:: It's important that each selected folder matches the requirements as documented in the expected log file formats: :ref:`csa.callengine.supportedproducts`. .. only:: external You can upload new files by dragging them over to the dropzone and then clicking *Upload*. .. image:: images/csa/callengine/upload_files_external.png :align: center CSA will then load the files and automatically detect the product type. The list of files available for analysis is the listed under *Available files*. You can see the file name, size and product type. .. image:: images/csa/callengine/available_files_external.png :align: center .. note:: If the product type is not automatically detected, the file you have uploaded may be corrupt or it is not supported by the tool. For the list of the supported products and log file formats please refer to this document: :ref:`csa.callengine.supportedproducts`. You can select one or more files at once and hit **Run Analysis** button to start the analysis. As discussed in the following `Analysis`_ section, the output shown will differ for single or multiple file analysis. Packet captures (PCAPs) ^^^^^^^^^^^^^^^^^^^^^^^ Packet capture files (.pcap, .pcapng) have their own product type and can be used in several ways when selected for analysis: 1. Single standalone pcap will be result in single product (pcap) analysis. 2. Multiple pcap files without any other product result in multiple pcap files analysis. Each pcap is treated individually. 3. Selecting one or more pcaps along with *one* other product will result in the pcaps being associated with that one product and the information extracted from pcap files will be used in the main product analysis. This is useful when the packet captures were upload separately to main product archive. 4. Selecting multiple products and one or more pcaps will result in pcaps being analyzed individually as we can't know to which product to link them to. .. note:: Any pcap file that is contained within a folder or archive that was identified as specific product will be automatically associated with this product. If multiple products each having one or more pcaps need to be analyzed at once, the way to do it is to create an archive for each product containing the product logs and the pcaps and then select the archive with particular product type for analysis. The above only applies when pcap files are individually selected and identified as *pcap* product type. .. only:: internal Reference analysis ------------------- CSA also provides a set of analysis examples that show the communication flows for specific scenarios. This is helpful when trying to understand certain signaling flow within a single product or across multiple products and can assist as reference when comparing with problematic flow that engineer is troubleshooting in a service request. List of reference analysis can be accessed via *Reference analysis* link on the right part of the Log analysis input section. .. image:: images/csa/callengine/csa_callengine_reference_analysis.png :align: center The reference analysis section provides a list of stored examples with a flow description and the products analyzed. Next to the analysis link there is also a download button to download the original log archives if needed. .. image:: images/csa/callengine/csa_callengine_reference_analysis_detail.png :align: center Analysis -------- Files selected for analysis are parsed for system information and communication flows. If more than one file (product) was selected, the communication flows are correlated across these products and later on displayed in such way. Additionally, we try to automatically detect common configuration mistakes or known defects using our Diagnostic signatures. The analysis display is organized in a top-to-bottom flow including the following sections, which are described in more detail below: 1. Diagnostic signatures (both CSA native and external integrations) 2. Multiple file analysis overview and analysis switcher (only when multiple products are selected) 3. System information 4. Log overview 5. Detail view Top menu navigation ^^^^^^^^^^^^^^^^^^^ When analysis is done and the page scrolls below the initial input portion of the page, the top menu navigation bar is shown to allow for quicker navigation between sections. .. only:: internal .. image:: images/csa/callengine/csa_callengine_top_menu_bar.png :align: center .. only:: external .. image:: images/csa/callengine/csa_callengine_top_menu_bar_external.png :align: center On the right part of the bar there is a timezone selector. By default, all the timestamps within CSA analysis (log overview, detail sections) are normalized to UTC. User has the option to change the timezone in which to see the timestamps and this value will be remembered. .. only:: internal Next to the timezone selector there is the name of the file that was analyzed. This is helpful to keep track when switching between multiple analysis pages. The link icon button on the right provides a direct link to this analysis (copy to clipboard) for the purpose of sharing in case notes or between engineers for easy collaboration. Analysis is stored in database and can be access with the direct link without the need for the original log files. Diagnostic signatures ^^^^^^^^^^^^^^^^^^^^^ If any issues were found by diagnostic signatures, the first thing displayed to the user is the summary view of these issues. If no issues are found, the page scrolls down to `System information`_ section if one file was selected for analysis, or to `Multiple analysis options`_ section if more files were selected. .. image:: images/csa/callengine/csa_callengine_diagnostic_signatures.jpg :align: center At the time of analysis first the CSA own diagnostic signatures are shown and other external diagnostic signatures sources are called in the background. This is indicated by the messages below and could result in more signatures matched once the process finishes: .. image:: images/csa/callengine/csa_callengine_additional_diag.png :align: center Clicking on any of the items shown will display more details about the issue including the more detailed issue description, appearance in logs and action plan to fix or further troubleshoot the issue. If there is a known defect related to the issue, or there is external document available, the link will be shown. .. image:: images/csa/callengine/csa_callengine_diagnostic_signature_detail.jpg :align: center Some diagnostic signatures may contain a direct link *Show in analysis* to the communication flow in log analysis itself. To get to the log analysis, you can alternatively scroll down to `System information`_ section or click on **Continue to analysis** link above the Diagnostic signature list. If you're interested in seeing what other diagnostic signatures were run, but did not find any issues, you can select other *Result type* in the control panel on the left. More information about the specific result type is shown when hovering over the ? button on the top right of the diagnostic signatures list. Additionally there is a toggle button to only show defect-related diagnostic signatures. .. image:: images/csa/callengine/csa_callengine_ds_filter.jpg :align: center All diagnostic signatures are listed :ref:`here ` Multiple analysis options ^^^^^^^^^^^^^^^^^^^^^^^^^ When more files were selected for analysis, we analyze the logs and present the information in 2 ways: 1. **Combined analysis** - Here we correlate the communication flows across multiple products and show the flow across these products. There is less specific product analysis shown. 2. **Individual product analysis** - All the data analyzed on individual product is shown, but no correlation to other products is done. .. image:: images/csa/callengine/csa_callengine_analysis_options.jpg :align: center Idea here is that combine analysis should provide the high level view on the flow, and may help with identifying which product may be causing issue. Then, the individual analysis can be reviewed for more information on that product. You can swap between the different analysis options at any time. Selecting an analysis will move the page to `System information`_ section. System information ^^^^^^^^^^^^^^^^^^ This section shows system status and configuration gathered from the product logs, which is often used for troubleshooting. The output varies for different products. .. image:: images/csa/callengine/csa_callengine_system_information.jpg :align: center **CMS console** If live.json is present in the CMS archive uploaded to the tool, the CMS console is shown and works the same as the actual MMP console of the CMS server, when you connect to it over console or SSH. .. note:: Not all the MMP commands are supported, and it will be indicated by the console if unsupported command is entered. .. only:: internal **CMS Dial Plan Tester** The CMS Dial Plan tester is available in the Dial plan tab under System information. Just enter a URI to test and the page will dynamically show what rules are matching and which ones not .. image:: images/csa/callengine/cms_dial_plan_tester_1.png :align: center For the incoming call handling the tester takes into account configured users and spaces and when no match is found it continues to the call forwarding and outbound calls .. image:: images/csa/callengine/cms_dial_plan_tester_2.png :align: center Limitations 1. Tenant and CallBridge scope are not taken into account 2. Dial transforms are not evaluated Log overview ^^^^^^^^^^^^ The section displays the communication flows found in logs organised in tabs. Each tab contains the list of all the flows found that can be sorted and/or searched through using the search input field. With multiple files selected for analysis, the amount of information shown depends based on type of analysis chosen. See the above `Multiple analysis options`_ section for more information. **Combined analysis** .. image:: images/csa/callengine/csa_callengine_log_overview_combined.jpg :align: center **Single product analysis** .. image:: images/csa/callengine/csa_callengine_log_overview_single.jpg :align: center Clicking on any item in the table for some of the tabs will display a detailed view for the communication flow as shown in next section. Detail ^^^^^^ After clicking on item in the table in `Log overview`_ section more details are shown for the particular communication flow. Below are some examples for SIP/H323 calls. **Single product analysis - SIP call** General call information .. image:: images/csa/callengine/csa_callengine_detail_single_call1.jpg :align: center RTP streams linked to SIP call .. image:: images/csa/callengine/csa_callengine_detail_single_call2.jpg :align: center Signaling .. image:: images/csa/callengine/csa_callengine_detail_signaling.jpg :align: center **Multiple products analysis - ladder diagram of SIP call** .. image:: images/csa/callengine/csa_callengine_detail_multiple_ladder.jpg :align: center STUN ^^^^ This section explains how to read the tables in the STUN tab. **Allocate Requests** .. image:: images/csa/callengine/csa_callengine_stun_allocate_request.png :align: center In the above screenshot, we can see that 173.38.220.60:64267 has sent an Allocate Request to 213.33.49.40. |br| 213.33.49.40 has responded with an Allocate Success Response with the NATted address of 173.38.220.60 (XOR-MAPPED-ADDRESSED) and the address of the relay and the port allocated by the relay. When clicking on one of the rows, we can see the all the STUN messages related to the allocation in a ladder: .. image:: images/csa/callengine/csa_callengine_stun_binding_details.png :align: center When hovering the icons, additional information is provided: .. image:: images/csa/callengine/csa_callengine_stun_allocate_hover.png :align: center **Permission Requests** .. image:: images/csa/callengine/csa_callengine_stun_permission_request.png :align: center In the above screenshot, we can see that 10.58.9.191 has sent a Create Permission Request to 10.52.254.54 to allow traffic from the below addresses to be forwarded to 10.58.9.191:17728 : |br|- 192.168.1.4:2438 |br|- 2.34.97.130:2438 |br|- 10.51.28.179:37336 |br|- 173.38.168.144:24006 |br| |br| A Create Permission Success Response has been sent by 10.52.254.54 to 10.58.9.191. **Binding Requests** .. image:: images/csa/callengine/csa_callengine_stun_binding_request_1.png :align: center The green checkbox highlighted in blue means that 213.33.49.40:24007 has sent a Binding Request to 213.33.49.40:24001 while the green checkbox highlighted in orange means that a Binding Success Response has been sent from 213.33.49.40:24001 to 213.33.49.40:24007. If the checkbox highlighted in orange was red, it would mean that no Binding Success Response has been received by 213.33.49.40:24007. When clicking on one of the rows, we can see the details of the stream: .. image:: images/csa/callengine/csa_callengine_stun_binding_details.png :align: center .. image:: images/csa/callengine/csa_callengine_stun_binding_request_2.png :align: center The highlighted checkboxes in the above means that 60.60.60.100:2338 has sent a Bind Request to 10.10.10.10:2330 and has received a Binding Success Response. .. image:: images/csa/callengine/csa_callengine_stun_binding_request_3.png :align: center The highlighted checkboxes in the above means that 10.10.10.10:2330 has sent a Bind Request with the Use-Candidate attribue to 60.60.60.100:2338 and has received a Binding Success Response. .. image:: images/csa/callengine/csa_callengine_stun_binding_from_indication.png :align: center In the above screenshot, when hovering over the sheet icon, the IP address and port of the host will be displayed. |br| This means that the binding request sent from 173.38.168.144:24002 to 10.58.9.191:17064 has been initiated by a STUN Send Indication from 2.34.97.130:2326. |br| We can verify it by: |br|- clicking on a row to get the details .. image:: images/csa/callengine/csa_callengine_stun_binding_from_indication_details.png :align: center |br|- take note of the the transaction ID |br|- open the packet capture |br|- filter with "stun.type==0x0016 and data.data contains [transactionID]". The transactionID should be edited to insert ":" after every 2 charachters .. image:: images/csa/callengine/csa_callengine_stun_binding_from_indication_ws_1.png :align: center We can see that the first 4 hex is 0001 of the data (highlighted in blue). This means that the Send Indication message contains a STUN Binding Request message. A couple of hexadecimal after (just after the STUN magic cookie) we can find the transaction ID. |br|- We can now filter the packet capture with "stun.id == [transactionID]" .. image:: images/csa/callengine/csa_callengine_stun_binding_from_indication_ws_2.png :align: center |br| We can see that the message type is 0x0001 (Binding Request) and the transaction ID is the same. This message is the Binding Request sent from 173.38.168.144:24002 to 10.58.9.191:17064 initiated by the Send Indication. **Channel Bind Requests** .. image:: images/csa/callengine/csa_callengine_stun_channel_binding.png :align: center In the screenshot above, we can see that 10.10.254.18:36654 sent a Channel Bind Request to 10.10.254.10:3478 and received a Channel Bind Success Response. The channel number that will be used fot Channel Data is 0x4000. .. only:: internal SIPp ^^^^ This section explains how to use callengine's SIPp menu and re-produce SIP call per callleg on SIPp server. |br| Before using SIPp, please read the following document: |techdoc|. It will help to understand how you should setup your lab environment to send and receive calls with SIPp server. .. |techdoc| raw:: html "Example how SIPp helps to confirm cause of the issue" Based on call leg and message direction callengine generates UAC or UAS SIPp scenarios: |br|- UAC scenario is where SIPp is initiator of the call (send INVITE message). |br|- UAS scenario is where SIPp is waiting first SIP message from lab side (receive INVITE message). SIPp example: |br| if you plan to re-produce B2B call, you should have a configured B2B solution in your lab. |br| Lab setup in this example: |br| - Endpoint (1080@evlab.net) is registered on CUCM. |br| - Configured B2B solution on EXP-C and E. |br| SIPp server will be able to send a call to 1080@evlab.net as destination URI and stream selected RTP media. SIPp server will initiate a call to EXP-E with ip address 10.48.80.131 on port 5060. |br| Call flow will be SIPp-EXPe-EXPc-CUCM-Endpoint (B2B call). Select a call from the list of calls at “Calls” tab: .. image:: images/csa/callengine/csa_callengine_sipp_select_call_from_calls.png :align: center Select call leg and press on “SIPP menu” link: .. image:: images/csa/callengine/csa_callengine_sipp_link.png :align: center In the screenshot below, SIPP Options window is pop-up: .. image:: images/csa/callengine/csa_callengine_sipp_destination_uri.png :align: center We can set destination sip URI, that URI should be configured in your lab environment. It’s not mandatory field if we have UAS scenario. SIPp server will make a call to provided destination SIP URI. RTP stream can be chosen at RTP information section, because of SIPp limitation we can choose only one stream per call simulation. Selected RTP stream will be send by SIPp server to our destination URI. If no pcap file has been found or no stream has been selected, in this case SIPp uses audio and video mirroring(loopback) and loop back audio and video RTP streams received from remote endpoint. Once “Generate” button is pressed, SIPp scenario will be created: .. image:: images/csa/callengine/csa_callengine_sipp_edit.png :align: center In the screenshot above, we can edit our created scenario directly in window. Customer’s call may contain some messages which are useless for SIPp test, we can easily remove messages from the scenario. Just delete the entire section with message: |br|- for messages which will be sent: … |br|- for messages which will be received: … |br| We can download edited scenario by pressing “Download” button in xml format or proceed further with execution scenario on SIPp server by pressing “Execute” button. Once “Execute” button is pressed, “Execute SIPp scenario” window will be pop-up: .. image:: images/csa/callengine/csa_callengine_sipp_execute.png :align: center In the screenshot above, we can choose preconfigured SIPp server or add your own server’s parameters. |br| For UAC scenarios: to make an outbound call form SIPp server, we have to set ip address, source port, username, password of SIPp server, destination IP and port of remote system. |br| For UAS scenarios: to make an inbound call to SIPp server, we have to set ip address, SIPp port (SIPp server will wait for a call on that port) username, password of SIPp server. Destination IP and port of remote system are not mandatory for inbound calls. |br| Once all parameters are configured, we can press “Execute” button to execute SIPp call. Once SIPp finished a call, new window will be opened with log information: .. image:: images/csa/callengine/csa_callengine_sipp_log_screen.png :align: center .. only:: internal TCP/UDP streams ^^^^^^^^^^^^^^^ This section explains how to read the table in the TCP/UDP streams tab. .. image:: images/csa/callengine/csa_callengine_tcp_table_overview.png :align: center |br| The table lists all the TCP and UDP streams seen in the packet capture. |br| A checkmark is displayed in the 2-way communication whenever packets are sent in both directions. |br| Whenever a stream contains TLS messages or OCSP traffic, a magnifier icon or a checkmark will be respectively displayed in the appropriate column. |br| When the streams contains TLS traffic, clicking on the row will move the view to a ladder diagram: .. image:: images/csa/callengine/csa_callengine_tls_ladder.png :align: center |br| Note that clicking on a message which has a magnifier next to it will give more information about the message. Clicking on a Client Hello or Server Hello message will list the Cipher Suites contained in the message while clicking on a Certificate message will display the certificate chain sent. |br||br| When click on a streams which has the OCSP check mark, a ladder diagram will also be displayed in the same way with the OCSP request/response related to that TLS stream. .. image:: images/csa/callengine/csa_callengine_tls_ladder_w_ocsp.png :align: center