Overview

Global settings are those settings which are applied globally across all groups and are not group specific. User can apply following type of settings through Global Settings window of Scenario GUI.

  • Logs and Reports
  • Transaction
  • HTTP
  • SSL
  • Network Simulation
  • Sync Points
  • Advanced

These features are explained in detail below:

Logs and Reports

To setup logs and reports, click the Logs and Reports tab under the Global Settings section on Scenario Setting window, the Logs and Reports window is displayed.

The Logs and Reports setting window has the following options:

General Settings

Sample Interval

The time duration between two data samples is the sample interval. The default unit is second and the default sample interval is 10 sec

Enable Debug Trace

A debug trace is required to investigate when a script is not being executed properly.  When enabled, the trace is available to debug following:

  • Issues with script execution
  • Bugs or defects

On the Dashboard GUI, go to View -> Logs -> Debug Trace Log in the left panel.

Debug Trace Log is displayed in a window that contains the details of all the user activities. This provides the detailed information on how the Script Test is running.

Enable save nvm file param value: When enabled, nvm file parameter value is saved. 

Reports

Auto Generate Stats Report: When enabled a stats report is auto-generated with following information:

  • Report Type: Select a report type – tabular, word or HTML.
  • Template Name: List of available templates (having graphs information)
  • Report Set Name: Name of the report set
  • X-Axis Time Format: Time format for X-Axis either ‘Elapsed’ or ‘Absolute’.
  • Time: Time selection for generating compare report:
    • Total Run
    • Phases
    • Specified Time Interval

Generate NetCloud Compare Report after completion of the test: When enabled, a compare report is generated after the completion of test.

  • Template Name: List of available templates (having graphs information)
  • Report Set Name: Name of the report set
  • Include Controller: Enable this to include the controller
  • Time: Time selection for generating compare report:
    • Total Run
    • Phases
    • Specified Time Interval

Send test completion report at the end of the test: When enabled a stats report is emailed to specified recipients after test completion with following information:

  • TR number
  • Machine and Controller
  • Scenario
  • Start time of TestRun
  • Duration of TestRun

Percentile Data

A percentile is the value of a variable below which a certain percent of observations fall. So, the 20th percentile is the value (or score) below which 20 percent of the observations may be found. This setting is basically used to have the graphical representation of the percentile report in the GUI analysis window. User needs to select the Generate Percentile Data checkbox and specify the duration after which the percentile data is to be generated. On selecting this check box, the Map Percentile Definition File section is enabled with the following options. 

  • URL Response Time
  • Page Response Time
  • Session Response Time
  • Individual Transaction Response Time
  • All Transaction Response Time

User can change the percentile-associated graph from the drop down list according to the requirement.

Note: When the user uses the same PDF for more than one case, NetStorm throws an error while starting the test.

Event Log

Event log is a log of occurrence or happening of a particular task or program, such as the completion of an asynchronous input/output operation.  A task may wait for an event or any of a set of events or it may (request to) receive asynchronous notification that the event has occurred. User can also filter events based on requirements. To enable event log, select the Enable Event check box.

  • Event Log Levels: This specifies which level of events are to be logged. There are 5 levels – Information, warning, minor, major, and critical. For example: If level is warning, all events with level warning or higher level are logged. Events are further filtered using filter level before logging in the event log file.
  • Filter Criteria: This is used to filter out events based on two criteria. First is ‘Do not filter’ (in this case, events are not filtered) and second is ‘Filter based on the events as per event definition file’ (in this case, filters on events are applied based on the event definition file). On selecting the second option, user needs to specify the event definition file from the drop-down list.
  • Use Log Manager for all events and to save data API: Enable Log manager starts NetStorm logging manager process. Each process sends events to log manager by using TCP/IP. Filtering happens at system level (default). Each process writes event in the event log file directly. Filtering happens at process level. User can specify the port or use the assigned port.
  • Use System Assigned Manager Port: When enabled, system assigned port is used. Else, user needs to specify a port

Test Notification

Send test completion report at the end of the test: When enabled, the test completion report is emailed to specified recipients.

Advanced Settings

  • Enable Scenario group based test metric(s): When enabled, dashboard displays Test metrics data based on specific groups of the Scenario.
  • Enable Page Based Stats: When enabled, Real Browser User (RBU) page stats metrics are displayed on dashboard.
  • Enable online generation of drill down reports, Page dump reports: When enabled, drill down reports and page dump reports are generated in real-time (for current running Test Run or Sessions). A user can specify “maximum delay” (in seconds) to dump data in the database.
  • Enable Server IP address based metrics: It helps to get the resolved IPs at run time in web-dashboard from which the request is being served (Sent/Sec).

Transactions

Here user can perform transaction related settings. There are two sub-sections under this:

Pages As Transactions

This option is used to enable or disable the transaction. It contains following options:

  • Do not consider page as transaction: When enabled, page is not considered as a transaction.
  • Consider each page as a transaction: When enabled, each page is considered as a specific transaction.
  • Consider each page’s success or failure as distinct transactions: When enabled, each page’ success or failure is considered as a transactions.
  • Consider each page status as different transactions: When enabled, status of each page is considered as a distinct transaction for that page.
  • Do not add ‘tx_’ prefix in transaction name: When enabled, ‘tx_’ is not prefixed with the transaction names. This option is available in all cases except ‘do not consider page as a transaction’ scenario.

Advanced

This sub-section contains some advanced settings for transactions.

  • Enable transaction cumulative metrics: When enabled, the transaction cumulative graphs are added to the test metrics and user can plot them onto dashboard. The most common metrics are – Requests Completed & Requests successful in HTTP requests metrics.
  • Enable transaction grouping in transaction detail: To enable transaction grouping in transaction details, select the transaction group from the drop-down list.

HTTP

To setup HTTP settings, click the HTTP tab under the Global Settings section on Scenario Setting window, the HTTP window is displayed. There are two sub-sections within this – URL Encoding and Advanced.

URL Encoding

URL encoding is a mechanism of encoding information in a URI.

  • Encode space in URL and query parameters of URL: User can specify how the space can be encoded in URL and query parameter of URL. It has value + or %20 (from drop down list).
  • Additional characters to be encoded in URL: It has value none or characters which user want to encode in URL other than space .e.g. {, }, |, % etc.
  • Additional characters to be encoded in query parameters of URL: It has value same or different characters which user want to encode in query parameter .e.g. {.}, |, % etc.

Advanced

  • Enable cookie(s) with: When enabled, cookies received in the HTTP response are stored and used in the next HTTP request.
  • with expires attribute: Selecting “ignored” ignores the expiry of the cookie. Selecting “in the past then delete” deletes the cookie whose life span has expired already.
  • Enable redirection upto depth: When enabled, redirection upto given depth is allowed for any url based on the location header received in HTTP response.
  • Use parent URL method for redirected URL: When enabled, parent method (GET/POST) is used in the redirected URL request else, GET method is used.

SSL

Secure Sockets Layer (SSL) is the standard security technology for establishing an encrypted link between a web server and a browser. This link ensures that all data passed between the web server and browsers remain private and integral.

Override SSL version for specified hosts: It overrides the existing default SSL version with the selected SSL version for specified hosts. To add a recorded host name and SSL version, click the Add button.

Select the recorded Host name and the SSL version from the list

Network Simulation

Network simulation is a technique where a program models the behavior of a network either by calculating the interaction between the different network entities (hosts/packets, etc.) using mathematical formulas, or actually capturing and playing back observations from a production network.

To enable network simulation, select the Network Simulation check box. There are other options also:

  • Enable Network Simulation: When enabled, user is simulated on different bandwidth and location.
  • Network Jitter: Jitter is defined as a variation in the delay of received packets. The sending side transmits packets in a continuous stream and spaces them eventually apart. Because of network congestion, improper queuing, or configuration errors, the delay between packets can vary instead of remaining constant.
    • Jitter (%)
      • Forward: It is any deviation in, or displacement of, the signal pulses in a high-frequency digital signal from server to client.
      • Backward: It is any deviation in, or displacement of, the signal pulses in a high-frequency digital signal from client to server.
  • Adverse Factor Latency:Adverse factor for latency and packet loss: It is the variable percentage for latency and packet loss from server to client and vice versa.

Sync Points

SyncPoint helps to see how a particular transaction of an application responds with loads. SyncPoint instructs VUsers to wait till all VUsers come to a certain point and proceed together.

When enabled, user can configure and apply the SyncPoint settings

Example: If user is testing banking application and wants to test performance of the system if 1000 users are transferring money from their account at a time. As, all users are doing their task differently, so SyncPoint before the transaction can be inserted. NetStorm first wait for 1000 users to get collected at a given point (before the debit transaction), and when 1000 users are collected then ALL USERS hits the transaction at the same time.

SyncPoint Types

  1. Standard SyncPoints: Standard SyncPoints are – Start of Transaction, Page and Script
  2. Custom SyncPoints: Custom SyncPoints are defined using C-API

SyncPoint Release Policy

Following attributes are to be provided for policy:

  • Provide release policy for SyncPoints: Sets the number of VUsers that must arrive at a Sync Point before the NetStorm releases the VUsers. For instance, suppose user is testing a scenario of 50 VUsers and want a particular operation to be executed simultaneously by 10 VUsers. Every time 10 VUsers arrive at the SyncPoint, they are released.
  • Provide GUI interface to release it prematurely: Immediate release for any SyncPoint at any state. For example, if any SyncPoint has Release Target VUsers as 1000 and SyncPoint collected VUsers are less than 1000 and user want to release the SyncPoint then user can use this feature.

By default, ENABLE_SYNC_POINT is disabled i.e. with 0 value. So, SyncPoint setting would not be visible.

To setup Sync Points, click the Sync Points tab under the Global Settings section on Scenario Setting window, the Sync Points window is displayed.

34

In case user selected the Enable SyncPoints Settings checkbox then all other features are displayed.

35

SyncPoint Attributes

Overall Timeout (Overall timeout for releasing sync point since first VUser arrived at sync point (OATO))

Specify the maximum timeout for running a Sync Point. It determines the maximum wait time (in seconds) for each Sync Point to come to release state from the time when first VUser reached the Sync Point. If the Sync Point is not released within the timeout period, then all the VUsers are released from the Sync Point.

Inter Arrival Timeout (Maximum time allowed between two VUsers to arrive at sync point (IATO))

Specify the maximum time between two VUsers arriving at SyncPoint. It determines the maximum wait time (in seconds) for each VUser to arrive at a Sync Point. After each VUser arrives at the Sync Point, it waits up to timeout for the next VUser to arrive. If the next VUser does not arrive within the timeout, then all the VUsers are released from the Sync Point. Each time a new VUser arrives, the timer is reset to zero.

SyncPoints Management

Group Name

Specify the group to define the SyncPoint. It can be ALL or any defined group name. Undefined group is ignored with warning.

Type

Specify the type of SyncPoint from the following options:

  • Start Transaction
  • Start Page
  • Start Script
  • Start Custom SyncPoint

Name

Specify the name of the object (transaction/page/script/custom) on which SyncPoint is applied.

Active/Inactive

Either enable (Yes) or disable (No) SyncPoint.

Participating VUsers Pct 

Specify how many percent (%) of users participates in SyncPoint. Default value is 10.

Release Target Vusers

Specify how many users should reach SyncPoint for releasing of SyncPoint. Since SyncPoint can be reached many times, user can provide this for each time using comma separated value. If it is same, then * can be used. But * should be used only once and that too at the end.

Examples:

  • 100
  • 200,300,*
  • 100,100,100,500,*

Scripts

Specify all the scripts that are part of SyncPoint. Multiple script names can be provided as comma separated. For example, if page login is part of two scripts CheckoutWithAccount and CheckoutWithCard, then it has both scripts. In case, group is not ALL, then it can be only one script as one group maps to one script only.

Advanced

There are following options under this section:

No of Virtual Machine(s): Virtual machines are child processes that are created by NetStorm parent process to distribute the load (users/sessions) among them.

There are two fields first one for count and second one to select either Machine or CPU.

(i) CPU: Number of Virtual Machines = Given count x Number of CPUs present in machine

(ii) Machine: Number of Virtual Machines = Given count

Example:

  • When user enter number of process as 2 and initiated on “MACHINE”

It takes 2 processes on machine. Here, number of processes made is 3, i.e. 2 processes    mentioned and 1 parent process. So that makes it 3 processes.

Formula: Total number of processes = Number of MACHINES + 1

  • When user enter number of process as 2 and initiated on “CPU”

If user provides this value (assume that the number of CPUs working are 2), then the number of processes that are generated would be 5, i.e. 2 processes per CPU and 1 parent process.

       Formula: Total number of processes = 2 * (Number of CPUs) + 1

Partition Setting: Using partition settings, user can specify duration of the partitions. It has two options:

  • Auto: Partition is created of duration 8 hours and it is synced with first partition.
  • Enabled: For manual partitioning, select ‘Enabled’ and provide the size of partition (duration). It has two options – to either sync or not sync on first switch.

Auto scale Cleanup: With this setting, a user can perform auto scale cleanup from UI. A new GDF is created on partition switch and a message is sent to get inactive node to NDC in specified time interval.

Copy Script in test run: When enabled, all the scripts are copied in the testrun. When “Exclude Dump” option is enabled, dump files are skipped while copying the script in test run.

Optimize Ethernet packet flow: User can optimize the Ethernet packet flow by using the following options. By default, 10 packets are exchanged between client and server.

  • Handshake Merge Ack: It reduces 1 packet, 9 packets will be exchanged between client and server (packet 3 and 4 will be merged into single packet). 
  • Data Merge Ack: It reduces 1 packet, 9 packets will be exchanged between client and server (packet 7 and 8 will be merged into single packet).
  • Close by Reset: It reduces 2 packets, 8 packets will be exchanged between client and server (packet 8, 9, and 10 will be merged into single packet).

User/Session distribution (%) over NVMs

There is another section for configuring user/session distribution over NVMs

It contains following options:

  • Distribution for maximum performance. A scenario group may be distributed over 1 or more NVMs: All Users/Sessions is distributed over 1 or more NVMs as per setting in Number of virtual Machine.
  • Distribution for maximum isolation among scripts. All scenario groups using same script is executed only by one NVM: Users/Sessions of same script is assigned to one NVM (Number of Virtual machines= different number of scripts used in scenario file).
  • User specifies NVM Id for each scenario group: Users/Sessions of same cluster id is assigned to one NVM (Number of Virtual Machines = different number of cluster IDs).

Advanced Setting

There are following options under this section:

Virtual User Thread Pool:

It is a collection of Virtual users where each VUser is considered as a thread. It is used for ramping up/ramping down users as specified in scenario when script is run in thread mode. User needs to provide following inputs

  • Initial number of threads:  Initial number of threads to be used in test
  • Increment By: How many threads to be increased when the initial gets exhausted
  • Max: Maximum number of threads to be used for a test
  • Thread stack size: Stack of each thread in KBs.

Maximum number of dynamic hosts: Maximum number of different hosts allowed to connect.

Set CavMon epoch year: Set the epoch year of CavMon. Epoch time is a system for describing instants in time, defined as the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970, not counting leap seconds.

Maximum Number of Concurrent Sessions: It defines number of concurrent sessions need to be executed. Only defined number of sessions will run parallel, rest will be in Blocked state.

Continue test on generator failures: By selecting this check box, the user can continue to run the test even if the generators fail to start. These are the following options under this section:

  • Stop test on controller if one (or more) of the following thresholds are breached
    • Minimum percentage of healthy generators needed for test to start: The user needs to provide the minimum percentage of generators that are required to run the test.
    • Minimum percentage of healthy generators during a test: This percentage of generators decides whether the controller continues or fails while the test is running.
  • Stop test on generator if one (or more) of the following thresholds are breached
    • Stop test on generator if connection with controller does not establish in: In case of the test triggered on generator, the user needs to provide the time (in seconds) for which the controller must wait for the generators to start test.
    • Stop test on generator if missed progress samples equal: The user needs to provide the number of progress samples. The test on generator stops if it is unable to send this number.

TCP Connection Port Selection Settings

Socket Binding is the process of allocating a Port number to a Socket to address a network within a system.

Mode for Port Selection: Selection of Port is either ‘Sequential’ or ‘Random’ mode. By default, it is Sequential Mode.

Number of retries on socket Bind Failure: Number of retries to be attempted once bind failed on socket. After retry attempt exhaustion, the provided action is implemented.

Action to be taken on failure: When retry attempts are exhausted, action is taken according to the option applied. Options can be ‘Enable’ or ‘Disable’.

  • Disable: Continue test, stop execution of page.
  • Enable: Stop test (as of previous behavior on bind fail).

Capture TCP Dump for Controller/Generators Communication troubleshooting: By selecting this check box, the user can capture TCP dump. These are the following options under this section:

  • Capture TCP Dump on Controller: Select this check box to capture the TCP dump on controller.
  • Capture TCP Dump on Generator: Select this check box to capture the TCP dump on generator.
    • Always: Select this option to always capture the TCP dump on generator.
    • On Connection failure from Generator to Controller: Select this option only when there is connection failure from generator to controller.
  • Data Connection / Controller Connection: Select these options to enable them.
  • TCP Dump Duration: Provide the time duration (in seconds) of TCP dump.

IO Vector Settings: IO Vector settings is the configuration setting of IO vector used by NetCloud for sending request on socket.

  • Initial Vector Size: Initial number of allocated IO vector. Cannot be less than 100.
  • Increment By: Increment size for reallocating IO vectors. Should be greater than 0 and less than max.
  • Maximum Size: Maximum numbers of IO vectors that can be allocated for whole test.