Overview

Global settings are those settings that are applied globally across all groups and are not group-specific. User can apply the following types of settings through the Global Settings window of Scenario GUI.

  • Logs and Reports
  • Transaction
  • HTTP
  • SSL
  • Network Simulation
  • Sync Points
  • Advanced

These features are explained in detail below:

Logs and Reports

To set up logs and reports, click the Logs and Reports tab under the Global Settings section on the Scenario Setting window, the Logs and Reports window is displayed.

The Logs and Reports setting window has the following options:

General Settings

  • Sample Interval: The time duration between two data samples is the sample interval. The default unit is second and the is now ‘Auto’ by default instead of 0.
  • Enable Debug Trace: A debug trace is required to investigate when a script is not being executed properly.  When enabled, the trace is available to debug following:
    • Issues with script execution
    • Bugs or defects

On the Dashboard GUI, go to View > Logs > Debug Trace Log in the left panel.

Debug Trace Log is displayed in a window that contains the details of all the user activities. This provides the detailed information on how the Script Test is running.

  • Enable save nvm file param value: When enabled, nvm file parameter value is saved. 
  • Exclude all failed Page, Transaction and Session from Metrics: When enabled, failed pages, failed transactions, and failed sessions information is excluded from the report.
  • Exclude all Stopped Page, Transaction and Session from Metrics: When enabled, stopped pages, failed transactions, and failed sessions information is excluded from the report

Reports

  • Auto Generate Stats Report:When enabled, a complete report is generated and saved. User can specify the format of report using report type which could be Tabular / Word / HTML:
    • Template: Report templates have specific graphs information. For example: TransactionReportsTemplate, SessionReportTemplate etc. Few predefined templates are bundled with the product however users can create their own template.
    • Report Set Name: Name of the report set
    • X-Axis Time Format: Time format has two options – ‘Elapsed’ or ‘Absolute’. For example – ‘Absolute’ will plot 16:00 hr to 17:00 hr on the X-Axis as opposed to 0 to 1 hr when ‘Elapsed’ is selected.
    • Time: This specifies the time or the phase for which the report needs to be generated.
      • Total Run
      • Phases
      • Specified Time Interval
  • Send test completion report at the end of the test:After the test completion, a summary is emailed to the recipients with following information:
    • TestRun number
    • Machine and Controller
    • Scenario
    • Start time of TestRun
    • Duration of TestRun
  • Send Report as an attachment: This allows the user to send the “auto-generated report” as an attachment along with the test completion summary.

Percentile Data

Percentile data provides the insight into the distribution of metrics values. For example, what percentage of offending transactions has contributed to a high average response time. When enabled, the percentile data is generated:

  • at the end of test,
  • at the end of Run Phase,
  • at every specified interval (default, 5 minutes).

A percentile is the value of a variable below which a certain percent of observations fall. So, the 20th percentile is the value (or score) below which 20 percent of the observations may be found. This setting is basically used to have the graphical representation of the percentile report in the GUI analysis window. User needs to select the Generate Percentile Data checkbox and specify the duration after which the percentile data is to be generated. On selecting this check box, the Map Percentile Definition File section is enabled with the following options. 

  • URL Response Time
  • Page Response Time
  • Session Response Time
  • Individual Transaction Response Time
  • All Transaction Response Time

User can change the percentile-associated graph from the drop down list according to the requirement.

Note: When the user uses the same PDF for more than one cases, NetStorm throws an error while starting the test.

Event Log

Events are generated by some predefined rules which indicate occurrences or happening of a particular incident. Event log is a collection of these incidents..

  • Event Log Levels: The predefined rules also specify the event log levels. These are: Debug, Information, Warning, Minor, Major, and Critical (debug being the lowest and critical being the highest). For example: If level is “Warning”, all events with level “Warning” or higher level are logged.
  • Filter Criteria: This is used to filter out events based on two criteria. First is ‘Do not filter’ (in this case, events are not filtered) and second is ‘Filter based on the events as per event definition file’ (in this case, filters on events are applied based on the event definition file). On selecting the second option, user needs to specify the event definition file from the drop-down list.
  • Use Log Manager for all events and to save data API: Enable Log manager starts NetStorm logging manager process. Each process sends events to log manager by using TCP/IP. Filtering happens at system level (default). Each process writes event in the event log file directly. Filtering happens at process level. User can specify the port or use the assigned port.
  • Use System Assigned Manager Port: When enabled, system assigned port is used. Else, user needs to specify a port.

Test Notification

  • Send test completion report at the end of the test: When enabled, the test completion report is emailed to specified recipients.

HTTP Response Code

Using this section, the user can apply HTTP response code in the scenario. This section displays system defined response code and user defined response code. The user can add and apply a user defined HTTP code.

Add User Defined Response Code

To add a response code:

  1. Select the Use User Defined Response Code option in the HTTP Response Code section.

2. Click Add. The Add User Defined Response Code window is displayed:

3. Enter the status code and status text.

4. Click button.

5. Click OK after you have added all the status codes.

Updating a User-Defined Response Code

To update an existing user-defined response code:

  1. Click . The Update User Defined Response Code window is displayed.
  2. Update the status text.

Note: If you update the status code, a new row of code will be added.

  1. Click button.
  2. Click OK after you have updated the status codes. A Override Confirmation message is displayed:

5. Click Yes.

Deleting a User-Defined Response Code

To delete an existing user-defined response code, select the required status code and click .

Advanced Settings

  • Enable Scenario group based test metric(s): When enabled, dashboard displays Test metrics data based on specific groups of the Scenario.
  • Enable Page Based Stats: When enabled, Real Browser User (RBU) page stats metrics are displayed on dashboard.
  • Enable online generation of drill down reports, Page dump reports: When enabled, drill down reports and page dump reports are generated in real-time (for current running Test Run or Sessions). A user can specify “maximum delay” (in seconds) to dump data in the database.
  • Enable Server IP address based metrics: It helps to get the resolved IPs at run time in web-dashboard from which the request is being served (Sent/Sec).

Transactions

Here user can perform transaction related settings. There are two sub-sections under this:

Pages As Transactions

This option is used to enable or disable the transaction. It contains following options:

  • Do not consider page as transaction: When selected, no transactions are created automatically for any page. However, user can create custom transactions within the scripts.
  • Consider each page as a transaction: When selected, the transactions are automatically created for each page. For example: For Home page, a transaction named “tx_Home” will be created.
  • Consider each page’s success or failure as distinct transactions: When selected, each page’s success or failure is considered as a transaction. For example: For Home page, with status such as Success and Fail, two transaction names will be created – “Home_Success” and “Home_Fail”.
  • Consider each page status as different transactions: When selected, status of each page is considered as a distinct transaction for that page. For example: For page status Home_Success, HomeConFail, and findflight_PartialHdr, transaction names will be “tx_success” and “tx_failuretype” where failure type is ConFail and PartialHeader.
  • Do not add ‘tx_’ prefix in transaction name: By default, a transaction name begins with a prefix “tx_”. When this option is selected, “tx_” prefix is not added.

Advanced

This sub-section contains some advanced settings for transactions.

  • Enable transaction cumulative metrics: When enabled, the transaction cumulative graphs are added to the test metrics and user can plot them onto dashboard. The most common metrics are – Requests Completed & Requests successful in HTTP requests metrics.
  • Enable transaction grouping in transaction detail: To enable transaction grouping in transaction details, select the transaction group from the drop-down list.

HTTP

To setup HTTP settings, click the HTTP tab under the Global Settings section on Scenario Setting window, the HTTP window is displayed. There are two sub-sections within this – URL Encoding and Advanced.

URL Encoding

URL encoding is a mechanism of encoding information in a URI.

  • Encode space in URL and query parameters of URL: User can specify how the space can be encoded in URL and query parameter of URL. It has value + or %20 (from drop down list).
  • Additional characters to be encoded in URL: It has value none or characters which user want to encode in URL other than space .e.g. {, }, |, % etc.
  • Additional characters to be encoded in query parameters of URL: It has value same or different characters which user want to encode in query parameter .e.g. {.}, |, % etc.

Advanced

  • Enable cookie(s) with: When enabled, cookies received in the HTTP response are stored and used in the next HTTP request.
    • Enable cookie for same domain & path: Only those cookies that are having the same domain and path in the next HTTP request(s), will be sent.

Suppose server set below cookies :

HTTP/1.1 200 OK

Set-Cookie: token1=value1; Domain=www.server1.com; Path=/index.html; Expires=Wed, 13 Jan 2021 22:23:01 GMT

Set-Cookie: token2=value2; Domain=www.server2.com; Path=/; Expires=Wed, 13 Jan 2021 20:50:02 GMT

Set-Cookie: token3=value3; Domain=www.server3.com; Path=/test/; Expires=Wed, 13 Jan 2021 20:50:02 GMT

And if next requests are for “https://www.server1.com/index.html” and “https://www.example.com/test/index.html” . As per selected mode it will validate domain and path of next requests and send matched cookies with next requests.

    GET /index.html HTTP/1.1
   Host: www.server1.com
   Cookie: token1=value1
   …….

   GET /test/index.html HTTP/1.1
   Host: www.example.com
   …….

    • Enable cookie for all domain & path: All the cookies irrespective of any domain and path in the next HTTP request(s), will be sent.

    GET /index.html HTTP/1.1
   Host: www.server1.com
   Cookie: token1=value1
   Cookie: token2=value2
   Cookie: token3=value3
   …….

  GET /test/index.html HTTP/1.1
   Host: www.example.com
   Cookie: token1=value1
   Cookie: token2=value2
   Cookie: token3=value3
   …….

    • Enable cookie for same path: All the cookies that are having the same path irrespective of the domain in the next HTTP request(s), will be sent.

    GET /index.html HTTP/1.1
   Host: www.server1.com
   Cookie: token1=value1
   Cookie: token2=value2
   …….

    GET /test/index.html HTTP/1.1
   Host: www.example.com
   Cookie: token2=value2
   Cookie: token3=value3
   …….

    • Enable cookie for same domain: All the cookies that are having the same domain irrespective of the path in the next HTTP request(s), will be sent.

    GET /index.html HTTP/1.1
   Host: www.server1.com
   Cookie: token1=value1
   …….
  
   GET /test/index.html HTTP/1.1
   Host: www.example.com
   …….

  • With expires attribute: Selecting “ignored” ignores the expiry of the cookie. Selecting “in the past then delete” deletes the cookie whose life span has expired already.
  • Enable redirection upto depth: When enabled, redirection upto given depth is allowed for any url based on the location header received in HTTP response.
  • Use parent URL method for redirected URL: When enabled, parent method (GET/POST) is used in the redirected URL request else, GET method is used.

SSL

Secure Sockets Layer (SSL) is the standard security technology for establishing an encrypted link between a web server and a browser. This link ensures that all data passed between the web server and browsers remain private and integral.

Override SSL version for specified hosts: It overrides the existing default SSL version with the selected SSL version for specified hosts. To add a recorded host name and SSL version, click the Add button.

Select the recorded Host name and the SSL version from the list

Network Simulation

Network simulation is a technique where a program models the behavior of a network either by calculating the interaction between the different network entities (hosts/packets, etc.) using mathematical formulas, or actually capturing and playing back observations from a production network.

To enable network simulation, select the Network Simulation check box. There are other options also:

  • Enable Network Simulation: When enabled, user is simulated on different bandwidth and location.
  • Network Jitter: Jitter is defined as a variation in the delay of received packets. The sending side transmits packets in a continuous stream and spaces them eventually apart. Because of network congestion, improper queuing, or configuration errors, the delay between packets can vary instead of remaining constant.
    • Jitter (%)
      • Forward: It is any deviation in, or displacement of, the signal pulses in a high-frequency digital signal from server to client.
      • Backward: It is any deviation in, or displacement of, the signal pulses in a high-frequency digital signal from client to server.
  • Adverse Factor Latency:Adverse factor for latency and packet loss: It is the variable percentage for latency and packet loss from server to client and vice versa.

Sync Points

SyncPoint helps to see how a particular transaction of an application responds with loads. SyncPoint instructs VUsers to wait till all VUsers come to a certain point and proceed together.

When enabled, user can configure and apply the SyncPoint settings

Example: If user is testing banking application and wants to test performance of the system if 1000 users are transferring money from their account at a time. As, all users are doing their task differently, so SyncPoint before the transaction can be inserted. NetStorm first wait for 1000 users to get collected at a given point (before the debit transaction), and when 1000 users are collected then ALL USERS hits the transaction at the same time.

Note: There is a separate link for Syncpoint User manual.

Advanced

There are following options under this section:

No of Virtual Machine(s): Virtual machines are child processes that are created by NetStorm parent process to distribute the load (users/sessions) among them.

There are two fields first one for count and second one to select either Machine or CPU.

(i) CPU: Number of Virtual Machines = Given count x Number of CPUs present in machine. Selecting the CPU option displays the Number of Users/Sessions field where you can specify the minimum number of users in a session. By default, 50 users are available per session, as highlighted below:

Note: If you change the value from CPU to MACHINE, the system will retain the last value configured with the CPU option, when MACHINE is again changed to CPU.

(ii) Machine: Number of Virtual Machines = Given count

Example:

  • When user enter number of process as 2 and initiated on “MACHINE”: It takes 2 processes on machine. Here, number of processes made is 3, i.e. 2 processes    mentioned and 1 parent process. So that makes it 3 processes.

Formula: Total number of processes = Number of MACHINES + 1

  • When user enter number of process as 2 and initiated on “CPU”: If user provides this value (assume that the number of CPUs working are 2), then the number of processes that are generated would be 5, i.e. 2 processes per CPU and 1 parent process.

       Formula: Total number of processes = 2 * (Number of CPUs) + 1

Partition Setting: Using partition settings, user can specify duration of the partitions. It has two options:

  • Auto: Partition is created of duration 8 hours and it is synced with first partition.
  • Enabled: For manual partitioning, select ‘Enabled’ and provide the size of partition (duration). It has two options – to either sync or not sync on first switch.

Auto scale Cleanup: With this setting, a user can perform auto scale cleanup from UI. A new GDF is created on partition switch and a message is sent to get inactive node to NDC in specified time interval.

Copy Script in test run: When enabled, all the scripts are copied in the testrun. When “Exclude Dump” option is enabled, dump files are skipped while copying the script in test run.

Optimize Ethernet packet flow: User can optimize the Ethernet packet flow by using the following options. By default, 10 packets are exchanged between client and server.

  • Handshake Merge Ack: It reduces 1 packet, 9 packets will be exchanged between client and server (packet 3 and 4 will be merged into single packet). 
  • Data Merge Ack: It reduces 1 packet, 9 packets will be exchanged between client and server (packet 7 and 8 will be merged into single packet).
  • Close by Reset: It reduces 2 packets, 8 packets will be exchanged between client and server (packet 8, 9, and 10 will be merged into single packet).

User/Session distribution (%) over NVMs

There is another section for configuring user/session distribution over NVMs

It contains following options:

  • Distribution for maximum performance. A scenario group may be distributed over 1 or more NVMs: All Users/Sessions is distributed over 1 or more NVMs as per setting in Number of virtual Machine.
  • Distribution for maximum isolation among scripts. All scenario groups using same script is executed only by one NVM: Users/Sessions of same script is assigned to one NVM (Number of Virtual machines= different number of scripts used in scenario file).
  • User specifies NVM Id for each scenario group: Users/Sessions of same cluster id is assigned to one NVM (Number of Virtual Machines = different number of cluster IDs).

Advanced Setting

There are following options under this section:

Virtual User Thread Pool: It is a collection of Virtual users where each VUser is considered as a thread. It is used for ramping up/ramping down users as specified in scenario when script is run in thread mode. User needs to provide following inputs

  • Initial number of threads:  Initial number of threads to be used in test
  • Increment By: How many threads to be increased when the initial gets exhausted
  • Max: Maximum number of threads to be used for a test
  • Thread stack size: Stack of each thread in KBs.

Maximum number of dynamic hosts: Maximum number of different hosts allowed to connect.

Set CavMon epoch year: Set the epoch year of CavMon. Epoch time is a system for describing instants in time, defined as the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970, not counting leap seconds.

Maximum Number of Concurrent Sessions: It defines number of concurrent sessions need to be executed. Only defined number of sessions will run parallel, rest will be in Blocked state.

TCP Connection Port Selection Settings: Socket Binding is the process of allocating a Port number to a Socket to address a network within a system.

Mode for Port Selection: Selection of Port is either ‘Sequential’ or ‘Random’ mode. By default, it is Sequential Mode.

Number of retries on socket Bind Failure: Number of retries to be attempted once bind failed on socket. After retry attempt exhaustion, the provided action is implemented.

Action to be taken on failure: When retry attempts are exhausted, action is taken according to the option applied. Options can be ‘Enable’ or ‘Disable’.

  • Disable: Continue test, stop execution of page.
  • Enable: Stop test (as of previous behavior on bind fail).