Test Plan Execution

After generating test cases and creating test plans, a user needs to execute these to check the correctness. This section describes in detail how a user can execute the test cases, view the dashboard, and perform other actions.

Execute

To execute a test plan, go to Test Plan Execution > Execute. The following window is displayed.

The Active test run table displays the details of the test runs that are ongoing. For the test runs that are completed, a user can view their test plans in the Test Plan Summary table.

Execute (Instantly)

In the Test Plan Summary table, the user can select the check boxes against the test plans that need to be executed. Click Execute. The ‘Execute’ dialog box is displayed.

Provide the following details, and then click Execute:

  • Name and Description: The user can edit the name and description of the test plan.
  • Profile: Select the profile from the drop-down list.
  • SelectorProfile: Select a ‘Selector Mapping profile’ from the drop-down list or enter the SelectorProfile name.
  • Cluster: Enable or disable the cluster option. When disabled, the test runs on the current machine only. When enabled, the test runs on the current machine as well as the machines that are active in cluster configuration.
  • DataSet: Select the DataSet from the drop-down list.
  • Release: Select the appropriate release from the drop-down list.
  • Build: Select the build number in which the test is running from the drop-down list.
  • Max Execution Time: Select the ‘Unlimited’ option or provide the time (in hours, minutes, and seconds) for which the test case should be executed.
  • Test Case Filter: The user can select any priority (from P1 to P5). In this case, test cases with the selected priorities are executed. Selecting ‘All’ executes all the test cases.

Note: By default, maximum 10 test cases are executed at once. For example, if there are a total of 23 test cases and the user executes all of them, the 10 test cases that have the higher priority / smaller script length (specified as per the ‘Profile’ selected by the user) are executed first. Then the next 10 test cases are executed. Then the remaining three test cases are executed.

Schedule

A user can also choose to execute the test cases at a different specified time. To do that, in the Test Plan Summary table, select the check boxes against the test plans that need to be executed, and then click Schedule. The ‘Scheduler’ window is displayed.

Provide the details in each of the following sections, and then click Schedule.

Description Info

Expand the ‘Description Info’ section.

Provide the description details of the test case, such as name, release, build, description, profile, and cluster (enable/disable).

Timing Info

Expand the ‘Timing Info’ section. The user has three options of time to specify the schedule – hourly, weekly, and monthly. First, specify the maximum execution time. Select the ‘Unlimited’ option or provide the time (in hours, minutes, and seconds) for which the test case should be executed. Then specify the test case filter. The user can select any priority (from P1 to P5). In this case, test cases with the selected priorities are executed. Selecting ‘All’ executes all the test cases. Now, select any of the time period tabs.

Hourly

Select the ‘Every’ option and provide the number of hours in the text box. The test case is scheduled to execute at this specified frequency. Or select the ‘Daily At’ option and provide the time. The test case is scheduled to execute daily at this specified time of the day.

In the ‘Schedule Expiry Date’ field, provide the date and time at which the execution must stop.

Weekly

Select any of the days, and then provide the start time at which the test case should execute.

In the ‘Schedule Expiry Date’ field, provide the date and time at which the execution must stop.

Monthly

Specify the day and month in which the test case should be executed. For example, if the user wants a test case to be executed on 7th day every 4 months, type 7 in the ‘Day’ field and 4 in the ‘of every Month(s)’ field.

Provide the start time at which the test case should execute.

In the ‘Schedule Expiry Date’ field, provide the date and time at which the execution must stop.

Enter Mailing Information

Expand the ‘Enter Mailing Information’ section.

A user has the option to receive the report of a test case execution through email after the execution is completed. To configure this, provide the email subject, receiver’s email address, and message in the corresponding text boxes. The user can provide more than one receiver’s email address separated by a semicolon (;).

Execution History

This section displays the test case executions that are completed. To view this section, go to Test Plan Execution > Execution History. The following window is displayed.

The table lists all the test cases whose executions are completed. The user can view the details, such as active status, TR number, TR name, execution time, release & build numbers, test plan name, total test cases, test cases that succeeded or failed during execution, duration, and the actions that can be performed.

Actions

A user can perform the following actions.

View Test Execution Report

Click  icon to view the test execution report.

At the top of this report, a user can view the test plan name, total number of test cases, execution start time, and duration. To view the report of any active test case, select an active test case from the drop-down list, and then click View. The user can also export this report in an excel format by clicking  icon.

This report displays the various stages of a test case execution, which are as follows:

  • Executed
  • Executing
  • Pending
  • Passed
  • Failed
  • Completed

Each stage is displayed in the form of a dial. Clicking a dial displays the test cases in that particular stage in a status table below the dial (as displayed in the image above).

The ‘status table’ displays the number of test cases in that particular stage, script name, test case number, result, flag, time, execution duration, error type, message, and replay. Fewer details are displayed in some stages.

The user can click  icon to view the sequence.

To open the script corresponding to the test case, click the  icon.

Click a script name to download its script.

Click  icon to view the aggregate failure report.

Click  icon to pause / resume the execution.

Click  icon to create a test plan. Provide the test plan name and description. Click Save / Save & Run.

Click  icon to export the result in excel format.

Click  icon to view the coverage report.

Click  icon to show/hide column filters.

Click  icon to stop the running test.

Click  icon to select columns which should be displayed in the table.

Select a test case, and then click  icon to open and add a tag.

Select a test case, and then click icon to archive the test case.

Select a test case, and then click  icon to delete the test case.

Select a test case, and then click  icon to add a comment.

While viewing failed test cases, the user can click a message in the ‘Message’ column to view the comparison between the baseline view and the current view. The baseline view is a pagedump which is captured when a test case (script) is recorded. The current view is the pagedump which is captured when a test case (script) is executed.

Execute Test Case

In the ‘Execution History’ window, click  icon to execute a test case. The ‘Execute’ dialog box is displayed.

Provide the release and build numbers. The user can click Execute Failed Cases button to execute all the failed test cases in that test plan. To execute all the test cases in that test plan, click the Execute All Cases button.

Stop / Resume Test

In the ‘Execution History’ window, click  icon to stop a running test. Click  icon to resume a stopped test.

Execution Profile

Execution profile guides NetTest how to execute a test plan. By creating a profile, a user pre-defines some configuration settings that are used when a test plan is executed. To view the profiles that are created, go to Test Plan Execution > Execution Profile. The following window is displayed.

To create a profile, click the Create button at the upper-right corner. The following window is displayed.

Provide the following details, and then click Save.

  • Name: Name of the profile.
  • Description: Provide a description for better understanding.
  • Parallel Execution: It is the number of test cases that will be executed concurrently while selecting this profile.
  • Execution Order: Choose from the following two options.
    • High to Low priority: Selecting this option executes the test cases first that have the higher priority.
    • Smaller to Longer Execution Time: Selecting this option sorts the test cases according to their execution time (based on the script length), and then executes them. The test cases with the smaller execution time are executed first.
  • Browser: A user can specify browser and their allocation in term of %. The user can either select the ‘Auto’ option or select the browsers from the drop-down list and provide its percentage. Default is ‘Auto’ in which NetTest automatically assigns the browser as per the NV data.
  • Host Mapping: This option helps to map the recorded hosts in scripts with other hosts.

Once a profile is created, it is displayed in the ‘Execution Profile’ window. To edit a profile, click  icon. The following window is displayed. Edit the fields as required, and then click Save.

The user can delete a profile by clicking the corresponding  icon.

Execution Dashboard

In NetTest dashboard, all the information related to tests are visualized with the help of graphs and tables. A user can view this information release wise and build wise. The user can view the collective status of test cases in terms of the following:

  • Passed test cases
  • Failed test cases
  • Not Tested test cases

The window has following sections – Dashboard, Build, Category, and Test Plan.

For every build of any release, the stats are displayed with the help of bar graph in the Build section. And for every release, stats are displayed with the help of pie chart based on same groups in the Dashboard section. Here, the user can view priority wise stats of test cases.

The user can choose to view the release number and the executed or overall status of the test cases from the drop-down lists at the upper-right corner of this window.

In the ‘Build’ section, a user can select the TimeWise option from the drop-down list to view the stats according to a specific time period instead of build numbers. After selecting the TimeWise option, select ‘Overall’ to view the stats for overall duration of that release. Or select ‘Specified’, and then provide the start and end time to view the stats for a time duration. In the ‘Bucket’ option, the user can select from Daily, Weekly, or Monthly. Click Apply to apply these filters and view the stats.

In the Category section, a user can view the different categories and the stats as per those categories.

In the Test Plan section, the user can view the different test plan names and the stats as per those test plans.

In these sections, clicking any category or test plan name loads the dashboard as per that category or test plan.

To view the Coverage Report, click  icon.

Scheduled Data

Go to Test Plan Execution > Scheduled Data to view the scheduled data. This section displays the test cases that a user chooses to execute at a different specified time (see the Schedule section).

The table displays the details of the scheduled data, such as name, build, release, schedule, expiry time, description, and action. The user can delete a scheduled data by clicking the corresponding  icon.

Coverage

Go to Test Plan Execution > Coverage to view the coverage report. This report contains the overall as well as page wise report. Click (>) icon to scroll and view the page wise reports horizontally. Clicking any tab loads the corresponding report.

By clicking the ‘Overall’ tab, a user can view the coverage report of all the test cases. When the user clicks a page wise report, the corresponding report is displayed.

Each tab has the report name, percentage, and covered/not covered test cases (in parenthesis) written in it. For example, in the above image in the tab for ‘Address’ page report, the details written in it are – ‘Address’, ‘75.0%’, and ‘(3/4)’. When the user clicks this tab, the corresponding report is displayed. The user can also view at the right side of the window that out of total four test cases, three are covered (in green color) and one is not covered (in red color).

Click  icon to expand the view to display all the page wise reports at once.