1 - Overview

RealLoad: Unique Key Features

What is RealLoad?

RealLoad is an Enterprise tool to perform next generation Load Testing, Synthetic Monitoring and Regression Testing.

At the core of the RealLoad product is a universal measurement interface that supports to capture data of any kind.

Due to the universal architecture, a ‘Test’ created with RealLoad can be run as Load Test, as Synthetic Monitoring Test, and as Regression Test.

The product is highly scalable and can be used to run tests from one simulated user up to an almost unlimited strength with millions of concurrent users - and this for Load Tests, for Synthetic Monitoring Tests, and for Regression Tests.

The executed tests can be either generated automatically by using the wizards, or alternatively programmed by hand, with all libraries and interfaces being fully documented in detail.

Regardless of what type of test you are running, all data reported to the universal interface during a test are displayed directly in real time in form of statistics and charts.

RealLoad itself is written in Java - but can execute tests written in any programming language, since the universal measurement interface is file-based and supports any programming language that is capable to write data into a file.

2 - Getting Started

Getting started with RealLoad

This section of the documentation will walk you through the first steps in order to getting started with the RealLoad product.

Follow the next sections of the document and you should be able to run your first basic test script using the SaaS Evaluation scenario within 20 minutes.

Let us know if encounter any issues while getting started, as that will help us updating this documentation to make it as clear and user friendly as possible. Please email us at support@realload.com

2.1 - Deployment Types & Architecture

Chose the right deployment type for you.

First of all you’ll have to decide which RealLoad deployment type better suits your needs. These guidelines should assist you in making an informed decision.

Architecture

There a few components that make up the RealLoad solution. Most components are available under an SaaS model which doesn’t require any infrastructure commitment from your side.

If required by security or other constraints, some or all components can be deployed on your own infrastructure. Please reach out to us to discuss requirements if you’re planning to go down this road.

The key components that make up RealLoad’s solution are:

The Portal Server

  • Exposes the main GUI to end users.
  • All tasks to maintain Load Testing or Synthetic Monitoring scripts are performed here.
  • Load Tests are can be triggered via the Portal.
  • The Synthetic Monitoring dashboard is visible here and shows the current state of configured tests.

Measuring Agent

  • Load Test and Synthetic Monitoring scripts are executed from Measuring Agents.
  • For Synthetic Monitoring purposes, you can use shared or dedicated agents to execute your tests from.
  • The Measuring Agent needs to be reachable from RealLoad’s portal server and needs to be able to reach the servers to be load tested or monitored.
  • You can have multiple Measuring Agents, for example to execute Synthetic Monitoring tests from different locations.

There are 2 options for your to run AWS cloud based Measuring Agents:

  • Using our Cloud Credits: The relevant Virtual Machines are spun up from our portal under our own AWS account and you pay a usage fee using Cloud Credits you’ve previously purchased from us.
  • By starting pre-configured AWS AMIs under your own AWS account. You’ll be charged directly for AWS related costs.

Cluster Controller (Optional)

  • If your Load Testing requirements exceed the load generating capacity of a single Measuring Agent, you can combine multiple agents in a cluster using a Cluster Controller.
  • Like measuring agents, Cluster Controllers can be hosted in the cloud or on your own infrastructure, as per your requirements.

Proxy Recorder (Optional)

  • You can use the Proxy Recorder component in order to develop test scripts by recording HTTP traffic while it transits through it.
  • The Proxy Recorder can be run in the cloud, as a service on your own infrastructure or directly on your Desktop using the Desktop Companion.

Typical deployment scenarios

Load Testing using Cloud Credits

The RealLoad portal gives you the ability to launch load generators (Measuring Agents) in the AWS cloud with a click of button and the execute a load test.

Key features and requirements are:

  • You’re not required to have an AWS account, as the Measuring Agent is run under RealLoad’s AWS account.
  • You pay for the AWS and RealLoad licensing costs using previously purchased Cloud Credits.
  • The application to be load tested needs to be reachable from the Measuring Agent’s public IP address, assigned by AWS at launch.

Load Testing using Cloud Credits

Load Testing by running the Measuring Agent behind perimeter firewall

Using Cloud Credits is not an option if the servers to be performance tested are not exposed to the internet. In this scenario, the Measuring Agent will need to be deployed behind perimeter firewalls as follows:

  • In AWS: You can launch our pre-configured AMIs under your AWS account and attach them to your VPCs.
  • If preferred, Measuring Agent instances can be launched using our convenient Desktop Companion application.
  • If running on non-AWS infrastructure: Install the Measuring Agent on a supported Linux Distribution.

Key features and requirements are:

  • The Measuring Agent needs to be reachable from the internet based Portal Server. You’ll need to configure firewall and NAT rules on your perimeter firewall to allow communication.
  • You’ll need to update the Measuring Agent software from time to time, to stay compatible with updates on the Portal Server.
  • You’ll need to purchase a Load Testing license to suit your load requirements.

Load Testing using managed agent

Synthetic Monitoring of Internet exposed services

Typically Synthetic monitoring is use to monitor services that are exposed to the internet. RealLoad uses Measuring Agents to implement the monitoring functionality.

We recommend executing Synthetic Monitoring tests from at least 2 Measuring Agents, to cater for transitory local disruptions (like network issues or agent software updates) which might trigger false positive alerts.

There are two types of agents you can use for monitoring purposes:

  • Shared Agents: These are Monitoring Agents shared by multiple customers. Using these agents is included in the basic cost of the monitoring service.
  • Dedicated Agents: If you need to execute monitoring from a specific location or you don’t want to use a shared agent, we can setup a dedicated agent for you. This will incur additional costs, plz reach out to us to discuss.

Key features and requirements are:

  • The application or service to be monitored needs to be reachable from the internet.
  • You’ll need to subscribe to a Synthetic Monitoring plan. Reach out to us for pricing.

Synthetic Monitoring using Shared Agents

Synthetic Monitoring of services running behind perimeters firewalls

Should you have the need to monitor services within your perimeter, this can be arranged by deploying Measuring Agents within your networks.

Key features and requirements are:

  • You’ll need to deploy two measuring agents behind the firewall perimeter. For AWS deployments your can use our existing AMIs, for other infrastructure install the Measuring Agent on a supported Linux Distribution.
  • You’ll need to update the Measuring Agent software from time to time, to stay compatible with updates on the Portal Server.
  • The Measuring Agents need to be reachable from the internet based Portal Server. You’ll need to configure firewall and NAT rules on your perimeter firewall to allow communication.
  • You’ll need to subscribe to a Synthetic Monitoring plan. Reach out to us for pricing.

Synthetic Monitoring using Internal Agents

2.2 - Portal Sign Up and Login

To get started you’ll need to setup and account for yourself at the RealLoad portal. See here how…

If you already have an account, you can login at https://portal.realload.com

Signing Up - Free Account Features

In order to login to the portal you’ll first have to setup an account. Go to the portal URL and click on the Sign Up button or go this URL: https://portal.realload.com/SignUp

You’ll need to provide:

  • Email address.
  • Mobile phone number.
  • No credit card required, it’s completely free.

Account deletion

To have your account deleted, please send an email to support@realload.com. We’ll then reach out to you to confirm deletion.

Your account will be immediately disabled and all your data will be removed within 90 days, including backups.

Sign Up Process

Step 1 of 4

Provide your details, including email and mobile number. All data you enter in this form are only required to verify your identity and will not be shared with any third parties.

Step 2 of 4

Validate your email address.

Step 3 of 4

Validate your mobile number

Step 4 of 4

Configure your nickname and password

Welcome Page with Examples

Once you signed up, you will be automatically loggedIn and the welcome page will be displayed. Three examples are shown which are also available online:

2.3 - Execute a Simple HTTP/S Test

This section gets you kickstarted with a simple HTTP/S test.

This example shows how a Simple HTTP/S Test can be defined as a RealLoad ‘Test’ and executed as both a Load Test Job and a Monitoring Job.

Pre-requisites

To configure and execute this simple test you’ll need:

  • Access to the Real Load portal. If you haven’t signed up, do so first: Sign Up
  • Approx. 20..30 minutes of your time
  • A cup of tea or coffee

1. Invoke the HTTP Test Wizard and Create a New Session

Invoke HTTP Test Wizard and Create New Session

Enter the Headline that briefly describes what the test does. Enter Session Headline

2. Save the (empty) Session

It’s best to save the empty session now. To do this, you must select a Resource Set of a Project. Alternatively, you can also create a new ‘Project’ and/or ‘Resource Set’. ‘Resource Sets’ are something like sub-directories of a Project, which contain all the files necessary to define and execute a test. Click ‘Save Session’

Enter the file name for the session, e.g. HttpSession_TestDownloadServer.json and save the session. Enter File Name of Session

In this example, a new Project named ‘HTTP Tests’ is created which also contains a new Resource Set named ‘Download Server’. Create new Project and Resource Set

Result: Empty saved Session

3. Add URLs to the Session

Click on the ‘Add’ dropdown and select ‘URL’. Dropdown ‘Add’ - Select ‘URL’

Enter the URL and click on the ‘+’ icon at ‘Verify HTTP Response’. Enter URL

At ‘Verify HTTP Status Code’ select ‘200 OK’ and then click the ‘Add URL’ button. Select ‘200 OK’ and add URL

You can then add additional URLs to the session. In this example it looks like this: Session with 3 URLs

4. Debug the Session

Before converting the session into a RealLoad ‘Test’ you should Debug the Session first. This way you can check whether all URL calls are working correctly. Click ‘Debug Session’. Invoke the Session Debugger

This invokes a powerful Remote Debugger that runs always on a Measuring Agent. As shown at the top right of the following image you can also select an alternative Measuring Agent on which the debugger will run. In the Remote Debugger you can also make changes to the HTTP/S session, which take effect immediately, which means that the session is automatically synchronized between the debugger and the HTTP Test Wizard.

In the debugger, click Next Step until all URLs have been executed successfully and the end of the session has been reached - or until an error occurs. If an error occurs, exit the debugger, then repair the URL (= HTTP/S request), and then invoke the debugger again. Debug the Session

After debugging is successful, return to the HTTP Test Wizard. Return to HTTP Test Wizard

5. Save the (debugged) Session

Save the debugged Session

6. Generate and Compile the Code, and Define the RealLoad ‘Test’

Click ‘Generate Code’

Generate and Compile the Code

After the source code is compiled, click ‘Define New Test’. Click ‘Define New Test’

Enter the Test Description, and click ‘Define New Test’. Enter Test Description

The RealLoad ‘Test’ is now defined. From here you can now create both a Load Test Job and a Monitoring Job. Result: RealLoad ‘Test’ Defined

Note: In case you (later) modify the session in the HTTP Test Wizard, you will have to generate and compile the source code again.

7. Define and Execute a Load Test Job

In the RealLoad ‘Test’ click ‘Define Test Job’. Click ‘Define Test Job’

In next menu click ‘Continue’ … Click ‘Continue’

… And select the Measuring Agent on which the Load Test Job will be executed, then click ‘Define Load Test Job’. Click ‘Define Load Test Job’

The Load Test Job is now defined. Click ‘Start Test Job’. Click ‘Start Test Job’

Enter the settings of the Load Test Job here and then click ‘Start Test Job’. We recommend the following settings for the first load test run:

  1. Number of Users: 5
  2. Max. Test Duration (Seconds): 60
  3. Max. Loops per User: [unlimited]
  4. Loop Iteration Delay (Milliseconds): 1000
  5. Ramp Up Time (Seconds): 10

Enter Settings and Click ‘Start Test Job’

After a few seconds the Load Test Job is in the state ‘Running’. Click ‘Monitor Jobs’. Job is ‘Running’, Click ‘Monitor Jobs’

The real-time data of the running Load Test Job is now displayed. Real-Time Data of running Load Test Job

Click on ‘Analyze Result’ after the Load Test Job is completed. Load Test Job completed, Click ‘Analyze Result’

The Load Test Job result is then displayed. Load Test Job Result

To view the Job Output Files navigate to ‘Load Test Jobs’ and click in the dropdown of the Job on ‘Job Log Files’. Click ‘Job Log Files’

Select the file users.out which contains the Log Output of the Job. Select File ‘users.out’

If you would like to run the Load Test Job again with the same or changed settings, click on ‘Clone Job’ in the Job dropdown and then start the cloned job. Clone Load Test Job

8. Define a Monitoring Job

If this is your first Monitoring Job, you must first create a Monitoring Group. Navigate to Monitoring, click the ‘Configuration’ tab and then click ‘Add Monitoring Group’. There is also a help available for configuring the Synthetic Monitoring. Navigate to Monitoring Configuration

Enter the ‘Group Title’ and select at least one ‘Measuring Agent’ on which the Monitoring Job(s) will be executed. Then click ‘Add Monitoring Group’. Add Monitoring Group

In the ‘Monitoring Group’ click ‘Monitoring Jobs’ and then click ‘Add Monitoring Job’. Add Monitoring Job

Select the ‘Test’ of the Monitoring Job. Select ‘Test’ of Monitoring Job

Click ‘Define Monitoring Job’. Define Monitoring Job

Enable the execution of the Monitoring Group and navigate to ‘Dashboard’. Enable Monitoring Group and Navigate to Dashboard

The Monitoring Job is now defined and will be executed periodically. For additional help configuring monitoring (e.g. adding ‘Alert Devices’), see Monitoring Help. Monitoring Job is Defined

Tip: If you modify the RealLoad ‘Test’, the corresponding Monitoring Job is not automatically updated to avoid unexpected corruption of the Monitoring Job. After you have verified the modified ‘Test’ by running a small Load Test Job, you must manually update the Monitoring Job. Update the Monitoring Job

Done, congrats, you’ve run your first test with Real Load.

9. Conclusion and Prospects

As you have seen, a RealLoad Test can be run as both a Load Test Job and a Monitoring Job.

This example was deliberately kept simple so that you can become familiar with the RealLoad product. To create a test with a web browser session across multiple web pages, you can use a Remote Proxy Recorder to record the test case and then convert the recording into an HTTP Test Wizard session.

Additionally, a Test Job Template can also be defined from any ‘Load Test Job’, which can then be part of a Test Suite. This means that you can add multiple ‘Test Job Templates’ to a Test Suite and execute them in a single run as a Regression Test.

To learn more we suggest heading to the User Guide section where you’ll find detailed documentation in relation to the steps outlined in this document.

2.4 - Convert a Selenium IDE Test to a RealLoad Test

This example shows how to convert a Selenium IDE Test to RealLoad Test.

This example shows how a Selenium IDE Test - which was recorded with a Chrome or Firefox web browser - can be converted into a RealLoad ‘Test’. The converted RealLoad ‘Test’ can then be executed both as a Load Test Job and as a Monitoring Job. In addition, the converted test can also be part of a Test Suite for performing Regression Tests.

In most cases, the conversion can be done easily with just a few mouse clicks - without the need for any special settings.

At the end of each executed test run (of the converted RealLoad test), a final screenshot of the simulated web browser session is taken for each virtual user so that you can check whether the web browser session of the virtual user has been fully executed.

Tip: If possible, always use the Chrome web browser for Selenium IDE recording, as Chrome sessions are simulated much faster (= more realistic) than Firefox sessions.

Pre-requisites

To configure and execute this test you’ll need:

  • Access to the Real Load portal. If you haven’t signed up, do so first: Sign Up
  • Approx. 20..30 minutes of your time
  • A cup of tea or coffee

1. Export the Selenium IDE Test as JUnit Test

Export the your recorded Selenium IDE Test as Java JUnit file and save the file to any folder on your local device. Note that the 3-dot dropdown icon may initially be hidden and may only appear when you are near to the Selenium ‘Test Case’ area. Do not choose any export option. Export Java JUnit File

Export Java JUnit File

Export Java JUnit File

2. Upload the exported Selenium JUnit File

At Projects menu click on the ‘Developer Tools’ dropdown and select ‘Convert Selenium IDE Test to Real Load Test’. Select ‘Convert Selenium IDE Test to Real Load Test’

Then upload the file by ‘Upload File’ or by dragging and dropping the file into the dashed area. Upload the File

3. Convert the Uploaded JUnit File

Click on the ‘Convert’ button after uploading the file. The converted Java code is then displayed. Then click on the ‘Save & Compile’ button. Convert Selenium IDE Test

You have to save the converted code in a Resource Set of a Project. You can also first create a new Project with a new Resource Set. Save Converted JUnit File

After saving the converted code, you will be automatically redirected to the Compile Java File dialog.

4. Compile the Converted JUnit File

The JAR files required to compile are already preselected, i.e. you need only click the ‘Compile’ button. Compile Converted JUnit File

After successful compilation, click ‘Define or Update Selenium by JUnit Test’. Define or Update Selenium by JUnit Test

You will then be automatically redirected to the Define JUnit & Selenium Test dialog.

5. Define the Selenium RealLoad ‘Test’

Select the JUnit @Test method (in this case only one will be shown) and click ‘Define Test’. Define Selenium RealLoad ‘Test’

In an intermediate step you can enter an (optional) test description. Enter Test Description

Now the Selenium RealLoad ‘Test’ is defined from which you can define a Load Test Job and/or a Monitoring Job. Result: Defined RealLoad ‘Test’

6. Note about Measuring Agents

Please note that not all Measuring Agents support the execution of Selenium jobs. You can ‘ping’ a Measuring Agent at application level to find out whether Selenium jobs are supported: Ping Measuring Agent

Example 1: Measuring Agent ‘Demo Agent 1 (CH)’ does not support Selenium jobs Ping ‘Demo Agent 1 (CH)’

Example 2: Measuring Agent ‘Demo Agent 2B (CH)’ supports Selenium jobs Ping ‘Demo Agent 2B (CH)’

3 - Release Notes

RealLoad Release Notes

4.8.34 | 2024-02-20

  • Synthetic Monitoring API Documentation
    • Full Synthetic Monitoring API documentation is now available to support developers to implement their own version of a real-time dashboard for Synthetic Monitoring that can be displayed on their companies screen.
    • The powerful API also supports receiving data for drawing charts and sparklines. In addition, the API documentation contains helpful information about implementation and runtime behavior.
    • See https://kb.realload.com/docs/api/remoteusermonitoringapi/

4.8.31 | 2024-01-18

4.8.30 | 2023-12-27

  • New Features:
    • Synthetic Monitoring: Added configurable performance alert thresholds for the measured time per loop (= average passed session time). Two performance alert threshold values (in milliseconds) can be configured per monitoring job. One to trigger a warning and another to trigger an error. If one of these thresholds is reached or exceeded, a performance warning or performance error is added to the test result, which has the same effect as measuring a “normal” error with a severity of “Warning” or “Error”.
  • Portal Server version 4.8.30 requires now Measuring Agent and Cluster Controller version 4.5.11

4.8.29 | 2023-12-17

  • New Features:
    • Synthetic Monitoring: Added real-time Sparklines (microcharts) to the Real Time Dashboard. The sparklines with a time range of the last 24 hours show the measured availability for monitoring groups, and for monitoring jobs the measured availability and the measured response times (milliseconds per loop). The displayed time range of the sparklines can be zoomed by clicking on the chart while holding down the mouse button and panning to the right or left in the chart. After releasing the mouse button, the zoom of the corresponding area appears. If necessary, you can then zoom in again to narrow down the area further.

4.8.28 | 2023-12-14

  • New Features:
    • Slightly redesigned Synthetic Monitoring user interface.
      • Real-Time Dashboard: The dashboard supports now to show only monitoring groups or only monitoring jobs - or both of them (as before).
      • Real-Time Dashboard: The displayed availability of Monitoring Groups and Jobs for the last 24 hours is now a hyperlink that, when clicked, displays the corresponding statistical chart.
      • All Synthetic Monitoring menus and popups have been revised to make them more descriptive and user-friendly.
  • Bug Fixes:
    • Synthetic Monitoring: Webhook alert calls which return a HTTP status code 202 (instead of 200) are no longer logged as failed.

4.8.27 | 2023-12-07

  • New Features:
    • Synthetic Monitoring: Support for delayed alert notifications: Alert notifications can now be triggered immediately (as before) or (new) only after one or several repeats. This means that short-period errors and downtimes can be excluded from alert notifications. This does not affect the measured statistics such as availability and detected errors. Delayed alert notification settings can be configured separately for each “Alert Group”, whereby multiple alerts groups can be assigned to one or multiple “Monitoring Groups” and “Monitoring Jobs”.

4.8.26 | 2023-11-28

  • New Features:
    • Synthetic Monitoring: Added support for ‘Monitoring Downtimes’. A Monitoring Downtime is a window of time during which synthetic monitoring is temporarily disabled. Multiple monitoring downtimes can be defined for all monitoring groups, or for specific monitoring groups and also for specific monitoring jobs. In addition, a monitoring downtime can be repeated daily, weekly or monthly.

4.8.24 | 2023-11-12

With this new release we have reached the point where a general load testing tool has been crossed with a monitoring tool and also with a regression testing tool - with both the monitoring tool and the regression testing tool are capable of executing test jobs with hundreds or thousands of concurrent users.

  • New Features:
    • Real Load Test Suites
      • Test suites are the grouping of multiple test job templates and are executed as a single “run” with different test scenarios.
      • Due to the universal architecture, test jobs of any kind can be part of a test suite (HTTP Test Wizard Jobs, JUnit Jobs, Selenium Jobs ..).
      • The results of the test suite “runs” are archived and can be compared with each other.
      • In addition, the result of a particular test job of a test suite run can be compared with the result of the same test job executed by other test suite runs.
      • All test jobs of a test suite support the powerful ability to be executed with hundreds or thousands of concurrent users and can even executed on a cluster of measuring agents.
      • The test jobs of a test suite are grouped into “execution groups”, where the test jobs of an execution group can be executed in sequential or parallel order.
      • The execution groups can be nested, whereby a separate measuring agent or measuring agent cluster can be defined for each execution group.
      • A “run” of a test suite can be triggered either interactively or via the Remote User API.
    • The following functions have been added to Remote User API:
      • getTestExecutionSuites
      • getCurrentlyRunningTestExecutionSuiteIds
      • startTestExecutionSuiteTestRun
      • getTestExecutionSuiteTestRunResultDetail
      • getTestExecutionSuiteTestRunTestJobOutputFileContent
      • deleteTestExecutionSuiteTestRun
    • Tests: Delete Test: A warning message will now be displayed if the test is referenced by a Monitoring Job or by a Test Suite.
    • Tests: Option “Delete All Unreferenced Tests” added.
    • Load Test Jobs: The display of load test jobs can now filtered by Project and Resource Set.
  • Bug Fixes:
    • Starting a load test job failed in rare cases due to a synchronization problem.
  • Portal Server version 4.8.24 requires now Measuring Agent and Cluster Controller version 4.5.10

4.8.23 | 2023-09-18

  • New Features:
    • Added support for ‘Test Job Templates’.
    • The following functions have been added to Remote User API:
      • getTestjobTemplates
      • defineNewTestjobFromTemplate
      • submitTestjob
      • makeTestjobReadyToRun
      • startTestjob
      • getMeasuringAgentTestjobs
      • getTestjobOutDirectoryFilesInfo
      • getTestjobOutDirectoryFile
      • saveTestjobOutDirectoryFileToProjectTree
      • deleteTestjob

4.8.21 | 2023-09-13

  • New Features:
    • Improved integration of JUnit Test with the Real Load architecture. As already described earlier, any JAR file that contains JUnit test can be uploaded to the portal server and executed directly as a test, i.e. without conversion. However, if you are programming a JUnit test from scratch, you can benefit from the additional enhanced features of the Real Load architecture. These additional features include better log output of the executed JUnit test, support for “User Input Fields” (configurable test input parameters), and the measurement and display of additional statistics obtained during test execution.
    • After logging into the portal server, a new project called “Developer Examples” will appear, which contains an example of a JUnit test and shows how a JUnit test can be fully integrated into the Real Load architecture.

4.8.20 | 2023-09-10

  • New Features:
    • Added support for executing Selenium IDE tests with Google Chrome.
    • When a Measuring Agent is pinged at application level, it is displayed whether the Measuring Agent supports the execution of Selenium tests.
  • Changes:
    • The Selenium version sent to Measuring Agents is now 4.12.1 instead of 4.10.0. However, this only applies to tests that are newly created.
  • Portal Server version 4.8.20 requires now Measuring Agent version 4.5.9 when Selenium IDE tests with Google Chrome are executed.

4.8.19 | 2023-09-04

  • New Features:
    • Added support for executing Selenium IDE tests. Selenium IDE tests recorded on your local device can be executed as monitoring job and as load test job - by exporting them in the Selenium IDE as ‘Java JUnit’ test and then importing them into the Real Load portal. The imported tests support also ‘User Input Fields’ (configurable options when starting a test) and take a final screenshot of the (virtual) Web browser display just at the end of each test run - which can then be viewed in the Real Load portal. Currently only tests recorded with Firefox are supported. Support for tests recorded with Chrome and Edge will be added soon.
    • Added support for executing JUnit 4 tests. JAR files which contain ready-to-use JUnit @Test(s) can be uploaded to the Real Load portal and run directly as a monitoring job and as a load test job. The benefit of running JUnit tests on the Real Load infrastructure is firstly that they can be run as a “monitoring job” and secondly that they can be run as a “load test job” with many hundreds of JUnit tests, running concurrently in parallel (or even with many thousands of JUnit tests running in parallel, through the use of a cluster of Real Load Measurement Agents). However, this is usually only useful for JUnit tests which are testing or stressing network services of any kind.
  • Portal Server version 4.8.19 requires now Measuring Agent and Cluster Controller version 4.5.8

4.8.17 | 2023-07-29

  • New Features:
    • Synthetic Monitoring: Support of ‘Pagerduty V2’ and ‘Slack’ Webhook alert devices added.
    • Test Results: The selected test result can now exported to a PDF and to a MS Word document.
    • ‘HTTPS Server Certificate Tool’ added.
  • Changes:
    • HTTP Test Wizard: The URL HTTP timeout can now specified in milliseconds (instead of seconds).

4.8.14 | 2023-06-29

  • New Features:
    • Revised Sign-Up: When sign-up, two shared Measuring Agents are now assigned to the new user account.
    • For customers (user accounts) who do not operate their own HTTP/S Remote Proxy Recorder, a temporary HTTP/S Remote Proxy Recorder from a pool is now assigned each time they sign-in.
    • Synthetic Monitoring: A warning sign is now displayed for Monitoring Groups for which no Measuring Agent have been defined.
  • Portal Server version 4.8.14 requires now HTTP/S Remote Proxy Recorder version 1.1.3, and Measuring Agent and Cluster Controller version 4.5.7

4.8.13 | 2023-06-20

  • New Features:
    • HTTP Test Wizard ‘URL Plugins’ can now be assigned in one step to multiple or all URLs of a HTTP/S session.
  • Changes:
    • Sent SMS messages now have the sender ID ‘RealLoad’ instead of a (technical) phone number.
  • Bug Fixes:
    • Synthetic Monitoring: No alert was send at monitoring group level when an execution timeout occurred.

4.8.12 | 2023-06-07

  • New Features:
    • Measuring Agents which are shared with other Portal Server user accounts (other customers) are now marked with [P] = ‘partial shared’ or [S] = ‘fully shared’ in the Measuring Agents menu. Measuring Agents that are operated exclusively for a specific customer do not have such a mark.
    • The ‘My Account -> Price Plan’ menu shows now the number of sent SMS monitoring alerts of the current month, and also of the last 12 months.
    • Price Plan limits for Synthetic Monitoring have been implemented and are now shown in the ‘My Account -> Price Plan’ menu.
  • Changes:
    • The primary Mobile Number for main user accounts and team member accounts no longer needs to be unique across all Real Load accounts.
    • Measuring Agent Housekeeping: The data of executed jobs older than 30 days are now deleted on Measuring Agents (monitoring jobs + load test jobs).
  • Bug Fixes:
    • Team member accounts with read/only privileges could make changes to the Synthetic Monitoring configuration.

4.8.10 | 2023-05-11

  • New Features:
    • HTTP Test Wizard
      • ‘Conditional Jump’ session element added. For example, this new session element allows a webpage to be called in a loop until the received content contains a certain value, or until a maximum number of conditional jumps is exceeded. Adding multiple conditional jumps per session is supported, with the ability to jump backwards or forwards. The condition of whether to jump and where to jump can be determined by variable values.
      • Java JVM warm-up before starting a test. Tests that run with only one or a few users and loops are now more accurately measuring URL response times by performing a Java JVM warm-up before starting the test.
      • The log-data of measured errors have been improved (less but more meaningful data).
    • Projects Menu
      • The preview of the test results shows now additionally the ‘Avg. Passed Session Time’.
    • Load Test Jobs Menu
      • A new delete option ‘All Completed Jobs - Except the Last Ones’ has been added.
    • Test Results Menu
      • The ‘Avg. Passed Session Time’ is now show in the Summary Statistic.
      • Searching for Text values in an Error Detail ‘Log’ and in an Error Detail ‘Context’ is now supported.
  • Changes:
    • HTTP Test Wizard
      • Errors reported at URL level are no longer additional reported at Measurement Group level.
    • Test Execution
      • The number of stored errors is now limited to 2000 fully stored errors plus 8000 partial stored errors (partial stored: without error log and without error context).
  • Portal Server version 4.8.10 requires now Measuring Agent and Cluster Controller version 4.5.6
  • To take advantage of the new features you have to re-generate and re-compile the Java source code in the HTTP Test Wizard.

4.8.8 | 2023-04-05

  • New Features:
    • Menu Measuring Agents
      • Measuring Agent Log File: The log file is displayed faster in the browser and supports now searching for text fragments. The time zone of the log file is now also displayed.
      • List Remote Test Jobs: The display of the Job Type was added (‘Load Test’ or ‘Monitoring’).
  • Bug Fixes:
    • In rare cases the start a job failed on Measuring Agents.
    • In rare cases the Synthetic Monitoring Availability Chart did show wrongly a green bar (OK) instead of a red bar (Error).
  • Portal Server version 4.8.8 requires now Measuring Agent and Cluster Controller version 4.5.5

4.8.7 | 2023-03-26

  • New Features:
    • Synthetic Monitoring
      • “Statistics” Submenu added which shows statistics and charts of monitoring groups and monitoring jobs, such as availability, cumulated error time and performance. The time range can be selected arbitrary. In addition, single, historical measurements can be selected and their details displayed (e.g. to analyze a measured error).
      • The Real Time Dashboard shows now for the monitoring jobs the availability of the last 24 hours in percent.

4.8.4 | 2023-03-03

  • New Feature:
    • The Synthetic Monitoring Real Time Dashboard shows now the (average) performance of the last executed monitoring jobs in the unit of milliseconds per passed loop. Background information: A monitoring job can be defined with a strength of up to 5 concurrent users and with up to 5 test-loops per user. This value shows the average execution time of the successful executed test-loops overall concurrent users.

4.8.3 | 2023-02-28

  • New Features:
    • Synthetic Monitoring added: The same scripts used for load testing can now also be used to periodically check the uptime and performance of your critical business transactions. This means that a Monitoring Job can be defined from an already existing Test (similar as to define a Load Test Job). The Monitoring Jobs can be combined to Monitoring Groups whereby each Monitoring Group can have its own settings. Multiple Measuring Agents can be assigned to a Monitoring Group in order that the assigned Monitoring Jobs are executed from several, different locations (for example from New York, Berlin and Singapore). Supported Monitoring Alert Devices are Email, SMS and Webhooks.
    • WebSocket API for Synthetic Monitoring added: This WebSocket API enable developers to implement their own version of a Synthetic Monitoring “Real Time Dashboard”.
  • Portal Server version 4.8.3 requires now Measuring Agent and Cluster Controller version 4.5.4

4.7.8 | 2022-11-06

  • New Features:
    • Desktop Companion: An ‘OS X Intel Installation Kit’ and an ‘OS X ARM Installation Kit’ are now available at https://download.realload.com
    • Remote Proxy Recorder
      • Support of Client Certificates (PKCS12 files) that can uploaded to the Portal Server and applied for target server authentication on all Remote Proxy Recorders. The uploaded client certificates are automatically transmitted to the Remote Proxy Recorder(s) when connecting from the Portal Server to a Remote Proxy Recorder.
      • The log files of Remote Proxy Recorders can now be viewed in the Portal Server Web UI.
      • ‘Ping’ and ‘Connect’ to a Remote Proxy Recorder in the Portal Server Web UI now show an error message for outdated/incompatible Remote Proxy Recorder versions.
      • Any changes made at Remote Proxy Recorder settings for ‘Hosts Recording Filter’, ‘URL Filter’ and ‘URL Filter Quick Setting’ are now saved on the Portal Server and reapplied when connecting from the Portal Server to any Remote Proxy Recorder.
  • Bug Fixes:
    • HTTP/S session recording (with Remote Proxy Recorder and Desktop Companion) failed on Apple macOS and iOS client devices.
    • HTTP Test Wizard URL Explorer - Variables Wizard: variable names for JSON HTTP-request ‘variable assigners’ were not displayed.
    • HTTP Test Wizard Generate Code: The “Java Class Name” can now have a lowercase letter at the beginning.
  • Portal Server version 4.7.8 now requires Remote Proxy Recorder version 1.1.1

4.7.6 | 2022-10-17

  • New Features:
    • HTTP Test Wizard: Variable Assigner for ‘General Request Header Field’ added.
    • HTTP Test Wizard: Select Multiple Session Elements - ‘URL Filter Quick Settings’ menu added.
    • HTTP Test Wizard Debugger: Support to set/update the ‘Recorded Data’ of an executed URL (shown in the HTTP Test Wizard URL Explorer). This functionality can be used for further test optimization (e.g. if the content of a tested URL has been changed on the server side, or if the URL response is as yet unknown).
    • HTTP Test Wizard Debugger: Menu ‘Search in executed URLs’ added.
  • Bug Fixes:
    • HTTP Test Wizard URL Explorer | Variables Wizard: Bug Fix for automatically applied variable assigners which replace URL-Encoded/Decoded values.
    • Measuring Agent - Data Collector: Bug Fix for supporting large measured Error Details.
  • Portal Server version 4.7.6 requires now Measuring Agent and Cluster Controller version 4.5.2

4.7.3 | 2022-8-28

  • New Features:
    • HTTP Test Wizard: New menu “URL Explorer” added. The URL Explorer shows all the details of the recorded HTTP session data and also supports searching within it. Furthermore, the URL Explorer also contains a “Variables Wizard” which displays all distinct values clearly sent in an HTTP session and makes it much easier to extract and assign session variables.
    • HTTP Test Wizard / Review and Edit URL Settings: This menu has been revised and enhanced. Two new auto-configurations wizards are available with which the content of HTTP responses can be automatically checked for keywords and with which the parallel execution of HTTP calls can be configured automatically.
    • HTTP Test Wizard / Debugger: UI improved.
    • A new “Text Phrase Tool” has added. This tool determines the most meaningful human-readable text phrases or keywords from any given content (HTML/JSON/XML). This tool has also been integrated into the HTTP Test wizard.
    • Remote Proxy Recorder: “URL Filter Quick Setting” menu added.
    • Remote Proxy Recorder: “Convert Recording to HTTP Test Wizard Session” improved and enhanced.
    • Portal Server (all menus): Large screen support has been improved.
    • Admin Menu: Mobile Companion API integrated and “Mobile Companion App - Test & Logs” menu added.
  • Bug Fixes:
    • HTTP Test Wizard / Generate Code: The measured statistic IDs of executed load tests have now the same values as the indexes of the HTTP Test Wizard session elements.
    • Load test programs generated by the HTTP Test Wizard did not correctly measure the response time in some cases when an HTTP response had no response content (bug fix in com.dkfqs.tools.jar, new version 2.3.0).
  • Portal Server version 4.7.3 requires now Measuring Agent and Cluster Controller version 4.5.0, and Remote Proxy Recorder version 1.0.0

4.6.8 | 2022-5-19

  • Bug Fixes:
    • Adding of cluster members to AWS/EC2 ‘Cluster Controller’ instances which have launched by cloud credits did fail.
    • The status of cluster jobs was not updated by the real-time monitor.
    • Deleting of test jobs did fail if the corresponding cluster was no longer defined.
    • Starting of cloud instances is now prevented if the portal’s internal monitoring system is no longer working.
  • Portal Server version 4.6.8 requires now Measuring Agent version 4.2.0 (bug fix: incorrect measurement results were reported sporadically)

4.6.7 | 2022-5-18

  • New Features:
    • AWS/EC2 ‘Remote Proxy Recorders’ can now be started by cloud credits.
    • Preview of ‘Remote Proxy Recorder CA Root Certificate’ added when downloading a remote proxy recorder CA root certificate.
    • 40 seconds countdown to left navigation added when launching EC2 instances by cloud credits.

4.6.5 | 2022-5-10

  • New Features:
    • Support for team member accounts added:
      • Depending on the license (price plan), a normal user (main user) can also define sub-users (so-called team member accounts).
      • Team members can login as usual, with their email and password.
      • Team members can either have the same rights as the main user or alternatively only have read rights.
      • The team members (including the main user) can communicate with each other via push messages. The portal also shows which team member is currently online. Offline team members receive an email instead of an online push notification.
      • A list of all team members is provided for each team member (inclusive the main user), which contains the profile text and profile image of each team member.
      • Team member accounts are not portal-wide ‘public users’, and cannot act as public technical expert.
  • Bug Fixes:
    • Deleting of test jobs did fail if the corresponding measuring agent was no longer defined.

4.6.4 | 2022-4-18

  • New Features:
    • Support of special licenses to top up the amount of ‘Cloud Credits’.
    • License restrictions on the maximum number of Measuring Agents and Remote Proxy Recorders no longer apply when such components are launched by spending Cloud Credits.
    • The Web Browser interface has been optimized to support large screens.

4.6.2 | 2022-4-12

  • New Features:
    • Support of ‘Sign in with Microsoft’ added.

4.6.0 | 2022-4-10

  • New Features:
    • The list of excluded AWS/EC2 regions can now be edited in the admin menu.
    • The additional data transfer costs of AWS/EC2 instances which are started via the Portal Server are now billed via Cloud Credits.
    • A new Measuring Agent version 4.1.0 is available which is able to automatically configure the Java memory of the ‘Data Collector’ and the Java memory of executed Java test programs.
  • Portal Server version 4.6.0 supports Measuring Agent version 4.0.4 and version 4.1.0

In order to enable the automatic Java memory configuration of a Measuring Agent 4.1.0, the following Java program arguments must be set in the startup script: -autoAdjustMemory -osReservedMemory 1GB

Furthermore, the Java memory of the Measuring Agent should be set in the startup script as shown in the table below:

OS Physical Memory Measuring Agent Java -Xmx setting
<2 GiB 256m
2..3 GiB 512m
4..7 GiB 512m
8..15 GiB 1536m
16..31 GiB 3072m
32..63 GiB 4096m
64..96 GiB 6144m
>96 GiB 8192m
Odd number of GiB should be rounded up (e.g. 7.7 = 8 = 1536m).

Example: sudo -H -u dkfqs bash -c ‘CLASSPATH=/home/dkfqs/agent/bin/bcpkix-jdk15on-160.jar:/home/dkfqs/agent/bin/bcprov-jdk15on-160.jar:/home/dkfqs/agent/bin/bctls-jdk15on-160.jar:/home/dkfqs/agent/bin/DKFQSMeasuringAgent.jar;export CLASSPATH;nohup java -Xmx512m -DdkfqsMeasuringAgentProperties=/home/dkfqs/agent/config/measuringagent.properties -Dnashorn.args="–no-deprecation-warning" com.dkfqs.measuringagent.internal.StartDKFQSMeasuringAgent -autoAdjustMemory -osReservedMemory 1GB 1>/home/dkfqs/agent/log/MeasuringAgent.log 2>&1 &’

4.5.0 | 2022-3-27

  • New Features:
    • Amazon AWS/EC2 instances of ‘Measuring Agents’ and ‘Cluster Controllers’ can now be started directly in the Portal Server - without the customer needing an Amazon AWS account. The costs of such started AWS/EC2 instances are charged to the customer via so-called ‘Cloud Credits’, whereby the number of ‘Cloud Credits’ can be purchased as an integral part of a license or can also be purchased separately. For customers who register for the first time at the Portal Server, a free amount of ‘Cloud Credits’ will be credited to allow them try out this functionality. Note: It is still fully supported to launch Amazon AWS/EC2 instances of ‘Measuring Agents’ and ‘Cluster Controllers’ with your own Amazon AWS account by using the standalone tool ‘Desktop Companion’.
    • User Top Navigation: Link to Download Server and to Real Load Store added (both links are manageable by the administrator menu).
    • HTTP Test Wizard: Review and Edit URLs menu added.
    • Launched Cloud Instances menu added.
    • Cloud Credit Statement menu added.
    • Measuring Agents & Cluster Controllers menu: The ‘List of Predefined AWS/EC2 Measuring Agent AMIs’ shows no longer incompatible AMI versions.
    • The following new functions have been added for administrators:
      • User Accounts menu: Cloud Credit Statement menu per user added.
      • Server Settings menu: AWS/EC2 Cloud Provider API Settings section added.
      • Customize Auth Users Navigation menu: Download Components and Shop sections added.
      • Cloud Providers & Instance Type Credit Costs menu added.
      • Inspect AWS/EC2 Cloud Instances menu added.
      • Launched Cloud Instances menu added.
      • Cloud Credit Transactions menu added.

Advantages and disadvantages of ‘Cloud Credits’ versus launching AWS/EC2 instances by the ‘Desktop Companion’:

Benefits of ‘Cloud Credits’:

  • Full cost control. Cloud instances are automatically terminated after a selected time.
  • Easy to use. Seamless integration with the portal server.
  • No Amazon AWS account required and therefore no security issue for unauthorized AWS access.

Advantages of the ‘Desktop Companion’:

  • Lower costs for cloud instances compared to ‘Cloud Credits’(AWS self-cost are charged only).
  • Suitable for customers who use cloud instances frequently and are aware of AWS/EC2 costs involved.

4.4.4 | 2022-1-30

  • New Features:
    • Load-releasing Clusters of Measuring Agents are now supported. In order to operate a cluster, a separate process/component is required (so-called “Cluster Controller”) which is not part of the portal server and is operated by the customer himself. Cluster-Features:
      • Multiple clusters can be registered in the portal server using the UI, and the cluster members (measuring agents) can be assigned to the clusters with simple clicks.
      • Depending on the power of the Cluster Controller, a cluster can have several 100 members, which allows to perform load tests of almost unlimited strength (up to millions of concurrent users).
      • A cluster job can be started as easily as a normal job. The only difference is that a cluster is selected for execution instead of a single measuring agent.
      • Optionally, the content of input files can be automatically divided among the cluster members (e.g. if they contain user account credentials).
      • Cluster jobs support all runtime features like normal jobs: suspend job, resume job, stop job and kill job. Entering annotations at runtime is also supported.
      • The real-time display of cluster jobs is exactly the same as for normal jobs, but the statistics displayed at real time already contain the combined values of all cluster members. In addition, a table about the current operating status of each cluster member is available.
      • After a cluster job has been completed, the test result contains the combined values of all cluster members (analogous to the real-time display). In addition, the individual test results of the cluster members can also be displayed.
    • The following functions have been added to the “Remote User API”:
      • getMinRequiredMeasuringAgentVersion
      • getMinRequiredClusterControllerVersion
      • getMinRequiredProxyRecorderVersion
      • getMeasuringAgentClusters
      • getClusterControllers
      • pingClusterController
      • addMeasuringAgentCluster
      • addMemberToMeasuringAgentCluster
      • removeMemberFromMeasuringAgentCluster
      • pingMeasuringAgentClusterMembers
      • setMeasuringAgentClusterActive
      • deleteMeasuringAgentCluster
    • Portal Server “Measuring Agents” menu: The AWS/EC2 popup highlights now incompatible AMI versions.
    • Portal Server “Test Jobs” menu: The log files of executed jobs can now directly copied to the project tree.
  • Bug Fixes:
    • A test job can now also have the state “execution failed”.
  • Portal Server version 4.4.4 requires now Measuring Agent version 4.0.4

4.3.23 | 2021-11-13

  • New Features:
    • HTTP Test Wizard: New Session Element ‘Outbound IP Address’ added. In order that this element can be used, multiple valid IP addresses must be assigned to the network interface of the Measuring Agent(s).
    • HTTP Test Wizard: Support of specific HTTP processing timeout per URL added.
    • Test Results: Support of annotation and annotation events added. The annotation and annotation events can be added at runtime during the test and are shown in the test result. In addition, annotation events can also be reported via the All Purpose Interface and generated, for example, by HTTP Test Wizard plug-ins.
  • Bug Fixes:
    • The cached HTTP Test Wizard Session in the Portal Server UI is now updated when a variable is defined, modified or deleted, and when a variable extractor or a variable assigner is deleted.
    • Invalid Java code was generated by the HTTP Test Wizard if the error handling of URLs was set to ‘Continue as usual’.
    • Performance bottleneck fixed in com.dkfqs.tools.http.HTTPClient (occurred since previous version 4.3.22).
    • An incorrect error exception was thrown in com.dkfqs.tools.crypto.EncryptedSocket when an SSL handshake timeout occurred
    • Realtime charts sometimes showed inexact measurement results.
  • Portal Server version 4.3.23 requires now Measuring Agent version 3.9.33 and DKFQS Tools version 2.2.25

4.3.22 | 2021-10-24

  • Documentation: All Purpose Interface added.
  • The All Purpose Interface has been extended by 3 new statistics types that can be declared at runtime:
    • average-and-current-value : An average and current value
    • efficiency-ratio-percent : An efficiency ratio in percent (0..100%)
    • throughput-time-chart : A chart of a throughput per second
  • The measurement results of the HTTP Test Wizard now contain the following additional test-specific values (if the executed HTTP Test Wizard session contains URL session elements):
    • Total Bytes Sent
    • Total Bytes Received
    • Network Throughput in Mbps (real-time: current value, test result: average value and chart)
    • Average TCP Connect Time in milliseconds (real-time: + latest value)
    • Average SSL Handshake Time in milliseconds (real-time: + latest value)
    • HTTP Keep-Alive Efficiency (0..100%)
  • Bug fix: The time in the name of the test result files is now always set in the time zone of the portal server, regardless of which time zone the Measuring Agents are operated.
  • Portal Server version 4.3.22 requires now Measuring Agent version 3.9.32 and DKFQS Tools version 2.2.24

4.3.21 | 2021-09-18

  • Searching for a text in all files of the project tree is now supported.
  • Support to upload files by drag and drop added.
  • Bug fix for assigning variables to HTTP requests in HTTP Test Wizard debugger.
  • Bug fix on real time statistics and test results if the measured unit is other than ms.
  • User profile images can now have a max size of 400 KB instead of 200 KB.
  • CA Root Certificates of HTTP/S Proxy Recorder(s) can now be downloaded via the Portal Server UI.
  • Documentation: User Guide added.
  • SNMP Plug-In published at https://portal.realload.com/publicPublishedPlugins

Portal Server Installation / Ubuntu 20: The “fontconfig” package has to be installed in order that the captcha generator is working:

sudo apt-get update
sudo apt-get install fontconfig

4.3.20 | 2021-08-07

  • HTTP Test Wizard Plug-Ins can now be published and are then available to other users.
  • The following HTTP Test Wizard Plugin fields are now protected by algorithms and cannot be manually modified by the JSON editor:
    • pluginId
    • authorNickname
  • A (new) “Resources Library” project is automatically created for each user in the Project Tree which contains by default the following “Resource Sets”:
    • “HTTP Test Wizard Plug-Ins”: by default empty
    • “Java”: contains always the latest version of com.dkfqs.tools.jar
    • “PowerShell”: contains always the latest version of DKFQSLibrary2.psm1
  • The “Resources Library” project contains common resources which are used by multiple projects. The users can add additional resource sets and files as needed to this project.
  • The file com.dkfqs.tools.jar is no longer copied to the corresponding resource set when a HTTP Test Wizard test is generated. Instead of this the generated test contain now a reference to “Resources Library / Java / com.dkfqs.tools.jar”.
  • New created HTTP Test Wizard Plug-Ins contain now by default a reference to the resource file “Resources Library / Java / com.dkfqs.tools.jar”.
  • More than 50 minor bugs have been fixed and some improvements have been made to the portal user interface.
  • Portal Server version 4.3.20 requires now Measuring Agent version 3.9.31
  • Existing tests and plug-ins should be upgraded to use com.dkfqs.tools.jar version 2.2.21 (located at “Resource Sets / Java”). This means that the tests have to be generated and defined once again.
  • The tuning parameters of Linux operating systems on which “Measuring Agents” run must be increased:

in /etc/security/limits.conf add or modify:

# TCP/IP Tuning
# =============
* soft     nproc          262140
* hard     nproc          262140
* soft     nofile         262140
* hard     nofile         262140
root soft     nproc          262140
root hard     nproc          262140
root soft     nofile         262140
root hard     nofile         262140

Enter: systemctl show -p TasksMax user-0

output: TasksMax=8966

if you get a value less than 262140 then add in /etc/systemd/system.conf

# System Tuning
# =============
DefaultTasksMax=262140

Reboot the system and verify the settings. Enter: ulimit -n

output: 262140

Enter: systemctl show -p TasksMax user-0

output: TasksMax=262140

4.3.18 | 2021-07-06

  • Support of Licenses added. Multiple licenses per user account are supported. New licenses can be entered via the Admin Menu and via the User Menu. Users whose license has expired can enter a new license during sign-in.
  • Pluggable Architecture for License Providers implemented.
  • Remote Admin API and Remote User API added. The corresponding API Authentication Tokens can be generated via the Admin Menu and via the User Menu. See API Documentation.
  • Unix Time Tool in User Menu added.
  • User accounts that are since a long time expired are now automatically deleted - including all user specific data. The number of days between the expiry date and the deletion date can be configured in the Admin Menu.
  • Deleted user accounts can now be viewed in the Admin Menu.
  • Portal Server version 4.3.18 requires now Measuring Agent version 3.9.30

4.3.14 | 2021-04-08

  • Support of multiple Price Plans / Limits for User Accounts such as Disk-Space, Number of Measuring Agents, Account expires time … (Admin Menu).
  • Access to Measuring Agents can now (optional) protected by an Authentication Token (password).
  • Configurable Default Price Plan for Sign Up (Admin Menu).
  • Configurable HTML content for Sign Up steps 1 to 4 and for (new) Sign Up completed “Welcome Page” (Admin Menu).
  • Test jobs are now digitally signed. This means that the following job settings on a Measuring Agent cannot be modified after the job has been transmitted to the Measuring Agent: type of job, local test job ID, number of users, maximum test duration.

4 - Synthetic Monitoring

Synthetic Monitoring - Introduction

Synthetic Monitoring is an essential part of Digital Experience Monitoring, where organizations can detect their service outages or performance degrade proactively by periodically running the tests.

RealLoad provides an easy way of configuring synthetic tests with an outcome of accurate metrics, alerting and SLA details.

RealLoad Synthetic Monitoring Features

  • Due to the universal architecture, RealLoad Tests of any kind can be run as a monitoring job (HTTP Test Wizard tests, JUnit tests, Selenium tests, etc.).
  • Synthetic monitoring jobs can also run with many virtual users.
  • Support for performance warning thresholds.
  • Support for delayed alarm notifications.
  • Supported alert devices: Email, SMS and WebHooks (including Pagerduty and Slack).

Below illustrates Synthetic Tests configurations, Alerts configurations, Real-Time Dashboard, Statistics with SLA measured.

Define a Test

You have first to define a RealLoad ‘Test’ before you can add it to Synthetic Monitoring. The following 3 examples show how a test can be defined:

After you can see all the tests you have defined like below under Tests page

Configure Synthetic Monitoring

Next step is to create Monitoring Groups and add & define tests under each group as jobs like below, Agents will be available for selection based on how many agents are already configured on different geographical locations, please refer here for more details

Synthetic Real-Time Dashboard

Now the created monitoring groups with its monitoring jobs will be available under the Real-Time Dashboard like below with the important metrics and also have the option to drill down into test results

Custom dashboards can be created on third party products like Grafana, using our WebSocket API

Alerting

Next explains how to configure the Alerting, below illustrates how to configure alerting for created Monitoring Groups or Jobs

Statistics with Performance & Availability

Next explains about the statistics data, which will be the source for many KPIs , for example to calculate the defined SLA for an application or an API service

5 - API

Portal Server APIs

The portal server has three APIs:

  • Remote Admin API
  • Remote User API
  • Remote User Monitoring API

All APIs require so-called “Authentication Tokens” to verify the access authorization. If 5 or more invalid “Authentication Tokens” are sent within 60 seconds, the corresponding remote IP address is blocked for 30 minutes.

5.1 - Remote Admin API

Portal Server Remote Admin API Specification

Generating an Authentication Token

To perform a Remote Admin API call, you must first generate an “Admin API Auth Token” in the Administrator Menu of the portal. When generating the token you can enter a purpose (only used as information) and also restrict the remote IP addresses for which the token is valid. You can also specify whether the token has read/write access or read/only access.

“alt attribute”

API Functions

The API supports the following functions (so-called “actions”):

  • getAllUserAccounts (Get all user accounts of the portal server)
  • getAllPricePlans (Get all price plans of the portal server)
  • addLicenseCertificateToUser (Add a (new) license certificate to an user)
  • getServerSettings (Get the server settings)
  • setServerMaintenanceMode (Turn the server maintenance mode on or off)
URL HTTP Method
https://portal.realload.com/RemoteAdminAPI POST
All data are sent and received in JSON data format. The “authTokenValue” and the “action” must always be sent when an API call is made.

Example

API HTTP/S Request

curl -v --request POST --header "Content-Type: application/json" --data "@getAllUserAccounts.json"  https://portal.realload.com/RemoteAdminAPI

API Request Data

{
  "authTokenValue": "8mKSz1UzaQg17kfu",
  "action": "getAllUserAccounts"
}

API Response Data

{"allUserAccountsArray":
[{"userId":13,"nickname":"DKF","firstName":"Max","lastName":"Fischer","primaryEmail":"max@dkfqa.com","primarySMSPhone":"+41771111111","secondaryEmail":"","secondarySMSPhone":"","accountBlocked":false,"accountCreateTime":1538556183756,"lastLoginTime":1625181623869,"lastLoginIP":"127.0.0.1","pricePlanId":1,"accountExpiresTime":-1,"pricePlanTitle":"Unlimited"},{"userId":18,"nickname":"AX","firstName":"Alex","lastName":"Fischer","primaryEmail":"alexfischer66@yahoo.com","primarySMSPhone":"+41781111111","secondaryEmail":"","secondarySMSPhone":"","accountBlocked":false,"accountCreateTime":1539874749677,"lastLoginTime":1616111301975,"lastLoginIP":"127.0.0.1","pricePlanId":1,"accountExpiresTime":-1,"pricePlanTitle":"Unlimited"},{"userId":22,"nickname":"Kes","firstName":"Kesorn","lastName":"Fischer","primaryEmail":"gsklsta@yahoo.com","primarySMSPhone":"+66000000000","secondaryEmail":"","secondarySMSPhone":"","accountBlocked":false,"accountCreateTime":1605303204754,"lastLoginTime":1624389324770,"lastLoginIP":"127.0.0.1","pricePlanId":6,"accountExpiresTime":-1,"pricePlanTitle":"BASIC1"},{"userId":48,"nickname":"BET","firstName":"Bettina","lastName":"Meier","primaryEmail":"b123456@lucini.id.au","primarySMSPhone":"+61404905702","secondaryEmail":"","secondarySMSPhone":"","accountBlocked":false,"accountCreateTime":1623719604561,"lastLoginTime":-1,"lastLoginIP":"","pricePlanId":6,"accountExpiresTime":1625061600000,"pricePlanTitle":"BASIC1"}],
"isError":false}

If the API call is successful, then the response field “isError” is false. If a numeric field has a value of -1 (minus one), this means “no data” or “unlimited” depending on the context.

getAllUserAccounts

Specific Request Fields:

  • [none]

Specific Error Flags:

  • [none]

getAllPricePlans

Specific Request Fields:

  • [none]

Specific Error Flags:

  • [none]

addLicenseCertificateToUser

Specific Request Fields:

  • mapToUserEmailAddress
  • mapToUserMobilePhone
  • licenseProvider
  • licenseCertificate

The license is successfully assigned to a user if either mapToUserEmailAddress or mapToUserMobilePhone matches to a user account.

Specific Error Flags:

  • writeAccessError
  • mapToUserError
  • licenseProviderError
  • licenseCertificateError
  • licenseCertificateAlreadyAddedError
  • pricePlanError

JSON Request Example:

{
  "authTokenValue":"8mKSz1UzaQg17kfu",
  "action":"addLicenseCertificateToUser",
  "licenseProvider": "Real Load Pty Ltd / nopCommerce",
  "mapToUserEmailAddress": "max@dkfqa.com",
  "mapToUserMobilePhone": "+41771111111",
  "licenseCertificate": "-----BEGIN CERTIFICATE-----\r\nMIIEnjCCA4agAwIBAgIEyDnukzANBgkqhkiG9w0BA ...... Hn/UMGAGRB6xF4w+TewYqTAZrdhi/WLyYwg==\r\n-----END CERTIFICATE-----"
}

JSON Response Example (Success Case):

{"licenseId":12,"cloudCreditLicenseId":-1,"userId":13,"isCloudCreditsLicense":false,"isError":false}

JSON Response Example (Error Case):

{"isError":true,"genericErrorText":"","writeAccessError":false,"licenseProviderError":false,"mapToUserError":false,"pricePlanError":false,"licenseCertificateAlreadyAddedError":false,"licenseCertificateError":true}

getServerSettings

Specific Request Fields:

  • [none]

Specific Error Flags:

  • [none]

JSON Response Example:

{
  "isServerMaintenanceMode":false,
  "isSignInSelectPricePlanFromMultipleValidLicenseCertificates":true,
  "isSignInExpiredAccountCanEnterLicenseCertificate":true,
  "isSignUpEnabled":true,
  "isSignUpRequiresInvitationTicket":false,
  "signUpDefaultPricePlanId":2,
  "signUpDefaultAccountExpiresInDays":14,
  "deleteExpiredUserAccountsAfterDays":183,
  "isError":false
}

setServerMaintenanceMode

Specific Request Fields:

  • serverMaintenanceMode

Specific Error Flags:

  • writeAccessError

JSON Request Example:

{
  "authTokenValue":"8mKSz1UzaQg17kfu",
  "action":"setServerMaintenanceMode",
  "serverMaintenanceMode":true
}

JSON Response Example (Success Case):

{"isServerMaintenanceMode":true,"isError":false}

5.2 - Remote User API

Portal Server Remote User API Specification

Generating an Authentication Token

To perform a Remote User API call, you must first sign in into the portal and generate an “API Authentication Token”. When generating the token you can enter a purpose (only used as information) and also restrict the remote IP addresses for which the token is valid.

“alt attribute”

“alt attribute”

API Functions

The API supports the following functions (so-called “actions”):

Common Functions:

  • getUserAccountInfo (Get information about the own user account)
  • getPricePlanInfo (Get information about the current price plan)

Projects, Resource Sets and Files Functions:

  • getProjectTree (Get the project tree inclusive all resource sets and all file information)
  • createProject (Create a new project)
  • deleteProject (Delete a project)
  • getResourceSetsOfProject (Get all resource sets of a project, inclusive all file information)
  • createResourceSet (Create a new resource set)
  • deleteResourceSet (Delete a resource set)
  • getFilesInfoOfResourceSet (Get all files information of a resource set)
  • createFile (Create or overwrite a file)
  • getFile (Get the content of a file and the file information)
  • deleteFile (Delete a file)

Measuring Agents Functions:

  • getMeasuringAgents (Get all defined measuring agents)
  • getMinRequiredMeasuringAgentVersion (Get the minimum required measuring agent version)
  • addMeasuringAgent (Add a new measuring agent)
  • pingMeasuringAgent (Ping a measuring agent)
  • setMeasuringAgentActive (Set the state of a measuring agent to active or inactive)
  • deleteMeasuringAgent (Delete a measuring agent)

Measuring Agent Cluster Functions:

  • getMeasuringAgentClusters (Get all defined measuring agent clusters)
  • getClusterControllers (Get all cluster controllers and for each cluster controller a list of measuring agent clusters which are referencing the cluster controller)
  • getMinRequiredClusterControllerVersion (Get the minimum required cluster controller version)
  • pingClusterController (Ping a cluster controller)
  • addMeasuringAgentCluster (Add a new measuring agent cluster)
  • addMemberToMeasuringAgentCluster (Add a member to a measuring agent cluster)
  • removeMemberFromMeasuringAgentCluster (Remove a member from a measuring agent cluster)
  • pingMeasuringAgentClusterMembers (Ping the cluster members of a measuring agent cluster via cluster controller)
  • setMeasuringAgentClusterActive (Set the state of a measuring agent cluster to active or inactive)
  • deleteMeasuringAgentCluster (Delete a measuring agent cluster)

HTTP/S Remote Proxy Recorders Functions:

  • getProxyRecorders (Get all defined HTTP/S proxy recorders)
  • getMinRequiredProxyRecorderVersion (Get the minimum required HTTP/S proxy recorder version)
  • addProxyRecorder (Add a new HTTP/S proxy recorder)
  • pingProxyRecorder (Ping a HTTP/S proxy recorder)
  • deleteProxyRecorder (Delete a HTTP/S proxy recorder)
  • getProxyRecordersClientCertificates (Get all client certificates for HTTP/S proxy recorders)
  • addProxyRecordersClientCertificate (Add a client certificate for HTTP/S proxy recorders)
  • setProxyRecordersClientCertificateActive (Set a client certificate for HTTP/S proxy recorders to active or inactive)
  • modifyProxyRecordersClientCertificate (Modify a client certificate for HTTP/S proxy recorders)
  • deleteProxyRecordersClientCertificate (Delete a client certificate for HTTP/S proxy recorders)

Load Test Jobs Functions:

  • getTestjobTemplates (Get all test job templates)
  • defineNewTestjobFromTemplate (Define new test job from template)
  • submitTestjob (Submit a ‘defined’ test job to the measuring agent or to the cluster controller)
  • makeTestjobReadyToRun (Turn a remote test job which is in the state ‘submitted’ into the state ‘ready to run’)
  • startTestjob (Turn a remote test job which is in the state ‘ready to run’ into the state ‘running’)
  • getMeasuringAgentTestjobs (Get from a measuring agent all test jobs, inclusive their states)
  • getTestjobOutDirectoryFilesInfo (Get from test job information about all files located in a measuring agent job or cluster controller job ‘out’ subdirectory)
  • getTestjobOutDirectoryFile (Get from a measuring agent or from a cluster controller the content of a file located in a testjob/cluster-job out subdirectory in base64 format)
  • saveTestjobOutDirectoryFileToProjectTree (Get from a measuring agent or from a cluster controller the content of a file located in a testjob/cluster-job out subdirectory and save it in a resource set of the project tree)
  • deleteTestjob (Delete a test job locally and also on the measuring agent or cluster controller)

Test Execution Suites Functions:

  • getTestExecutionSuites (Get all test execution suites with a summary of all results of the executed test runs)
  • getCurrentlyRunningTestExecutionSuiteIds (Get the ID(s) of all currently running test execution suites)
  • startTestExecutionSuiteTestRun (Start a new run of a test execution suite)
  • getTestExecutionSuiteTestRunResultDetail (Get a test run result detail of a test execution suite)
  • getTestExecutionSuiteTestRunTestJobOutputFileContent (Get the content of a test job output file of a test execution suite run)
  • deleteTestExecutionSuiteTestRun (Delete a test run of a test execution suite)
URL HTTP Method
https://portal.realload.com/RemoteUserAPI POST
All data are sent and received in JSON data format. The “authTokenValue” and the “action” must always be sent when an API call is made.

Example

API HTTP/S Request

curl -v --request POST --header "Content-Type: application/json" --data {\"authTokenValue\":\"jPmFClqeDUXaEk8Q274q\",\"action\":\"getUserAccountInfo\"}  https://portal.realload.com/RemoteUserAPI

API Request Data

{
  "authTokenValue": "jPmFClqeDUXaEk8Q274q",
  "action": "getUserAccountInfo"
}

API Response Data

{
  "userAccountInfo":{
    "userId":48,
    "nickname":"BET",
    "firstName":"Bettina",
    "lastName":"MeierHans",
    "primaryEmail":"bettina@meierhans.id.au",
    "primarySMSPhone":"+61401111111",
    "secondaryEmail":"",
    "secondarySMSPhone":"",
    "accountBlocked":false,
    "accountCreateTime":1623719604561,
    "lastLoginTime":1625348376450,
    "lastLoginIP":"127.0.0.1",
    "pricePlanId":6,
    "accountExpiresTime":1625953109397,
    "pricePlanTitle":"BASIC1"
    },
  "isError":false
}

If the API call is successful, then the response field “isError” is false. If a numeric field has a value of -1 (minus one), this means “no data” or “unlimited” depending on the context.

getUserAccountInfo

Specific Request Fields:

  • [none]

Specific Error Flags:

  • [none]

getPricePlanInfo

Specific Request Fields:

  • [none]

Specific Error Flags:

  • [none]

JSON Response Example:

{
  "pricePlanInfo":{
    "pricePlanId":6,
    "title":"BASIC1",
    "description":"",
    "isDeprecated":false,
    "lastModified":1625348413042,
    "maxDiskSpaceMB":1024,
    "maxSubUserAccounts":0,
    "maxMeasuringAgentsOwnedByUser":3,
    "maxRemoteProxyRecordersOwnedByUser":3,
    "executeLoadJobsEnabled":true,
    "executeMonitoringJobsEnabled":false,
    "apiAccessEnabled":true,
    "maxStartLoadJobsLast24h":24,
    "maxUsersPerLoadJob":500,
    "maxDurationPerLoadJob":1800
  },
  "isError":false
}

The unit for “maxDurationPerLoadJob” is seconds.

getProjectTree

Specific Request Fields:

  • [none]

Specific Error Flags:

  • [none]

JSON Response Example:

{
  "projectsArray": [
    {
      "projectId": 97,
      "projectName": "Common",
      "projectDescription": "",
      "resourceSetsArray": [
        {
          "resourceSetId": 154,
          "resourceSetName": "Input Files",
          "resourceSetDescription": "",
          "filesArray": [
            {
              "fileName": "InputFile.txt",
              "fileSize": 233,
              "fileHashCode": 1873256029,
              "fileLastModified": 1613835992073
            }
          ]
        },
        {
          "resourceSetId": 155,
          "resourceSetName": "Jar Files",
          "resourceSetDescription": "",
          "filesArray": [
            {
              "fileName": "com.dkfqs.tools.jar",
              "fileSize": 578087,
              "fileHashCode": -2033420926,
              "fileLastModified": 1613838181727
            }
          ]
        },
        {
          "resourceSetId": 156,
          "resourceSetName": "Plug-Ins",
          "resourceSetDescription": "",
          "filesArray": [
            {
              "fileName": "HttpSessionPlugin_ChangeCopyright.json",
              "fileSize": 5321,
              "fileHashCode": 1958407366,
              "fileLastModified": 1613838287871
            }
          ]
        }
      ]
    },
...
...
...
  ],
  "isError": false
}

createProject

Specific Request Fields:

  • projectName
  • projectDescription (optional)

Response Fields:

  • projectId

Specific Error Flags:

  • projectNameError
  • diskSpaceLimitExceededError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"createProject",
  "projectName":"My New Project",
  "projectDescription": "Created by API call"
}

JSON Response Example (Success Case):

{"projectId":113,"isError":false}

JSON Response Example (Error Case):

{"isError":true,"genericErrorText":"","diskSpaceLimitExceededError":false,"projectNameError":true}

deleteProject

Specific Request Fields:

  • projectId
  • moveToTrash (optional, default: false)

Specific Error Flags:

  • projectIdError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteProject",
  "projectId":113,
  "moveToTrash":false
}

getResourceSetsOfProject

Specific Request Fields:

  • projectId

Specific Error Flags:

  • projectIdError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getResourceSetsOfProject",
  "projectId":97
}

JSON Response Example (Success Case):

{
  "resourceSetsArray": [
    {
      "resourceSetId": 154,
      "resourceSetName": "Input Files",
      "resourceSetDescription": "",
      "filesArray": [
        {
          "fileName": "InputFile.txt",
          "fileSize": 233,
          "fileHashCode": 1873256029,
          "fileLastModified": 1613835992073
        }
      ]
    },
    {
      "resourceSetId": 155,
      "resourceSetName": "Jar Files",
      "resourceSetDescription": "",
      "filesArray": [
        {
          "fileName": "com.dkfqs.tools.jar",
          "fileSize": 578087,
          "fileHashCode": -2033420926,
          "fileLastModified": 1613838181727
        }
      ]
    },
    {
      "resourceSetId": 156,
      "resourceSetName": "Plug-Ins",
      "resourceSetDescription": "",
      "filesArray": [
        {
          "fileName": "HttpSessionPlugin_ChangeCopyright.json",
          "fileSize": 5321,
          "fileHashCode": 1958407366,
          "fileLastModified": 1613838287871
        }
      ]
    }
  ],
  "isError": false
}

createResourceSet

Specific Request Fields:

  • projectId
  • resourceSetName
  • resourceSetDescription (optional)

Response Fields:

  • resourceSetId

Specific Error Flags:

  • projectIdError
  • resourceSetNameError
  • diskSpaceLimitExceededError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"createResourceSet",
  "projectId":97,
  "resourceSetName":"My New Resource Set",
  "resourceSetDescription": "Created by API call"
}

JSON Response Example (Success Case):

{"resourceSetId":172,"isError":false}

JSON Response Example (Error Case):

{"isError":true,"genericErrorText":"","resourceSetNameError":true,"projectIdError":false,"diskSpaceLimitExceededError":false}

deleteResourceSet

Specific Request Fields:

  • projectId
  • resourceSetId
  • moveToTrash (optional, default: false)

Specific Error Flags:

  • projectIdError
  • resourceSetIdError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteResourceSet",
  "projectId":97,
  "resourceSetId":172,
  "moveToTrash":false
}

getFilesInfoOfResourceSet

Specific Request Fields:

  • projectId
  • resourceSetId

Specific Error Flags:

  • projectIdError
  • resourceSetIdError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getFilesInfoOfResourceSet",
  "projectId":23,
  "resourceSetId":143
}

JSON Response Example:

{
  "filesArray": [
    {
      "fileName": "DKFQSLibrary2.psm1",
      "fileSize": 16339,
      "fileHashCode": -1503445747,
      "fileLastModified": 1603566144851
    },
    {
      "fileName": "powershell-http-bern2.ps1",
      "fileSize": 12900,
      "fileHashCode": -1174212096,
      "fileLastModified": 1603566162094
    },
    {
      "fileName": "TestResult_powershell-http-bern2Neu_2020-10-24@21-06-04.json",
      "fileSize": 14395,
      "fileHashCode": -951574615,
      "fileLastModified": 1603566379097
    },
    {
      "fileName": "TestResult_powershell-http-bern2Neu_2020-10-24@21-09-45.json",
      "fileSize": 55128,
      "fileHashCode": 1499924815,
      "fileLastModified": 1603566591322
    }
  ],
  "isError": false
}

createFile

Specific Request Fields:

  • projectId
  • resourceSetId
  • fileName
  • fileContentB64 (the content of the file, in Base64 format)

Response Fields:

  • fileName
  • fileSize
  • fileHashCode
  • fileLastModified

Specific Error Flags:

  • projectIdError
  • resourceSetIdError
  • fileNameError
  • diskSpaceLimitExceededError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"createFile",
  "projectId":23,
  "resourceSetId":143,
  "fileName":"test.txt",
  "fileContentB64":"VGhpcyBpcyB0aGUgY29udGVudCBvZiB0aGUgZmlsZS4=" 
}

JSON Response Example (Success Case):

{
  "fileName":"test.txt",
  "fileSize":32,
  "fileHashCode":-1460278014,
  "fileLastModified":1625423562384,
  "isError":false
}

JSON Response Example (Error Case):

{"isError":true,"genericErrorText":"","projectIdError":false,"resourceSetIdError":false,"diskSpaceLimitExceededError":false,"fileNameError":true}

getFile

Specific Request Fields:

  • projectId
  • resourceSetId
  • fileName

Response Fields:

  • fileName
  • fileContentB64 (the content of the file, in Base64 format)
  • fileSize
  • fileHashCode
  • fileLastModified

Specific Error Flags:

  • projectIdError
  • resourceSetIdError
  • fileNameError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getFile",
  "projectId":23,
  "resourceSetId":143,
  "fileName":"test.txt"
}

JSON Response Example (Success Case):

{
  "fileName":"test.txt",
  "fileContentB64":"VGhpcyBpcyB0aGUgY29udGVudCBvZiB0aGUgZmlsZS4=",
  "fileSize":32,
  "fileHashCode":-1460278014,
  "fileLastModified":1625423562384,
  "isError":false
}

deleteFile

Specific Request Fields:

  • projectId
  • resourceSetId
  • fileName
  • moveToTrash (optional, default: false)

Response Fields:

  • fileDeleted (a flag which is true if the file was deleted)

Specific Error Flags:

  • projectIdError
  • resourceSetIdError
  • fileNameError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteFile",
  "projectId":23,
  "resourceSetId":143,
  "fileName":"test.txt",
  "moveToTrash":false
}

JSON Response Example (Success Case):

{"fileDeleted":true,"isError":false}

JSON Response Example (Error Case):

{"isError":true,"genericErrorText":"","projectIdError":false,"resourceSetIdError":false,"fileNameError":true}

getMeasuringAgents

Specific Request Fields:

  • [none]

Response Fields:

  • agentId (the unique measuring agent id)
  • createdBySystem (normally false, true = the user cannot modify or delete the measuring agent)
  • ownerUserId (always the same as the user account id)
  • agentActive (flag: if false then the availability of the measuring agent is not monitored)
  • agentDescription
  • agentHost
  • agentPort
  • authToken (the authentication token to access the measuring agent, or an empty string = no access protection | don’t confuse it with the API authTokenValue)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getMeasuringAgents"
}

JSON Response Example (Success Case):

{
  "measuringAgentsArray": [
    {
      "agentId": 46,
      "createdBySystem": false,
      "ownerUserId": 13,
      "agentActive": true,
      "agentDescription": "Local Agent",
      "agentHost": "127.0.0.1",
      "agentPort": 8080,
      "authToken": "OrKmpkbyNWEHok"
    },
    {
      "agentId": 49,
      "createdBySystem": false,
      "ownerUserId": 13,
      "agentActive": false,
      "agentDescription": "Rasberry 1",
      "agentHost": "192.168.0.51",
      "agentPort": 8080,
      "authToken": ""
    },
    {
      "agentId": 50,
      "createdBySystem": false,
      "ownerUserId": 13,
      "agentActive": true,
      "agentDescription": "Ubuntu 10",
      "agentHost": "192.168.0.110",
      "agentPort": 8080,
      "authToken": ""
    },
    {
      "agentId": 51,
      "createdBySystem": false,
      "ownerUserId": 13,
      "agentActive": true,
      "agentDescription": "Ubuntu 11",
      "agentHost": "192.168.0.111",
      "agentPort": 8080,
      "authToken": ""
    }
  ],
  "isError": false
}

getMinRequiredMeasuringAgentVersion

Specific Request Fields:

  • [none]

Response Fields:

  • minRequiredMeasuringAgentVersion (the minimum required measuring agent version)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getMinRequiredMeasuringAgentVersion"
}

JSON Response Example (Success Case):

{
  "minRequiredMeasuringAgentVersion":"3.9.34",
  "isError":false
}

addMeasuringAgent

Specific Request Fields:

  • agentDescription (must be unique across all measuring agents of the user, cannot be an empty string)
  • agentHost
  • agentPort
  • agentActive (flag: if false then the availability of the measuring agent is not monitored)
  • agentAuthToken (the authentication token to access the measuring agent, or an empty string = no access protection)

Response Fields (JSON object “measuringAgent”):

  • agentId (the unique measuring agent id)
  • createdBySystem (always false for this function)
  • ownerUserId (always the same as the user account id)
  • agentActive (flag: if false then the availability of the measuring agent is not monitored)
  • agentDescription
  • agentHost
  • agentPort
  • authToken (the authentication token to access the measuring agent, or an empty string = no access protection)

Specific Error Flags:

  • agentDescriptionError
  • agentHostError
  • agentPortError
  • maxNumberMeasuringAgentsLimitExceededError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"addMeasuringAgent",
  "agentDescription":"Ubuntu 12",
  "agentHost":"192.168.0.112",
  "agentPort":8080,
  "agentActive": true,
  "agentAuthToken": "nixda"
}

JSON Response Example (Success Case):

{
  "measuringAgent": {
    "agentId": 53,
    "createdBySystem": false,
    "ownerUserId": 13,
    "agentActive": true,
    "agentDescription": "Ubuntu 12",
    "agentHost": "192.168.0.112",
    "agentPort": 8080,
    "authToken": "nixda"
  },
  "isError": false
}

pingMeasuringAgent

Specific Request Fields:

  • agentId

Response Fields (JSON object “agentResponse”):

  • pingFromRemoteIp
  • pingFromRemoteUserId
  • productVersion (measuring agent version | don’t confuse with portal server version)
  • limitMaxUsersPerJob (limit of the measuring agent, -1 = unlimited | don’t confuse with price plan limit)
  • limitMaxJobDurationSeconds (limit of the measuring agent, -1 = unlimited | don’t confuse with price plan limit)
  • osName
  • osVersion
  • javaVersion
  • javaVendor
  • javaMaxMemory
  • systemTime
  • deltaTimeMillis
  • agentStartupTimeStamp
  • httpExecuteTimeMillis

Specific Error Flags:

  • agentIdError
  • agentAccessDeniedError
  • agentVersionOutdatedError
  • agentNotReachableError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"pingMeasuringAgent",
  "agentId":48
}

JSON Response Example (Success Case):

{
  "agentResponse": {
    "pingFromRemoteIp": "83.150.39.44",
    "pingFromRemoteUserId": 13,
    "productVersion": "3.9.30",
    "limitMaxUsersPerJob": 500,
    "limitMaxJobDurationSeconds": 300,
    "osName": "Linux",
    "osVersion": "4.15.0-136-generic",
    "javaVersion": "11.0.1",
    "javaVendor": "Oracle Corporation",
    "javaMaxMemory":"2048 MB",
    "systemTime": 1625513238236,
    "deltaTimeMillis": 841,
    "agentStartupTimeStamp": 1622836702172,
    "httpExecuteTimeMillis": 247
  },
  "isError": false
}

JSON Response Example (Error Case 1):

{
  "isError": true,
  "genericErrorText": "API V1 request to 192.168.0.51:8080 timed out",
  "agentIdError": false,
  "agentAccessDeniedError": false,
  "agentNotReachableError": true,
  "agentVersionOutdatedError": false
}

JSON Response Example (Error Case 2):

{
  "isError": true,
  "genericErrorText": "Min. measuring agent version required: 3.9.30",
  "agentIdError": false,
  "agentAccessDeniedError": false,
  "agentNotReachableError": false,
  "agentVersionOutdatedError": true
}

setMeasuringAgentActive

Specific Request Fields:

  • agentId
  • agentActive

Response Fields:

  • [none]

Specific Error Flags:

  • agentIdError
  • agentAccessDeniedError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"setMeasuringAgentActive",
  "agentId":46,
  "agentActive":false
}

JSON Response Example (Success Case):

{"isError":false}

deleteMeasuringAgent

Specific Request Fields:

  • agentId

Response Fields:

  • [none]

Specific Error Flags:

  • agentIdError
  • agentAccessDeniedError
  • agentDeleteDeniedError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteMeasuringAgent",
  "agentId":54
}

JSON Response Example (Success Case):

{"isError":false}

getMeasuringAgentClusters

Specific Request Fields:

  • [none]

Response Fields (JSON array “measuringAgentClustersArray”):

  • clusterId (the unique cluster id)
  • createdBySystem (normally false, true = the user cannot modify or delete the cluster)
  • ownerUserId (always the same as the user account id)
  • clusterActive (flag: if false then the availability of the cluster is not monitored)
  • clusterDescription
  • controllerHost (the hostname or IP address of the cluster controller)
  • controllerPort (the IP port of the cluster controller)
  • controllerAuthToken (the authentication token to access the cluster controller, or an empty string = no access protection | don’t confuse it with the API authTokenValue)
  • clusterMembersArray
    • clusterMemberId (the unique cluster member id)
    • loadFactor (integer 0..100: the default load factor of this cluster member)
    • agentId (the referenced measuring agent id)
    • agentActive (flag: if false then the availability of the measuring agent is not monitored)
    • agentDescription
    • agentHost
    • agentPort
    • agentAuthToken (the authentication token to access the measuring agent, or an empty string = no access protection | don’t confuse it with the API authTokenValue)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getMeasuringAgentClusters"
}

JSON Response Example:

{
  "measuringAgentClustersArray": [
    {
      "clusterId": 11,
      "createdBySystem": false,
      "ownerUserId": 13,
      "clusterActive": true,
      "clusterDescription": "C1",
      "controllerHost": "192.168.0.50",
      "controllerPort": 8083,
      "controllerAuthToken": "aberaber",
      "clusterMembersArray": [
        {
          "clusterMemberId": 59,
          "loadFactor": 1,
          "agentId": 64,
          "agentActive": true,
          "agentDescription": "Agent 1",
          "agentHost": "192.168.0.10",
          "agentPort": 8080,
          "agentAuthToken": "OrKmAAbyNWEHok"
        },
        {
          "clusterMemberId": 60,
          "loadFactor": 1,
          "agentId": 59,
          "agentActive": true,
          "agentDescription": "Ubuntu 10",
          "agentHost": "192.168.0.110",
          "agentPort": 8080,
          "agentAuthToken": "asc7jhacab"
        },
        {
          "clusterMemberId": 61,
          "loadFactor": 1,
          "agentId": 60,
          "agentActive": true,
          "agentDescription": "Ubuntu 11",
          "agentHost": "192.168.0.111",
          "agentPort": 8080,
          "agentAuthToken": "66ascascsdac"
        }
      ]
    },
    {
      "clusterId": 14,
      "createdBySystem": false,
      "ownerUserId": 13,
      "clusterActive": true,
      "clusterDescription": "C2",
      "controllerHost": "192.168.0.50",
      "controllerPort": 8083,
      "controllerAuthToken": "aberaber",
      "clusterMembersArray": [
        {
          "clusterMemberId": 66,
          "loadFactor": 1,
          "agentId": 56,
          "agentActive": true,
          "agentDescription": "Test System",
          "agentHost": "192.168.0.60",
          "agentPort": 8080,
          "agentAuthToken": "aberdoch"
        },
        {
          "clusterMemberId": 67,
          "loadFactor": 1,
          "agentId": 59,
          "agentActive": true,
          "agentDescription": "Ubuntu 10",
          "agentHost": "192.168.0.110",
          "agentPort": 8080,
          "agentAuthToken": "asc7jhacab"
        }
      ]
    }
  ],
  "isError": false
}

getClusterControllers

Specific Request Fields:

  • [none]

Response Fields (JSON array “clusterControllersArray”):

  • controllerHost (the hostname or IP address of the cluster controller)
  • controllerPort (the IP port of the cluster controller)
  • controllerAuthToken (the authentication token to access the cluster controller)
  • measuringAgentClustersArray (an array of measuring agent clusters which are referencing this cluster controller)
    • clusterId
    • clusterDescription
    • clusterActive

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getClusterControllers"
}

JSON Response Example:

{
  "clusterControllersArray": [
    {
      "controllerHost": "192.168.0.33",
      "controllerPort": 8083,
      "controllerAuthToken": "2fasdtfffe",
      "measuringAgentClustersArray": [
        {
          "clusterId": 11,
          "clusterDescription": "C1",
          "clusterActive": 1
        },
        {
          "clusterId": 13,
          "clusterDescription": "C2",
          "clusterActive": 1
        },
        {
          "clusterId": 14,
          "clusterDescription": "C3",
          "clusterActive": 1
        }
      ]
    },
    {
      "controllerHost": "192.168.0.50",
      "controllerPort": 8083,
      "controllerAuthToken": "asfsdgh763",
      "measuringAgentClustersArray": [
        {
          "clusterId": 15,
          "clusterDescription": "C4",
          "clusterActive": 1
        },
        {
          "clusterId": 16,
          "clusterDescription": "C7",
          "clusterActive": 1
        }
      ]
    }
  ],
  "isError": false
}

getMinRequiredClusterControllerVersion

Specific Request Fields:

  • [none]

Response Fields:

  • minRequiredClusterControllerVersion (the minimum required cluster controller version)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getMinRequiredClusterControllerVersion"
}

JSON Response Example:

{
  "minRequiredClusterControllerVersion":"4.0.4",
  "isError":false
}

pingClusterController

Specific Request Fields:

  • controllerHost (the cluster controller host name or IP address)
  • controllerPort (the cluster controller IP port)
  • controllerAuthToken (the authentication token to access the cluster controller, or an empty string = no access protection)

Response Fields (JSON object “controllerResponse”):

  • pingFromRemoteIp
  • pingFromRemoteUserId
  • productVersion (cluster controller version | don’t confuse with portal server version)
  • osName
  • osVersion
  • javaVersion
  • javaVendor
  • javaMaxMemory
  • systemTime
  • deltaTimeMillis
  • controllerStartupTimeStamp
  • httpExecuteTimeMillis
  • clusterControllerOutdated

Specific Error Flags:

  • controllerHostError
  • controllerPortError
  • controllerVersionOutdatedError
  • controllerNotReachableError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"pingClusterController",
    "controllerHost":"192.168.0.50",
    "controllerPort":8083,
    "controllerAuthToken":"hagsajjs99"
}

JSON Response Example (Success Case):

{
  "controllerResponse": {
    "pingFromRemoteIp": "192.168.0.100",
    "pingFromRemoteUserId": 13,
    "productVersion": "4.0.4",
    "osName": "Linux",
    "osVersion": "4.15.0-135-generic",
    "javaVersion": "11.0.1",
    "javaVendor": "Oracle Corporation",
    "javaMaxMemory": "512 MB",
    "systemTime": 1643406118552,
    "deltaTimeMillis": 1120,
    "controllerStartupTimeStamp": 1643322597013,
    "httpExecuteTimeMillis": 249,
    "clusterControllerOutdated": false
  },
  "isError": false
}

JSON Response Example (Error Case 1):

{
  "isError": true,
  "genericErrorText": "API call pingGetControllerInfo failed. Error code = 18, Error message = Invalid authentication token",
  "controllerHostError": false,
  "controllerVersionOutdatedError": false,
  "controllerNotReachableError": true,
  "controllerPortError": false
}

JSON Response Example (Error Case 2):

{
  "isError": true,
  "genericErrorText": "Min. cluster controller version required: 4.0.4",
  "controllerHostError": false,
  "controllerVersionOutdatedError": true,
  "controllerNotReachableError": false,
  "controllerPortError": false
}

addMeasuringAgentCluster

Specific Request Fields:

  • clusterActive (flag: if false then the availability of the cluster is not monitored)
  • clusterDescription (must be unique across all measuring agent clusters and all measuring agents of the user, cannot be an empty string)
  • controllerHost (the cluster controller host name or IP address)
  • controllerPort (the cluster controller IP port)
  • controllerAuthToken (the authentication token to access the cluster controller, or an empty string = no access protection)
  • clusterMembersArray (an array of cluster members - can also be empty)
    • agentId (the referenced measuring agent id)
    • loadFactor (integer 0..100: the load factor of this cluster member, recommended value = 1)

Response Fields:

  • clusterId (the unique cluster id)
  • clusterMembersArray (the array of cluster members)
    • clusterMemberId (the unique cluster member id)
    • agentId (the referenced measuring agent id)
    • loadFactor (integer 0..100: the load factor of this cluster member)

Specific Error Flags:

  • clusterDescriptionError
  • controllerHostError
  • controllerPortError
  • agentIdError
  • loadFactorError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"addMeasuringAgentCluster",
  "clusterActive":true,
  "clusterDescription":"C7",
  "controllerHost":"192.168.0.50",
  "controllerPort":8083,
  "controllerAuthToken":"aberaber",
  "clusterMembersArray":[
    {
      "agentId":59,
      "loadFactor":1
    },
    {
      "agentId":60,
      "loadFactor":1
    }
  ]
}

JSON Response Example (Success Case):

{
  "clusterId":16,
  "clusterMembersArray":[
    {
      "clusterMemberId":71,
      "agentId":59,
      "loadFactor":1
    },
    {
      "clusterMemberId":72,
      "agentId":60,
      "loadFactor":1
    }
  ],
  "isError":false
}

JSON Response Example (Error Case):

{
  "isError": true,
  "genericErrorText": "Invalid agentId = 101",
  "controllerHostError": false,
  "agentIdError": true,
  "controllerPortError": false,
  "loadFactorError": false,
  "clusterDescriptionError": false
}

addMemberToMeasuringAgentCluster

Specific Request Fields:

  • clusterId
  • agentId (the referenced measuring agent id)
  • loadFactor (integer 0..100: the load factor of this cluster member, recommended value = 1)

Response Fields (JSON object “clusterMember”):

  • clusterMemberId (the unique cluster member id)
  • agentId
  • loadFactor

Specific Error Flags:

  • clusterIdError
  • clusterAccessDeniedError
  • clusterModifyDeniedError
  • agentIdError
  • agentAccessDeniedError
  • loadFactorError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"addMemberToMeasuringAgentCluster",
  "clusterId":17,
  "agentId":64,
  "loadFactor":1
}

JSON Response Example:

{
  "clusterMember": {
    "clusterMemberId": 75,
    "agentId": 64,
    "loadFactor": 1
  },
  "isError": false
}

removeMemberFromMeasuringAgentCluster

Specific Request Fields:

  • clusterId
  • clusterMemberId

Response Fields:

  • [none]

Specific Error Flags:

  • clusterIdError
  • clusterAccessDeniedError
  • clusterModifyDeniedError
  • clusterMemberIdError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"removeMemberFromMeasuringAgentCluster",
  "clusterId":17,
  "clusterMemberId":75
}

JSON Response Example:

{"isError":false}

pingMeasuringAgentClusterMembers

Specific Request Fields:

  • clusterId

Response Fields (JSON object “controllerResponse”):

  • productVersion (cluster controller version)
  • clusterConnectResult (the connect information to the cluster members)
    • measuringAgentClusterMemberArray (the array of cluster members)
      • clusterMemberId
      • loadFactor
      • agentId
      • agentActive
      • agentDescription
      • agentHost
      • agentPort
      • agentAuthToken
    • connectSuccessfulClusterMemberArray (the array of cluster member ids to which the connection was successful established)
    • connectFailedClusterMemberArray (the array of cluster members to which the connection has failed)
      • clusterMemberId
      • errorMessage
    • clusterConnectStartTimestamp
    • clusterConnectDurationMillis
  • clusterActionResult (the ping result of the cluster members)
    • actionSuccessfulClusterMemberArray (the array of cluster member ids which have performed the ping to the measuring agent)
    • actionFailedClusterMemberArray (the array of cluster members which have not performed the ping to the measuring agent)
      • clusterMemberId
      • errorMessage
    • jsonResponseClusterMemberArray (the array of cluster member which have performed the ping)
      • clusterMemberId
      • jsonResponseObject (the pong response of the cluster member)
        • productVersion (measuring agent product version)
        • systemTime
        • deltaTimeMillis (the OS time difference in milliseconds between the cluster controller and the measuring agent)
        • osName
        • osVersion
        • javaVersion
        • javaVendor
        • javaMaxMemory
        • samplingGranularityMillis (the data collector sampling granularity in milliseconds)
        • isError (boolean flag, normally always false)
        • measuringAgentOutdated (a boolean flag, true = measuring agent product version is outdated)
    • clusterActionStartTimestamp
    • clusterActionDurationMillis
  • httpExecuteTimeMillis
  • clusterControllerOutdated

Specific Error Flags:

  • clusterIdError
  • clusterAccessDeniedError
  • controllerVersionOutdatedError
  • controllerNotReachableError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"pingMeasuringAgentClusterMembers",
  "clusterId":16
}

JSON Response Example (Success Case):

{
  "controllerResponse": {
    "productVersion": "4.0.4",
    "clusterConnectResult": {
      "measuringAgentClusterMemberArray": [
        {
          "clusterMemberId": 71,
          "loadFactor": 1,
          "agentId": 59,
          "agentActive": true,
          "agentDescription": "Ubuntu 10",
          "agentHost": "192.168.0.110",
          "agentPort": 8080,
          "agentAuthToken": "agsdhagsj"
        },
        {
          "clusterMemberId": 72,
          "loadFactor": 1,
          "agentId": 60,
          "agentActive": true,
          "agentDescription": "Ubuntu 11",
          "agentHost": "192.168.0.111",
          "agentPort": 8080,
          "agentAuthToken": "nvbjnvbnn"
        }
      ],
      "connectSuccessfulClusterMemberArray": [
        71,
        72
      ],
      "connectFailedClusterMemberArray": [],
      "clusterConnectStartTimestamp": 1643410829270,
      "clusterConnectDurationMillis": 79
    },
    "clusterActionResult": {
      "actionSuccessfulClusterMemberArray": [
        71,
        72
      ],
      "actionFailedClusterMemberArray": [],
      "jsonResponseClusterMemberArray": [
        {
          "clusterMemberId": 71,
          "jsonResponseObject": {
            "productVersion": "4.0.4",
            "systemTime": 1643410829340,
            "deltaTimeMillis": -10,
            "osName": "Linux",
            "osVersion": "5.4.0-92-generic",
            "javaVersion": "11.0.1",
            "javaVendor": "Oracle Corporation",
            "javaMaxMemory": "2048 MB",
            "samplingGranularityMillis": 4000,
            "isError": false,
            "measuringAgentOutdated": false
          }
        },
        {
          "clusterMemberId": 72,
          "jsonResponseObject": {
            "productVersion": "4.0.4",
            "systemTime": 1643410829351,
            "deltaTimeMillis": -10,
            "osName": "Linux",
            "osVersion": "5.4.0-92-generic",
            "javaVersion": "11.0.1",
            "javaVendor": "Oracle Corporation",
            "javaMaxMemory": "2048 MB",
            "samplingGranularityMillis": 4000,
            "isError": false,
            "measuringAgentOutdated": false
          }
        }
      ],
      "clusterActionStartTimestamp": 1643410829349,
      "clusterActionDurationMillis": 43
    },
    "httpExecuteTimeMillis": 1778,
    "clusterControllerOutdated": false
  },
  "isError": false
}

JSON Response Example (Error Case / Partly failed):

{
  "controllerResponse": {
    "productVersion": "4.0.4",
    "clusterConnectResult": {
      "measuringAgentClusterMemberArray": [
        {
          "clusterMemberId": 71,
          "loadFactor": 1,
          "agentId": 59,
          "agentActive": true,
          "agentDescription": "Ubuntu 10",
          "agentHost": "192.168.0.110",
          "agentPort": 8080,
          "agentAuthToken": "marderzahn"
        },
        {
          "clusterMemberId": 72,
          "loadFactor": 1,
          "agentId": 60,
          "agentActive": true,
          "agentDescription": "Ubuntu 11",
          "agentHost": "192.168.0.111",
          "agentPort": 8080,
          "agentAuthToken": "marderzahn"
        }
      ],
      "connectSuccessfulClusterMemberArray": [
        72
      ],
      "connectFailedClusterMemberArray": [
        {
          "clusterMemberId": 71,
          "errorMessage": "Connection refused (Connection refused)"
        }
      ],
      "clusterConnectStartTimestamp": 1643414272214,
      "clusterConnectDurationMillis": 97
    },
    "clusterActionResult": {
      "actionSuccessfulClusterMemberArray": [
        72
      ],
      "actionFailedClusterMemberArray": [],
      "jsonResponseClusterMemberArray": [
        {
          "clusterMemberId": 72,
          "jsonResponseObject": {
            "productVersion": "4.0.4",
            "systemTime": 1643414272310,
            "deltaTimeMillis": -8,
            "osName": "Linux",
            "osVersion": "5.4.0-92-generic",
            "javaVersion": "11.0.1",
            "javaVendor": "Oracle Corporation",
            "javaMaxMemory": "2048 MB",
            "samplingGranularityMillis": 4000,
            "isError": false,
            "measuringAgentOutdated": false
          }
        }
      ],
      "clusterActionStartTimestamp": 1643414272311,
      "clusterActionDurationMillis": 21
    },
    "httpExecuteTimeMillis": 1769,
    "clusterControllerOutdated": false
  }
}

setMeasuringAgentClusterActive

Specific Request Fields:

  • clusterId
  • clusterActive

Response Fields:

  • [none]

Specific Error Flags:

  • clusterIdError
  • clusterAccessDeniedError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"setMeasuringAgentClusterActive",
  "clusterId":16,
  "clusterActive":true
}

JSON Response Example (Success Case):

{"isError":false}

deleteMeasuringAgentCluster

Specific Request Fields:

  • clusterId

Response Fields:

  • [none]

Specific Error Flags:

  • clusterIdError
  • clusterAccessDeniedError
  • clusterDeleteDeniedError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteMeasuringAgentCluster",
  "clusterId":16
}

JSON Response Example (Success Case):

{"isError":false}

getProxyRecorders

Specific Request Fields:

  • [none]

Response Fields (JSON array “proxyRecordersArray”):

  • recorderId (the unique proxy recorder id)
  • createdBySystem (normally false, true = the user cannot modify or delete the proxy recorder)
  • ownerUserId (always the same as the user account id)
  • recorderDescription
  • recorderProxyHost
  • recorderProxyPort (HTTP and HTTPS port of the proxy)
  • recorderProxyAuthUsername (proxy authentication username, or an empty string = no proxy authentication required)
  • recorderProxyAuthPassword (proxy authentication password)
  • recorderControlPort (the proxy recorder control port)
  • recorderControlAuthToken (the authentication token to access the proxy recorder control port, or an empty string = no access protection | don’t confuse it with the API authTokenValue)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getProxyRecorders"
}

JSON Response Example:

{
  "proxyRecordersArray": [
    {
      "recorderId": 3,
      "createdBySystem": false,
      "ownerUserId": 13,
      "recorderDescription": "Erster",
      "recorderProxyHost": "192.168.0.40",
      "recorderProxyPort": 8082,
      "recorderProxyAuthUsername": "",
      "recorderProxyAuthPassword": "",
      "recorderControlPort": 8081,
      "recorderControlAuthToken": ""
    },
    {
      "recorderId": 4,
      "createdBySystem": false,
      "ownerUserId": 13,
      "recorderDescription": "proxy.realload.com",
      "recorderProxyHost": "proxy.realload.com",
      "recorderProxyPort": 8082,
      "recorderProxyAuthUsername": "max.meier",
      "recorderProxyAuthPassword": "123456",
      "recorderControlPort": 8081,
      "recorderControlAuthToken": "aZujkl97zuwert"
    }
  ],
  "isError": false
}

getMinRequiredProxyRecorderVersion

Specific Request Fields:

  • [none]

Response Fields:

  • minRequiredProxyRecorderVersion (the minimum required HTTP/S proxy recorder version)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getMinRequiredProxyRecorderVersion"
}

JSON Response Example (Success Case):

{
  "minRequiredProxyRecorderVersion":"0.2.2",
  "isError":false
}

addProxyRecorder

Specific Request Fields:

  • recorderDescription (must be unique across all HTTP/S proxy recorders of the user, cannot be an empty string)
  • recorderProxyHost
  • recorderProxyPort (HTTP and HTTPS port of the proxy)
  • recorderProxyAuthUsername (proxy authentication username, or an empty string = no proxy authentication required)
  • recorderProxyAuthPassword (proxy authentication password, applied if recorderProxyAuthUsername is not an empty string)
  • recorderControlPort (the proxy recorder control port)
  • recorderControlAuthToken (the authentication token to access the proxy recorder control port, or an empty string = no access protection)

Response Fields (JSON object “proxyRecorder”):

  • recorderId (the unique HTTP/S proxy recorder id)
  • createdBySystem (always false for this function)
  • ownerUserId (always the same as the user account id)
  • recorderDescription
  • recorderProxyHost
  • recorderProxyPort
  • recorderProxyAuthUsername
  • recorderProxyAuthPassword
  • recorderControlPort
  • recorderControlAuthToken

Specific Error Flags:

  • recorderDescriptionError
  • recorderProxyHostError
  • recorderProxyPortError
  • recorderControlPortError
  • maxNumberProxyRecordersLimitExceededError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"addProxyRecorder",
  "recorderDescription":"My New Proxy Recorder",
  "recorderProxyHost":"192.168.0.148",
  "recorderProxyPort":8082,
  "recorderProxyAuthUsername":"max.meier",
  "recorderProxyAuthPassword":"123456",
  "recorderControlPort":8081,
  "recorderControlAuthToken":"aZujkl97zuwert"
}

JSON Response Example (Success Case):

{
  "proxyRecorder": {
    "recorderId": 10,
    "createdBySystem": false,
    "ownerUserId": 13,
    "recorderDescription": "My New Proxy Recorder",
    "recorderProxyHost": "192.168.0.148",
    "recorderProxyPort": 8082,
    "recorderProxyAuthUsername": "max.meier",
    "recorderProxyAuthPassword": "123456",
    "recorderControlPort": 8081,
    "recorderControlAuthToken": "aZujkl97zuwert"
  },
  "isError": false
}

pingProxyRecorder

Specific Request Fields:

  • recorderId

Response Fields (JSON object “pongResponse”):

  • pingFromRemoteIp
  • pingFromRemoteUserId
  • productVersion (the remote proxy recorder version | don’t confuse with portal server version)
  • recorderComponentVersion (the proxy recorder component version | don’t confuse with portal server version)
  • isRecording
  • recordHostFilter
  • numRecordedElements
  • osName
  • osVersion
  • javaMemoryMB
  • javaVersion
  • javaVendor
  • systemTime
  • deltaTimeMillis
  • httpExecuteTimeMillis

Specific Error Flags:

  • recorderIdError
  • recorderAccessDeniedError
  • recorderNotReachableError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"pingProxyRecorder",
  "recorderId":4
}

JSON Response Example (Success Case):

{
  "pongResponse": {
    "pingFromRemoteIp": "83.150.39.44",
    "pingFromRemoteUserId": 13,
    "productVersion": "0.2.0",
    "recorderComponentVersion": "1.1.0",
    "isRecording": false,
    "recordHostFilter": "www.dkfqa.com",
    "numRecordedElements": 0,
    "osName": "Linux",
    "osVersion": "5.4.0-74-generic",
    "javaMemoryMB": 2048,
    "javaVersion": "11.0.1",
    "javaVendor": "Oracle Corporation",
    "systemTime": 1625529858405,
    "deltaTimeMillis": 790,
    "httpExecuteTimeMillis": 88
  },
  "isError": false
}

JSON Response Example (Error Case):

{
  "isError": true,
  "genericErrorText": "connect timed out",
  "recorderNotReachableError": true,
  "recorderIdError": false,
  "recorderAccessDeniedError": false
}

deleteProxyRecorder

Specific Request Fields:

  • recorderId

Response Fields:

  • [none]

Specific Error Flags:

  • recorderIdError
  • recorderAccessDeniedError
  • recorderDeleteDeniedError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteProxyRecorder",
  "recorderId":10
}

JSON Response Example (Success Case):

{"isError":false}

getProxyRecordersClientCertificates

Specific Request Fields:

  • [none]

Response Fields:

  • certificateId (unique ID)
  • ownerUserId (always the own user ID)
  • recorderId (always -1 = apply for all proxy recorders)
  • originalCertFileName (without disk path)
  • certFileType (always PKCS#12)
  • certFileDataB64 (the file data in base64 format)
  • certPassword
  • certDescription
  • certActive (if false then do not transmit the certificate to the proxy recorders)
  • applyForHost
  • applyForPort

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getProxyRecordersClientCertificates"
}

JSON Response Example (Success Case):

{
  "clientCertificatesArray": [
    {
      "certificateId": 1,
      "ownerUserId": 13,
      "recorderId": -1,
      "originalCertFileName": "fischer@dkfqa.com.p12",
      "certFileType": "PKCS#12",
      "certFileDataB64": "MIIRQQ.....CAA=",
      "certPassword": "12345678",
      "certDescription": "Fischer's Cert",
      "certActive": true,
      "applyForHost": "192.168.0.100",
      "applyForPort": 499
    },
    {
      "certificateId": 3,
      "ownerUserId": 13,
      "recorderId": -1,
      "originalCertFileName": "miller@dkfqa.com.p12",
      "certFileType": "PKCS#12",
      "certFileDataB64": "MIIRXB.....CCAA=",
      "certPassword": "abcdefgh",
      "certDescription": "",
      "certActive": false,
      "applyForHost": "www.werwaswo.com",
      "applyForPort": 443
    }
  ],
  "isError": false
}

addProxyRecordersClientCertificate

Specific Request Fields:

  • recorderId (always -1 = apply for all proxy recorders)
  • certFileName (without disk path)
  • certFileType (always PKCS#12)
  • certFileDataB64 (the file data in base64 format)
  • certPassword
  • certDescription (optional, but not null)
  • certActive (if false then do not transmit the certificate to the proxy recorders)
  • applyForHost
  • applyForPort

Response Fields:

  • certificateId (the new unique ID)
  • ownerUserId (always the own user ID)
  • recorderId (always -1 = apply for all proxy recorders)
  • originalCertFileName (without disk path)
  • certFileType (always PKCS#12)
  • certDescription
  • certActive
  • applyForHost
  • applyForPort

Specific Error Flags:

  • certDescriptionError
  • applyForHostError
  • applyForPortError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"addProxyRecordersClientCertificate",
  "recorderId": -1,
  "certFileName": "felix@dkfqa.com.p12",
  "certFileType": "PKCS#12",
  "certFileDataB64": "MIIRQQIBAzCCEQ.....3dUWQICCAA=",
  "certPassword": "mammamia",
  "certDescription":"",
  "certActive": true,
  "applyForHost": "www.dkfqa.com",
  "applyForPort": 443
}

JSON Response Example (Success Case):

{
  "addedClientCertificate": {
    "certificateId": 5,
    "ownerUserId": 13,
    "recorderId": -1,
    "originalCertFileName": "felix@dkfqa.com.p12",
    "certFileType": "PKCS#12",
    "certDescription": "",
    "certActive": true,
    "applyForHost": "www.dkfqa.com",
    "applyForPort": 443
  },
  "isError": false
}

setProxyRecordersClientCertificateActive

Specific Request Fields:

  • certificateId
  • certActive (if false then do not transmit the certificate to the proxy recorders)

Response Fields:

  • certificateId
  • ownerUserId (always the own user ID)
  • recorderId (always -1 = apply for all proxy recorders)
  • originalCertFileName (without disk path)
  • certFileType (always PKCS#12)
  • certDescription
  • certActive
  • applyForHost
  • applyForPort

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"setProxyRecordersClientCertificateActive",
  "certificateId": 5,
  "certActive": false
}

JSON Response Example (Success Case):

{
  "updatedClientCertificate": {
    "certificateId": 5,
    "ownerUserId": 13,
    "recorderId": -1,
    "originalCertFileName": "felix@dkfqa.com.p12",
    "certFileType": "PKCS#12",
    "certDescription": "",
    "certActive": false,
    "applyForHost": "www.dkfqa.com",
    "applyForPort": 443
  },
  "isError": false
}

modifyProxyRecordersClientCertificate

Specific Request Fields:

  • certificateId
  • certDescription (optional but not null)
  • applyForHost (required)
  • applyForPort (required)

Response Fields:

  • certificateId
  • ownerUserId (always the own user ID)
  • recorderId (always -1 = apply for all proxy recorders)
  • originalCertFileName (without disk path)
  • certFileType (always PKCS#12)
  • certDescription
  • certActive
  • applyForHost
  • applyForPort

Specific Error Flags:

  • certDescriptionError
  • applyForHostError
  • applyForPortError

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"modifyProxyRecordersClientCertificate",
  "certificateId": 5,
  "certDescription": "My Cert",
  "applyForHost": "www.dkfqa.com",
  "applyForPort": 444
}

JSON Response Example (Success Case):

{
  "updatedClientCertificate": {
    "certificateId": 5,
    "ownerUserId": 13,
    "recorderId": -1,
    "originalCertFileName": "felix@dkfqa.com.p12",
    "certFileType": "PKCS#12",
    "certDescription": "My Cert",
    "certActive": false,
    "applyForHost": "www.dkfqa.com",
    "applyForPort": 444
  },
  "isError": false
}

deleteProxyRecordersClientCertificate

Specific Request Fields:

  • certificateId

Response Fields:

  • [none]

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteProxyRecordersClientCertificate",
  "certificateId": 5
}

JSON Response Example (Success Case):

{"isError": false}

JSON Response Example (Error Case):

{"isError": true, "genericErrorText": "Invalid certificate Id"}

getTestjobTemplates

Specific Request Fields:

  • [none]

Response Fields:

  • testjobTemplatesArray

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getTestjobTemplates"
}

JSON Response Example (Success Case):

{
  "testjobTemplatesArray": [
    {
      "testjobTemplate": {
        "templateId": 10,
        "templateName": "SimpleTest 1VU",
        "templateDescription": "",
        "testjobProperties": {
          "testjobResultFileName": "",
          "executeMeasuringAgentClusterInfo": "",
          "testjobConcurrentUsers": "1",
          "testjobDescription": "SimpleTest",
          "testjobDebugMeasuring": "false",
          "testjobMaxTestDuration": "30",
          "isClusterJob": "",
          "testjobRampUpTime": "10",
          "localTestjobId": "",
          "testjobMaxLoopsPerUser": "1",
          "productVersion": "4.8.23",
          "remoteTestjobId": "",
          "testjobAdditionalArguments": "",
          "definedFromTestId": "384",
          "testjobSignature": "",
          "testjobDelayPerLoop": "1000",
          "testjobExitCode": "",
          "createTimeStamp": "1694549684261",
          "userInputFieldsFileName": "",
          "testjobType": "load test",
          "enableAutomaticFileSync": "true",
          "testjobState": "invalid",
          "isClusterControllerJobProperties": "",
          "testjobDebugExecution": "false",
          "executeOnUserMeasuringAgentId": "",
          "definedFromTimeZoneId": "Europe/Berlin",
          "executeMeasuringAgentInfo": "",
          "executeOnMeasuringAgentClusterId": ""
        },
        "hasUserInputFields": false
      },
      "testIsHealthy": true
    },
    {
      "testjobTemplate": {
        "templateId": 11,
        "templateName": "SimpleTestWithInputFields 10VU",
        "templateDescription": "",
        "testjobProperties": {
          "testjobResultFileName": "",
          "executeMeasuringAgentClusterInfo": "",
          "testjobConcurrentUsers": "10",
          "testjobDescription": "SimpleTestWithInputFields",
          "testjobDebugMeasuring": "false",
          "testjobMaxTestDuration": "60",
          "isClusterJob": "",
          "testjobRampUpTime": "10",
          "localTestjobId": "",
          "testjobMaxLoopsPerUser": "5",
          "productVersion": "4.8.23",
          "remoteTestjobId": "",
          "testjobAdditionalArguments": "",
          "definedFromTestId": "385",
          "testjobSignature": "",
          "testjobDelayPerLoop": "1000",
          "testjobExitCode": "",
          "createTimeStamp": "1695047319785",
          "userInputFieldsFileName": "InputFields_SimpleTestWithInputFields.json",
          "testjobType": "load test",
          "enableAutomaticFileSync": "true",
          "testjobState": "invalid",
          "isClusterControllerJobProperties": "",
          "testjobDebugExecution": "true",
          "executeOnUserMeasuringAgentId": "",
          "definedFromTimeZoneId": "Europe/Berlin",
          "executeMeasuringAgentInfo": "",
          "executeOnMeasuringAgentClusterId": ""
        },
        "hasUserInputFields": true,
        "userInputFieldsFile": {
          "useShortNames": false,
          "magicPattern": "UserInputFields",
          "productVersion": "4.8.23",
          "userInputFieldsArray": [
            {
              "useShortNames": false,
              "productVersion": "4.8.23",
              "label": "Input Host",
              "variableName": "vHost",
              "inputType": "string",
              "defaultValue": "126.3.4.5",
              "currentValue": "126.3.4.5"
            },
            {
              "useShortNames": false,
              "productVersion": "4.8.23",
              "label": "Input Port",
              "variableName": "vPort",
              "inputType": "integer",
              "defaultValue": "444",
              "currentValue": "444"
            }
          ]
        }
      },
      "testIsHealthy": true
    }
  ],
  "isError": false
}

defineNewTestjobFromTemplate

Specific Request Fields:

  • templateId
  • measuringAgentOrClusterId
  • isCluster (boolean)
  • jobDescription

Response Fields:

  • newTestjobId

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"defineNewTestjobFromTemplate",
  "templateId":10,
  "measuringAgentOrClusterId":65,
  "isCluster":false,
  "jobDescription": "Simple Test"
}

JSON Response Example (Success Case):

{ 
  "newTestjobId":33533,
  "isError":false
}

submitTestjob

Specific Request Fields:

  • localTestjobId

Response Fields:

  • agentResponse (for measuring agents), or
  • controllerResponse (for cluster controllers)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"submitTestjob",
  "localTestjobId":33534
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "agentResponse": {
    "remoteTestjobId": 24129,
    "httpExecuteTimeMillis": 338
  },
  "isError": false
}

makeTestjobReadyToRun

Specific Request Fields:

  • localTestjobId

Response Fields:

  • agentResponse (for measuring agents), or
  • controllerResponse (for cluster controllers)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"makeTestjobReadyToRun",
  "localTestjobId":33534
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "agentResponse": {
    "testjobProperties": {
      "testjobResultFileName": "",
      "executeMeasuringAgentClusterInfo": "",
      "testjobConcurrentUsers": "1",
      "testjobDescription": "Simple Test",
      "testjobDebugMeasuring": "false",
      "testjobMaxTestDuration": "30",
      "isClusterJob": "false",
      "testjobRampUpTime": "10",
      "localTestjobId": "33534",
      "testjobMaxLoopsPerUser": "1",
      "productVersion": "4.8.18",
      "remoteTestjobId": "24129",
      "testjobAdditionalArguments": "",
      "definedFromTestId": "384",
      "testjobSignature": "Z9B963yIj232nht1DIX5aSRt8m77QuXpvVA5736Tl9Wc7UZ7jOiU3l6UC7y3Nx6CXDDZB4qUg4FHDtTuHsD9xd0T/CSq/g==",
      "testjobDelayPerLoop": "1000",
      "testjobExitCode": "",
      "createTimeStamp": "1695050460395",
      "userInputFieldsFileName": "",
      "testjobType": "load test",
      "enableAutomaticFileSync": "true",
      "testjobState": "ready to run",
      "isClusterControllerJobProperties": "",
      "testjobDebugExecution": "false",
      "executeOnUserMeasuringAgentId": "65",
      "definedFromTimeZoneId": "Europe/Berlin",
      "executeMeasuringAgentInfo": "agent2.realload.com:8080",
      "executeOnMeasuringAgentClusterId": ""
    },
    "httpExecuteTimeMillis": 2871
  },
  "isError": false
}

startTestjob

Specific Request Fields:

  • localTestjobId

Response Fields:

  • agentResponse (for measuring agents), or
  • controllerResponse (for cluster controllers)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"startTestjob",
  "localTestjobId":33534
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "agentResponse": {
    "testjobProperties": {
      "testjobResultFileName": "",
      "executeMeasuringAgentClusterInfo": "",
      "testjobConcurrentUsers": "1",
      "testjobDescription": "Simple Test",
      "testjobDebugMeasuring": "false",
      "testjobMaxTestDuration": "30",
      "isClusterJob": "false",
      "testjobRampUpTime": "10",
      "localTestjobId": "33534",
      "testjobMaxLoopsPerUser": "1",
      "productVersion": "4.8.18",
      "remoteTestjobId": "24129",
      "testjobAdditionalArguments": "",
      "definedFromTestId": "384",
      "testjobSignature": "Z9B963yIj232nht1DIX5aSRt8m77QuXpvVA5736Tl9Wc7UZ7jOiU3l6UC7y3Nx6CXDDZB4qUg4FHDtTuHsD9xd0T/CSq/g==",
      "testjobDelayPerLoop": "1000",
      "testjobExitCode": "",
      "createTimeStamp": "1695050460395",
      "userInputFieldsFileName": "",
      "testjobType": "load test",
      "enableAutomaticFileSync": "true",
      "testjobState": "running",
      "isClusterControllerJobProperties": "",
      "testjobDebugExecution": "false",
      "executeOnUserMeasuringAgentId": "65",
      "definedFromTimeZoneId": "Europe/Berlin",
      "executeMeasuringAgentInfo": "agent2.realload.com:8080",
      "executeOnMeasuringAgentClusterId": ""
    },
    "httpExecuteTimeMillis": 249
  },
  "isError": false
}

getMeasuringAgentTestjobs

Specific Request Fields:

  • measuringAgentId
  • synchronizeLocalTestjobsState (boolean)

Response Fields:

  • agentResponse

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getMeasuringAgentTestjobs",
  "measuringAgentId":65,
  "synchronizeLocalTestjobsState":true
}

JSON Response Example (Success Case):

{
  "agentResponse": {
    "allTestjobsArray": [
      {
        "testjobProperties": {
          "testjobResultFileName": "TestResult_RealloadTest_2023-09-09@23-45-34.json",
          "executeMeasuringAgentClusterInfo": "",
          "testjobConcurrentUsers": "1",
          "testjobDescription": "RealloadTest",
          "testjobDebugMeasuring": "false",
          "testjobMaxTestDuration": "60",
          "isClusterJob": "false",
          "testjobRampUpTime": "10",
          "localTestjobId": "30717",
          "testjobMaxLoopsPerUser": "1",
          "productVersion": "4.8.18",
          "remoteTestjobId": "15240",
          "testjobAdditionalArguments": "",
          "definedFromTestId": "379",
          "testjobSignature": "NeYJG4vf8/joAwz4nR4Fie3W0Qr7ijps25ONuk6F5BLghuD8/clPL9mBKc/ogQiic7wI8OXOf53Mi4YDCpg4dQzuaWQ==",
          "testjobDelayPerLoop": "1000",
          "testjobExitCode": "0",
          "createTimeStamp": "1694295923799",
          "userInputFieldsFileName": "",
          "testjobType": "load test",
          "enableAutomaticFileSync": "true",
          "testjobState": "completed",
          "isClusterControllerJobProperties": "",
          "testjobDebugExecution": "true",
          "executeOnUserMeasuringAgentId": "65",
          "definedFromTimeZoneId": "Europe/Berlin",
          "executeMeasuringAgentInfo": "agent2.realload.com:8080",
          "executeOnMeasuringAgentClusterId": ""
        },
        "testProperties": {
          "referencedResourceFilesList": [
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "com.dkfqs.selenium.jar"
            },
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "commons-io-2.13.0.jar"
            },
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "hamcrest-core-1.3.jar"
            },
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "com.dkfqs.tools.jar"
            },
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "junit-4.13.2.jar"
            },
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "selenium_bundle-4.12.1.jar"
            }
          ],
          "jsonConfigObject": {
            "productVersion": "4.8.20",
            "jUnitConfigExecutionClassArray": [
              {
                "productVersion": "4.8.20",
                "executingClassName": "RealloadTest",
                "testMethodNamesArray": [
                  "realload"
                ]
              }
            ],
            "seleniumWebBrowserType": "Firefox"
          },
          "executingScript": "RealloadTest.jar",
          "executingScriptResourceFileList": "",
          "createTimeStamp": "1694293491984",
          "testDescription": "",
          "definedFromTestplanId": "251",
          "productVersion": "4.8.20",
          "definedFromProjectId": "137",
          "visibleTestName": "RealloadTest",
          "scriptType": "Selenium4byJUnit4",
          "testId": "379"
        }
      },
      {
        "testjobProperties": {
          "testjobResultFileName": "TestResult_SimpleTest_2023-09-18@17-01-08.json",
          "executeMeasuringAgentClusterInfo": "",
          "testjobConcurrentUsers": "1",
          "testjobDescription": "Simple Test",
          "testjobDebugMeasuring": "false",
          "testjobMaxTestDuration": "30",
          "isClusterJob": "false",
          "testjobRampUpTime": "10",
          "localTestjobId": "33533",
          "testjobMaxLoopsPerUser": "1",
          "productVersion": "4.8.18",
          "remoteTestjobId": "24112",
          "testjobAdditionalArguments": "",
          "definedFromTestId": "384",
          "testjobSignature": "JYstx+myVNqbyVMr24u4Pi8IXkLGH0iXUquRh5+NL0veKs4DRJX/w2ZT77wLXF70J9YmTPzR7g==",
          "testjobDelayPerLoop": "1000",
          "testjobExitCode": "0",
          "createTimeStamp": "1695049264730",
          "userInputFieldsFileName": "",
          "testjobType": "load test",
          "enableAutomaticFileSync": "true",
          "testjobState": "completed",
          "isClusterControllerJobProperties": "",
          "testjobDebugExecution": "false",
          "executeOnUserMeasuringAgentId": "65",
          "definedFromTimeZoneId": "Europe/Berlin",
          "executeMeasuringAgentInfo": "agent2.realload.com:8080",
          "executeOnMeasuringAgentClusterId": ""
        },
        "testProperties": {
          "referencedResourceFilesList": [
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "com.dkfqs.tools.jar"
            }
          ],
          "jsonConfigObject": {},
          "executingScript": "SimpleTest.jar",
          "executingScriptResourceFileList": "",
          "createTimeStamp": "1694549112155",
          "testDescription": "",
          "definedFromTestplanId": "251",
          "productVersion": "4.8.21",
          "definedFromProjectId": "137",
          "visibleTestName": "SimpleTest",
          "scriptType": "OpenJDK11Jar",
          "testId": "384"
        }
      },
      {
        "testjobProperties": {
          "testjobResultFileName": "TestResult_SimpleTest_2023-09-18@17-51-53.json",
          "executeMeasuringAgentClusterInfo": "",
          "testjobConcurrentUsers": "1",
          "testjobDescription": "Simple Test",
          "testjobDebugMeasuring": "false",
          "testjobMaxTestDuration": "30",
          "isClusterJob": "false",
          "testjobRampUpTime": "10",
          "localTestjobId": "33534",
          "testjobMaxLoopsPerUser": "1",
          "productVersion": "4.8.18",
          "remoteTestjobId": "24129",
          "testjobAdditionalArguments": "",
          "definedFromTestId": "384",
          "testjobSignature": "Z9B963yIj232nht1DIX5aSRt8m77QuXYjP9rn4OBSq/g==",
          "testjobDelayPerLoop": "1000",
          "testjobExitCode": "0",
          "createTimeStamp": "1695050460395",
          "userInputFieldsFileName": "",
          "testjobType": "load test",
          "enableAutomaticFileSync": "true",
          "testjobState": "completed",
          "isClusterControllerJobProperties": "",
          "testjobDebugExecution": "false",
          "executeOnUserMeasuringAgentId": "65",
          "definedFromTimeZoneId": "Europe/Berlin",
          "executeMeasuringAgentInfo": "agent2.realload.com:8080",
          "executeOnMeasuringAgentClusterId": ""
        },
        "testProperties": {
          "referencedResourceFilesList": [
            {
              "projectId": 130,
              "testplanId": 203,
              "fileName": "com.dkfqs.tools.jar"
            }
          ],
          "jsonConfigObject": {},
          "executingScript": "SimpleTest.jar",
          "executingScriptResourceFileList": "",
          "createTimeStamp": "1694549112155",
          "testDescription": "",
          "definedFromTestplanId": "251",
          "productVersion": "4.8.21",
          "definedFromProjectId": "137",
          "visibleTestName": "SimpleTest",
          "scriptType": "OpenJDK11Jar",
          "testId": "384"
        }
      }
    ],
    "httpExecuteTimeMillis": 141
  },
  "isError": false
}

getTestjobOutDirectoryFilesInfo

Specific Request Fields:

  • localTestjobId

Response Fields:

  • filesInfoArray (for measuring agents), or
  • controllerResponse (for cluster controllers)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getTestjobOutDirectoryFilesInfo",
  "localTestjobId":33534
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "measuringAgentData": {
    "agentId": 65,
    "createdBySystem": false,
    "ownerUserId": 13,
    "agentActive": true,
    "agentDescription": "Demo Agent 2",
    "agentHost": "agent2.realload.com",
    "agentPort": 8080,
    "authToken": "************",
    "sharedMeasuringAgentTemplateId": -1,
    "isLaunchedByCloudCredits": false
  },
  "filesInfoArray": [
    {
      "fileName": "DataCollector.err",
      "fileSize": 0,
      "lastModified": 1695051456261,
      "mimeType": "application/octet-stream"
    },
    {
      "fileName": "DataCollector.out",
      "fileSize": 5647,
      "lastModified": 1695052318516,
      "mimeType": "application/octet-stream"
    },
    {
      "fileName": "DataCollector.pid",
      "fileSize": 8,
      "lastModified": 1695051456273,
      "mimeType": "application/octet-stream"
    },
    {
      "fileName": "DataCollector.port",
      "fileSize": 6,
      "lastModified": 1695051457729,
      "mimeType": "application/octet-stream"
    },
    {
      "fileName": "DataCollector.state",
      "fileSize": 2,
      "lastModified": 1695052318512,
      "mimeType": "application/octet-stream"
    },
    {
      "fileName": "TestResult_SimpleTest_2023-09-18@17-51-53.json",
      "fileSize": 7421,
      "lastModified": 1695052315488,
      "mimeType": "application/json"
    },
    {
      "fileName": "user_1_statistics.out",
      "fileSize": 5037,
      "lastModified": 1695052313788,
      "mimeType": "application/octet-stream"
    },
    {
      "fileName": "users.out",
      "fileSize": 1923,
      "lastModified": 1695052314000,
      "mimeType": "application/octet-stream"
    }
  ],
  "isError": false
}

getTestjobOutDirectoryFile

Specific Request Fields:

  • localTestjobId
  • clusterMemberId (optional, -1 = no cluster member file)
  • fileName

Response Fields:

  • filesInfoArray (for measuring agents), or
  • controllerResponse (for cluster controllers)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getTestjobOutDirectoryFile",
  "localTestjobId":33534,
  "fileName":"users.out"
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "measuringAgentData": {
    "agentId": 65,
    "createdBySystem": false,
    "ownerUserId": 13,
    "agentActive": true,
    "agentDescription": "Demo Agent 2",
    "agentHost": "agent2.realload.com",
    "agentPort": 8080,
    "authToken": "************",
    "sharedMeasuringAgentTemplateId": -1,
    "isLaunchedByCloudCredits": false
  },
  "fileContentB64": "MjAyMy0wOS0xOCAxNTo1MTo1Mi42MjYgfCBtYWluIHwgSU5GTyB8IE1heC4gSmF2YSBNZW1vcnkgPSAzMzAwIE1CCjIwMjMtMDktMTggMTU6NTE6NTIuNjU1IHwgbWFpbiB8IElORk8gfCBUaW1lIFpvbmUgPSBDb29yZGluYXRlZCBVbml2ZXJzYWwgVGltZSArMDA6MDAKMjAyMy0wOS0xOCAxNTo1MTo1Mi42NjggfCBtYWluIHwgSU5GTyB8IC0tLSB2dnYgLS0tIHJlc291cmNlIGZpbGVzIC0tLSB2dnYgLS0tCjIwMjMtMDktMTggMTU6NTE6NTIuNjY5IHwgbWFpbiB8IElORk8gfCAtLS0gXl5eIC0tLSByZXNvdXJjZSBmaWxlcyAtLS0gXl5eIC0tLQoyMDIzLTA5LTE4IDE1OjUxOjUyLjY2OSB8IG1haW4gfCBJTkZPIHwgLS0tIHZ2diAtLS0gZ2VuZXJpYyBhcmd1bWVudHMgLS0tIHZ2diAtLS0KMjAyMy0wOS0xOCAxNTo1MTo1Mi42NjkgfCBtYWluIHwgSU5GTyB8IHVzZXJzID0gMQoyMDIzLTA5LTE4IDE1OjUxOjUyLjY2OSB8IG1haW4gfCBJTkZPIHwgZHVyYXRpb24gPSAzMAoyMDIzLTA5LTE4IDE1OjUxOjUyLjY3MCB8IG1haW4gfCBJTkZPIHwgbWF4TG9vcHMgPSAxCjIwMjMtMDktMTggMTU6NTE6NTIuNjcwIHwgbWFpbiB8IElORk8gfCBkZWxheVBlckxvb3AgPSAxMDAwCjIwMjMtMDktMTggMTU6NTE6NTIuNjcwIHwgbWFpbiB8IElORk8gfCByYW1wdXBUaW1lID0gMTAKMjAyMy0wOS0xOCAxNTo1MTo1Mi42NzAgfCBtYWluIHwgSU5GTyB8IGRhdGFPdXRwdXREaXIgPSAvaG9tZS9ka2Zxcy9hZ2VudC91c2Vyc0RhdGEvVXNlcl8xMy9UZXN0Sm9icy9UZXN0am9iXzI0MTI5L291dAoyMDIzLTA5LTE4IDE1OjUxOjUyLjY3MCB8IG1haW4gfCBJTkZPIHwgZGVidWdFeGVjID0gZmFsc2UKMjAyMy0wOS0xOCAxNTo1MTo1Mi42NzEgfCBtYWluIHwgSU5GTyB8IGRlYnVnRGF0YSA9IGZhbHNlCjIwMjMtMDktMTggMTU6NTE6NTIuNjcxIHwgbWFpbiB8IElORk8gfCBkZXNjcmlwdGlvbiA9IFNpbXBsZSBUZXN0CjIwMjMtMDktMTggMTU6NTE6NTIuNjcxIHwgbWFpbiB8IElORk8gfCAtLS0gXl5eIC0tLSBnZW5lcmljIGFyZ3VtZW50cyAtLS0gXl5eIC0tLQoyMDIzLTA5LTE4IDE1OjUxOjUyLjkyMSB8IG1haW4gfCBJTkZPIHwgSlZNIFdhcm11cCBwZXJmb3JtZWQgaW4gMjM2IG1zCjIwMjMtMDktMTggMTU6NTE6NTIuOTc3IHwgbWFpbiB8IElORk8gfCAtLS0gdnZ2IC0tLSBzcGVjaWZpYyBhcmd1bWVudHMgLS0tIHZ2diAtLS0KMjAyMy0wOS0xOCAxNTo1MTo1Mi45OTQgfCBtYWluIHwgSU5GTyB8IHRjcFRpbWVvdXQgPSAxMDAwMAoyMDIzLTA5LTE4IDE1OjUxOjUyLjk5NSB8IG1haW4gfCBJTkZPIHwgc3NsVGltZW91dCA9IDUwMDAKMjAyMy0wOS0xOCAxNTo1MTo1Mi45OTcgfCBtYWluIHwgSU5GTyB8IGh0dHBUaW1lb3V0ID0gMzAwMDAKMjAyMy0wOS0xOCAxNTo1MTo1Mi45OTcgfCBtYWluIHwgSU5GTyB8IC0tLSBeXl4gLS0tIHNwZWNpZmljIGFyZ3VtZW50cyAtLS0gXl5eIC0tLQoyMDIzLTA5LTE4IDE1OjUxOjUzLjAwMiB8IG1haW4gfCBJTkZPIHwgW1N0YXJ0IG9mIFRlc3RdCjIwMjMtMDktMTggMTU6NTE6NTMuMDAzIHwgVXNlci0xIHwgSU5GTyB8IFVzZXIgc3RhcnRlZAoyMDIzLTA5LTE4IDE1OjUxOjUzLjAyNiB8IHBlcmlvZGljLXRocmVhZCB8IElORk8gfCBUaHJlYWQgc3RhcnRlZAoyMDIzLTA5LTE4IDE1OjUxOjUzLjAyNiB8IFVzZXItMSB8IElORk8gfCBTdGFydCBleGVjdXRlIHNlc3Npb24gbm8uIDEgLi4uCjIwMjMtMDktMTggMTU6NTE6NTMuNTk3IHwgVXNlci0xIHwgSU5GTyB8IC4uLiBFbmQgZXhlY3V0ZSBzZXNzaW9uIG5vLiAxIHN1Y2Nlc3NmdWwgcGFzc2VkCjIwMjMtMDktMTggMTU6NTE6NTMuNjQ5IHwgVXNlci0xIHwgSU5GTyB8IFVzZXIgbm9ybWFsIHRlcm1pbmF0ZWQKMjAyMy0wOS0xOCAxNTo1MTo1NC4wMDMgfCBtYWluIHwgSU5GTyB8IFtFbmQgb2YgVGVzdF0K",
  "isError": false
}

saveTestjobOutDirectoryFileToProjectTree

Specific Request Fields:

  • localTestjobId
  • clusterMemberId (optional, -1 = no cluster member file)
  • fileName
  • saveAsFileName
  • projectId
  • resourceSetId

Response Fields:

  • fileHashCode

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"saveTestjobOutDirectoryFileToProjectTree",
  "localTestjobId":33534,
  "fileName":"users.out",
  "saveAsFileName":"users.out",
  "projectId":137,
  "resourceSetId":251
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "fileHashCode": 4557110,
  "isError": false
}

deleteTestjob

Specific Request Fields:

  • localTestjobId
  • deleteAlsoRemoteTestjob (boolean)

Response Fields:

  • localTestjobState (before the testjob was delete)
  • agentResponse (for measuring agents), or
  • controllerResponse (for cluster controllers)

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteTestjob",
  "localTestjobId":33534,
  "deleteAlsoRemoteTestjob":true
}

JSON Response Example (Success Case):

{
  "isClusterJob": false,
  "localTestjobState": "completed",
  "agentResponse": {
    "httpExecuteTimeMillis": 143
  },
  "isError": false
}

getTestExecutionSuites

Specific Request Fields:

  • [none]

Response Fields:

  • testExecutionSuitesArray
    • testExecutionSuite
    • testRunResultsArray

Specific Error Flags:

  • [none]

Values of testRunResultsArray[n].executionState :

  • EXECUTION_STATE_INITIALIZED = 0;
  • EXECUTION_STATE_RUNNING = 1;
  • EXECUTION_STATE_RUN_COMPLETED = 2;
  • EXECUTION_STATE_RUN_FAILED = 3;
  • EXECUTION_STATE_RUN_ABORTED = 4;

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getTestExecutionSuites"
}

JSON Response Example (Success Case):

{
  "testExecutionSuitesArray": [
    {
      "testExecutionSuite": {
        "testExecutionSuiteId": 2,
        "testExecutionSuiteName": "First Suite",
        "testExecutionSuiteDescription": "",
        "topExecuteOnMeasuringAgentOrClusterId": "m-145",
        "topExecutionOrder": 2,
        "totalNumberOfTestjobs": 3,
        "isTestExecutionSuiteHealthy": true
      },
      "testRunResultsArray": [
        {
          "userId": 13,
          "subUserId": -1,
          "testRunId": 1,
          "executionState": 2,
          "annotationArray": [],
          "testRunStartTimestamp": 1699716014207,
          "testRunLastUpdatedTimestamp": 1699716054368,
          "numTestjobTemplatesToExecute": 3,
          "numTestjobTemplatesExecutionPassed": 3,
          "numTestjobTemplatesExecutionFailed": 0,
          "totalSessionsPassed": 8,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 5295,
          "totalSamplesPassed": 12,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 1,
          "maxPendingSamples": 1
        },
        {
          "userId": 13,
          "subUserId": -1,
          "testRunId": 2,
          "executionState": 2,
          "annotationArray": [],
          "testRunStartTimestamp": 1699716108571,
          "testRunLastUpdatedTimestamp": 1699716151826,
          "numTestjobTemplatesToExecute": 3,
          "numTestjobTemplatesExecutionPassed": 3,
          "numTestjobTemplatesExecutionFailed": 0,
          "totalSessionsPassed": 8,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 5503,
          "totalSamplesPassed": 12,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 1,
          "maxPendingSamples": 1
        }
      ]
    },
    {
      "testExecutionSuite": {
        "testExecutionSuiteId": 3,
        "testExecutionSuiteName": "Second Suite Long Run",
        "testExecutionSuiteDescription": "",
        "topExecuteOnMeasuringAgentOrClusterId": "c-27",
        "topExecutionOrder": 1,
        "totalNumberOfTestjobs": 2,
        "isTestExecutionSuiteHealthy": true
      },
      "testRunResultsArray": [
        {
          "userId": 13,
          "subUserId": -1,
          "testRunId": 1,
          "executionState": 2,
          "annotationArray": [],
          "testRunStartTimestamp": 1699715808192,
          "testRunLastUpdatedTimestamp": 1699715867468,
          "numTestjobTemplatesToExecute": 2,
          "numTestjobTemplatesExecutionPassed": 2,
          "numTestjobTemplatesExecutionFailed": 0,
          "totalSessionsPassed": 44,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 4400,
          "totalSamplesPassed": 44,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 4,
          "maxPendingSamples": 4
        },
        {
          "userId": 13,
          "subUserId": -1,
          "testRunId": 2,
          "executionState": 2,
          "annotationArray": [],
          "testRunStartTimestamp": 1699715952212,
          "testRunLastUpdatedTimestamp": 1699716012124,
          "numTestjobTemplatesToExecute": 2,
          "numTestjobTemplatesExecutionPassed": 2,
          "numTestjobTemplatesExecutionFailed": 0,
          "totalSessionsPassed": 44,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 4400,
          "totalSamplesPassed": 44,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 4,
          "maxPendingSamples": 3
        },
        {
          "userId": 13,
          "subUserId": -1,
          "testRunId": 3,
          "executionState": 2,
          "annotationArray": [],
          "testRunStartTimestamp": 1699717843777,
          "testRunLastUpdatedTimestamp": 1699717905674,
          "numTestjobTemplatesToExecute": 2,
          "numTestjobTemplatesExecutionPassed": 2,
          "numTestjobTemplatesExecutionFailed": 0,
          "totalSessionsPassed": 43,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 4300,
          "totalSamplesPassed": 43,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 4,
          "maxPendingSamples": 3
        }
      ]
    }
  ],
  "isError": false
}

getCurrentlyRunningTestExecutionSuiteIds

Specific Request Fields:

  • [none]

Response Fields:

  • testExecutionSuiteIdArray

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getCurrentlyRunningTestExecutionSuiteIds"
}

JSON Response Example (Success Case):

{
  "testExecutionSuiteIdArray": [
    3
  ],
  "isError": false
}

startTestExecutionSuiteTestRun

Specific Request Fields:

  • testExecutionSuiteId
  • testRunAnnotation (optional, can also be an empty string)
  • topExecuteOnMeasuringAgentOrClusterId (the ’top execution’ measuring agent id or cluster id)
  • topExecuteOnCluster (boolean, if true: top execution = cluster id, or false: top execution = measuring agent id)

Response Fields:

  • newTestRunId

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"startTestExecutionSuiteTestRun",
  "testExecutionSuiteId":3,
  "testRunAnnotation":"",
  "topExecuteOnMeasuringAgentOrClusterId":143,
  "topExecuteOnCluster":false
}

JSON Response Example (Success Case):

{
  "newTestRunId": 5,
  "isError": false
}

JSON Response Example (Error Case):

{
  "isError": true,
  "genericErrorText": "Test execution suite currently running"
}

getTestExecutionSuiteTestRunResultDetail

Specific Request Fields:

  • testExecutionSuiteId
  • testRunId

Response Fields:

  • testRunResultDetail
    • testRunResult
    • executedByUserNickname
    • testjobsOutFilesArray

Specific Error Flags:

  • [none]

Values of executionState :

  • EXECUTION_STATE_INITIALIZED = 0;
  • EXECUTION_STATE_RUNNING = 1;
  • EXECUTION_STATE_RUN_COMPLETED = 2;
  • EXECUTION_STATE_RUN_FAILED = 3;
  • EXECUTION_STATE_RUN_ABORTED = 4;

Values of logLevel :

  • LOG_DEBUG = 4;
  • LOG_INFO = 7;
  • LOG_WARN = 8;
  • LOG_ERROR = 9;
  • LOG_FATAL = 10;

Values of elementType :

  • ELEMENT_TYPE_TESTJOB_TEMPLATE = 1;
  • ELEMENT_TYPE_EXECUTION_GROUP = 10;

Values of executionOrder :

  • EXECUTION_ORDER_SEQUENTIAL = 1;
  • EXECUTION_ORDER_PARALLEL = 2;

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getTestExecutionSuiteTestRunResultDetail",
  "testExecutionSuiteId":3,
  "testRunId":5
}

JSON Response Example (Success Case):

{
  "testRunResultDetail": {
    "testRunResult": {
      "userId": 13,
      "subUserId": -1,
      "testExecutionSuite": {
        "testExecutionSuiteId": 3,
        "testExecutionSuiteName": "Second Suite Long Run",
        "testExecutionSuiteDescription": "",
        "topExecuteOnMeasuringAgentOrClusterId": "c-27",
        "topExecutionOrder": 1,
        "testExecutionElementsArray": [
          {
            "productVersion": "4.8.24",
            "elementType": 10,
            "elementId": 737867696,
            "elementInactive": false,
            "executionGroupName": "Top Group",
            "executionGroupDescription": "",
            "executionOrder": 2,
            "executeOnMeasuringAgentOrClusterId": "",
            "executionElementsArray": [
              {
                "productVersion": "4.8.24",
                "elementType": 1,
                "elementId": 1040705161,
                "elementInactive": false,
                "templateId": 18
              },
              {
                "productVersion": "4.8.24",
                "elementType": 1,
                "elementId": 493179849,
                "elementInactive": false,
                "templateId": 18
              }
            ]
          }
        ]
      },
      "testjobTemplatesArray": [
        {
          "templateId": 18,
          "templateName": "Simple Test 2VU Long Run",
          "templateDescription": "",
          "testjobProperties": {
            "testjobResultFileName": "",
            "executeMeasuringAgentClusterInfo": "",
            "testjobConcurrentUsers": "2",
            "testjobDescription": "Simple Test 2VU Long Run",
            "testjobDebugMeasuring": "false",
            "testjobMaxTestDuration": "30",
            "isClusterJob": "",
            "testjobRampUpTime": "1",
            "localTestjobId": "",
            "testjobMaxLoopsPerUser": "-1",
            "productVersion": "4.8.24",
            "remoteTestjobId": "",
            "testjobAdditionalArguments": "",
            "definedFromTestId": "384",
            "testjobSignature": "",
            "testjobDelayPerLoop": "3000",
            "testjobExitCode": "",
            "createTimeStamp": "1697224057753",
            "userInputFieldsFileName": "",
            "testjobType": "load test",
            "enableAutomaticFileSync": "true",
            "testjobState": "invalid",
            "isClusterControllerJobProperties": "",
            "testjobDebugExecution": "false",
            "executeOnUserMeasuringAgentId": "",
            "definedFromTimeZoneId": "Europe/Berlin",
            "executeMeasuringAgentInfo": "",
            "executeOnMeasuringAgentClusterId": ""
          },
          "hasUserInputFields": false
        }
      ],
      "testRunId": 5,
      "executorLog": {
        "userId": 13,
        "testExecutionSuiteId": 3,
        "testRunId": 5,
        "logEntriesArray": [
          {
            "timestamp": 1699726961703,
            "logLevel": 7,
            "threadName": "TestSuiteExecutor-S:3-R:5",
            "testExecutionElementId": -1,
            "logMessage": "Test run #5 started",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726963704,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-G:737867696",
            "testExecutionElementId": 737867696,
            "logMessage": "Execution Group 'Top Group' started",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726963706,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:1040705161",
            "testExecutionElementId": 1040705161,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' execution started",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726963708,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:493179849",
            "testExecutionElementId": 493179849,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' execution started",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726963731,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:1040705161",
            "testExecutionElementId": 1040705161,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' test job defined with id = 34442",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726963753,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:493179849",
            "testExecutionElementId": 493179849,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' test job defined with id = 34443",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726964081,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:1040705161",
            "testExecutionElementId": 1040705161,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' test job submitted to 'Shared Agent 2', remote test job id = 791",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726964128,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:493179849",
            "testExecutionElementId": 493179849,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' test job submitted to 'Shared Agent 2', remote test job id = 792",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726968458,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:493179849",
            "testExecutionElementId": 493179849,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' started on 'Shared Agent 2', remote test job id = 792",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699726968466,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:1040705161",
            "testExecutionElementId": 1040705161,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' started on 'Shared Agent 2', remote test job id = 791",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699727005910,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:493179849",
            "testExecutionElementId": 493179849,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' execution successful",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699727006912,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:1040705161",
            "testExecutionElementId": 1040705161,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' execution successful",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699727012111,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:1040705161",
            "testExecutionElementId": 1040705161,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' execution terminated",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699727012112,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-T:493179849",
            "testExecutionElementId": 493179849,
            "logMessage": "Test Job Template 'Simple Test 2VU Long Run' execution terminated",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699727012113,
            "logLevel": 7,
            "threadName": "TestSuiteExecutorGroup-S:3-R:5-G:737867696",
            "testExecutionElementId": 737867696,
            "logMessage": "Execution Group 'Top Group' terminated",
            "stacktraceArray": []
          },
          {
            "timestamp": 1699727014114,
            "logLevel": 7,
            "threadName": "TestSuiteExecutor-S:3-R:5",
            "testExecutionElementId": -1,
            "logMessage": "Test run #5 done. Execution state = completed",
            "stacktraceArray": []
          }
        ]
      },
      "executionState": 2,
      "annotationArray": [],
      "testRunStartTimestamp": 1699726961701,
      "testRunLastUpdatedTimestamp": 1699727012113,
      "numTestjobTemplatesToExecute": 2,
      "numTestjobTemplatesExecutionPassed": 2,
      "numTestjobTemplatesExecutionFailed": 0,
      "totalSessionsPassed": 44,
      "totalSessionsFailed": 0,
      "totalSessionsPassedTimeSum": 4400,
      "totalSamplesPassed": 44,
      "totalSamplesFailed": 0,
      "maxConcurrentUsers": 4,
      "maxPendingSamples": 2,
      "testRunExecutionGroupResultArray": [
        {
          "testExecutionSuiteId": 3,
          "testRunId": 5,
          "executionGroupElementId": 737867696,
          "containsMeasuringAgent": true,
          "measuringAgent": {
            "agentId": 143,
            "createdBySystem": true,
            "ownerUserId": 13,
            "agentActive": true,
            "agentDescription": "Shared Agent 2",
            "agentHost": "192.168.0.50",
            "agentPort": 8081,
            "sharedMeasuringAgentTemplateId": 5,
            "isLaunchedByCloudCredits": false
          },
          "containsMeasuringAgentCluster": false,
          "executionState": 2,
          "executionStartTimestamp": 1699726963704,
          "executionEndTimestamp": 1699727012113,
          "numTestjobTemplatesExecutionPassed": 2,
          "numTestjobTemplatesExecutionFailed": 0,
          "totalSessionsPassed": 44,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 4400,
          "totalSamplesPassed": 44,
          "totalSamplesFailed": 0
        }
      ],
      "testRunTestjobResultArray": [
        {
          "testExecutionSuiteId": 3,
          "testRunId": 5,
          "testExecutionGroupElementId": 737867696,
          "testjobTemplateElementId": 493179849,
          "testjobTemplate": {
            "templateId": 18,
            "templateName": "Simple Test 2VU Long Run",
            "templateDescription": "",
            "testjobProperties": {
              "testjobResultFileName": "TestResult_SimpleTest_2023-11-11@19-22-49.json",
              "executeMeasuringAgentClusterInfo": "",
              "testjobConcurrentUsers": "2",
              "testjobDescription": "Simple Test 2VU Long Run",
              "testjobDebugMeasuring": "false",
              "testjobMaxTestDuration": "30",
              "isClusterJob": "false",
              "testjobRampUpTime": "1",
              "localTestjobId": "34443",
              "testjobMaxLoopsPerUser": "-1",
              "productVersion": "4.8.24",
              "remoteTestjobId": "792",
              "testjobAdditionalArguments": "",
              "definedFromTestId": "384",
              "testjobSignature": "",
              "testjobDelayPerLoop": "3000",
              "testjobExitCode": "",
              "createTimeStamp": "1697224057753",
              "userInputFieldsFileName": "",
              "testjobType": "load test",
              "enableAutomaticFileSync": "true",
              "testjobState": "invalid",
              "isClusterControllerJobProperties": "",
              "testjobDebugExecution": "false",
              "executeOnUserMeasuringAgentId": "143",
              "definedFromTimeZoneId": "Europe/Berlin",
              "executeMeasuringAgentInfo": "Shared Agent 2",
              "executeOnMeasuringAgentClusterId": ""
            },
            "hasUserInputFields": false
          },
          "executionState": 2,
          "executionStartTimestamp": 1699726963708,
          "executionEndTimestamp": 1699727005710,
          "jobStartTimestamp": 1699726969618,
          "jobEndTimestamp": 1699727003517,
          "totalSessionsPassed": 22,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 2200,
          "totalSamplesPassed": 22,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 2,
          "maxPendingSamples": 1
        },
        {
          "testExecutionSuiteId": 3,
          "testRunId": 5,
          "testExecutionGroupElementId": 737867696,
          "testjobTemplateElementId": 1040705161,
          "testjobTemplate": {
            "templateId": 18,
            "templateName": "Simple Test 2VU Long Run",
            "templateDescription": "",
            "testjobProperties": {
              "testjobResultFileName": "TestResult_SimpleTest_2023-11-11@19-22-49.json",
              "executeMeasuringAgentClusterInfo": "",
              "testjobConcurrentUsers": "2",
              "testjobDescription": "Simple Test 2VU Long Run",
              "testjobDebugMeasuring": "false",
              "testjobMaxTestDuration": "30",
              "isClusterJob": "false",
              "testjobRampUpTime": "1",
              "localTestjobId": "34442",
              "testjobMaxLoopsPerUser": "-1",
              "productVersion": "4.8.24",
              "remoteTestjobId": "791",
              "testjobAdditionalArguments": "",
              "definedFromTestId": "384",
              "testjobSignature": "",
              "testjobDelayPerLoop": "3000",
              "testjobExitCode": "",
              "createTimeStamp": "1697224057753",
              "userInputFieldsFileName": "",
              "testjobType": "load test",
              "enableAutomaticFileSync": "true",
              "testjobState": "invalid",
              "isClusterControllerJobProperties": "",
              "testjobDebugExecution": "false",
              "executeOnUserMeasuringAgentId": "143",
              "definedFromTimeZoneId": "Europe/Berlin",
              "executeMeasuringAgentInfo": "Shared Agent 2",
              "executeOnMeasuringAgentClusterId": ""
            },
            "hasUserInputFields": false
          },
          "executionState": 2,
          "executionStartTimestamp": 1699726963706,
          "executionEndTimestamp": 1699727006711,
          "jobStartTimestamp": 1699726969515,
          "jobEndTimestamp": 1699727003517,
          "totalSessionsPassed": 22,
          "totalSessionsFailed": 0,
          "totalSessionsPassedTimeSum": 2200,
          "totalSamplesPassed": 22,
          "totalSamplesFailed": 0,
          "maxConcurrentUsers": 2,
          "maxPendingSamples": 1
        }
      ]
    },
    "executedByUserNickname": "DKF",
    "testjobsOutFilesArray": [
      {
        "elementId": 1040705161,
        "outFileInfoArray": [
          {
            "fileName": "DataCollector.err",
            "fileSize": 0,
            "lastModified": 1699727006724,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.out",
            "fileSize": 6537,
            "lastModified": 1699727006726,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.pid",
            "fileSize": 7,
            "lastModified": 1699727006736,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.port",
            "fileSize": 6,
            "lastModified": 1699727006737,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.state",
            "fileSize": 1,
            "lastModified": 1699727006738,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "TestResult_SimpleTest_2023-11-11@19-22-49.json",
            "fileSize": 11913,
            "lastModified": 1699727006739,
            "mimeType": "application/json"
          },
          {
            "fileName": "users.out",
            "fileSize": 5507,
            "lastModified": 1699727006741,
            "mimeType": "application/octet-stream"
          }
        ]
      },
      {
        "elementId": 493179849,
        "outFileInfoArray": [
          {
            "fileName": "DataCollector.err",
            "fileSize": 0,
            "lastModified": 1699727005722,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.out",
            "fileSize": 6264,
            "lastModified": 1699727005725,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.pid",
            "fileSize": 7,
            "lastModified": 1699727005733,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.port",
            "fileSize": 6,
            "lastModified": 1699727005734,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "DataCollector.state",
            "fileSize": 1,
            "lastModified": 1699727005735,
            "mimeType": "application/octet-stream"
          },
          {
            "fileName": "TestResult_SimpleTest_2023-11-11@19-22-49.json",
            "fileSize": 11850,
            "lastModified": 1699727005736,
            "mimeType": "application/json"
          },
          {
            "fileName": "users.out",
            "fileSize": 5507,
            "lastModified": 1699727005738,
            "mimeType": "application/octet-stream"
          }
        ]
      }
    ]
  },
  "isError": false
}

getTestExecutionSuiteTestRunTestJobOutputFileContent

Specific Request Fields:

  • testExecutionSuiteId
  • testRunId
  • testjobTemplateElementId (don’t confuse with testjobTemplateId)
  • fileName

Response Fields:

  • fileContentB64

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"getTestExecutionSuiteTestRunTestJobOutputFileContent",
  "testExecutionSuiteId":3,
  "testRunId":5,
  "testjobTemplateElementId":1040705161,
  "fileName":"users.out"
}

JSON Response Example (Success Case):

{
  "fileContentB64": "MjAyMy0xMS0xMSAxO ... uZCBvZiBUZXN0XQo=",
  "isError": false
}

deleteTestExecutionSuiteTestRun

Specific Request Fields:

  • testExecutionSuiteId
  • testRunId

Response Fields:

  • [none]

Specific Error Flags:

  • [none]

JSON Request Example:

{
  "authTokenValue":"jPmFClqeDUXaEk8Q274q",
  "action":"deleteTestExecutionSuiteTestRun",
  "testExecutionSuiteId":3,
  "testRunId":5
}

JSON Response Example (Success Case):

{
  "isError": false
}

5.3 - Remote User Monitoring API

Portal Server Remote User Monitoring API Specification

Abstract

The “Remote User Monitoring API” is a WebSocket API that supports developers to implement their own version of a Synthetic Monitoring “Real Time Dashboard”.

Similar to the Remote User API there is also an “API Authentication Token” required to access the API.

“alt attribute”

API Functions

The API supports the following functions (so-called “actions”):

  • pingPortalServer : Ping the Portal Server at API level.
  • subscribeToHeartbeat : Subscribe to a periodic timestamp which is received every second.
  • unsubscribeFromHeartbeat : Cancel subscription to ‘subscribeToHeartbeat’.
  • subscribeToPeriodicAliveThread: Same functionality as ‘subscribeToHeartbeat’, but the time interval (in seconds) can be specified arbitrarily.
  • unsubscribeFromPeriodicAliveThread: Cancel subscription to ‘subscribeToPeriodicAliveThread’.
  • subscribeToRealtimeMonitor : Subscribe to events when a Monitoring Group or a Monitoring Job is created, modified or deleted, and subscribe to be notified each time when a Monitoring Group or a Monitoring Job is executed.
  • unsubscribeFromRealtimeMonitor : Cancel subscription to ‘subscribeToRealtimeMonitor’.
  • subscribeToMonitoringAlerts : Subscribe to monitoring alerts.
  • unsubscribeFromMonitoringAlerts : Cancel subscription to ‘subscribeToMonitoringAlerts’.
  • subscribeToGroupExecutionHistoryCache : Subscribe to a monitoring group’s execution history cache, which contains the measured data over the last 24 hours. The received data can be used to draw a chart or sparkline of the corresponding monitoring group.
  • unsubscribeFromAllGroupExecutionHistoryCaches : Cancel subscription to ‘subscribeToGroupExecutionHistoryCache’ for all monitoring groups.
  • subscribeToJobExecutionHistoryCache : Subscribe to a monitoring job’s execution history cache, which contains the measured data over the last 24 hours. The received data can be used to draw a chart or sparkline of the corresponding monitoring job.
  • unsubscribeFromAllJobExecutionHistoryCaches : Cancel subscription to ‘subscribeToJobExecutionHistoryCache’ for all monitoring jobs.
WebSocket URL
wss://portal.realload.com/RemoteUserMonitoringAPI
HTTP Request Header Fields
x-api-key: <API Authentication Token> required
subscribeToHeartbeat: true optional
subscribeToRealtimeMonitor: true optional

The subscribed data are received in JSON format. The received JSON values are never null.

Runtime Behavior

  • When the WebSocket connection is closed, all subscriptions are terminated.
  • When you send an “action”, the action is confirmed with a response frame with the name of the action repeated (except for “pingPortalServer”).
  • If you send an invalid “action”, the WebSocket connection will be closed.
  • The WebSocket connection doesn’t last forever. It will be closed after a few hours by the Portal Server. You then have to reestablish the WebSocket connection and renew the subscriptions.

Example:

Send: {"action":"subscribeToHeartbeat"}
Received: {"action":"subscribeToHeartbeat","isError":false}
Received: {"action":"heartbeat","timestamp":1708466543962,"isError":false}
Received: {"action":"heartbeat","timestamp":1708466544963,"isError":false}
Received: {"action":"heartbeat","timestamp":1708466545963,"isError":false}

Implementation Recommendations

  • We recommend that you implement two hash maps on the client side which contain the current configuration data of the monitoring jobs and monitoring groups - based on the actions of onMonitoringGroupEvent and onMonitoringJobEvent received from subscribeToRealtimeMonitor (hash map keys = Monitoring Group ID, Monitoring Job ID).
  • Preferably use a WebSocket client framework that can detect the closure of the WebSocket connection to the server. If this is not possible, use “subscribeToPeriodicAliveThread” or “pingPortalServer”.

pingPortalServer

Example:

Send: {"action":"pingPortalServer"}
Received: {"action":"pongFromPortalServer","portalServerVersion":"4.8.35","portalServerTimestamp":1708468300516,"isError":false}

subscribeToHeartbeat

  • Send action : “subscribeToHeartbeat”
{
  "action":"subscribeToHeartbeat"
}
  • Received action: “heartbeat”
{
  "action":"heartbeat",
  "timestamp":1677521478771,
  "isError":false
}

subscribeToPeriodicAliveThread

  • Send action : “subscribeToPeriodicAliveThread”
{
  "action":"subscribeToPeriodicAliveThread",
  "periodicIntervalSeconds":60
}
  • Received action: “periodicAlive”
{
  "action":"periodicAlive",
  "timestamp":1708469366167,
  "isError":false
}

subscribeToRealtimeMonitor

  • Send action : “subscribeToRealtimeMonitor”
{
  "action":"subscribeToRealtimeMonitor"
}
  • Received actions:
    • onMonitoringGroupEvent(The configuration data of a monitoring group)
    • onMonitoringJobEvent(The configuration data of a monitoring job)
    • onMonitoringGroupExecuted(The execution result of a monitoring group)
    • onMonitoringJobExecuted(The execution result of a monitoring job)
    • onMonitoringGroupTemporarilyEvent(Notification whether a monitoring group is temporarily disabled)
    • onMonitoringJobTemporarilyEvent(Notification whether a monitoring job is temporarily disabled)

As first you will receive instantly the configuration data of all already defined Monitoring Groups and Monitoring Jobs, and also the data of the last Execution Result of the Monitoring Groups and Monitoring Jobs (if they have been executed in the past).

After that you will receive update information of all Monitoring Groups and Monitoring Jobs when they are new-defined/modified/deleted, and you will receive also update information about the result of executed Monitoring Groups and Monitoring Jobs.

Because monitoring groups and jobs can be temporarily disabled by configuring one or multiple “Monitoring Downtime(s)” you receive the actions “onMonitoringGroupTemporarilyEvent” and “onMonitoringJobTemporarilyEvent”. “alt attribute”

Notifications for Monitoring Groups

  • Received action: “onMonitoringGroupEvent”
  • monitoringGroupAction:
    • “groupExists”
    • “groupAdded”
    • “groupModified”
    • “groupDeleted”
{
  "action": "onMonitoringGroupEvent",
  "monitoringGroupId": 34,
  "monitoringEvent": {
    "eventTimestamp": 1708541771774,
    "monitoringGroup": {
      "monitoringGroupId": 34,
      "ownerUserId": 13,
      "title": "First Group",
      "description": "",
      "maxDataStorageDays": 14,
      "agentType": "agent",
      "executionEnabled": true,
      "executionInterval": 300,
      "executionTimeout": 300,
      "currentExecutionState": "executing",
      "lastExecutionStartTime": 1708541763897
    },
    "monitoringGroupAction": "groupExists"
  },
  "isError": false
}

The executionInterval and the executionTimeout are in seconds. The lastExecutionStartTime has a value of -1 (minus one) if the Monitoring Group was never executed. The agentType has currently no meaning.

Any modification of a Monitoring Group made by the Portal Server Web Interface such as modify executionEnabled or title effects that a new frame with monitoringGroupAction = “groupModified” is received.

Notification for Monitoring Jobs

  • Received action: “onMonitoringJobEvent”
  • monitoringJobAction:
    • “jobExists”
    • “jobAdded”
    • “jobModified”
    • “jobDeleted”
{
  "action": "onMonitoringJobEvent",
  "monitoringGroupId": 36,
  "monitoringJobId": 79,
  "monitoringEvent": {
    "eventTimestamp": 1708541771790,
    "monitoringJobProperties": {
      "performanceErrorAlertThresholdMillis": "-1",
      "enableJobExecution": "true",
      "performanceWarningAlertsEnabled": "false",
      "testjobConcurrentUsers": "1",
      "testjobDelayPerLoop": "1000",
      "testjobDebugMeasuring": "false",
      "performanceErrorAlertsEnabled": "false",
      "enableAutomaticFileSync": "false",
      "performanceWarningAlertThresholdMillis": "-1",
      "testjobMaxTestDuration": "60",
      "testjobDebugExecution": "false",
      "testjobRampUpTime": "5",
      "definedFromMonitoringGroupId": "36",
      "testjobMaxLoopsPerUser": "1",
      "productVersion": "4.8.35",
      "monitoringJobId": "79",
      "monitoringJobTitle": "Local_Cinema",
      "lastExecutionStartTime": "1708541731138",
      "testjobAdditionalArguments": "",
      "definedFromTestId": "399",
      "testjobUserInputFieldsValuesJsonArray": "[]",
      "monitoringJobDescription": ""
    },
    "monitoringTestProperties": {
      "referencedResourceFilesList": [
        {
          "projectId": 130,
          "testplanId": 203,
          "fileName": "com.dkfqs.tools.jar"
        }
      ],
      "jsonConfigObject": {},
      "executingScript": "Local_Cinema.jar",
      "executingScriptResourceFileList": "",
      "createTimeStamp": "1708293851731",
      "testDescription": "",
      "definedFromTestplanId": "287",
      "productVersion": "4.8.34",
      "definedFromProjectId": "137",
      "visibleTestName": "Local_Cinema",
      "scriptType": "OpenJDK11Jar",
      "testId": "399"
    },
    "monitoringTestIsHealthy": true,
    "monitoringJobAction": "jobExists"
  },
  "isError": false
}

The testjobMaxTestDuration and the testjobRampUpTime are in seconds. The testjobDelayPerLoop are in milliseconds. The lastExecutionStartTime has a value of -1 (minus one) if the Monitoring Job was never executed.

Any modification of a Monitoring Job made by the Portal Server Web Interface such as modify enableJobExecution or monitoringJobTitle effects that a new frame with monitoringJobAction = “jobModified” is received.

Notification for Monitoring Group Execution

  • Received action: “onMonitoringGroupExecuted”
  • oldEvent: true or false | Note: the oldEvent is true at the first time (if the monitoring group was executed in the past) and then will always be false each time when the monitoring group is executed.
{
  "action": "onMonitoringGroupExecuted",
  "monitoringGroupId": 34,
  "monitoringEvent": {
    "eventTimestamp": 1708541771806,
    "oldEvent": true,
    "groupExecutionResult": {
      "magicPattern": "MonitoringGroupExecutionResult",
      "productVersion": "4.8.35",
      "userId": 13,
      "monitoringGroupId": 34,
      "groupExecutionId": 39412,
      "resultTimestamp": 1708541492973,
      "monitoringJobIdToLocalTestjobIdArray": [
        {
          "monitoringJobId": 60,
          "localTestjobId": 73932
        },
        {
          "monitoringJobId": 61,
          "localTestjobId": 73934
        },
        {
          "monitoringJobId": 62,
          "localTestjobId": 73933
        }
      ],
      "groupSystemStatus": 0,
      "groupCombinedTestjobState": "completed",
      "groupExecutionLogArray": [
        "2024-02-21 19:51:03.915 |  | INFO | Monitoring Group Execution Thread started",
        "2024-02-21 19:51:32.972 |  | INFO | Monitoring Job Id 60 executed",
        "2024-02-21 19:51:32.972 |  | INFO | Monitoring Job Id 62 executed",
        "2024-02-21 19:51:32.972 |  | INFO | Monitoring Job Id 61 executed",
        "2024-02-21 19:51:32.973 |  | INFO | Monitoring Group Execution Thread terminated"
      ],
      "numberOfPassedSamples": 10,
      "numberOfFailedSamples": 0,
      "numberOfPassedLoops": 6,
      "numberOfFailedLoops": 0,
      "sumErrorsSeverityWarning": 0,
      "sumErrorsSeverityError": 0,
      "sumErrorsSeverityFatal": 0
    }
  },
  "hasGroupExecutionStatistic": true,
  "groupExecutionStatistic": {
    "userId": 13,
    "monitoringGroupId": 34,
    "fromTimestamp": 1708455371812,
    "toTimestamp": 1708541771812,
    "systemAvailabilityPercent": 100,
    "measuredAvailabilityPercent": 100,
    "measuredFaultyTimeSumSeconds": 0
  },
  "isError": false
}

groupSystemStatus:

  • 0: Healthy = All monitoring jobs of the monitoring group have been started on all measuring agents.
  • -1 (minus one): Partial Malfunction = Some monitoring jobs of the monitoring group have not been started on some measuring agents.
  • -2 (minus two): Malfunction = No monitoring jobs of the monitoring group have been started on any measuring agent.

groupCombinedTestjobState

  • “completed”: All test jobs of the monitoring group have been successfully executed on all measuring agents.
  • “partial failed”: The execution of some of the test jobs of the monitoring group did fail on some measuring agents.
  • “failed”: The execution of all of the test jobs of the monitoring group did fail on all measuring agents.

Hint for groupCombinedTestjobState : Note that is value does not reflect if any warnings or errors have been measured by the test jobs.

The Reference Implementation on the Real Load Portal Server shows the measured (sumErrorsSeverityError + sumErrorsSeverityFatal) simple as “number of measured errors”.

Notification for Monitoring Job Execution

  • Received action: “onMonitoringJobExecuted”
  • oldEvent: true or false | Note: the oldEvent is true at the first time (if the monitoring job was executed in the past) and then will always be false each time when the monitoring job is executed.
{
  "action": "onMonitoringJobExecuted",
  "monitoringGroupId": 36,
  "monitoringJobId": 79,
  "monitoringEvent": {
    "eventTimestamp": 1708541771849,
    "oldEvent": true,
    "jobExecutionResult": {
      "magicPattern": "MonitoringJobExecutionResult",
      "productVersion": "4.8.35",
      "userId": 13,
      "monitoringGroupId": 36,
      "groupExecutionId": 39417,
      "monitoringJobId": 79,
      "localTestjobId": 73939,
      "resultTimestamp": 1708541748100,
      "systemStatus": 0,
      "jobExecutionLogArray": [
        "2024-02-21 19:55:31.137 |  | INFO | Monitoring Job Execution Thread started",
        "2024-02-21 19:55:31.495 | Shared Agent 1 | INFO | Job submitted to measuring agent, remote testjob id = 19015",
        "2024-02-21 19:55:38.034 | Shared Agent 1 | INFO | Job executed on measuring agent, exit code = process executed successfully",
        "2024-02-21 19:55:40.014 | Shared Agent 2 | INFO | Job submitted to measuring agent, remote testjob id = 4864",
        "2024-02-21 19:55:46.446 | Shared Agent 2 | INFO | Job executed on measuring agent, exit code = process executed successfully",
        "2024-02-21 19:55:48.100 |  | INFO | Monitoring Job Execution Thread terminated"
      ],
      "combinedTestjobState": "completed",
      "hasCombinedTestResult": true,
      "combinedTestResultFileName": "TestResult_MonitoringJob_79_2024-02-21@19-55-48.json",
      "combinedTestResultSummaryStatistic": {
        "productVersion": "4.8.35",
        "localTestjobId": 73939,
        "remoteTestjobId": 39417,
        "samplingGranularity": 4000,
        "jobStartTimestamp": 1708541736385,
        "jobEndTimestamp": 1708541744994,
        "osCpuLoadPercent": -1,
        "numberOfPassedLoopsTimeSum": 178,
        "numberOfConcurrentUsers": 0,
        "currentlyPendingSamples": 0,
        "numberOfPassedSamples": 2,
        "numberOfFailedSamples": 0,
        "numberOfPassedLoops": 2,
        "numberOfFailedLoops": 0,
        "numberOfMeasuredErrors": 0,
        "numErrorsSeverityWarning": 0,
        "numErrorsSeverityError": 0,
        "numErrorsSeverityFatal": 0
      },
      "averagePassedLoopTime": 89,
      "sumErrorsSeverityWarning": 0,
      "sumErrorsSeverityError": 0,
      "sumErrorsSeverityFatal": 0
    }
  },
  "hasJobExecutionStatistic": true,
  "jobExecutionStatistic": {
    "userId": 13,
    "monitoringGroupId": 36,
    "monitoringJobId": 79,
    "fromTimestamp": 1708455371849,
    "toTimestamp": 1708541771849,
    "systemAvailabilityPercent": 100,
    "measuredAvailabilityPercent": 100,
    "measuredFaultyTimeSumSeconds": 0,
    "averagePassedLoopTime": 93,
    "hasPassedLoopTimePercentileArray": false
  },
  "isError": false
}

systemStatus:

  • 0: Healthy = The monitoring job have been started on all measuring agents.
  • -1 (minus one): Partial Malfunction = The monitoring job was not started on some measuring agents.
  • -2 (minus two): Malfunction = The monitoring job was not started on any measuring agent.

combinedTestjobState

  • “completed”: The testjob was successfully executed on all measuring agents.
  • “partial failed”: The execution of the testjob did fail on some measuring agents.
  • “failed”: The execution of the testjob did fail on all measuring agents.

Hint for combinedTestjobState : Note that is value does not reflect if any warnings or errors have been measured by the test jobs.

The Reference Implementation of the Monitoring Dashboard on the RealLoad Portal Server shows the measured (sumErrorsSeverityError + sumErrorsSeverityFatal) simple as “number of measured errors”.

Notification for Monitoring Group Temporarily Event

  • Received action: “onMonitoringGroupTemporarilyEvent”
  • monitoringGroupTemporarilyAction: “groupExecutionTemporarilyNotDisabled” or “groupExecutionTemporarilyDisabled”
{
  "action": "onMonitoringGroupTemporarilyEvent",
  "monitoringGroupId": 34,
  "monitoringEvent": {
    "eventTimestamp": 1708541771851,
    "monitoringGroup": {
      "monitoringGroupId": 34,
      "ownerUserId": 13,
      "title": "First Group",
      "description": "",
      "maxDataStorageDays": 14,
      "agentType": "agent",
      "executionEnabled": true,
      "executionInterval": 300,
      "executionTimeout": 300,
      "currentExecutionState": "executing",
      "lastExecutionStartTime": 1708541763897
    },
    "monitoringGroupTemporarilyAction": "groupExecutionTemporarilyNotDisabled"
  },
  "isError": false
}

Notification for Monitoring Job Temporarily Event

  • Received action: “onMonitoringJobTemporarilyEvent”
  • monitoringJobTemporarilyAction: “jobExecutionTemporarilyNotDisabled” or “jobExecutionTemporarilyDisabled”
{
  "action": "onMonitoringJobTemporarilyEvent",
  "monitoringGroupId": 36,
  "monitoringJobId": 79,
  "monitoringEvent": {
    "eventTimestamp": 1708541771854,
    "monitoringJobProperties": {
      "performanceErrorAlertThresholdMillis": "-1",
      "enableJobExecution": "true",
      "performanceWarningAlertsEnabled": "false",
      "testjobConcurrentUsers": "1",
      "testjobDelayPerLoop": "1000",
      "testjobDebugMeasuring": "false",
      "performanceErrorAlertsEnabled": "false",
      "enableAutomaticFileSync": "false",
      "performanceWarningAlertThresholdMillis": "-1",
      "testjobMaxTestDuration": "60",
      "testjobDebugExecution": "false",
      "testjobRampUpTime": "5",
      "definedFromMonitoringGroupId": "36",
      "testjobMaxLoopsPerUser": "1",
      "productVersion": "4.8.35",
      "monitoringJobId": "79",
      "monitoringJobTitle": "Local_Cinema",
      "lastExecutionStartTime": "1708541731138",
      "testjobAdditionalArguments": "",
      "definedFromTestId": "399",
      "testjobUserInputFieldsValuesJsonArray": "[]",
      "monitoringJobDescription": ""
    },
    "monitoringJobTemporarilyAction": "jobExecutionTemporarilyNotDisabled"
  },
  "isError": false
}

subscribeToMonitoringAlerts

  • Send action : “subscribeToMonitoringAlerts”
{
  "action":"subscribeToMonitoringAlerts"
}
  • Received action: “onMonitoringAlert”
  • alertLevel: “Monitoring Group”, “Monitoring Job” or “Monitoring Job Measuring Agent”
  • alertType: “New Alert”, “Modified Alert”, “Repeated Alert” or “Canceled Alert”
{
  "action": "onMonitoringAlert",
  "monitoringEvent": {
    "eventTimestamp": 1708547148410,
    "monitoringAlert": {
      "userId": 13,
      "alertLevel": "Monitoring Group",
      "alertType": "New Alert",
      "alertId": 579,
      "alertContext": {
        "alertIdUpdateCounter": 0,
        "currentSystemStatus": "Healthy",
        "currentExecutionState": "Successful",
        "currentNumMeasuredWarnings": 0,
        "currentNumMeasuredErrors": 2,
        "previousSystemStatus": "Healthy",
        "previousExecutionState": "Successful",
        "previousNumMeasuredWarnings": 0,
        "previousNumMeasuredErrors": 0
      },
      "monitoringGroupId": 36,
      "monitoringGroupExecutionResult": {
        "magicPattern": "MonitoringGroupExecutionResult",
        "productVersion": "4.8.35",
        "userId": 13,
        "monitoringGroupId": 36,
        "groupExecutionId": 39525,
        "resultTimestamp": 1708547148390,
        "monitoringJobIdToLocalTestjobIdArray": [
          {
            "monitoringJobId": 79,
            "localTestjobId": 74083
          }
        ],
        "groupSystemStatus": 0,
        "groupCombinedTestjobState": "completed",
        "groupExecutionLogArray": [
          "2024-02-21 21:25:31.365 |  | INFO | Monitoring Group Execution Thread started",
          "2024-02-21 21:25:48.383 |  | INFO | Monitoring Job Id 79 executed",
          "2024-02-21 21:25:48.390 |  | INFO | Monitoring Group Execution Thread terminated"
        ],
        "numberOfPassedSamples": 0,
        "numberOfFailedSamples": 2,
        "numberOfPassedLoops": 0,
        "numberOfFailedLoops": 2,
        "sumErrorsSeverityWarning": 0,
        "sumErrorsSeverityError": 2,
        "sumErrorsSeverityFatal": 0
      }
    }
  },
  "isError": false
}

subscribeToGroupExecutionHistoryCache

  • Send action : “subscribeToGroupExecutionHistoryCache”
{
  "action": "subscribeToGroupExecutionHistoryCache",
  "monitoringGroupId": 36
}

Error Case

{
  "action": "subscribeToGroupExecutionHistoryCache",
  "isError": true,
  "genericErrorText": "subscribeToGroupExecutionHistoryCache failed: java.lang.IllegalArgumentException: Invalid monitoringGroupId : 35"
}

Received actions:

  • onMonitoringGroupExecutionHistoryInitialEntries(history data of the last 24 hours)
  • onMonitoringGroupExecutionHistoryNewEntry(last update of history data)
{
  "action": "onMonitoringGroupExecutionHistoryInitialEntries",
  "monitoringGroupId": 36,
  "monitoringEvent": {
    "eventTimestamp": 1708548777246,
    "historyEntriesArray": [
      {
        "t": 1708463444623,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 0,
        "f": 0,
        "gid": 37851
      },
      {
        "t": 1708463504557,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 0,
        "f": 0,
        "gid": 37853
      },
      {
        "t": 1708463563635,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 0,
        "f": 0,
        "gid": 37854
      },
      {
        "t": 1708548588515,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 2,
        "f": 0,
        "gid": 39554
      },
      {
        "t": 1708548648466,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 2,
        "f": 0,
        "gid": 39555
      },
      {
        "t": 1708548708464,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 2,
        "f": 0,
        "gid": 39557
      },
      {
        "t": 1708548768449,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "w": 0,
        "e": 2,
        "f": 0,
        "gid": 39558
      }
    ]
  },
  "isError": false
}
{
  "action": "onMonitoringGroupExecutionHistoryNewEntry",
  "monitoringGroupId": 36,
  "monitoringEvent": {
    "eventTimestamp": 1708549188525,
    "historyEntry": {
      "t": 1708549188468,
      "ge": 60,
      "ss": 0,
      "ct": "completed",
      "w": 0,
      "e": 2,
      "f": 0,
      "gid": 39566
    }
  },
  "isError": false
}
  • t : Unix-like timestamp
  • ge : Monitoring group execution interval (seconds)
  • ss : Group system status: 0 = HEALTHY, -1 = PARTIAL MALFUNCTION, -2 = MALFUNCTION
  • ct : Group combined test job state: “completed”, “partial failed” or “failed”
  • w : Sum of errors with severity warning
  • e : Sum of errors with severity error
  • f : Sum of errors with severity fatal
  • gid : Group execution ID

subscribeToJobExecutionHistoryCache

  • Send action : “subscribeToJobExecutionHistoryCache”
{
  "action": "subscribeToJobExecutionHistoryCache",
  "monitoringJobId": 79
}

Error Case

{
  "action": "subscribeToJobExecutionHistoryCache",
  "isError": true,
  "genericErrorText": "subscribeToJobExecutionHistoryCache failed: java.lang.IllegalArgumentException: Invalid monitoringJobId : 80"
}

Received actions:

  • onMonitoringJobExecutionHistoryInitialEntries(history data of the last 24 hours)
  • onMonitoringJobExecutionHistoryNewEntry(last update of history data)
{
  "action": "onMonitoringJobExecutionHistoryInitialEntries",
  "monitoringGroupId": 36,
  "monitoringJobId": 79,
  "monitoringEvent": {
    "eventTimestamp": 1708551216713,
    "historyEntriesArray": [
      {
        "t": 1708464823243,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": 118,
        "w": 0,
        "e": 0,
        "f": 0,
        "lid": 71889,
        "h": true
      },
      {
        "t": 1708464882694,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": 107,
        "w": 0,
        "e": 0,
        "f": 0,
        "lid": 71890,
        "h": true
      },
      {
        "t": 1708464943399,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": 96,
        "w": 0,
        "e": 0,
        "f": 0,
        "lid": 71891,
        "h": true
      },
      {
        "t": 1708550987656,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": -1,
        "w": 0,
        "e": 2,
        "f": 0,
        "lid": 74186,
        "h": true
      },
      {
        "t": 1708551047722,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": -1,
        "w": 0,
        "e": 2,
        "f": 0,
        "lid": 74187,
        "h": true
      },
      {
        "t": 1708551108599,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": -1,
        "w": 0,
        "e": 2,
        "f": 0,
        "lid": 74191,
        "h": true
      },
      {
        "t": 1708551167586,
        "ge": 60,
        "ss": 0,
        "ct": "completed",
        "al": -1,
        "w": 0,
        "e": 2,
        "f": 0,
        "lid": 74192,
        "h": true
      }
    ]
  },
  "isError": false
}
{
  "action": "onMonitoringJobExecutionHistoryNewEntry",
  "monitoringGroupId": 36,
  "monitoringJobId": 79,
  "monitoringEvent": {
    "eventTimestamp": 1708551228189,
    "historyEntry": {
      "t": 1708551228183,
      "ge": 60,
      "ss": 0,
      "ct": "completed",
      "al": -1,
      "w": 0,
      "e": 2,
      "f": 0,
      "lid": 74193,
      "h": true
    }
  },
  "isError": false
}
  • t : Unix-like timestamp
  • ge : Monitoring group execution interval (seconds)
  • ss : System status: 0 = HEALTHY, -1 = PARTIAL MALFUNCTION, -2 = MALFUNCTION
  • ct : Combined test job state: “completed”, “partial failed” or “failed”
  • al : Average passed loop time (milliseconds), -1 = no data
  • w : Sum of errors with severity warning
  • e : Sum of errors with severity error
  • f : Sum of errors with severity fatal
  • lid : Local test job ID
  • h: Has combined test result: true or false

Other Received Actions

You will also receive other actions than previously described above. These are confirmations that you can ignore. Example:

{
    "action":"subscribeToRealtimeMonitor",
    "isError":false
}

6 - User Guide

Portal Server User Guide

Thank you for using the Real Load product.

On this first page of the user guide you will learn:

  • How to perform a simple HTTP/S test
  • The basic concepts of the product
  • How to determine the system capacity of a stressed server

Projects Menu

“alt attribute”

After you have signed in to the portal server, you will see a navigation bar whose first menu on the left is Projects. This is a file browser that displays all of your files. The view is divided into Projects and Resource Sets which contain the files for executing your tests and the measured test results. You can also store additional files in a Resource Set which contain e.g. instructions on how a test should be performed.

Among other things, you can:

  • Create, clone and delete projects.
  • Create, clone and delete resource sets (subdirectories of projects).
  • Upload and download files, create new files, display the content of files, edit and delete files.

There is also a recycle bin from which you can restore deleted projects, resources sets and files.

“alt attribute”

HTTP Test Wizard

“alt attribute”

Tests can be created in a number of different ways. You can even develop test programs from scratch that measure anything and use any protocol.

To keep it simple in the first step, we introduce the “HTTP Test Wizard”, with which you can easily define HTTP/S sessions and run them as tests.

Test Preparation

  1. Navigate to the “Projects Menu” and create a new Project and a new Resource Set.
  2. Navigate to the “HTTP Test Wizard”.
  3. Create a New Session in the “HTTP Test Wizard”.
  4. Save the New Session to the Project and Resource Set which you have created before.

“alt attribute”

You now have an empty session to which you can add HTTP requests (URL calls).

Test Creation

Now add one or more URLs to the test. If you want to subdivide the URLs into individual groups, first add a “Measuring Group”, then add the associated URLs and then again, add the next “Measuring Group” with the corresponding URLs.

If you define a “Measuring Group”, the (separately measured) response time of the group is measured overall URLs of the group.

“alt attribute”

Entering a URL is easy. Select the HTTP method (GET, POST, …) and enter the absolute URL. If you want to execute an HTTP POST method, first select the requests content type and then enter the data of the post request. Finally click at the “Add URL” button at the bottom of the dialog.

After you’ve created the test, it looks similar to this:

“alt attribute”

Save the test again (save session) and then debug the test.

Debugging the Test

In order to debug a test, at least one “Measuring Agent” must be available. This is because it is a remote debugger.

If you click on “Debug Session”, the test is automatically transmitted to one of your measuring agents and the debugger is started. If you have registered several measuring agents, you can select in the debugger at the top right on which measuring agent the debug session will be executed.

“alt attribute”

During debugging, please pay close attention to the HTTP status code you get from the URLs. For example, if you receive a 404 status code (not found), you have probably entered a faulty url. In such a case, cancel the debug session, correct the URL and then start the debugger once again.

As you may have already noticed, the HTTP status code of the URLs has not yet been checked by the debugger. Return to the HTTP Session Wizard and configure the corresponding HTTP status code for each URL (section “Verify HTTP Response”). Don’t forget to click at the “Modify URL” button at the bottom of the dialog.

“alt attribute”

Debug your test one last time and then save it.

Generating the Code

Generating the code is pretty easy. First click on “Generate Source Code” and then on “Compile & Generate JAR”.

“alt attribute”

“alt attribute”

Then click on “Define New Test”.

Defining the Test

Defining a test with the “HTTP Test Wizarzd” is also quite easy. Optionally, you can enter here a test description. Then click on “Define Test”.

“alt attribute”

After you have clicked on “Define Test” the test is created and the view changes from the “HTTP Test Wizarzd” menu to the “Tests” menu.

Tests Menu

The Test Menu shows all tests that you have defined - across all Projects and Resource Sets. You can also filter the view according to Projects and Resource Sets, and sort the tests in different ways.

“alt attribute”

Note that a test is something like a bracket that only contains references to the files that are required for the test execution. There a no files stored inside the test itself.

On the one hand, this means that a test become invalid/corrupted if you delete the corresponding resource files of the test in the Projects Menu. On the other hand, this also means that all changes to files that are saved in the tree of the Projects Menu are immediately applied to the test.

Each test has a base location from which it was defined (Project and Resource Set) and to where also the test results are saved.

A test itself can reference its resource files in two ways:

  • Resource Files that reside in the base location of the test (the test core files).
  • Referenced Files that are located in other Projects / Resource Sets. These are typically libraries, plug-ins and input files which are used/shared by several, different tests.

Defining a Test Job

After you have clicked on a test on “Define Test Job” and then on “Continue”, the intermediate “Define Test Job” menu is displayed.

“alt attribute”

In contrast to a “test”, all referenced files of the test are copied into the “test job”. For this reason there is also the option “Synchronize Files at Start of Job” which you should always have switched on. At “Additional Job Arguments” you can enter test-specific command line arguments that are transferred directly to the test program or test script. However, you usually do not need to enter anything.

After you have clicked on “Define Load Test Job” the test job is created and the view changes from the “Tests” menu to the “Test Jobs” menu.

Test Jobs Menu

In this menu all test jobs are displayed with their states. The green point at the top right next to the title indicates that all measurement agents that are set to active can also be reached. The color of the point changes to yellow or red if one or more measuring agents cannot be reached / are not available.

“alt attribute”

A test job can have one of the following state:

  • invalid: The job is damaged and can only be deleted.
  • defined: The job is defined locally in the portal server, but has not yet been transmitted to a masuring agent. Jobs that are in this state can still be modified.
  • submitted: The job was transmitted to a measuring agent, but is not yet ready to be started.
  • ready to run: The job on the measuring agent is ready to start (the corresponding data collector process is already running on the measuring agent, but the job itself is not yet started).
  • start failed: The start of the job has failed.
  • running: The job is currently running and can be monitored at real time.
  • completed: A job that was previously running is now completed (done).

As soon as a job is in the “defined” state, it has a “local job Id”. If the job is then transmitted to a measuring agent, the job has additionally a “remote job Id”.

Starting a Test Job

After you have clicked on “Start Test Job” in a test job you can select/modify the measuring agent on which the job will be executed and configure the job settings.

“alt attribute”

Input Fields:

  • Measuring Agent: The measuring agent on which the test is executed.
  • Number of Users: The number of simulated users that are started.
  • Max. Test Duration: The maximum test duration.
  • Max. Loops per User: The maximum number of sessions executed per user (the number of iterations of the test per user). Each time a new session of a user is started, the user’s session context is reset.
  • Loop Iteration Delay: The delay time after a session of a user has ended until the next session of the user is started.
  • Ramp Up Time: The length of time at the beginning of the test until all simulated users are started. Example: with 20 simulated users and a time of 10 seconds, a new user is started each 0.5 seconds.
  • Additional Arguments: Additional values which are transferred on the command line when the test script is started. These arguments are test specific. For tests that were created with the “HTTP Test Wizard” you can specify for example the following values: “-tcpTimeout 8000 -sslTimeout 5000 -httpTimeout 60000” (TCP connect timeout / SSL handshake timeout / HTTP processing timeout) which are considered by the executed URL calls and override the default values.
  • Debug Execution: This option effects that detailed information are written to the log file of the test. For example variable values which have been extracted from input files or from HTTP responses as well as variable values which are assigned at runtime. Only activate this option if you have problems with the execution of the test.
  • Debug Measuring: Effects that the Data Collector of the Measuring Agent writes the JSON objects of the DKFQS All Purpose Interface to its log file. This option can be enabled to debug self-developed tests that have been written from scratch.

Normally you do not have to enter any “Additional Arguments” and leave “Debug Execution” and “Debug Measuring” switched off.

After clicking on “Start Test Job”, the job is started on the measuring agent and the status of the job is now “Running”. Then click on “Monitor Jobs”.

“alt attribute”

Real-Time Monitor Menu

The real-time display shows all currently running jobs including their measured values and measured errors.

“alt attribute”

You can also suspend a running job for a while and resume it later. However this has no effect on the “Max. Test Duration”.

“alt attribute”

After the job is completed you can click on “Analyze Result”. The view changes then to the “Test Results” menu.

“alt attribute”

Test Results Menu

The “Test Results” menu is a workspace into which you can load multiple test results. You can switch back and forth between the test results. As in the “Real-Time Monitor” menu, all measured values and all measured errors are displayed. In addition, percentile statistics and diagrams of error types are also displayed in this menu.

This menu enables you also to combine several test results into a so-called “load curve” - and thus to determine the maximum number of users that a system such a Web server can handle (see next chapter).

“alt attribute”

The Summary Statistic of a test result contains some interesting values:

  • Avg. Measured Samples per Second: These are the number of successful measured values per second, counted over the whole test (in other products also so-called as “hits per second”).
  • Max. Concurrent Users: These are the maximum number of concurrent user that have been really reached during the test (which may differ from the “Number of Users” defined when starting the test).
  • Max. Pending Samples: This is the maximum amount of requests to the server, for which no immediately response has been received, measured over the whole test (the maximum traffic jam of the requests).
  • Average CPU Load: This is the average CPU load in percent on the measuring agent (load generator) which was captured during the execution of the test. If this value is greater than 95% the test is invalid because the measuring agent itself was overloaded.

Since several thousand to several million response times can be measured in a very short time during a test, the successfully measured response times are summarized in the response time diagrams at 4-second intervals. For this reason, a minimum value, a average value and a maximum value is displayed for each 4-second interval in such diagrams.

However, this summarization is not performed for percentile statistics and for measured errors. Every single measured value is taken into account here.

Determining System Capacity

The maximum capacity of a system, such as the maximum number of users that a web server can handle, can be determined by a so-called “load curve”.

To obtain such a load curve, you must repeat the same test several times, by increasing the number of users with each test. For example a test series with 10, 50, 100, 200, 400, 800, 1200 and 1600 users.

The easiest way to repeat a test is to clone a test job. You can then enter the higher number of users when starting the test.

“alt attribute”

A measured load curve looks like this, for example:

“alt attribute”

As you can see, the throughput of the server increases linearly up to 400 users - with the response times remaining more or less the same (Avg. Passed Session Time). Then, with 800, 1200 and 1600 users, only individual errors are measured at first, then also many errors, with the response times now increasing sharply.

This means that the server can serve up to 400 users without any problems.
But could you operate the server with 800 users if you accept longer response times? With 800 users, 745,306 URL calls were successfully measured, with only 50 errors occurring. To find it out, let’s compare the detailed response times of “Page 1” of 400 users with 800 users.

Response Times of “Page 1” at 400 Users: “alt attribute”

Response Times of “Page 1” at 800 Users: “alt attribute”

The 95% percentile value at 400 users is 224 milliseconds and increases to 1952 milliseconds at 800 users. Now you could say that it just takes longer. However, if you look at the red curve of the outliers, these are only one time a little bit more than 1 second at 400 users, but often more than 8 seconds at 800 users. Conclusion: The server cannot be operated with 800 users because it is then overloaded.

Now let’s do one last test with 600 users. Result:

“alt attribute”

The throughput of the server at 600 users is a little bit higher than at 400 users and also little bit higher than at 800 users. No errors were measured.

Response Times of “Page 1” at 600 Users: “alt attribute”

The 95% percentile value at 600 users is 650 milliseconds, and there are only two outliers with a little bit more than one second. Final Conclusion: The server can serve up to 600 Users, but no more.

Troubleshooting a Test

In rare cases it can happen that a test does not measure anything (neither measurement results nor errors). In this case you should either wait until the test is finished or stop it directly in the “Real Time Monitor” menu.

Then you can then acquire the test log files in the “Test Jobs” menu and search for errors.

“alt attribute”

“alt attribute”

6.1 - AWS Measuring Agents

How to launch AWS based Measuring Agents

This document describes how to launch Measuring Agents in AWS as EC2 instances. Readers are assumed to be familiar with AWS EC2 terms used in here, in particular if launching Measuring Agents manually.

AWS based Measuring Agents can be launched in three ways:

  1. Via the RealLoad Portal
  2. Via our Desktop Companion App
  3. Via the AWS Console

1. Launching AWS Measuring Agents via the RealLoad Portal

Launching AWS Measuring Agents via the RealLoad portal is the easiest of the three options. You don’t need an AWS account for this, but RealLoad ‘Clouds Credits’ are required. If you run out of Cloud Credits, you can purchase new ones in the RealLoad Shop: https://shop.realload.com/

Launching an AWS Measuring Agent using Cloud Credits

Navigate in the Portal to ‘Measuring Agents & Cluster Controllers’. Navigate to ‘Measuring Agents & Cluster Controllers’

The click the AWS icon: click the AWS icon

Click on the rocket icon of an AWS region: Click the rocket icon of an AWS region

Select the ‘AWS/EC2 Instance Type’ (in this example ’t3.xlarge’) and the number of hours until the instance will be terminated (in this example 3 hours). Then click on the ‘Launch Instance’ button: Select Instance Type and number of hours

You will then receive a confirmation notification that the instance has been started. Now you have to wait at least 40 seconds until the instance gets into the running state. Launch instance confirmation notification

The Measuring Agent is now registered in the portal. You should then ‘Ping’ it to make sure it is functional. If the Ping fails, wait another minute and then try again. Measuring Agent registered in the portal

Ping Measuring Agent

Premature Termination of an Instance - Refund of Cloud Credits

In this example, the Measuring Agent (cloud instance) was started for three hours, but the load test was performed successfully after just one and a half hours. In order to save Cloud Credits, the instance can be terminated early.

To terminate an instance launched by Cloud Credits navigate to ‘Launched Cloud Instances’: Navigate to ‘Launched Cloud Instances’

At the relevant instance, click the ‘Terminate Instance’ icon: Click ‘Terminate Instance’ icon

Click the ‘Terminate Instance’ button: Click ‘Terminate Instance’ button

The instance will now terminate. In this example you will receive 4 out of a total of 12 cloud credits back. Refund of Cloud Credits

Verifying the Cloud Credit Billing

Navigate to ‘Cloud Credit Statement’ to check your Cloud Credit Balance and Transactions: Refund of Cloud Credits

Refund of Cloud Credits

2. Launching AWS Measuring Agents via the Desktop Companion App

Launching AWS Measuring Agents via the Desktop Companion App requires that you have an AWS account.

No additional costs (beyond the costs charged by AWS) apply when using our Measuring Agent images.

Note that you need the rights to log in to the AWS Console to check whether all Measuring Agent instances that are no longer needed have been terminated.

First download, install and configure the Desktop Companion. The download links are published on https://download.realload.com

Configuring the Desktop Companion

To configure the Desktop Companion for launching AWS Measuring Agents you need:

  1. An AWS ‘Access Key’ of an AWS User which have the permission AmazonEC2FullAccess
  2. A RealLoad ‘API Authentication Token’

To generate a RealLoad ‘API Authentication Token’ sign in to the RealLoad Portal and navigate to ‘My Account’ - ‘API Authentication Tokens’: Navigate to ‘My Account’ - ‘API Authentication Tokens’

Click the ‘Add API Authentication Token’ button: Click ‘Add API Authentication Token’

Enter as purpose ‘Desktop Companion’ and click the ‘Add API Authentication Token’ button. Then copy the shown Token Value to the clipboard or to a text editor: Copy the shown Token Value

Now start the Desktop Companion and configure it. Click on the File tab and then click Settings. Start and configure Desktop Companion

In the General Tab enter:

  1. The Agent Secret: This is something like a password that protects the launched AWS Measuring Agents from unauthorized access. You can enter any text here, but it must not be empty.
  2. The Portal URL: This is the URL of the Portal Server API and should always be https://portal.realload.com/RemoteUserAPI
  3. The Authentication Token: Enter here the RealLoad ‘API Authentication Token’.
  4. Set the Refresh Interval to a value that is greater than 60 seconds.

Then click the ‘Test’ button near ‘Authentication Token’: Configure General Settings

Switch then to the AWS Tab and enter:

  1. The AWS Access Key ID
  2. The AWS Secret Access Key
  3. The Preferred Instance Type: We recommend that you enter t3.medium for your first attempt.
  4. Set the AWS EC2 Refresh Interval to a value that is greater than 60 seconds.
  5. Select at My AWS Regions your preferred AWS region(s): Multiple regions can be selected by CTRL-mouse-click. We recommend that you select eu-central-1 and/or us-west-1 for your first attempt.

Then click the ‘Test’ button near ‘AWS Secret Access Key’: Configure AWS Settings

Finally, click the ‘OK’ button. The configuration is now complete.

Launching an AWS Measuring Agent

In the Desktop Companion, go to the AWS tab and select an AMI of type ‘MA’ (MA = Measuring Agent). Then launch the instance with a right mouse click: Launch Instance

A window will appear in which you can check the configuration of the instance and set the instance auto-terminate time period. You can also choose a value of zero for the auto-terminate time period, which means that the instance will continue to run until you stop it. Confirm Launch Instance

The instance is then launched and in the state ‘pending’: Instance in ‘pending’ state

Now wait until the instance reaches the ‘running’ state. Then register the instance in the portal with a right mouse click: Register ‘running’ instance

The instance is now also displayed in ‘Portal Agents Registration’: Portal Agents Registration

Check in the RealLoad Portal whether the instance is running correctly by pinging the Measuring Agent at application level: Portal: Navigate to Measuring Agents

Portal: Ping Measuring Agent

Portal: Ping Measuring Agent - Result

Terminating an AWS Measuring Agent

Note that this section only applies to Measuring Agent instance which have launched via the Desktop Companion.

Two steps are necessary to terminate an instance:

  1. Terminate the instance in AWS
  2. Deregister the Measuring Agent in the portal server

To terminate an instance in AWS, perform a right mouse click in the Desktop Companion - Running AWS Images in “My Regions” at the corresponding instance and select ‘Terminate’: Terminate AWS Instance

The instance is then in the state ‘shutting-down’. Now you can deregister the Measuring Agent in the portal server by a right mouse click: Deregister Measuring Agent

Double-check no longer needed Instances

Because the Desktop Companion only shows Running AWS Images in “My Regions”, some instances launched in other AWS Regions may not be displayed.

Sign in to the AWS console to review Measuring Agent instances across all AWS regions. Select EC2 Global View: AWS Console: EC2 Global View

In EC2 Global View click Global search and select the the tag REAL_LOAD_AGENT: AWS Console: Global search

Select the operator Equals and set the value to true, then hit the return key: AWS Console: Global search

If the list displayed contains one or more instances, click on the instance(s): AWS Console: Global search

Memorize the instance ID and terminate the instance: AWS Console: Terminate Instance

Sign in into the RealLoad Portal and delete the corresponding Measuring Agent: RealLoad Portal: Delete Measuring Agent

3. Launching AWS Measuring Agents via the AWS Console

In EC2, first select the AWS Region where you want to launch the Measurement Agent instance.

Creating the Security Group

In order for the AMI to be reachable from the RealLoad Portal you’ll need to configure a Security Group allowing inbound TCP/IP V4 connections to port 8080 as a minimum. Optionally, you can also configure SSH access.

In the following example the security group “Real Load Agent Security Group” was created: AWS Console: Security Group

Launching a Measuring Agent Instance

When launching the instance click Browse more AMIs: AWS Console: Browse more AMIs

Then click Community AMIs, enter our (RealLoad) AWS account 775953462348 and select the Measuring Agent image: AWS Console: Community AMIs - Measuring Agent

Add the tag REAL_LOAD_AGENT = true AWS Console: Launch Instance - Add Tag

Select the Instance Type and set the (existing) Security Group “Real Load Agent Security Group”. Optionally you may also set a Key Pair: AWS Console: Launch Instance - Instance Type and Security Group

Scroll down and click Advanced Details: AWS Console: Launch Instance - Advanced Details

In Advanced Details scroll down to User Data. In order to protect access to your Measuring Agent we strongly recommend setting a non-default secret when launching the instance.

The secret can be set at launch time by providing User Data to the AMI as follows (replace “secret” with the secret of your choice):

#!/bin/sh
echo "AGENT_SECRET=secret" > /home/ec2-user/agent_secret.sh

AWS Console: Launch Instance - Agent Secret

Scroll down to the bottom of the AWS Console and click Launch Instance: AWS Console: Click Launch Instance

After launching the instance click on the instance ID: AWS Console: Click on Instance ID

Copy the Public IPv4 address of the instance to a text editor or to the clipboard: AWS Console: Copy Public IPv4 address

Sign In to the RealLoad Portal and add (=register) the Measuring Agent instance: RealLoad Portal: Add Measuring Agent

The Agent IP Port wil always be 8080. Enter the Auth Token which you have set as AGENT_SECRET when launching the instance. RealLoad Portal: Add Measuring Agent

Then ping the Measuring Agent at application level: RealLoad Portal: Ping Measuring Agent

RealLoad Portal: Ping Measuring Agent

And check the Measuring Agent Log File to verify that the Agent Secret was configured: RealLoad Portal: Measuring Agent Log File

RealLoad Portal: Measuring Agent Log File

The Measuring Agent is now ready for operation. Don’t forget to terminate the instance when you no longer need it.

Once you have terminated the instance, you should delete the corresponding Measuring Agent in the RealLoad Portal.

6.2 - HTTP Test Wizard

User Guide | HTTP Test Wizard

“alt attribute”

Overview and Functionality

The HTTP Tests Wizard supports you to create sophisticated tests in an easy way. You can compile (define) the entire test via the user interface and then generate a ready-to-run test program from this definition. A powerful debugger helps you to perfect your test.

As the name suggests, the HTTP Tests Wizard is optimized for the execution of HTTP/S tests. However, by using plug-ins, any other protocols can also be tested and measured (such as SMTP, DNS queries, DB-SQL queries, etc.).

HTTP Tests Wizard Features:

  • No programming required (except for self written plug-ins). You can create your test through the powerful user interface by assembling all required functionalities via simple dialogs.
  • The executed HTTP/S calls can be either entered manually our can be inported from a Proxy Recorder session.
  • Variable values (variables) can be extracted and assigned directly via the user interface.
  • Integrated Remote-Debugger: The defined test can be debugged in advance on any Measuring Agent - before generating the test program. The degugger supports also to define variables as well as to define variable-extractors and variable-assigners at the fly, which are adopted into the test.
  • After the test has been passed by the debugger, the defined test can be automatically converted into a performance-optimized test program.

Session and Session Elements

The test sequence is referred to as a so-called “Session”, whereby each simulated user repeatedly executes the session in a loop (so called “User Session” or “Session Loop”).

In order to define a test sequence, you can add various “Session Elements” to a session:

  • User Input Field: Displays an additional input field when starting the test and stores the entered value in a global variable.
  • Measurement Group: Measures the execution time over several session elements.
  • URL: Executes a HTTP request and receives the HTTP response. Measures the execution time, and verifies the HTTP response. The HTTP response can either received synchronously or asynchronously (see also URL Synchronisation Point). Supports “Bound to URL Plug-Ins” which, among other things, can modify HTTP request and postprocess HTTP responses.
  • URL Synchronisation Point: Synchronizes asynchronously received HTTP responses.
  • Delay Time: Delays the execution of the user session for an amount of time.
  • Basic Authentication: Adds a Basic Authentication username and password to all or selected URLs.
  • SSL/TLS Client Certificate: Adds a SSL/TLS Client Certificate to all or selected URLs.
  • General Request Header Field: Adds a HTTP request header field (for example “User-Agent”) to all or selected URLs.
  • Cookie Modifier: Sets/Extracts/Deletes cookies.
  • Plug-In: Initializes and executes a “Normal Session Element Plug-In”.
  • Input File: Reads an input file line by line, tokenizes the line and extracts the values of the tokens into variables.
  • Output File: Stores the values of variables into an output file.

“alt attribute”

Adding session elements is in most cases quite simple and self-explanatory, but it can be a bit challenging for URLs and Input Files. For this reason, adding of these two session elements is described in more detail in the next two sub-chapters.

Adding URLs

When adding an URL you have at least to select the HTTP Method (for example GET) and to enter the absolute URL (https://<host>/path).

Checking the HTTP response code and the HTTP response content is optional, but strongly recommended, as otherwise the test result may contain false positive results.

“alt attribute”

The following fields can be entered or selected:

  • General Area
    • HTTP Method: Select the HTTP method (GET, POST, …).
    • URL: Enter an absolute URL, inclusive (optional) query string parameters. Example: https://www.dkfqa.com/?v=1&w=2
    • Execute - synchronous or asynchronous: Select how the HTTP response is received. If you choose “asynchronous” then you have also to add an “URL Synchronisation Point” to the test sequence after the asynchronous executed URLs.
    • Error Handling: Select what happens if the HTTP request fails or if the HTTP response is invalid. “Final End” means that the whole test is aborted.
    • Enable Implicit Assigners ${<variable-name>}: If checked, then placeholders for variables are considered in the URL as well as in the HTTP request header and content. Example: URL = https://${vHost}/?v=1&w=2, Variable vHost = “www.realload.com”, Result = https://www.realload.com/?v=1&w=2 . Note insted of using inplicit Assigners you can also define Variable Assigers (see next chapter: Variables).
  • HTTP Request Content Area | Only fill in this area if the HTTP request contains a content, for example for POST requests.
    • Request Content Type: Select or enter the request content type. For example “application/x-www-form-urlencoded” or “application/json”.
    • Request Content Charset: Select or enter the request content charset - or let the field blank if not applicable.
    • Direct Value or Read From File: Select if the request content data are directly entered here or are read from a file.
    • Request Content Data: Enter the request content data (if they are not read from a file). Note: select first the Request Content Type before you enter the Request Content Data.
    • Also send Zero Content Length: If checked, but no content is available, the HTTP request header field “Content-Length: 0” is sent.
  • Additional HTTP Header Fields Area
    • You can enter additional HTTP request header fields here which are only applied for this URL. Note: additional HTTP request header fields that apply to all URLs should instead be defined by using a “General Request Header Field” session element.
  • Verify HTTP Response Area | Note: you should configure at least a HTTP response code in order to verify the test result.
    • Verify HTTP Status Code: Select the expected HTTP response code(s).
    • Verify Content Type: Select the expected HTTP response content type(s).
    • Verify Content Text: Enter text fragments that must be present in the HTTP response content, or let the fields blank if not applicable.
  • Plug-Ins Area
    • Here you can add “Bound to URL Plug-Ins” (see chapter “Plug-Ins”).

Adding Input Files

Input files are text files (*.txt, *.csv) whose content is read line by line during the test. Each line is divided into tokens from which values of variables are extracted. Empty lines are skipped. Note that the variables must first be defined before you can add an input file.

The following fields can be entered or selected:

  • Token Delimiter: The token delimiter. This is usually a single character, but can be also a string.
  • Comment Tag: Lines which are starting with the comment tag are skipped. You can place also the comment tag within a line which means that the remaining part of the line is ignored. The comment tag is usually a single character, but can be also a string.
  • Cyclic Read: If not checked, then the test will end when no further line can be read (at eof). On the other hand, if checked, the file is re-read after the end of file was reached.
  • Randomize Lines: if checked then the order of the lines is randomized each time when the file is read.
  • Trim Values: If checked then whitespace characters are removed from the start and end of the extracted values (tokens).
  • Remove Double Quotes: If checked then double quotes are removed from the extracted values (tokens).
  • Scope: Get Next Line per:
    • Global: Reads only a single one line of the file. The line is read at the start of the test.
    • User: Reads each time a line from the file when a simulated user is started.
    • User Session: Reads each time a line from the file when a simulated user starts a new session loop iteration.
  • 6x Variable - Token[Index]: You can configure up to 6 token indexes whose values are extracted into variables.

Variables

The HTTP Tests Wizard (as well as the debugger) supports to define variables and to extract and assign variable values from/to session elements. Variable definitions which are (remotely) made in the debugger are automatically synchronized with the HTTP Test Wizard at portal server side.

The data type of a variable is always a string, which can also be empty, but is usually never null.

Defining Variables

“alt attribute”

When defining a variable, the following attributes can be set:

  1. Variable Name
  2. Scope: “Global”, “User” or “User Session”
  3. Defailt Value (initial value)

For variables with the scope “Global” there is only one instance which is initialized when the test is started.

For variables that have the scope “User”, there is a separate instance for each simulated user, which is initialized when the user is started.

In the case of variables with the scope “User Session”, a new instance is created and initialized at the start of each iteration of the session loop. The visibility is resricted to the current session loop of the simulated user.

Variable Extractors and Assigers

Variable extractors are definitions which contain instructions about how to extract a value from a session element into a variable. Variable assigners, on the other hand, are definitions that contain instructions how to assign or replace a value of a session element.

The following two images show a variable extractor that extracts a dynamic value from URL[2], and a variable assigner which assigns the value to a JSON post request of URL[3].

“alt attribute”

“alt attribute”

While variables (almost) always have to be defined manually, the definition of variable extractors and assigners is usually done implicitly (semi-automatically), depending on the context of the session element.

For example, if you define an input file, you must first create the variables that contain the extracted values (for username and password in this example). If you then assign the line token numbers to the variables, the corresponding variable extractors are automatically created.

“alt attribute”

“alt attribute”

“alt attribute”

Note: In this example above, a separate username and password are read from the input file for each simulated user and assigned to the value of the corresponding variables. For this reason the variables have the scope “User”, and the input file has the scope “Get Next Line per User”.

Assigning and Extracting Variable Values to/from URLs

Apart from Implicit Assigners (see “Adding URLs”), variable values of URLs can only be extracted and assigned by using the debugger. However, this is not a problem as definitions made in the debugger are automatically synchronized with the portal server.

Extracting variable values from URLs during debugging is quite easy. After executing a URL in the debugger, click on the symbol of the HTTP response header or of the HTTP response content and then extract the value.

“alt attribute”

“alt attribute”

Assigning variable values to URLs is a little bit more tricky. You have to interrupt the HTTP request in the debugger in the middle - after the request is initialized, but before the request is send to the server.

For this you have temporary to enable the option “HTTP Request Breakpoint” in the debugger, before you execute the URL (Next Step):

“alt attribute”

After you have clicked on “Next Step” you can assign the variable value to the pending HTTP request:

“alt attribute”

Click on the symbol of the URL, or of the HTTP request header or of the HTTP request content, and then assign the variable value:

“alt attribute”

“alt attribute”

Plug-Ins

Plug-Ins are reusable HTTP Test Wizard extensions that are manually programmed. They can also be published by manufacturing users and imported by other users. Therefore, before you start programming your own plug-in, take first a look at the already published plug-ins.

Plug-In Types

There are 3 types of plug-ins:

  • Normal Session Element Plug-Ins: As the name suggests, such plug-ins are executed as a normal session element. This type of plug-in is versatile and can, among other things, process variables, abort the current session, the user or the entire test, or even perform independent measurements with any protocol.
  • Bound to URL Plug-Ins: Such plug-ins are linked to a URL session element and can change the HTTP request before sending it to the server and post-process the received HTTP response - such as extracting variable values or checking the content of the HTTP response. Such plug-ins can also abort the current session, the user, or the entire test.
  • Java Source Code Modifier Plug-Ins: These are special plug-ins that can subsequently modify the source code generated by the HTTP Test Wizard. Usually such plug-ins are only used as a workaround if the HTTP Test Wizard generates faulty or incomplete source code.

Importing Published Plug-Ins

“alt attribute”

“alt attribute”

After you have imported the plug-in you have to load and compile it - then save the compiled plug-in.

“alt attribute”

“alt attribute”

“alt attribute”

“alt attribute”

After you have saved the compiled plug-in you can add it to your HTTP Test Wizard session. Plug-ins of the type “Java Source Code Modifier Plug-Ins” do not have to be added, but can be called directly after the source code of the test program has been generated.

Developing own Plug-ins

Own plug-ins can be developed in Java 8 or 11. The following example shows a plug-in which decode a base64 encoded string.

“alt attribute”

After clicking at “New Plug-In” the plugin type has to be selected (in this example “Normal Session Element Plug-In”): “alt attribute”

At the “General Settings” tab you have at least to enter the Plug-In Title and the Java Class Name. You should als enter a Plug-In Description which can be formatted by simple HTML tags. The onInitialize Scope can be set to “Global”, as this plug-in does not require any initialization. The onExecute Scope is set to “User Session” in order that all kind of variabe scopes are supported. “alt attribute”

At the next tab - “Input & Output Values” - a plug-in input and a plug-in output (Java-)variable is defined for the plug-in method onExecute. The values of this two Java variables will later correspond with two HTTP Test Wizard variables (which may have a different variable name): “alt attribute”

When you click at the next tab - “Resources Files” - you will see that the Java library com.dkfqs.tools.jar is already added to the plugin. This is normal as all kind of plug-ins require this library. Here you may add also other Java libraries required by the plug-in - but in this case no other libraries are needed. “alt attribute”

Now at the “Source Code & Compile” tab, you can first generate the plug-in template. Then you have to extend the generated code with your own code, ie in this example you have to complete the Java import definitions and program the inner logic of the Java method onExecute. Then compile the plug-in.

“alt attribute”

“alt attribute”

At the last tab “Test & Save” you can first test the plug-in (remotely) on any Measuring Agent. To perform the test enter for example “SGVsbG8gV29ybGQ=” as input and you will see as output “Hello World”. “alt attribute”

“alt attribute”

Finally save the plug-in: “alt attribute”

After you have saved the plug-in click at the “Close” button. Then you will see the new plug-in in the plug-in list: “alt attribute”

Publishing Plug-Ins

If your plug-in can be used universally, it would be nice if you also publish it - to make it available to other users.

Note that you have to enable in your Profile Settings the option “Public In-App User” in order that you entitled to publish plug-ins.

Publishing plug-ins is especially useful for users who have additionally activated the option “Publish My Profile as Technical Expert” in their Profile Settings. This will significantly improve your visibility and competence.

“alt attribute”

“alt attribute”

6.3 - HTTP/S Remote Proxy Recorder

User Guide | HTTP/S Remote Proxy Recorder

By using an HTTP/S Proxy Recorder, the HTTP/S traffic from Web browsers and technical Web clients can be recorded and easily converted into a HTTP Test Wizard session.

The HTTP Test Wizard session can then be debugged and post-processed. Finally, an executable test program can be generated from the recorded session.

Functional Overview: Remote Proxy Recorder

“alt attribute”

An HTTP/S Remote Proxy Recorder has two service ports:

  • A Control Port that is addressed by the portal server.
  • A Proxy Port that records the data traffic from a Web browser or from a technical Web client.

All data traffic that passes through the proxy port is first decrypted by the Proxy Recorder and then encrypted again before it is forwarded to the target Web server(s).

In order to record a Web surfing session by a Web browser you have to start two different Web browser products on your local machine. For example:

  • A Chrome browser to operate the portal server (Web Browser 1), and
  • A Firefox browser to record the Web surfing session (Web Browser 2)

We recommend to use always Firefox as Web Browser 2.

To be able to record a Web surfing session, you have to reconfigure Web Browser 2:

  • Import the Proxy Server CA Root Certificate into the Web Browser 2.
  • Configure the Network Settings of Web Browser 2 to use the Proxy Server.

Additional note: Before you start recording a Web surfing session in Web Browser 2, you must always clear the browser cache.

Once the recording is completed you should undo the configuration changes in Web Browser 2 (restore the original network settings and delete the Proxy Server CA Root Certificate).

Registering a Proxy Recorder at the Portal

After you have started a HTTP/S Proxy Recorder as a cloud instance or installed it on one of your systems, you have to register it at the Portal Server.

“alt attribute”

“alt attribute”

You can set any user name and password for Proxy Authentication. The Control API Authentication Token was defined/set by the person who started or installed the Proxy Recorder.

Once you have registered the Proxy Recorder you can try to ping it on the application level:

“alt attribute”

“alt attribute”

Then try to connect to the Control Port of the Proxy Recorder. If this works, the Proxy Recorder is now ready for use.

“alt attribute”

“alt attribute”

Configuring Web Browser 2 for Recording

Import the Proxy Server CA Root Certificate

Click at the certificate icon and store the certificate at any location.

“alt attribute”

“alt attribute”

Then open the Web browser settings and import the certificate.

“alt attribute”

“alt attribute”

“alt attribute”

Since there is a small bug in some versions of Firefox, you have to close and reopen the Certificate Manager to view the imported certificate.

“alt attribute”

Configure the Network Settings

“alt attribute”

“alt attribute”

Recording a Web Surfing Session

Navigate with Web Browser 1 to the Proxy Recorder control menu.

“alt attribute”

Then configure the “Hosts Recording Filter” and click on the “Start Recording” icon.

“alt attribute”

In Web Browser 2, first clear the browser cache and then enter the URL where you want the recording to start.

“alt attribute”

“alt attribute”

In Web Browser 1 you can now see the recording of the first page. Before navigating to the next page in Web Browser 2, first insert a “Page Break” in Web Browser 1 with a brief description of the next page.

“alt attribute”

“alt attribute”

Then navigate to the next page in Web Browser 2.

“alt attribute”

In Web Browser 1 you can now see the recording of the next page.

“alt attribute”

Then continue as before. Insert a Page Break in Web Browser 1, navigate to a next page in Web Browser 2. and so on …

After you have done the recording click on the “Stop Recording” icon. Then convert the recording into an HTTP Test Wizard session.

“alt attribute”

Converting the Recorded Session

Here you can select different options. The HTTP Status Code Filter also supports * as a wildcard. An exclamation mark in the font of a value means “do not / exclude”.

The option Enable Parallel Execution of HTTP Requests should be enabled if you have recorded a session with a Web browser (as described above), but should be disabled if the recording was performed by a technical Web client.

“alt attribute”

Once you have clicked on the Convert button, the recorded session is converted and directly loaded into the HTTP Test Wizard.

“alt attribute”

After that debug the converted session in the HTTP Test Wizard and then save the HTTP Test Wizard session. Finally generate and execute the test.

7 - Browser Recorder extensions for both Chrome and Firefox

Browser Extension - Introduction

This browser extension will ask for the RealLoad proxy configurations, then automatically set Proxy in the browser. Using the extension user can clear cache, start recording, stop recording, clear recording and add page breaks while recording the user journeys in the browser.

Before you start installing the browser extension, Please make sure that you have already followed the steps mentioned in the doc HTTP/S Remote Proxy Recorder) , Registering a Proxy Recorder at the Portal and Import the Proxy Server CA Root Certificate.

Browser extension for Chrome

Please open the link https://chrome.google.com/webstore/detail/mlindhmkpdadnkdoicbmijgdecpbioai/ in Chrome and Add to Chrome.

After adding the extension to Chrome, the next step will be setting the proxy details for recording in the extension options like below.

Once you have entered all the necessary RealLoad proxy details and save, you are ready to use the Browser Recorder extension for Chrome

Now you can go ahead and record your browser sessions by pressing Start Recording, You can add page breaks while recording by adding the page break name and press icon marked in the below screenshot.

Once you are done with your recording, you can press Stop Recording, which will revert the configured RealLoad Proxy to the Browser Automatic Proxy.

Now when you sign in to the RealLoad portal https://portal.realload.com/, you can see the recordings under the configured HTTP/S Remote Proxy Recorder screen.

The same steps are described below for the extension for Firefox.

Browser extension for Firefox

Please open the link https://addons.mozilla.org/en-US/firefox/addon/realload-test-recorder/ in Firefox and Add to Firefox. Please make sure that you have checked this option “Allow this extension to run in Private Windows” during the installation.

After adding the add-ons to the firefox, the next step will be setting the proxy details for recording in the add-on preferences like below.

Once you have entered all the necessary RealLoad proxy details and save, you are ready to use the Browser Recorder extension for Firefox.

Now you can go ahead and record your browser sessions by pressing Start Recording, You can add page breaks while recording by adding the page break name and press icon marked in the below screenshot.

Once you are done with your recording, you can press Stop Recording, which will revert the configured RealLoad Proxy to the Browser Automatic Proxy.

Now when you sign in to the RealLoad portal https://portal.realload.com/, you can see the recordings under the configured HTTP/S Remote Proxy Recorder screen.

8 - Desktop Companion User Guide

Real Load Desktop companion - Introduction

The Desktop Companion is a small application that allows you to manage some features related to the Real Load Platform.

In particular the desktop companion allows you to:

  • Generate HTTP Load Test scripts from HTTP archives (.har) files
  • Locally run the Real Load Proxy Recorder
  • Perform some basic editing of test scripts
  • Upload test scripts directly to the Real Load Portal
  • Manage AWS Measuring Agents (launch, terminate, register with Real Load Portal)

See sections below to learn more about the Desktop Companion.

8.1 - Release Notes

Desktop Companion change log

0.26 | 6 Nov 2022

0.25 | 28 Oct 2022

0.24 | 21 Mar 2022

0.23 | 25 Jan 2022

0.22 | 13 Dec 2021

0.21 | 9 Dec 2021

0.20 | 3 Dec 2021

  • Usability: Added help links to online documentation.
  • Usability: Added various configuration checks and related alert dialogs.
  • Usability: On exit, added alert in relation to running AWS EC2 instances.
  • Usability: List of AWS instances is updated via a background thread, periodically.
  • Usability: List of registered Measuring Agents is updated via a background thread, periodically.
  • Usability: The AWS region where an EC2 instance is launched is automatically added to “My Regions” list.
  • Installer: Created a Windows full installer that includes a JRE.
  • Bugfix: Fixed an issue with HAR files containing POST requests with no content type.
  • Bugfix: Updated JavaFX to v17
  • Download URL JAR: https://download.realload.com/desktop_companion/RealLoadDesktopCompanion-0.20.jar
  • Download URL Windows full installer: https://download.realload.com/desktop_companion/RLDTC_windows-x64_0_20.exe

0.10 | 8 Nov 2021

8.2 - Desktop Companion - Pre-requisites

Read this first.

Before running the Desktop Companion make sure you satisfy the following pre-requisites:

JRE 11 (Mandatory to execute JAR file)

The Desktop Companion is a Java application currently delivered as a .jar file. At this stage there is no full installer for the application, which means that as a pre-requisite to run the application you’ll need to install a Java 11 JRE.

While it should run on any systems that support Java 11 and FXML, it was only tested on the following platforms:

Operating System JRE 11 Vendor and Build JRE download link Comments
Windows 10 Microsoft “11.0.11” 2021-04-20 https://docs.microsoft.com/en-au/java/openjdk/download
Windows 11 Microsoft https://docs.microsoft.com/en-au/java/openjdk/download
Apple OS X Microsoft “build 11.0.13+8-LTS” https://docs.microsoft.com/en-au/java/openjdk/download

AWS API Security Credentials (Optional)

In order to use the AWS integration features, you’ll need to prepare Security Credentials with AWS IAM.

Seach for “IAM” in the AWS search bar and click on “Users”:

Then add a new user:

Enter a suitable user name and select “Access key” as the AWS credential type.

On the next page set an appropriate policy for the user. Usin the “AmazonEC2FullAccess” provides sufficinet permissions.

Skip the “Add tags” screen and go to the “Review” screen and click on “Create user”. Make sure the permissions you’ve assigned to the user appear on this screen.

Finally take a note of the Access Key ID and Secret Access Key as thess need to be configured in the Real Load Desktop Companion.

Real Load Portal User API Authentication Token (Optional)

In order to register with the Real Load portal the AWS EC2 instances you’ve launched, the Desktop Companion requires that you configure an authentication token.

To create an authentication token proceed as follows:

Login to the Real Load Portal and click on the User -> API Authenticaiton Tokens menu.

Then click on the “Add API Authentication Tokens” button and enter a suitable purpose description. Optionally you can restrict the src IP address from where this token can be used.

Take note of the authentication Token Value as you’ll need to configure it in the Desktop Companion.

8.3 - Desktop Companion - Download and run

Where to donwload the application from and how to run.

Downloading the full installer (Windows and OS X)

The full installer includes a JRE as well as the Desktop Companion application itself. This is the preferred method of deploying the application, as it avoids issues that might occur because of JRE customization, etc…

Additionally the application can be started from the Windows start menu, as any other application.

The latest version of the full installer can be downloaded from here:

Downloading standalone JAR file only (Intel x64 architectures only)

You can download the latest version of the Desktop Companion as an executable .jar file from here: https://download.realload.com/desktop_companion/latest

There is no installer to be executed, the application can be launched by double clicking on the .jar file. If that doesn’t work, consider launching from the command line.

Windows: Launch from file explorer

If you’ve only got one JDK/JRE version installed on your Windows computer then it’s very likely that .jar files are associated with the correct JRE. In that case you should be able to launch the application by double clicking on the .jar file.

If the application doesn’t start it’s possible the .jar extensions is associated with the incorrect JRE or another application altogether. You can try to right click on the .jar file and the “Open with” from the context menu.

Then selecte the OpenJRE launcher:

… or alternatively navigate to the JRE 11 binary, as shown in this screenshot:

Similarly on Windows 11:

s

If the application doesn’t start, consider starting it from the command line as explained below.

Windows: Launch from command line

Open a Terminal window. First validate that the java binary in your path is a JRE 11 binary by running this command:

java -version

The output should indicate you’re running a JRE 11 binary, as shown here:

If the version of the JRE is 11 then you can run the following command from the folder where the Desktop Companion .jar file was downloaded:

java.exe -jar .\RealLoadCompanionFXML-0.1.jar

OS X (x64 only): Launch from finder

If you only have JRE 11 installed on your Mac, you should be able to launch the application simply by double clicking on its icon:

Alternatively right-click on the icon and “Opne With” Jar Launcher.

If this doesn’t work, refer below on how to launch from the command line.

OS X: Launch from command line

Open a Terminal window. First validate that the java binary in your path is a JRE 11 binary by running this command:

java -version

The output should indicate you’re running a JRE 11 binary, as shown here:

If a different JRE version is returned (like for example a JRE 8 binary) then you’ll need to specify the full path to the java 11 command. For example, assuming you installed the Microsoft OpenJDK 11, you would typically find it at this location:

/Library/Java/JavaVirtualMachines/microsoft-11.jdk/Contents/Home/bin/java

To run the Desktop Companion, assuming the .jar file is present in the current directory, run the following command from a Terminal window:

/Library/Java/JavaVirtualMachines/microsoft-11.jdk/Contents/Home/bin/java -jar RealLoadCompanionFXML-0.1.jar

8.4 - Desktop Companion - Settings Menu

Configure the Desktop Companion for integration with AWS and the Real Load Portal.

Various application settings are configurable in the File -> Settings menu.

General settings

  • Default HAR input folder: Location where to look for .har (HTTP Archives) files by default. This should be the location where your browser writes files to.

  • Default export folder: This is the location where Real Load HTTP test scripts will be exported to in JSON format.

  • Agent Secret: The Measuring Agent secret to authenticate connections from the Real Load Portal to the Cloud Agent instance (AWS EC2 and Azure instances in future).

User API settings

  • Portal URL: The endpoint of the Real Load portal User API. Unless you have an on-premise installation, use the default value https://portal.realload.com/RemoteUserAPI

  • Authentication Token: Enter here the authentication token that was generated by the Portal as part of the pre-requisites steps. Click on the “Test” button to test the API token.

  • Refresh Interval: How frequently should the list of Measuring Agents registered with the Real Cloud portal be refreshed. Any value 60 seconds or lower indicates no background refresh, you’ll need to manually trigger a refresh via context menu. By default set to 61 seconds (background refresh enabled).

AWS settings

  • AWS Access Key: Paste the AWS Access Key that was obained as part of the preparation steps.

  • Secret Access Key: Paster the AWS Secret Access key that was obained as part of the preparation steps. Use the “Test” button to validate the credentials.

  • Preferred Instance Type: The EC2 instance type to use when launching a new Mesauring Agent instance.

  • AWS EC2 Refresh Interval: How frequently should the list of AWS EC2 instances be refreshed. Any value 60 seconds or lower indicates no background refresh, you’ll need to manually trigger a refresh via context menu. By default set to 61 seconds (background refresh enabled).

  • My AWS Regions: Select the AWS regions you commonly use. To select multiple regions hold the CTRL key while selecting.

Proxy Recorder settings

  • Proxy Port: The TCP port on the local machine that will listen for incoming proxy connections. You’ll need to configure this port as the HTTP proxy in your browser.

  • Export CA Certificate: This exports the CA certificate used by reverse proxy to sign SSL certificates. The exported CA should be added as a trusted CA to the browser you’re planning to use to record HTTP requests.

The next settings should be left to default values, unless there are specific reasons for changing them.

  • Proxy Backend Server Start Port Range: Starting TCP source port range for connections generated by the Proxy.
  • Proxy Backend Server End Port Range: Ending (high) TCP source port range for connections generated by the Proxy.
  • Debug HTTP Headers: Enables capturing HTTP headers values in Proxy debug log.

8.5 - Desktop Companion - File Menu

Import HTTP archives and export test scripts locally or to the Portal.

Import HAR file

To import an HTTP Archive generate by a browser use the corresponding option in the file menu and select the relevant .har file. All requests present in the file will appear in the editor tab.

Export test script to JSON file

You can export your test script to a Real Load JSON session file by selecting the corresponding option in the file menu. The file will be saved in the folder you select.

You’ll then need to manually upload the file to a Resource Set in the Real Load Portal using the upload function.

Export test script to Portal

Test scripts can be directly uploaded to the Real Load Portal via the “Export session to portal…” menu. You’ll need to select the resource set to save the load test to:

You have 2 options:

  • Overwrite existing test script: If you select and existing HTTP test script without modifying the filename this will effectively overwrite it.
  • Select a resource set: On the other hand if you select a resource set (… “Web Test” in the above screenshot for example) you’ll then have to enter a filename for your new test script.

8.6 - Desktop Companion - Editor

Simple request editor.

In the request editor tab you’ll be able to perform some basic editing of the requests imported from an HAR file or recorded by the proxy recorder.

For more complex editing please use the HTTP wizard in the Real Cloud portal after uploading your test script there.

Overview

The Editor tab is split in 3 main parts:

  • HTTP Requests lists: This section shows all HTTP requests imported from an HAR or recorded by the Recording Proxy. The context menu accessible via a right click allows you to perform some basic editing functions.
  • Request details: When selecting a request some additional details about the request will appear, like values extracted from HTTP headers, etc…
  • Unique domains: In this section all unique domain names will be listed. Clicking on a domain will select all relevant rows in the requests table.

Delete Requests

You can mark requestes in the main editor window by clicking on them. Hold the ctrl keyty to mark multiple request or use the shift key to mark a range of requests.

Once marked you can delete the requests by right-clicking on a marked request and selecting “Delete selected” from the context menu:

Adding time delays

To add a time delay (currently hardcoded to 1 second) before or after a request, select the relevant context menu item while hovering over a request.

Time delays rows appear as requests of type “T” in the main editor window:

Domain based selection

In order to bulk select all requests belonging to specific domains select one or more domains (by holding the CTRL key). This will select corresponding requests in the requests list which can then be easilly deleted.

It is also possible to directly delete requests belonging to one or multiple domains using the context menu.

8.7 - Desktop Companion - Measuring Agents

Launch and terminate AWS based measuring agents and manage their registration with the Portal.

The Measuring Agents tab allows you to manage Cloud Based (… currently AWS) Measuring Agents. This section of the application will be of most use if you configured AWS credentials in the preferences section of the application.

If no AWS credentials are configured, you won’t be able to start and terminate AWS instances.

Listing Measuring Agents AMIs

In the left pane you’ll see a list of available AWS AMIs. You can further filter the list of AMIs by selecting a specific version and/or by selecteing an AWS region.

Launching an instance

To launch an new EC2 instance right-click on the relevant AMI and select “Launch”. A screen to confirm the launch will be displayed. Confirm by clicking on the “Launch” button.

When launching an instance in an AWS EC2 region that is not yet part of your “My Regions” list, the region will be automatically added to the list.

Launching a new instaance will automatically trigger a refresh of the AWS Measuring Agents list. It might take a few second for the list to update and the new instance to be reflected in it.

Then confirm the launch action:

Listing running EC2 instances

After launching an instance go to the top right part of the window listing the running instances and right click on “Refresh”. This will retrieve all running EC2 instances from the preferred AWS regions and display them in the table.

Registering an instance

To register an AWS instance with the Real Load Portal right-click on the instance and then select the “Register with portal” menu item. The instance ID will be used as the description in the Real Load Portal.

List registered instances

In the left bottom part of the window you’ll see the Measuring Agents currently registered on the Portal. To update the list right click and select the “Refresh” option.

De-register an instance

In order to de-register an instance from the Portal right click on the instance name and select De-register.

Terminate an instance

To terminate an AWS EC2 instance right click on the instance name.

8.8 - Desktop Companion - Proxy Recorder

Run the proxy recorder locally on your desktop.

The Desktop Companion allows you to run the Proxy Recorder locally on your desktop. Recorded requests will then appear in the Editor tab. Refer to the Proxy Recorder documentation for further information on how it works.

There are two main sections in this chapter:

  • How to configure your browser.
  • Hot to start/stop the recording process.

Configure your browser

You’ll have to configure the port the Real Load Proxy listens on (default: 18080) in your browser. We recommend to use a browser that allows you to configure proxy settings independently from the Operating System proxy settings. FireFox is a good candidate for this and we’ve documented its configuration in this section.

Import Reverse Proxy CA certificate

For the browser to trust SSL certificates issued by the Recording Proxy, you’ll need to import the Proxy’s CA certificate as follows:

  • Export the certificate to a file: In the Desktop Companion’s Proxy Setting tab, click the “Export CA Certificate. Then select a folder where the certificate is to be exported. It will be saved in file “RecProxyCert.cer”.

  • Import certificate: Navigate to the certificate settings in FireFox:

Then click on the “Auhtorities” tab, “Import” button and select the “RecProxyCert.cer” that you’ve just exported from the Desktop Companion.

Make sure you trust the certificate to identify websites:

Configure proxy settings

In the FireFox “Settings” and scroll to the bottom of the page where “Network Settings” are located . The below window should be displayed and configure the Port to be 18080 for both HTTP and HTTPS. Configure the host and port as shown in the below screenshot. Also tick the “Also use this proxy for HTTPS”.

Test browser configuration

Navigate to any SSL enabled page. You shouldn’t see any warnings about untrusted SSL certificates beings used.

If you check the certificate of the site you’re visiting, the issuer should be “Real Load Pty Ltd”, as shown in this screenshot.

Start the recording process

To start the recording process go to the Proxy Recorder tab and click on the “Start Recorder” button. Note that when the recording process is started, previous Proxy Recorder logs are purged.

Stop the recording process

To stop the recording process clik on the “Stop Recorder” button. Then navigate to the “Editor” window and you should see all recorded requests as in this example:

9 - Regression Testing - Test Suites

Regression Testing - Test Job Templates and Test Suites

Abstract

Regression testing is supported by Real Load using “Test Job Templates” and “Test Suites”.

Features of Real Load Test Suites:

  • Test suites are the grouping of multiple test job templates and are executed as a single “run” with different test scenarios.
  • Due to the universal architecture, test jobs of any kind can be part of a test suite (HTTP Test Wizard Jobs, JUnit Jobs, Selenium Jobs ..).
  • The results of the test suite “runs” are archived and can be compared with each other.
  • In addition, the result of a particular test job of a test suite run can be compared with the result of the same test job executed by other test suite runs.
  • All test jobs of a test suite support the powerful ability to be executed with hundreds or thousands of concurrent users and can even executed on a cluster of measuring agents.
  • The test jobs of a test suite are grouped into “execution groups”, where the test jobs of an execution group can be executed in sequential or parallel order.
  • The execution groups can be nested, whereby a separate measuring agent or measuring agent cluster can be defined for each execution group.
  • A “run” of a test suite can be triggered either interactively or via the Remote User API.

This seems a bit complicated at first glance. However, as you will see, creating and running test suites is fairly easy.

Test Job Templates

Test job templates can be created by clicking ‘Save as Template’ in the dropdown menu of any load test job:

You can freely choose the name of the template, but it must be unique.

The template is then created. For verification purposes, the test job properties are displayed which you can also change here.

The list of all templates will then be displayed. Here you can delete, clone and modify individual templates and you can create also new load test jobs from a template.

Test Suites

Adding a Test Suite

As soon as one or more test job templates are created, you can add a new test suite.

You can freely choose the name of the test suite, but it must be unique.

After that the test suite is created and the “Test Suite Editor” is automatically invoked.

Editing a Test Suite

It is recommended that you configure a measuring agent or cluster on which the test suite runs will be executed so that you are not asked about it every time you start a test suite run.

Only “Execution Groups” can be defined at the main level of a test suite. This means that you have first to add one or more execution groups to the test sequence.

At the top right of the editor area you can configure the “Main Execution Order”, which controls whether the execution groups at the main level are executed one after the other (= sequential ) or concurrently (= parallel).

Adding an Execution Group

Here you can specify on which measuring agent or cluster the execution group will be executed, whereby the default option “inherit from parent” is recommended. By setting the “Execution Order” you can determine whether the elements of the execution group are executed sequentially or in parallel.

Note: Execution groups can also be nested, which means that an execution group can contain several sub-execution groups, which can then contain sub-sub execution groups. The advantage of nesting execution groups is, among other things, that the measured data of the executed the test jobs are presented in aggregated form at the level of each execution group.

Adding Test Job Templates to an Execution Group

Finally, click the icon at the top left corner to return to the Test Suites Dashboard.

Executing a Test Suite “Run”

A test suite “run” can be triggered either interactively or via the Remote User API.

To start a “run” click on the green icon of the test suite - which means that all test jobs of the test suite are executed.

In the upper area of the green window, summarized/aggregated measurement results over all executed test jobs are displayed.

In the left area of the green window, the progress of the of the test suite execution is displayed in real time, whereby the measured values of the currently executed (load) test jobs are updated every 4 seconds.

Multiple test suites can be executed concurrently, but the same test suite cannot be executed twice at the same time. Note that closing the green window will not abort the execution of the test suite “run”.

Tip: If a load test job of a test suite takes several minutes to execute, you can also view the detailed statistics of the load test job in a second Web Browser window in the Real Time View. All you have to do is select the Real Time View in the second browser window.

After a test suite run is completed you can click on “Display Result”.

Test Suite “Run” Results

Test Suite Run Results are always historical data, whereby the execution groups and the test jobs are painted as they were at the time of execution.

In the upper area summarized/aggregated measurement results over all executed test jobs are displayed.

Clicking on the chart icon of a test job displays the “Test Result Details”. By clicking the file icon, you can view the contents of the test job output files.

Comparing Test Suite Run Results

The summarized/aggregated measurement results of the test suite runs can be compared across all or a selection of runs.

Furthermore, the measurement results of the individual test jobs can be compared across all or a selection of runs.

10 - Table Server

Table Server - Introduction

It is an API Service which holds all the API requests, which can be used to store the test data in table formats in a Database, which can be consumed/produced by both load test and synthetic monitoring tests. So it acts like a Test Data database where multiple scripts can consume common test data.

Currently, the Tests need Plugins to communicate with the Table Server API, but in near future we will come up with a solution where performance Engineers can write inline javascript in the test to communicate with the Table Server APIs.

Below are the API Requests that Table Server supports

  • Create a Table with the JSON input, an overwriteFlag flag will be optional here, if set to true, it will overwrite the existing table with the same name.
  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "createNewTable",
  "tableName": "Products",
  "columnNamesAndValues": [
        {
            "ProductGroup": "Icecream",   
            "ProductName": "Vanilla"
        },
        {
            "ProductGroup": "Pizza",   
            "ProductName": "Margareta"
        }
    ]
  }'
  • Create a Table with the CSV file data, an overwriteFlag flag will be optional here, if set to true, it will overwrite the existing table with the same name.
  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "createNewTableFromCSVFile",
  "uploadFileName": "Products.csv",
  "delimiter": ",",
  "uploadFileDataB64": "UHJvZHVjdEdyb3VwLFByb2R1Y3ROYW1lLFByaWNlDURyaW5rcyxDb2NhQ29sYSwxMApEcmlua3MsUGVwc2ksNwpEcmlua3MsN1VwLDgKRHJpbmtzLEZhbnRhLDkKUGl6emEsTWFyZ2FyZXRhLDEwDQ=="
  }'
  • Get All the Tables from Table Server with or without column details.
  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "getAllTables",
  "includeColumns": true
  }'
  • Create a single test data, can also pass an optional parameter “uniqueFlag” to 1, so that it will make sure that the column value is unique.
  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "insertColumn",
  "table": "Products",
  "column": "ProductGroup",
  "value": "Sweets"
  }'
  • Get a single test data, returns first row value for the given column and marks that as retrieved in the table. Supports an additional parameter rowIndex for getting the same value multiple times during the execution of the tests.
  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "retrieveColumn",
  "table": "Products",
  "column": "ProductName"
  }'
  • Update a single test data.

    curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
    --header 'Content-Type: application/json' \
    --data-raw '{
    "apiAuthToken": "{{apiAuthToken}}",
    "action": "updateColumn",
    "table": "Products",
    "column": "ProductGroup",
    "value": "Fruits",
    "rowIndex":1
    }'
    
  • Create multiple test data.

  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "insertRow",
  "table": "Products",
  "columnsAndValues": [
        {
            "ProductGroup": "Vegetables",      
            "ProductName": "Carrot"
        }
    ]
  }'
  • Get multiple test data, returns first row values for given columns and marks that as retrieved in the table. Supports an additional parameter rowIndex for getting the same value multiple times during the execution of the tests.
  curl --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "retrieveRow",
  "table": "Products",
  "columns": [
            "ProductGroup",    
            "ProductName"
    ]
  }'
  • Query a table with column names and values, if not provided Column Names and Values, the entire table data will be retrieved.
  curl -k --location --request POST 'https://{{tableserverURL}}:8084/Api' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "apiAuthToken": "{{apiAuthToken}}",
  "action": "queryTable",
  "table": "Products",
  "columnNamesAndValues": [
        {
            "ProductGroup": "Vegetables",   
            "ProductName": "Carrot"
        }
    ]
  }'
  • Export a table as CSV file data.
 curl --location --request POST 'https://{{tableserverURL}}:8084/Api' \
 --header 'Content-Type: application/json' \
 --data-raw '{
 "apiAuthToken": "{{apiAuthToken}}",
 "action": "exportTableToCSV",
 "table": "Products"
 }'

11 - Developer Guide

All Purpose Interface | Developer Guide

Abstract

This document explains:

  1. How to develop a test program from scratch.
  2. How to add self-programmed measurements to the HTTP Test Wizard (as plug-ins).

The product’s open architecture enables you to develop plug-ins, scripts and programs that measure anything that has numeric value - no matter which protocol is used!

The measured data are evaluated in real time and displayed as diagrams and lists. In addition to successfully measured values, also errors like timeouts or invalid response data can be collected and displayed in real time.

At least in theory, programs and scripts of any programming language can be executed, as long as such a program or script supports the All Purpose Interface.

In practice there are currently two options for integrating your own measurements into the Real Load Platform:

  1. Write an HTTP Test Wizard Plug-In in Java that performs the measurement. This has the advantage that you only have to implement a subset of the “All Purpose Interface” yourself:

    • Declare Statistic
    • Register Sample Start
    • Add Sample Long
    • Add Sample Error
    • [Optional: Add Counter Long, Add Average Delta And Current Value, Add Efficiency Ratio Delta, Add Throughput Delta, Add Test Result Annotation Exec Event]

    Such plug-ins can be developed quite quickly, as all other functions of the “All Purpose Interface” are already implemented by the HTTP Test Wizard.

    Tip: An HTTP Test Wizard session can also only consist of plug-ins, i.e. you can “misuse” the HTTP Test Wizard to only carry out measurements that you have programmed yourself: Plug-In Example

  2. Write a test program or from scratch. This can currently be programmed in Java or PowerShell (support for additional programming languages will be added in the future). This is more time-consuming, but has the advantage that you have more freedom in program development. In this case you have to implement all functions of the “All Purpose Interface”.

Interface Specification

Basic Requirements for all Programs and Scripts

The All Purpose Interface must be implemented by all programs and scripts which are executed on the Real Load Platform. The interface is independent of any programming language and has only three requirements:

  1. The executed program or script must be able to be started from a command line, and passing program or script arguments must be supported.
  2. The executed program or script must be able to read and write files.
  3. The executed program or script must be able to measure one or more numerical values.

All of this seems a bit trivial, but has been chosen deliberately. So that the interface can support almost all programming languages.

Generic Program and Script Arguments

Each executed program or script must support at least the following arguments:

  • Number of Users: The total number of simulated users (integer value > 0).
  • Test Duration: The maximum test duration in seconds (integer value > 0).
  • Ramp Up Time: The ramp up time in seconds until all simulated users are started (integer value >= 0). Example: If 10 users are started within 5 seconds then the first user is started immediately and then the remaining 9 users are started in (5 seconds / 9 users) = 0.55 seconds intervals.
  • Max Session Loops: The maximum number of session loops per simulated user (integer value > 0, or -1 means infinite number of session loops).
  • Delay Per Session Loop: The delay in milliseconds before a simulated user starts a next session loop iteration (integer value >= 0) – but not applied for the first session loop iteration.
  • Data Output Directory: The directory to which the measured data have to be written. In addition, other data can also written to this directory like for example debug information.

Implementation Note: The test ends if either the Test Duration is elapsed or if Max Session Loops are reached for all simulated users. Currently executed sessions are not aborted.

In addition, the following arguments are optional, but also standardized:

  • Description: A brief description of the test
  • Debug Execution: Write debug information about the test execution to stdout
  • Debug Measuring: Write debug information about the declared statistics and the measured values to stdout
Argument Java PowerShell
Number of Users -users number -totalUsers number
Test Duration -duration seconds -inputTestDuration seconds
Ramp Up Time -rampupTime seconds -rampUpTime seconds
Max Session Loops -maxLoops number -inputMaxLoops number
Delay Per Session Loop -delayPerLoop milliseconds -inputDelayPerLoopMillis milliseconds
Data Output Directory -dataOutputDir path -dataOutDirectory path
Description -description text -description text
Debug Execution -debugExec -debugExecution
Debug Measuring -debugData -debugMeasuring

Single-Threaded Scripts vs. Multiple-Threaded Programs

For scripts which don’t support multiple threads the Real Load Platform starts for each simulated user a own operating system process per simulated user. On the other hand, for programs which support multiple threads, only one operating system process is started for all simulated users.

Scripts which are not able to run multiple threads must support the following additional generic command line argument:

  • Executed User Number: The currently executed user (integer value > 0). Example: If 10 scripts are started then 1 is passed to the first started script, 2 is passed to the second started script, .. et cetera.
Argument PowerShell
Executed User Number -inputUserNo number

Specific Program and Script Arguments

Additional program and script specific arguments are supported by the Real Load Platform. Hoverer, their values are not validated by the platform.

Job Control Files

During the execution of a test the Real Load Platform can create and delete at runtime additional control files in the Data Output Directory of a test job. The existence, and respectively the absence of such control files must be frequently checked by the running script or program, but not too often to avoid CPU and I/O overload. Rule of thumb: Multi-threaded programs should check the existence of such files every 5..10 seconds. Single-threaded scripts should check such files before executing a new session loop iteration.

The following control files are created or removed in the Data Output Directory by the Real Load Platform:

  • DKFQS_Action_AbortTest.txt If the existence of this file is detected then the test executions must be aborted gracefully as soon as possible. Currently executed session loops are not aborted.
  • DKFQS_Action_SuspendTest.txt If the existence of this file is detected then the further execution of session loops is suspended until the file is removed by the Real Load Platform. Currently executed session loops are not interrupted on suspend. When resuming the test then the Ramp Up Time as passed as generic argument to the script or program must be re-applied. If a suspended test runs out of Test Duration then the test must end.

Testjob Data Files

When a test job is started by the Real Load Platform on a Measuring Agent, then the Real Load Platform creates at first for each simulated user an empty data file in the Data Output Directory of the test job:

Data File: user_<Executed User Number>_statistics.out

Example: user_1_statistics.out, user_2_statistics.out, user_3_statistics.out, .. et cetera.

After that, the test script(s) or test program is started as operating system process. The test script or the test program has to write the current state of the simulated user and measured data to the corresponding Data File of the simulated user in JSON object format (append data to the file only – don’t create new files).

The Real Load Platform component Measuring Agent and the corresponding Data Collector are listening to these data files and interpret the measured data at real-time, line by line as JSON objects.

“alt attribute”

Writing JSON Objects to the Data Files

The following JSON Objects can be written to the Data Files:

JSON Object Description
Declare Statistic Declare a new statistic
Register Execute Start Registers the start of a user
Register Execute Suspend Registers that the execution of a user is suspended
Register Execute Resume Registers that the execution of a user is resumed
Register Execute End Registers that a user has ended
Register Loop Start Registers that a user has started a session loop iteration
Register Loop Passed Registers that a session loop iteration of a user has passed
Register Loop Failed Registers that a session loop iteration of a user has failed
Register Sample Start Statistic-type sample-event-time-chart: Registers the start of measuring a sample
Add Sample Long Statistic-type sample-event-time-chart: Registers that a sample has measured and report the value
Add Sample Error Statistic-type sample-event-time-chart: Registers that the measuring of a sample has failed
Add Counter Long Statistic-type cumulative-counter-long: Add a positive delta value to the counter
Add Average Delta And Current Value Statistic-type average-and-current-value: Add delta values to the average and set the current value
Add Efficiency Ratio Delta Statistic-type efficiency-ratio-percent: Add efficiency ratio delta values
Add Throughput Delta Statistic-type throughput-time-chart: Add a delta value to a throughput
Add Error Add an error the test result
Add Test Result Annotation Exec Event Add an annotation event to the test result

Note that the data of each JSON object must be written as a single line which end with a \r\n line terminator.

Program Sequence

“alt attribute”

“alt attribute”

“alt attribute”

JSON Object Specification

Declare Statistic Object

Before the measurement of data begins, the corresponding statistics must be declared at runtime. Each declared statistic must have a unique ID. Multiple declarations with the same ID are crossed out by the platform.

Currently 5 types of statistics are supported:

  • sample-event-time-chart : This is the most common statistic type and contains continuously measured response times or any other continuously measured values of any unit. Information about failed measurements can also be added to the statistic. Statistics of this type are added to the ‘Overview Statistic’ area and can also displayed as a chart (see picture below).
  • cumulative-counter-long : This is a single counter whose value is continuously increased during the test. Statistics of this type are added to the ‘Test-Specific Values’ area.
  • average-and-current-value : This is a separately measured mean value and the last measured current value. Statistics of this type are added to the ‘Test-Specific Values’ area.
  • efficiency-ratio-percent : This is a measured efficiency in percent (0..100%). Statistics of this type are added to the ‘Test-Specific Values’ area.
  • throughput-time-chart : This is a measured throughput per second. Statistics of this type are added to the ‘Test-Specific Values’ area.

“alt attribute”

It’s also supported to declare new statistics at any time during test execution, but the statistics must be declared first, before the measured data are added.

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "DeclareStatistic",
  "type": "object",
  "required": ["subject", "statistic-id", "statistic-type", "statistic-title"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'declare-statistic'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "Unique statistic id"
    },
    "statistic-type": {
      "type": "string",
      "description": "'sample-event-time-chart' or 'cumulative-counter-long' or 'average-and-current-value' or 'efficiency-ratio-percent' or 'throughput-time-chart'"
    },
    "statistic-title": {
      "type": "string",
      "description": "Statistic title"
    },
    "statistic-subtitle": {
      "type": "string",
      "description": "Statistic subtitle | only supported by 'sample-event-time-chart'"
    },
    "y-axis-title": {
      "type": "string",
      "description": "Y-Axis title | only supported by 'sample-event-time-chart'. Example: 'Response Time'"
    },
    "unit-text": {
      "type": "string",
      "description": "Text of measured unit. Example: 'ms'"
    },
    "sort-position": {
      "type": "integer",
      "description": "The UI sort position"
    },
    "add-to-summary-statistic": {
      "type": "boolean",
      "description": "If true = add the number of measured and failed samples to the summary statistic | only supported by 'sample-event-time-chart'. Note: Synthetic measured data like Measurement Groups or Delay Times should not be added to the summary statistic"
    },
    "background-color": {
      "type": "string",
      "description": "The background color either as #hex-triplet or as bootstrap css class name, or an empty string = no special background color. Examples: '#cad9fa', 'table-info'"
    }
  }
}

Example: 
{
  "subject":"declare-statistic",
  "statistic-id":1,
  "statistictype":"sample-event-time-chart",
  "statistic-title":"GET http://192.168.0.111/",
  "statistic-subtitle":"",
  "y-axis-title":"Response Time",
  "unit-text":"ms",
  "sort-position":1,
  "add-to-summarystatistic":true,
  "background-color":""
}

After the statistics are declared then the activities of the simulated users can be started. Each simulated user must report the following changes of the current user-state:

  • register-execute-start : Register that the simulated user has started the test.
  • register-execute-suspend : Register that the simulated user suspend the execution of the test.
  • register-execute-resume : Register that the simulated user resume the execution of the test.
  • register-execute-end : Register that the simulated user has ended the test.

Register Execute Start Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterExecuteStart",
  "type": "object",
  "required": ["subject", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-execute-start'"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-execute-start","timestamp":1596219816129}

Register Execute Suspend Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterExecuteSuspend",
  "type": "object",
  "required": ["subject", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-execute-suspend'"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-execute-suspend","timestamp":1596219816129}

Register Execute Resume Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterExecuteResume",
  "type": "object",
  "required": ["subject", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-execute-resume'"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-execute-resume","timestamp":1596219816129}

Register Execute End Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterExecuteEnd",
  "type": "object",
  "required": ["subject", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-execute-end'"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-execute-end","timestamp":1596219816129}

Once a simulated user has started its activity it measures the data in so called ‘session loops’. Each simulated must report when a session loop iteration starts and ends:

  • register-loop-start : Register the start of a session loop iteration.
  • register-loop-passed : Register that a session loop iteration has passed / at end of the session loop iteration.
  • register-loop-failed : Register that a session loop iteration has failed / if the session loop iteration is aborted.

Register Loop Start Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterLoopStart",
  "type": "object",
  "required": ["subject", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-loop-start'"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-loop-start","timestamp":1596219816129}

Register Loop Passed Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterLoopPassed",
  "type": "object",
  "required": ["subject", "loop-time", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-loop-passed'"
    },
    "loop-time": {
      "type": "integer",
      "description": "The time it takes to execute the loop in milliseconds"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-loop-passed","loop-time":1451, "timestamp":1596219816129}

Register Loop Failed Object

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterLoopFailed",
  "type": "object",
  "required": ["subject", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-loop-failed'"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-loop-failed","timestamp":1596219816129}

Within a session loop iteration the samples of the declared statistics are measured. For sample-event-time-chart statistics the simulated user must report when the measuring of a sample starts and ends:

  • register-sample-start : Register that the measuring of a sample has started.
  • add-sample-long : Add a measured value to a declared statistic.
  • add-sample-error : Add an error to a declared statistic.

Register Sample Start Object (sample-event-time-chart only)

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "RegisterSampleStart",
  "type": "object",
  "required": ["subject", "statistic-id", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'register-sample-start'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "The unique statistic id"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"register-sample-start","statisticid":2,"timestamp":1596219816165}

Add Sample Long Object (sample-event-time-chart only)

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "AddSampleLong",
  "type": "object",
  "required": ["subject", "statistic-id", "value", "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'add-sample-long'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "The unique statistic id"
    },
    "value": {
      "type": "integer",
      "description": "The measured value"
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{"subject":"add-sample-long","statisticid":2,"value":105,"timestamp":1596219842468}

Add Sample Error Object (sample-event-time-chart only)

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "AddSampleError",
  "type": "object",
  "required": ["subject", "statistic-id", "error-subject", "error-severity",
  "timestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'add-sample-error'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "The unique statistic id"
    },
    "error-subject": {
      "type": "string",
      "description": "The subject or title of the error"
    },
    "error-severity": {
      "type": "string",
      "description": "'warning' or 'error' or 'fatal'"
    },
    "error-type": {
      "type": "string",
      "description": "The type of the error. Errors which contains the same error
    type can be grouped."
    },
    "error-log": {
      "type": "string",
      "description": "The error log. Multiple lines are supported by adding \r\n line terminators."
    },
    "error-context": {
      "type": "string",
      "description": "Context information about the condition under which the error occurred. Multiple lines are supported by adding \r\n line terminators."
    },
    "timestamp": {
      "type": "integer",
      "description": "Unix-like time stamp"
    }
  }
}

Example: 
{
  "subject":"add-sample-error",
  "statistic-id":2,
  "error-subject":"Connection refused (Connection refused)",
  "error-severity":"error",
  "error-type":"java.net.ConnectException",
  "error-log":"2020-08-01 21:24:51.662 | main-HTTPClientProcessing[3] | INFO | GET http://192.168.0.111/\r\n2020-08-01 21:24:51.670 | main-HTTPClientProcessing[3] | ERROR | Failed to open or reuse connection to 192.168.0.111:80 |
 java.net.ConnectException: Connection refused (Connection refused)\r\n",
  "error-context":"HTTP Request Header\r\nhttp://192.168.0.111/\r\nGET / HTTP/1.1\r\nHost: 192.168.0.111\r\nConnection: keep-alive\r\nAccept: */*\r\nAccept-Encoding: gzip, deflate\r\n",
  "timestamp":1596309891672
}

Note about the error-severity :

  • warning : After the error has occurred then the simulated user continues with the execution of the current session loop. Error color = yellow.
  • error : After the error has occurred then the simulated aborts the execution of the current session loop iteration, and starts the execution of the next session loop iteration. Error color = red.
  • fatal : After the error has occurred then the simulated user aborts any further execution of the test, which means that the test has ended for this simulated user. Error color = black.

Implementation note: After an error has occurred, the simulated user should wait at least 100 milliseconds before continuing his activities. This is to prevent that within a few seconds several thousand errors are measured and reported to the UI

Add Counter Long Object (cumulative-counter-long only)

For cumulative-counter-long statistics there is no such 2-step mechanism as for ‘sample-event-time-chart’ statistics. The value can simple increased by reporting a Add Counter Long object.

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "AddCounterLong",
  "type": "object",
  "required": ["subject", "statistic-id", "value"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'add-counter-long'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "The unique statistic id"
    },
    "value": {
      "type": "integer",
      "description": "The value to increment"
    }
  }
}

Example: 
{"subject":"add-counter-long","statistic-id":10,"value":2111}

Add Average Delta And Current Value Object (average-and-current-value only)

To update a average-and-current-value statistic the delta (difference) values of the cumulated sum and the delta (difference) of the cumulated number of values has to be reported. The platform calculates then the average value by dividing the cumulated sum by the cumulated number of values. In addition, the last measured value must also be reported.

{
  "$schema": "http://json-schema.org/draft/2019-09/schema",
  "title": "AddAverageDeltaAndCurrentValue",
  "type": "object",
  "required": ["subject", "statistic-id", "sumValuesDelta", "numValuesDelta", "currentValue", "currentValueTimestamp"],
  "properties": {
    "subject": {
      "type": "string",
      "description": "Always 'add-average-delta-and-current-value'"
    },
    "statistic-id": {
      "type": "integer",
      "description": "The unique statistic id"
    },
    "sumValuesDelta": {
      "type": "integer",
      "description": "The sum of delta values to add to the average"
    },
    "numValuesDelta": {
      "type": "integer",
      "description": "The number of delta values to add to the average"
    },
    "currentValue": {
      "type": "integer",
      "description": "The current value, or -1 if no such data is available"
    },
    "currentValueTimestamp": {
      "type": "integer",
      "description": "The Unix-like timestamp of the current value, or -1 if no such data is available"
    }
  }
}

Example: 
{
  "subject":"add-average-delta-and-current-value",
  "statistic-id":100005,
  "sumValuesDelta":6302,
  "numValuesDelta":22,
  "currentValue":272,
  "currentValueTimestamp":1634401774374
}