Performance Testing for a SignalR Web App using Jmeter

In our current project, we have a web application for playing weekly real-time quiz online between a group of people. The number of people playing concurrently depends on the number of people signed up for the current week game. So it was needed to test the current architecture to find its limits. Our web app primarily uses Azure SignalR for real-time communication. The web app is written in .NET Core 2.0. Azure SignalR primarily uses the WebSocket protocol to establish a persistent connection between the client and the server. After lots of deliberation, we ended up selecting JMeter to record the HTTP traffic as well as the WebSocket traffic and then mimic clients by using a tool called Redline13.

Note: If you are looking for replicating transport types like  Server-Sent Events or Long Polling in SignalR, this blog does not discuss it and only deals with WebSockets transport type in SignalR.

STEP 1: Recording All Http Requests

The first step is to record the HTTP traffic inside your website. The process of recording HTTP traffic is pretty straight-forward as explained in this blog post.
In case you are not familiar with JMeter, have a look at this post.

STEP 2: Recording All WebSocket Communication

Once Step 1 is done,  you have to record the WebSocket communication between the server and the client.

Unfortunately, there is no in-built support for capturing WebSocket traffic in Jmeter but third-party plugins are available for creating WebSocket connection, reading incoming message and sending a message to the server.

At the time of writing this blog, WebSocket Samplers by Peter Doornbosch is the most comprehensive of the lot and we used it.

To add a plugin to JMeter follow this blog post.

  • To view the WebSocket traffic in your SignalR app inspect the browser and go into the network tab and click on the WS connection that is used by the web app.
    In the sub-window that opened, click on the Frames tab and you can see the all the frames of data communication to and fro the server. Depending on the nature of your traffic through WebSocket use the various samplers provided in the plugin.
  • In SignalR, always the first frame to be sent from the client sent from a single write sampler is a HandshakeRequest: {“protocol”:”json”,”version”:1}

Note: All JSON messages must be terminated by the ASCII character 0x1E (record separator).

Without this character, the JMeter sampler won’t work properly since it denotes the end of the message. Thus the server will continue to listen for the continuation of the message. This character is not visible in all text editors but you could see it in the browser window as an opaque black triangle at the end of each frame.

STEP 3: Customizing JMeter Script According to user-behavior

Now that we have added all the network requests. According to the properties of your web app, you could use different Jmeter features to exactly replicate user behavior on the website.

  • For importing variables from a CSV sheet you could use the CSV Data Set Config.
  • Jmeter provides JMeter variables which can be used to have dynamic values in your requests.
  • For any logical calculation,s we can write scripts in between requests with any of the scripting languages supported like javascript, groovy and jexl3.
  • For processing response data you could add processors like JSR223 samplers or JSR223 pre/post processors.  
  • For adding delays between two request invocations you could use constant timers.
  • For looping or conditional request, invocations check out these logic controllers.
  • For extracting text from response, you can use regex extractor.
  • When using WebSocket samplers, the server occasionally sends ping messages to acknowledge the client and which should be returned using pong messages.
    This can be done using WebSocket Ping/Pong Frame Filter.
  • In SignalR applications, occasionally the signal ping messages are sent by the signalR server which can interfere with your WebSocket Single Read Sampler.
    So to filter out those incoming message frames you could use WebSocket Text Frame Filter.

          SignalR Ping message : – {“type”:3,”invocationId”:”0″,”result”:null}

STEP 4. : Increasing the users and testing in the cloud

Now that we have completed the request flow for one user, it’s time to magnify the number of users by increasing the thread count in the JMeter script.

  • The thread group has a property to specify thread count which denotes the number of users.
  • Also, we can specify the ramp-up time in seconds which denotes the time period in which all the user threads must start.
  • Save the JMeter script as a .jmx file.
  • Now to run JMeter script for load testing you can use your local machine if that suits you. We chose to run it on the cloud. Since we had a company AWS account we decided to go with  Redline13 which uses our AWS account where we can select a spot instance of any server to load the performance test. This can help in avoiding network bottleneck at your workplace.
  • The free version of Redline13 allows you to test but would not allow you to access JMeter logs if you have any. But the errors are displayed in the test page but up to only 500 in number so you can name your JMeter requests accordingly including any relevant JMeter variable to get its value in the error list. Our web app is hosted on Azure so we had our Application Insights in Azure dashboard to monitor web app health.
  • This blog post from Redline13 explains the steps in detail.
  • When you are using third-party plugins in JMeter scripts, it is important to add all the used plugins in the test configuration page in RedLine13 with the correct version.
  • Also if your using WebSockets in your JMeter script check the WebSockets checkbox in
    Jmeter Plugin box in the test configuration page.

Thus the combination of JMeter and Redline13 can be a right fit to performance test a complex web application for finding its limit scenarios.

Leave a Reply