JMeter Result Analysis using Datasette

Analyzing the results is one of the critical tasks for any performance engineers. Many of us are comfortable using the spreadsheet program if the data set is small. But if you have large data sets, it is very difficult to use spreadsheet programs. In that case, you can use the visualization app such as Grafana, or APM tools like Dynatrace or New Relic. In this blog post, I am going to show you, how you can export and publish your JMeter results to cloud, analyze and share it in a couple of steps. We are going to leverage Datasette tool to export and publish the JMeter results to Vercel.

What is Datasette?

Datasette is a tool for exploring and publishing data. It helps people take data of any shape or size and publish that as an interactive, explorable website and accompanying API.

Datasette is aimed at data journalists, museum curators, archivists, local governments and anyone else who has data that they wish to share with the world.

Here https://datasette.io/examples you can explore few example data sets.

Datasette GitHub Repo

Prerequisites 

Following are the prerequisites required to export and publish the JMeter test results to Vercel via Datasette.

1. Enable Windows WSL
2. Vercel account which is free to create
3. Vercel CLI
4. Python and PIP
5. Datasette and its utilities
6. JMeter test results

If you are using Windows 10, you will not be able to publish the results to Vercel. I have already raised an issue in Datasette repo.

Let's get started

Installing Datasette

Launch Windows Terminal and open Ubuntu and enter the below command to install Datasette.

pip install datasette

This will install Datasette in WSL. To verify, issue the command datasette --version.

Installing Vercel CLI

Follow the instructions mentioned here to install Vercel CLI.

Installing CSVS to SQLite

Output of JMeter result is CSV. Datasette uses SQLite to store the data. Before we publish the results to Vercel, we need to convert CSV into SQLite.

To install this utility, issue the below command in WSL.

pip install csvs-to-sqlite

There is a plugin called vega to visualize the JMeter results, but it can be installed later.

Converting CSV to SQLite

Now, it is time to convert the JMeter results to SQLite database. Copy the CSV file to the current directory.

To convert CSV to SQLite, issue the below command.

csvs-to-sqlite Run1.csv JMeterResults.db

CSV to SQLite

Now we have got the SQLite database which is ready to publish to Vercel.

Before we publish it to Vercel, let us publish it to localhost.

Publishing Results

To begin publishing to localhost, issue the below command. This will open the Datasette application in the available port.

datasette serve JMeterResults.db -p 0

Launch the localhost app where you can see the JMeter Results and uploaded Run1.csv file.

Now you can slice and dice the data to analyze the results. It is extremely fast and load time is in milliseconds.

But this data set doesn't have vega. Let us install vega plugin to visualize the charts.

Issue the below command to install vega.

datasette install datasette-vega

Now publish the database again and navigate to the app. You could see the chart for the JMeter results.

Publish to Vercel

Now let us take our game to the next level i.e. to the cloud. Open the Windows Ubuntu Terminal and issue the below command to login into your Vercel account.

Issue the below command to login into Vercel.

vercel login

This will prompt you to enter the email ID as shown below.

Vercel Login

Check your inbox and verify the login. After successful verification, you will get the below message.

Vercel Successful Login

Before publishing, we need to install Datasette plugin for Vercel using the below command.

pip install datasette-publish-vercel

Now, we are ready to publish the data to Vercel. To publish the database to Vercel, issue the below command.

datasette publish vercel JMeterResults.db --project jmeter --install datasette-vega

After successful deployment, you will see the link for your project. Here is the link which we published JMeter results to Vercel https://jmeter.vercel.app/.

Publishing Datasette

After deployment, you can slice and dice the data as shown. Datasette will work only for read queries. You can filter, export, visualize, and publish your data to the supported cloud environment in few steps.

Example filtering in Datasette.

Just below the above options, you can see the raw data with filter, sort, and facet options.

Conclusion

Datasette helps in exploring and publishing test results to local or cloud. It will help you to define a new perspective to view the test results to detect the anomalies. It is light-weight and fast. Due to its CLI nature, you can easily integrate in the CI/CD pipeline to deploy the dataset to your stakeholders. Datasette works with any performance testing tool. All you need is your raw data in CSV format or SQLite format.

Are you ready to explore Datasette? Please share your inputs in the comments.