As Big Data and real-time web apps keep getting bigger, NoSQL databases are more popular than ever for data storage. This is especially true for NoSQL’s most popular database – MongoDB.
Let’s take a step back for a moment…
When performance testing you usually need to check the entire application to measure the end-user experience. But it’s sometimes easier to test each database component separately if you want to check the response times for certain queries, the efficiency of horizontal scaling, replications, failovers and resilience, and to ensure the database is properly tuned you won’t hit a bottleneck or be denied service.
In today’s post, I’m going to guide you through MongoDB load testing with Apache JMeter, highlight frequently faced problems along with their workarounds, and explain and demonstrate some of MongoDB’s most common tasks.
JMeter first introduced support for MongoDB support in version 2.10, which was released in October 2013. Now we’re already on JMeter 2.13…here’s how MongoDB is being implemented:
MongoDB Source Config – a configuration element which creates a MongoDB connection using specified settings.
MongoDB Script – a sampler capable of sending requests to MongoDB. It assumes and requires the presence and correct configuration of MongoDB Source Config
Let’s take a look at how these test elements can be used for MongoDB testing. For today’s demo, I’ll be using the chúng tôi file – which can be found here: http://media.mongodb.org/zips.json. If you want to copy any of the examples below, just download the chúng tôi file to your computer and import it with this command:
mongoimport –db test –collection zips –file /path/to/zips.json
A quick explanation of each term…
mongoimport: a binary file located in the /bin folder of your MongoDB installation
“test”: MongoDB’s database, a container for “collections”
“zips”: MongoDB collection name
{ “_id” : “43747”, “city” : “JERUSALEM”, “loc” : [ -81.092088, 39.848831 ], “pop” : 2037, “state” : “OH” }
Want to visualize it more easily? Use tools like MongoVUE or Robomongo.
So let’s get started.
In order to execute MongoDB queries we need to:
You can leave other values as they are – the defaults are good to go. I’ll use the IP address of my MongoDB test server, and give “blazemeter” as the Mongo Source name.
Add the MongoDB Script, providing the:
Mongo Source name (in this case, “blazemeter”)
Database name (in this case, “test”)
Script. For this step, I’ll use the count of the elements in the “zips” collection, which is:
db.getCollection(“zips”).count()
View the Results Tree Listener to visualize the results
Let’s run the test and see how it goes:
As you can see, the MongoDB Script sampler response data is 29353.0. This is the actual count of all the documents in the “zips” collection of the “test” database.
IMPORTANT: There should not be any blank lines in the MongoDB Script Sampler. If you have a blank line after your MongoDB statement, the statement will still be chúng tôi you’ll only get the word “ok” in the response. If you’re getting “ok” double check that there aren’t any trailing blank lines in your script.
Make sure that line number “2” is NOT displayed.
Now let’s enter information on a city into the collection. You can re-use the same test plan but the “Script” area of the MongoDB Script sampler needs to be updated to something like this:
db.getCollection(‘zips’).insert({“city” : “BABYLON”, “loc” : [ 32.211100, 44.251500 ], “pop” : 300000, “state” : “IS” })
When you run the updated script in JMeter, you should be able to see the request details in the “Request” tab of the ‘View Results Tree Listener’
…and in the “Response Data” tab, you’ll be able to see the results.
If you change the MongoDB Script query back to db.getCollection(“zips”).count(), you should be able to get the updated documents count (incremented by 1). In my case, the count is: 29354.0
db.getCollection(‘zips’).findOne({“city”:”BABYLON”})
If you look into the Response Data tab of the View Results Tree Listener, you’ll be able to see the document that was inserted earlier.
It’s worth noting that I switched to the “JSON” tab for more user-friendly view.
Now, let’s measure the time it takes with a MongoDB Script sampler to get all the documents where the city is “NEW YORK”. This is how the MongoDB query should look:
db.getCollection(“zips”).find({“city” : “NEW YORK”})
When executed in Robomongo, this query returns 40 results within 0.103 seconds
Let’s try doing the same in JMeter. Enter the same request into the “Script” section of the MongoDB Script sampler:
Now let’s look at the “Response Data” tab:
It looks like that the sampler returns information on the health of the MongoDB server – rather than a list of documents which contain “city = NEW YORK” (as we wanted).
But Why?
When you look at the EvalResultsHandler class source, you can see that they can only be supported when cast to JSON. As the “NEW YORK” query returns more than one (actually 40!) documents, it needs to somehow be converted to JSON so the sampler can return it properly.
Fortunately there is a solution! You just need to append the “.toArray()” method to the end of the MongoDB query, like this:
db.getCollection(“zips”).find({“city” : “NEW YORK”}).toArray()
This will convert the response into something that can be cast to the JSON Array. This means the response will be displayed and used to extract the “interesting” bits – such as applying assertions, etc.
Now let’s see how to load test MongoDB with the help of JMeter. Surprisingly, it’s NOT RECOMMENDED to use the MongoDB Script Sampler for MongoDB load testing. The MongoDB Script uses the db.eval() method “under the hood” for executing queries and this is being phased out (as from the current version – MongoDB 3.0). Here are some reasons why:
Performance: the db.eval() method takes the global lock by default. This means that all read and write operations are blocked while your script is being evaluated
Compatibility: the db.eval() method won’t work on shared collections
Security: the db.eval() method requires too much privileges and your account might not have enough permissions to invoke it
Robustness: your scripts relying on the db.eval() method might stop working on next MongoDB release – when the method will be completely removed.
So here’s where it’s OK to use the MongoDB Script Sample:
For functional testing with one thread
If you need to change a huge amount of data and want to avoid overhead caused by passing gigabytes of data back and forth over the network.
However, if you need to create immense load, you should look at other options.
If you’re load testing MongoDB, it’s recommended to call the MongoDB Java Driver methods from the JSR223 Sampler, using “groovy” as a language.
Why JSR223 and groovy?
It’s the only scripting approach which provides performance comparable with native Java code. See Beanshell vs JSR223 vs Java JMeter Scripting: The Performance-Off You’ve Been Waiting For! for details on installing the groovy engine into JMeter – along with limitations and best practices.
If you’re not familiar with groovy – it’s worth knowing that it’s 100% compatible with Java, so a well-formed Java code will work like a charm!
So make sure you have:
a groovy-all-*.jar file in /lib folder of your JMeter installation
restarted JMeter to pick up the Groovy library
a MongoDB Source Config pointing to your MongoDB installation with the Source Name “blazemeter”
added the JSR223 Sampler
chosen “groovy” from “Language” drop down
And we’re good to go on to the next step.
First of all, we need to learn how to do the following:
Initialize the MongoDB connection based on the data source name
Get a count of documents in the collection
Insert a document
Assure that the count is incremented by 1
Find one document
Find multiple documents
Here’s how you access MongoDB from a groovy code:
import com.mongodb.DB; import org.apache.jmeter.protocol.mongodb.config.MongoDBHolder;
DB db = MongoDBHolder.getDBFromSource(“blazemeter”, “test”);
Where
If you need to provide credentials to access MongoDB, just add them as parameters:
DB db = MongoDBHolder.getDBFromSource(“blazemeter”, “test”, “username”, “password”);
So now you have an instance of the chúng tôi class, which is accessible as db. If you want to see all the available methods and fields, take a look at its JavaDOC. Let’s try to invoke MongoDB.getStats() – which is the equivalent of the dbstats MongoDB command. We’re interested in the string representation, so this is how the final line will look:
String result = db.getStats().toString();
Now we somehow need to return the resulting string. The JSR223 Sampler provides some predefined variables like ctx, vars, props, SampleResult,log, etc. These are all shorthands for relevant JMeter API classes instances, such as:
We’re particularly interested in the SampleResult as it gives control over the JSR223 Sampler and returns values like the Response Code, Response Message, and Response Body. To set the sampler response body, you’ll need to use the setResponseData()
So, here’s how it looks when we put everything together:
import com.mongodb.DB; import org.apache.jmeter.protocol.mongodb.config.MongoDBHolder;
DB db = MongoDBHolder.getDBFromSource(“blazemeter”, “test”); String result = db.getStats().toString() SampleResult.setResponseData(result.getBytes());
Here’s what you need to do:
This is what the full code will look like:
import com.mongodb.DB; import org.apache.jmeter.protocol.mongodb.config.MongoDBHolder; import com.mongodb.DBCollection;
DB db = MongoDBHolder.getDBFromSource(“blazemeter”, “test”); DBCollection collection = db.getCollection(“zips”);
Try to run the code yourself to see the count of documents in “zips” collection.
db.getCollection(‘zips’).insert({“city” : “BABYLON”, “loc” : [ 32.211100, 44.251500 ], “pop” : 300000, “state” : “IS” })
In the MongoDB Java API, you can do this via the DBCollection.insert() method – which takes the DBObject as a parameter and returns the WriteResult. Look at the code sample below to learn how to construct the DBObject instance.
We need to construct the following object:
For basic name/value pairs like “city” : “BABYLON”, you can use the BasicDBObject class
For arrays like “loc”, you can use BasicDBList class
This is how it looks when it’s all put together:
import com.mongodb.*
DB db = MongoDBHolder.getDBFromSource(“blazemeter”, “test”); DBCollection collection = db.getCollection(“zips”);
BasicDBObject object = new BasicDBObject();
WriteResult result = collection.insert(object, WriteConcern.ACKNOWLEDGED);
SampleResult.setResponseData(result.toString().getBytes());
This is how your response should look:
{ “serverUsed” : “/52.17.164.41:27017” , “n” : 0 , “connectionId” : 248 , “err” : null , “ok” : 1.0}
Now you can re-run the scenario ” How to Get the Count of the Documents in the Collection to make sure that the number of documents in the”zips” collection will be incremented by one.
Now let’s learn how to issue the “find()” command by using the JSR223 Sampler.
In order to find one document containing “city” : “BABYLON”, we need to invoke the DBCollection.findOne() method and pass the desired city name as a parameter. As we already know how to construct a DBObject to pass as a parameter, we just need to create one per query and add it to the DBCollection.findOne() method.
import com.mongodb.*
DB db = MongoDBHolder.getDBFromSource(“blazemeter”, “test”); DBCollection collection = db.getCollection(“zips”);
BasicDBObject query = new BasicDBObject(“city”, “BABYLON”);
SampleResult.setResponseData(result.toString().getBytes());
You should be able to see the document in the “Response Data” for the JSR223 Sampler in the View Results Tree Listener (just like in the MongoDB Script).
The query which returns multiple documents is pretty much the same as the query for one document. However, the DBCursor will be returned instead of the DBObject – so the results need to be handled a bit differently.
The DBCursor gives two methods which allow you to iterate through nested DBObjects:
hasNext() – checks whether another DBObject is available, it so – returns “true”, elsewise – “false”
next() – returns current DBObject and moves the cursor to the next one
Here’s how it looks when you put everything together:
import com.mongodb.*
DB db = MongoDBHolder.getDBFromSource(“blazemeter”, “test”); DBCollection collection = db.getCollection(“zips”);
BasicDBObject query = new BasicDBObject(“city”, “NEW YORK”); DBCursor cursor = collection.find(query);
while (cursor.hasNext()) {
}
SampleResult.setResponseData(resultBuilder.toString().getBytes());
By default, JMeter doesn’t print a lot into the chúng tôi file. However, if you have an issue that you’d like to get to the bottom of, you can increase the log level on a certain class or package. If you’d like to increase the details of the logging output of the MongoDB-related test elements, you can take one of two options:
Add the log_level.jmeter.protocol.mongodb=DEBUG line to the user.properties file (this can be found in the /bin folder of your JMeter installation). On the next JMeter instance, all classes under the jmeter.protocol.mongodb package will be set to the DEBUG level, which will provide additional information on what’s going on
Pass the desired log level as a command-line argument via the -L key as: jmeter -Ljmeter.protocol.mongodb=DEBUG
In any case, you’ll be able to see the extra debug output from the MongoDB test elements in the chúng tôi file
To get the best performance from the JSR223 Sampler, you just need to follow a few simple rules:
Keep the JSR223 scripts as files in the filesystem. This way they will get compiled into the bytecode and the overhead will be minimized
If, for some reason, point one isn’t possible or available, the groovy script can still be kept in the “Script” area. Just make sure that the “Compilation cache key” is set and unique for each JSR223 Sampler instance
Don’t refer to JMeter Variables as ${VariableName} in your groovy scripts as it will ruin all the benefits of the approach. Use vars.get(“VariableName”) instead.
Blazemeter systems are 100% JMeter compatible. They assume that you don’t need to do anything else except provide a groovy-all-*.jar along with your test script.
You can analyse your MongoDB load test sessions with Blazemeter’s reports. They are fully compatible with any sampler types – even when they are heavily customized.