For this example we’ll use CucumberJVM (AKA Cucumber for Java, here and here are install documents).  SoapUI (part of the ReadyAPI suite) is a powerful tool for testing SOAP or REST endpoints. SoapUI isn’t a BDD tool but for teams who like to use SoapUI for creating system tests, with some adjustment they can drive their SoapUI tests with a BDD framework. This allows SoapUI tests to be linked to features because we build software to create features, not to build APIs (dogs should wag tails after all, not the other way around). Although there are other alternatives achieving this without SoapUI, for teams committed to using SoapUI, here is how to combine the use of a BDD tool (Cucumber in this case) with SoapUI. This article takes you through the recipe of making Cucumber and SoapUI work together. It’s not necessary to use my source code but if you find it helpful, the code for a test project and a Spring Boot service maven project are on GitHub (Eclipse users, clone the repo in your workspace directory, then file->import…, select wizard for “existing maven project.”) If you get stuck, tweet me @LancerKind or contact me these other ways.

Conceptually, the relationship between components looks like:

Cucumber -> my.feature -> Step Definitions -> SoapUI Test -> Service 

This article guides you through the workflow of doing this:

A Develop feature file and understanding
B Build failing BDD Test

 1) check in the feature file with pending.
 2) stub service controller and develop SoapUI test
 3) build step definitions
 4) observe it fail
 5) Check in the code.

C Test Drive the code to change the stubbed service controller 
  into a working service
D Run BDD Test

Create a separate project for your BDD Tests

Let’s start with setting up a project for BDD tests. Although it’s best to mingle each unit test (micro tests) beside the product code it tests, macro tests like BDD tests are testing functionality should be kept in a separate project as features don’t map directly to a single class or method. Additionally, because the Cucumber step definitions are going to link to SoapUI dependencies which number over a hundred jars, chances are your app may not work due to library collisions and versions. (For example, my Spring Boot app couldn’t function with SoapUI dependencies in my class path.) So it’s necessary to insulate your BDD and SoapUI testing code from your application. So make a separate project with dependencies on Cucumber and SoapUI.

Separate BDD Tests from Product Code

For Cucumber, I downloaded the latest of these eight jars from the Maven repository:

And added them to the BDD Test project as external jars:

Using Eclipse add the external libraries SoapUI jars (there are more than shown in the snapshot below) located at the SoapUI install directory, in the java/app/lib:

Develop Feature File and Understanding

Get your amigos together and talk about what you want to build. Capture that understanding in a BDD feature file. (Syntax highlighting provided by the Natural plugin available in the Eclipse Marketplace.)

Click image clearly see the full picture.

Notice how Natural signals that there are step definitions missing with the caution symbols by steps that aren’t yet implemented. Test your cucumber environment by trying to run it.

Build failing BDD Test

It’s valuable to create the failing BDD test before doing any development because you’ll quickly get stuck if you’re understanding is incomplete. Although this feels uncomfortable if you’re not used to this, it’s high value! When you get stuck, go find someone to talk to clear up the misunderstanding. There’s no point to continuing down the path of development or you’ll build the wrong thing anyhow. Here is the feature file we used. Although there is an eclipse-cucumber plugin, I haven’t gotten the ‘select feature file, run as Cucumber Feature’ work flow working in some time. Although this isn’t great, it doesn’t matter since must people will be running their feature files via JUnit by creating a JUnit test case to execute BDD tests:

Right click on TestFeatures.java and “run as JUnit Test” to get the following result:

Notice the circle with the slash icon attached to the icons before the scenario names, signifying that the scenarios are pending. Also note the “Runs:” dashboard shows 14 gherkin keywords skipped. The math is a bit strange. Off camera I’ve another Scenario with a pending gherkin keyword so don’t be bothered to work out the algebra on why 14 on this example as you don’t have all the information. For many teams, the normal workflow is to check these pending tests in so that their build monitor can signal the team’s intent to the world. So that after a Sprint planning, all the .feature files can be checked into source and marked “pending,” at which point, the starting gun has been fired! and the Sprint has begun.

Take a look at the console output and you’ll see that Cucumber has left a present for you. A very valuable present if you’re not interested in fiddling with regular expressions.

Normally we’d move directly to creating step definitions for our .feature file but for this workflow we need to prepare a SoapUI test and then come back to our step definition.

Stub the Service API and Create SoapUI Test

Since SoapUI is at its heart an inspection device for existing services and a record and playback tool, you’ll need to build a stub interface that accepts the message and returns a hard coded response. So building only what is necessary, a Person model, Controller, and something to execute the server.

Click the pointer on the Demo Application Server and Start the server and smoke test the APIs.

This is adequate for bring SoapUI into the picture.

Create SoapUI Project

Launch ReadyAPI or SoapUI, create a project, and save the project file to your functional testing source directory.

SoapUI’s default name strings should all be changed to keep the SoapUI test understandable and paintable. Since the Soap project is designed to support the feature file, rename the project based on the feature name. Rename the request step based on the request API type. Then execute the step to observe that the request steps successfully connects to the API.

Select request step an rename it:

Save the SoapUI project file (.xml) next to the feature file it supports, and rename the project file to something sensible.

Play

Execute the request step by pressing the “play button” to see if it successfully connects to the API and show a green bar.

SoapUI shines as an interactive tool for quickly trying different requests and responses. Create a few prototypical tests for your endpoint and figure out how to get the request and responses working the way you want them. Hard code some of the values. When you think you’ve got the test right, add the SoapUI project properties in place of the hardcoded values. These properties (variables) will be the gateway that your feature file will use to pass along the values it needs to control test execution. Later, we’ll use these properties to pass information Cucumber’s step definitions to the SoapUI Test.

In this example, let’s think about something we need to check from our feature file, the presence of someone by name, and build that assertion on the request. Note the UI areas: Request, Response, and #3, the area where assertions are visible.

A nice feature of SoapUI is as soon as you save an assertion, it executes using the cached response and gives feedback if it passes muster by showing a green ball:

After doing the the same procedure for last name of Poppins:

Click the “play”button and run these tests against the service to get feedback that the test is formed correctly. After observing the test passed, make project properties for first and last name, and change the assertions to use those properties. Caution: There are a lot of types of properties. Test suites and test cases have properties too, so make sure to create and reference project properties.

Hardcode properties to the same hardcoded values for name.

Change the assertions to use the properties. Tap the Test button to check that the assertion is getting the right property, then save changes. Repeat the process for last name.

Since the Test button only tests against the cached message, click the play button and run the SoapUI test to check that the assertions are working correctly and the test is still passing.

Do are all the properties in place for the feature file? Actually no. We’re ignoring creating a request that uses passes along a room id as a project property. Also we need to control how to setup the circumstances for our feature file’s Given. For now, get this simpler test finished and executing from the Cucumber test, creating a full stack experience, and then we’ll talk about how to enhance this.

Cleanup

Clear the values of the project properties, run the tests, and observer the test case fails. The next time these tests will pass will be due to those properties being set via Cucumber. Save the SoapUI project as our BDD tests will use that project file to configure SoapUI for action..

Build Step Definitions

Like before, run execute the Cucumber test using JUnit. Copy the console output of the stubbed step definition and turn it into Java code. Execute the test again and you’ll get the same results as each of the stubbed steps are throwing a Pending exception.

Edit the step functions to use the SoapUI JUnit test runner (XXX link in Smart Bear docs) to use the project properties setup in the SoapUI test to accept input passed in from the feature file.

Click image to see full resolution.

Click to see image in full resolution.

The above test fails and why it fails is shown in the JUnit Failure trace: because “Kevin Bacon” isn’t the same as “Mary. (It appears there is a bug in the SoapUI JUnit error message because you should see “Bacon” populated in “Expected value: [].”)

Great! We’ve proven that we can drive a SoapUI with a feature file and make it fail. Now how about making it pass? Although we could update the stub API to return our brilliant actor “Kevin Bacon,” I’d save that for “Next Steps” and simply update the Then step in the feature file to be “Mary Poppins” and run the tests again so you see a green bar simply by adjusting the feature file.

Although our example isn’t finished covered all the basics needed to do BDD with SoapUI and have a working end-to-end proof of concept and have learned a ton doing it. All that’s left are some incremental learning to do in order to apply this to a product we’d release. Next Steps covers how to use this proof of concept to learn those remaining bits.

Next Moves

Adjust the SoapUI test and step definitions to connect to the service, pass in the room id, and search for “Kevin Bacon.”

• Just as we did above: use SoapUI to “play” with the API to form a request (as opposed to a response) with a room ID, get that working, then do this by passing a Project Property.  Here are a couple of visual hints: 1 and 2. Don’t forget to save the SoapUI Project when finished making changes.
• Enhance the step definitions we did above: use the project property in the Steps to pass the room ID
• Once you’ve got the above done, decide how to populate the service for the Given, then enhance the above definitions to create the situation demanded by the given.

Once you’ve got the first scenario whipped into shape, do the same for the rest of the Scenarios.  When all those BDD Tests are failing in a quality way, you’re done with BDD because:

The BDD tests executing the stubbed API as you want it to be used in production.  Check those tests in.  Your CI environment should signal that you’ve got some BDD tests crying to be made happy.  Congratulations!

Take off your BDD sombrero and put on your TDD hat and start building the solution that makes the BDD tests pass.

Test Drive functionality first, to support Givens and the other pieces demanded by your feature file. Work with an eye to making one BDD scenario happy, then another, then another until they are all passing. As each scenario is made to pass, check in the code so the team (and CI) has access to the latest work and integrate with it. This extreme continuous integration is a good habit to develop to get the developers prepared for Continuous Integration and Continuous Delivery. When all the BDD tests pass, the feature is done. If not, then there is a behavior/test-gap. At that point start a conversation with the product owner and team.

Conclusions

If your team is interested in doing BDD and SoapUI, here is a way to let your behavioral scenarios drive SoapUI tests. In the example presented here, that means making a new SoapUI project (.xml file) for each test. While it may be possible to combine all these SoapUI tests into one monolithic SoapUI project and use SoapUI test case properties instead of SoapUI project properties, but my gut feels it’s not a good idea to create a monolithic SoapUI project for a decades worth of SoapUI tests. But if you want to do it that way, I’m you can set those test case properties via the Java API too. For this article I used ReadyAPI.  Although I haven’t tried this, as far as I can tell this strategy works as documented with the open source version of SoapUI.

Whether you use SoapUI to build tests or as a manual exploration tool, you’ve got a way to keep them behavior driven. In a future post I’ll produce another version of this project using a Java REST testing library which may be a better fit for developers than SoapUI.

References

• Integrate SoapUI with Selenium and Cucumber
• SoapUI JUnit integration
• SoapUI download
• Cucumber JVM
• Cucumber-Eclipse

Troubleshooting & Tips

Here are some gotchas that I learned about.

Finding Soapui Assertions

It’s hard to find test assertions in SoapUI.  You’ll need to find this area to even add them.  Find the Assertion tab located in the bottom quarry *when* you’ve got your test case selected.

XXX assertion tab image

Having trouble finding your test case?  Another shortcoming of SoapUI is many of its icons aren’t intuitive. This is a REST test case icon:

XXX test case icon

When you click on that, the assertions tab should become visible.  Another problem may happen here where upon clicking the assertion tab, you cannot still see the assertions, but maybe only the toolbar for the assertions:

xxx image of the assertion toolbar

What’s happening in this case is that the UI layout isn’t functioning correctly or well.  What I did is maximize the size of the window, then patiently (it took a few tries) drag the pointer over to discover when the pointer changes to a pane-resizer and shrink the pane immediately above the assertion tab.

XXX pane resizer

Also I closed the log pane by clicking on the “log” button. If that doesn’t work either, click on the buttons on the assertion toolbar (like “+” for example) and that will trick the assertion pane open so you can see what your test is doing.

SOAPUI ASSERTION Never Fails

It’s easy to accidentally refer to the wrong property. When SoapUI executes, it caches the response at ${#TestCase#LastName}. This test will *never* fail and it won’t be using the property you’re using to communicate with Cucumber. So make sure you’re asserting on the property you intend.

XXx image of Cucumber linkage property.

Changing SOAPUI Property Not having an effect

When changing hardcoded SoapUI property values, if you leave the editor pane like the following (note the box around the field), the property hasn’t be updated and so it’s still operating with the old value. After changing the value, *press return* and the editor pane won’t have the box around the field, the property will actually be updated, and your test will be using the new value.

Path of the SoapUI project

I had two projects of the same name but at different locations. When I closed ReadyAPI and restarted, for some reason it didn’t re-open the last project I was working in. Instead it opened the first project I created with SoapUI (I noticed this because it didn’t have any of the properties I defined when I last was working with this tool.) To know the path of the project you’ve opened, look at the project properties pane in the lower left corner. You’ll need to fiddle with the column sizer to read value for the “File” property. Another way is to close all open projects and when you highlight projects in the Project Navigator pane, you’ll see the path in the empty editor area.

XXX image for navigator pane path in editor area.

Steps function not having affect on SoapUI Project Property

This symptom is that although the step function is setting the SoapUI project property, the BDD test kept failing and the property field was blank.

Check that your Step definition is spelling the SoapUI project property the same way as it’s named in the assertion that you’re using in SoapUI.  Once I was doing:

String [] properties = {"lastName="+lastName, "firstName="+ firstName};

soapUI.setProjectProperties(properties);

soapUI.run();

Instead of:

String [] properties = {"last_name="+lastName, "first_name="+ firstName};

Test Runner is stuck

If your ReadyAPI! license has expired, the test will “hang.” Check the console for a message like the below..