Product Lifecycle Management Blogs by SAP
Dive into product lifecycle management news, learn about digitalizing PLM for the digital supply chain, and stay informed with product updates from SAP.
cancel
Showing results for 
Search instead for 
Did you mean: 
tim_drury
Active Participant
0 Kudos


I’ve found soapUI to be an invaluable tool for testing web services in SAP Manufacturing Execution (ME) (SDN wiki page).  However, running tests manually is tedious so I began thinking of a way to automate the process.  Thankfully the soapUI developers were way ahead of me.  soapUI provides external runners to execute unit tests and load tests in a headless environment.  They’ve also provided an Apache Maven plugin to allow you to run soapUI tests from Maven (soapUI Maven plugin page).  Because Jenkins/Hudson can run Maven projects out of the box, it doesn’t take a software architect to figure out how to glue those pieces together…  In this blog, I’m using soapUI 3.5 Open Source edition, Apache Maven 2.2.1, and Hudson 1.385 running on Glassfish v3.
Creating a soapUI Project

Starting soapUI should present you with a screen that looks like this:



Let’s begin by right-clicking on Projects in the Navigator pane and selecting “New soapUI Project”.  Set the initial WSDL to http://www.27seconds.com/Holidays/US/Dates/USHolidayDates.asmx?WSDL and soapUI will supply a project name based on the WSDL URL; you can change that if you wish.



Click Ok.  Your project should appear in the Navigator and list all the web service operations available via that WSDL:



Although not strictly needed for this project, let’s introduce a project property which we can change during testing.  Properties allow you to change input values dynamically – i.e. when the test is run – instead of being forced to use the same value created within the soapUI IDE.  We use properties to let developers customize test data to their environment so that they don’t have to create an identical environment of the test author.

Expand the Properties pane below the Projects pane and select the Custom Properties tab:



Click on the Add Property icon on the tool bar – it’s the one on the far left with the “+” symbol.  Give your property a name:



Then give it an initial value:



We’ll be able to change this property when we run the test later.

Let’s open “Request 1” under the GetHalloweenDay operation which was created for us when the project was created by double-clicking on “Request 1” under the GetHalloweenDay operation in the Navigator.



We can run a quick test by throwing in a value for <dat:year> in the request,



and then running the request by clicking the green arrow icon in the Request toolbar:



The result, October 31st, 2010, is the correct value for Halloween in 2010.

Let’s try substituting our project property, YEAR, for the hard-coded year:



Running the request should yield the same result.
Creating a soapUI Unit Test

Right-click on “Request 1” under GetHalloweenDay and select “Add to Test Suite”.  Provide a name for the test suite:



Then you’ll be prompted to provide a name for the test case; I chose Halloween1 for my name:



Then you’ll be prompted to name the request and be allowed to specify some test assertions:



Finally, you’ll have your test suite.  Below is shown the request in the test suite:



Double-clicking on “TestSuite 1” in the Navigator pane will show you the test suite panel:



You can run the test suite by clicking on the green arrow in the upper left corner of the “TestSuite 1” panel:



It should execute successfully, but it’s not doing very much of interest because we haven’t told soapUI what to look for in a successful response.  Let’s do that now.  Re-open the “GetHalloweenDay – Request 1” request in the test suite and click on Assertions at the bottom left of the request window:



One assertion is already provided – that the SOAP response returned by the server is a valid response.  Let’s add a couple more.  Click the Add Assertion icon in the Assertions tool bar – it’s the one with the green and red circles with a “+” symbol.  Choose the “Not SOAP Fault” assertion:



This will cause the test to fail if the server responds with a SOAP fault.  Next, choose the “XPath Match” assertion:



which will open the XPath Match Configuration dialog:



We need to enter an XPath expression in the top text area and the expected result in the bottom text area.  Fortunately, soapUI provides some shortcuts to make this a little easier.  Click the Declare button and soapUI will pre-populate our XPath expression with namespace declarations available in the response area of the test request.  This only works if you have already sent the test request and received a response; if you haven’t done so, go back to the “GetHalloweenDay – Request 1” request and run the request.



Enter as an XPath Expression the following just below the namespace declarations:
/soap:Envelope/soap:Body/ns1:GetHalloweenDayResponse/ns1:GetHalloweenDayResult

Then click the button “select from current” which will run this XPath expression against the current result in the “GetHalloweenDay – Request 1” window.



This result may be good enough, but I worry that a developer in another time zone might get a different result and, potentially, cause the test to fail.  Let’s use a little more XPath to narrow the result down to just the month and day,
substring(/soap:Envelope/soap:Body/ns1:GetHalloweenDayResponse/ns1:GetHalloweenDayResult, 6, 5)

Press the “Select from current” button and you should see just the month and day:



Press the Save button to save this assertion.  Our test request should now have three assertions:



Let’s also create a test case which handles a failure scenario.  In the Navigator panel, right-click on GetHalloweenDay and create a new request named “Failure 1”.  For the year of that request, enter an invalid year such as “ABCD”:



Run the request and you should get a SOAP fault as the response:



Right-click on the “Failure 1” request in the Navigator and select “Add to TestCase”.  Choose the option to create a new TestCase in the existing test suite, “TestSuite 1”:



Name the test case:



Click Ok on the “Add Request to TestCase” dialog:



Since we’ve gone through some of these steps before I’m going to combine several steps into one picture.  The test case request window should appear after clicking Ok above.  Run the request and get the SOAP fault response, and then add an assertion to this test case for a SOAP fault response:



Now we have two test cases: “Halloween1” and “TestCase 2”.  Oops.  Let’s rename “TestCase 2” to “Halloween2” by right-clicking on “TestCase 2” and selecting Rename:



That looks better:



That should be enough of a meaningful project to build a Maven project around, and then we can run the Maven project via Hudson.

Let’s create a directory structure that represents the skeleton of a Maven project into which we can save our soapUI project.  Create the following directory structure:



and save your soapUI project into USHolidays/src/test/soapui.



Next, we’ll build a Maven POM that executes our tests.
Creating a Maven POM to run soapUI Tests

Below is a Maven POM which will run the soapUI plugin to execute the unit tests within our project.  For information about soapUI’s Maven plugin can be found here.
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
   <modelVersion>4.0.0</modelVersion>

   <name>USHoliday Web Service Test Module</name>
   <groupId>tdrury</groupId>
   <artifactId>usholidays.webservices.test</artifactId>
   <version>1.0-SNAPSHOT</version>
   <packaging>jar</packaging>
   <description>USHoliday web services soapUI integration tests</description>    
   <build>
      <plugins>           
         <plugin>
            <groupId>eviware</groupId>
            <artifactId>maven-soapui-plugin</artifactId>
            <version>3.6.1</version>
            <executions>
               <execution>
                  <id>USHolidayTests</id>
                  <goals>
                     <goal>test</goal>
                  </goals>
                  <phase>test</phase>          
               </execution>
            </executions>                      
            <configuration>
               <projectFile>src/test/soapui/USHolidayDates-soapui-project.xml</projectFile>
               <host>www.27seconds.com</host>
               <outputFolder>${project.build.directory}/surefire-reports</outputFolder>
               <junitReport>true</junitReport>
               <printReport>false</printReport>
               <projectProperties>
                  <projectProperty>YEAR=2011</projectProperty>
               </projectProperties>
            </configuration>
         </plugin>  
      </plugins>
   </build>         
</project>

Pay particular attention to the <projectProperties> element.  This is where we can set the soapUI project property, YEAR.  Above, it’s hardcoded to 2011, but this could easily be replaced by a Maven property which we’ll demonstrate later.

You should now be able to execute the project via:
mvn test

And, hopefully, you’ll see output similar to this:
[INFO] [soapui:test {execution: USHolidayTests}]
soapUI 3.6 Maven2 TestCase Runner
15:40:38,195 WARN  [DefaultSoapUICore] Missing folder [c:\work\java\USHolidays\.\ext] for external libraries
15:40:38,486 INFO  [DefaultSoapUICore] initialized soapui-settings from [C:\Users\i821885\soapui-settings.xml]
15:40:39,197 INFO  [WsdlProject] Loaded project from [file:c:/work/java/USHolidays/src/test/soapui/USHolidayDates-soapui-project.xml]
15:40:39,649 INFO  [SoapUITestCaseRunner] Running soapUI tests in project [USHolidayDates]
15:40:39,652 INFO  [SoapUITestCaseRunner] Running Project [USHolidayDates], runType = SEQUENTIAL
15:40:39,707 INFO  [SoapUITestCaseRunner] Running soapUI testcase [Halloween1]
15:40:39,728 INFO  [SoapUITestCaseRunner] running step [GetHalloweenDay - Request 1]
15:40:41,592 INFO  [SoapUITestCaseRunner] Assertion [SOAP Response] has status VALID
15:40:41,593 INFO  [SoapUITestCaseRunner] Assertion [Not SOAP Fault] has status VALID
15:40:41,594 INFO  [SoapUITestCaseRunner] Assertion [XPath Match] has status VALID
15:40:41,598 INFO  [SoapUITestCaseRunner] Finished running soapUI testcase [Halloween1], time taken: 1851ms, status: FINISHED
15:40:41,599 INFO  [SoapUITestCaseRunner] Running soapUI testcase [Halloween2]
15:40:41,600 INFO  [SoapUITestCaseRunner] running step [GetHalloweenDay - Failure 1]
15:40:41,754 INFO  [SoapUITestCaseRunner] Assertion [SOAP Response] has status VALID
15:40:41,754 INFO  [SoapUITestCaseRunner] Assertion [SOAP Fault] has status VALID
15:40:41,756 INFO  [SoapUITestCaseRunner] Finished running soapUI testcase [Halloween2], time taken: 153ms, status: FINISHED
15:40:41,757 INFO  [SoapUITestCaseRunner] Project [USHolidayDates] finished with status [FINISHED] in 2100ms

If this is the first time you’ve run Maven, you’ll have to wait until Maven – seemingly – downloads the entire Internet.
Some Possible Maven Issues

I can’t possibly go through all possible problems you may have running this project in Maven.  Mathematicians speculate there is no upper bound to possible Maven issues (Really! But I misplaced that link…).  I have to assume you have a working knowledge of Maven and have gotten it to run before or else this blog would be mind-numbingly dull.  However, here are a few possible problems and solutions.

1. I have to go through a proxy to access servers outside my company’s network.  When running soapUI via Maven, inform Maven of your company’s proxy (see http://maven.apache.org/guides/mini/guide-proxies.html😞
   <proxies>
      <proxy>
         <active>true</active>
         <protocol>http</protocol>
         <host>proxy</host>
         <port>8080</port>
         <nonProxyHosts>localhost</nonProxyHosts>
      </proxy>
   </proxies>

2. You need to add the soapUI Maven repository to your project (see http://www.soapui.org/Test-Automation/maven-2x.html😞
   <pluginRepository>
      <id>eviwarePluginRepository</id>
      <url>http://www.eviware.com/repository/maven2/</url>
   </pluginRepository>

3. Cannot find host www.27seconds.com.  This error can occur in two predominant cases: you don’t have your proxy correctly configured (see Issue #1) and/or you’re not using soapUI maven-soapui-plugin version 3.6.1.  Earlier versions of the plugin failed to use the Maven proxy when resolving host names.
Running the soapUI Maven Project in Hudson

At a minimum, make sure Hudson is configured with a JDK and Maven.



Click on the “Manage Hudson” link.



Then click on the “Configure System” link.  Set up your JDK and Maven installations.  The Ant installation is not required for this demonstration.



Now let’s create a Hudson project for our soapUI unit test.  Click the “New Job” link on the Hudson home page:



Provide a name for the project and make sure you’ve selected the Maven 2 option:



Here are my Hudson project properties:



Save this configuration.  Hudson will return you to the project page.

Let me make a quick note on continuous integration.  To simplify this blog entry I did not use a source code management tool so I have to instruct Hudson to build my project periodically.  Normally Hudson would monitor your source code management system and build a project whenever the source code has been changed (thus the term “continuous integration”).  Also, for a project such as this one which has no source code, it wouldn’t be unusual to instruct Hudson to run periodically instead of polling your source code management system since the code used to build the web service you’re testing may have been changed within another Hudson project.  Hudson provides other means to run this soapUI test project; e.g. you could have checked the “Build after other projects are built” checkbox and a text field would be provided where you could enter the name of your web service project you’re testing.  Then this test would run any time the web service project is built.

Click on “Build Now” to force Hudson to run Maven on your project.



After the build has run, you’ll see a link in the Build History box:



Click this link (mine is shown above as “Nov 16, 2010 5:18:14 PM”):



One thing is missing from this page.  Hudson normally displays a graph showing historical test results from unit tests.  Here’s one from an SAP ME module:



Hudson knows to do this when your Maven project has a test goal and has JUnit-formatted test results in the build target folder.  For some reason soapUI’s Maven plugin doesn’t trigger this feature in Hudson.  I’m fairly certain this used to work, so I need to submit a bug request to soapUI so they can look into this.

If you click the “Console Output” link you can see Hudson’s and Maven’s output during the build.  Hopefully your tests will have run successfully:

[INFO] [soapui:test {execution: USHolidayTests}]
soapUI 3.6 Maven2 TestCase Runner
13:30:55,300 WARN  [DefaultSoapUICore] Missing folder [c:\work\java\USHolidays\.\ext] for external libraries
13:30:55,538 INFO  [DefaultSoapUICore] initialized soapui-settings from [C:\Users\i821885\soapui-settings.xml]
13:30:56,251 INFO  [WsdlProject] Loaded project from [file:c:/work/java/USHolidays/src/test/soapui/USHolidayDates-soapui-project.xml]
13:30:56,547 INFO  [SoapUITestCaseRunner] Setting project property [YEAR] to [2010]
13:30:56,549 INFO  [SoapUITestCaseRunner] Running soapUI tests in project [USHolidayDates]
13:30:56,551 INFO  [SoapUITestCaseRunner] Running Project [USHolidayDates], runType = SEQUENTIAL
13:30:56,588 INFO  [SoapUITestCaseRunner] Running soapUI testcase [Halloween1]
13:30:56,596 INFO  [SoapUITestCaseRunner] running step [GetHalloweenDay - Request 1]
13:30:58,690 INFO  [SoapUITestCaseRunner] Assertion [SOAP Response] has status VALID
13:30:58,692 INFO  [SoapUITestCaseRunner] Assertion [Not SOAP Fault] has status VALID
13:30:58,692 INFO  [SoapUITestCaseRunner] Assertion [XPath Match] has status VALID
13:30:58,696 INFO  [SoapUITestCaseRunner] Finished running soapUI testcase [Halloween1], time taken: 2086ms, status: FINISHED
13:30:58,697 INFO  [SoapUITestCaseRunner] Running soapUI testcase [Halloween2]
13:30:58,697 INFO  [SoapUITestCaseRunner] running step [GetHalloweenDay - Failure 1]
13:31:02,565 INFO  [SoapUITestCaseRunner] Assertion [SOAP Response] has status VALID
13:31:02,566 INFO  [SoapUITestCaseRunner] Assertion [SOAP Fault] has status VALID
13:31:02,566 INFO  [SoapUITestCaseRunner] Finished running soapUI testcase [Halloween2], time taken: 3868ms, status: FINISHED
13:31:02,579 INFO  [SoapUITestCaseRunner] Project [USHolidayDates] finished with status [FINISHED] in 6025ms
[HUDSON] Archiving c:\work\java\USHolidays\pom.xml to c:\java\hudson\jobs\USHolidays\modules\tdrury$usholidays.webservices.test\builds\2010-12-03_13-30-45\archive\tdrury\usholidays.webservices.test\1.0-SNAPSHOT\pom.xml
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13 seconds
[INFO] Finished at: Fri Dec 03 13:31:02 EST 2010
[INFO] Final Memory: 17M/30M
[INFO] ------------------------------------------------------------------------
channel stopped
Finished: SUCCESS

Earlier we alluded to setting the soapUI project property, YEAR, to an adjustable value.  Let’s do that now.  Change your Maven POM so the <projectProperties> is set as follows:
               <projectProperties>
                  <projectProperty>YEAR=${test.year}</projectProperty>
               </projectProperties>

Developers can set the Maven property, ${test.year} in their settings.xml but for Hudson we set it in the Hudson project configuration.  In the Build section of the USHolidays Hudson project, change “Goals and options” by adding a Java property with the –D option:



If you force Hudson run the build via the “Build Now” link with this modified POM and Hudson settings, you’ll see the changes reflected in the console output in Hudson (note the line that says, “Setting project property [YEAR] to [2099]”):
[HUDSON] Recording test results
[INFO] [soapui:test {execution: USHolidayTests}]
soapUI 3.6 Maven2 TestCase Runner
13:11:16,457 WARN  [DefaultSoapUICore] Missing folder [c:\work\java\USHolidays\.\ext] for external libraries
13:11:16,765 INFO  [DefaultSoapUICore] initialized soapui-settings from [C:\Users\i821885\soapui-settings.xml]
13:11:17,463 INFO  [WsdlProject] Loaded project from [file:c:/work/java/USHolidays/src/test/soapui/USHolidayDates-soapui-project.xml]
13:11:18,033 INFO  [SoapUITestCaseRunner] Setting project property [YEAR] to [2099]
13:11:18,036 INFO  [SoapUITestCaseRunner] Running soapUI tests in project [USHolidayDates]
13:11:18,038 INFO  [SoapUITestCaseRunner] Running Project [USHolidayDates], runType = SEQUENTIAL
13:11:18,052 INFO  [SoapUITestCaseRunner] Running soapUI testcase [Halloween1]
13:11:18,093 INFO  [SoapUITestCaseRunner] running step [GetHalloweenDay - Request 1]
13:11:19,905 INFO  [SoapUITestCaseRunner] Assertion [SOAP Response] has status VALID
13:11:19,905 INFO  [SoapUITestCaseRunner] Assertion [Not SOAP Fault] has status VALID
13:11:19,905 INFO  [SoapUITestCaseRunner] Assertion [XPath Match] has status VALID
13:11:19,906 INFO  [SoapUITestCaseRunner] Finished running soapUI testcase [Halloween1], time taken: 1797ms, status: FINISHED



Summary

soapUI provides a nice development environment for authoring integration unit tests of your project’s web services.  If you then couple soapUI with Maven and Hudson, you can create a powerful, continuous testing environment for your application.
Resources


2 Comments