Mar 28

Testing WebServices Using JMeter

By Jim Yekrang IT World, Solutions, SQA No Comments »

Web Services are tested for throughput, efficiency, and response simulating real-world conditions. A well designed load-testing strategy can simulate real-world load and performance scenarios with minimal hassle and cost. User loads and network conditions of varying nature can be effortlessly created and replicated. Testing can be undertaken till the output charts show a performance range considered acceptable for an application of its nature. Load-testing results can hence be taken as a strong indicator of application performance in actual business environments.

Adding Thread Group: Thread group element controls the number of threads. Thread group allows the following controls:

1. Set the number of threads

2. Set the ramp-up period

3. Set the number of times to execute the test plan (Loop)

First control tells JMeter to execute the number of threads to execute the test.

Second control tells JMeter how long to “ramp-up” to the full number of thread chosen.

If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun.

Third control tells Jmeter how many times to execute the test plan.

Adding WebServices Requests: Samplers are used to tell Jmeter to send requests to a server. For load/stress testing of WebServices select WebService (Soap) Request sampler.

Step 1: On Thead Group use Add–> Sampler--> WebService (Soap) Request.

Step 2: Enter the WSDL URL and click on “Load WSDL”.

If the WSDL was loaded successfully, the “Web Methods” drop down should be populated. Otherwise it will prompt some error or exception.

Step 3: Select the web method and click “Configure”. The sampler populates the “URL”, Path and “SOAPAction” text fields. Change the proxy setting of the browser i.e. Http Proxy: localhost; Port: 8080.

Step 4: Paste the SOAP message in the “SOAP/XML-RPC Data” text area. We can optionally save the soap message to a file and browse to the location. We can also use message folder.

The sampler will randomly select files from a given folder and use the text for the soap message.

Jmeter1 (2)

Adding Listener to Store or view Test results: Listeners are use for storing all of the results of your HTTP requests in a file and presenting a visual model of the data. Select Add–>Listener–>View Results Tree (Select the type of graph you want to use as listener).

From run menu run the test plan. The result will be stored in the listener.

Load Testing Metrics and Parameters
The results obtained by load testing Web Services can potentially be reflected in terms of the following parameters.

Response time: It’s the most important parameter to reflect the quality of a Web Service. Response time is the total time it takes after the client sends a request till it gets a response.

Number of transactions passed/failed: This parameter simply shows the total number of transactions passed or failed.

Throughput: It’s measured in bytes and represents the amount of data that the virtual users receive from the server at any given second. We can compare this graph to the response-time graph to see how the throughput affects transaction performance.

Load size: The number of concurrent virtual users trying to access the Web Service at any particular instance in an interval of time.

CPU utilization: The amount of CPU time used by the Web Service while processing the request.

Memory utilization: The amount of memory used by the Web Service while processing the request.

Wait Time (Average Latency): The time it takes from when a request is sent until the first byte is received.

Sample Graph Result: Below is the sample graph result for a load test of webservices.

img_500_10_2_15

References: Apache JMeter by  Emily H. Halili

http://jakarta.apache.org/jmeter/usermanual/build-ws-test-plan.html

Tagged with: • • • • • • • • • • •
Dec 24

Software Testing – Manual or Automated?

By Jim Yekrang SQA 3 Comments »

Software Testing is an activity being part of the software development process aimed at evaluating a software item (system, subsystem, unit etc.) features (functionality, performance etc.) against the given set of system requirements.

Types of Testing: Manual and Automation Testing

Manual testing is the process of manually testing software for defects. It requires a tester to play the role of an end user, and use most of all features of the application to ensure correct behavior. To ensure completeness of testing, the tester often follows a written test plan that leads them through a set of important test cases.

Automation testing is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process

Automation testing can be performed on following types of testing:

  • Functional – Functional testing tests a system to verify that it meets its requirements.
  • Regression - testing that the behavior of the system has not changed after a major code change has occurred
  • Exception or Negative - forcing error conditions in the system.
  • Stress - determining the absolute capacities of the application and operational infrastructure. In short to test the reliability of the software
  • Performance - Performance testing is executed to determine how fast a system or sub-system performs under a particular workload.
  • Load - determining the points at which the capacity and performance of the system become degraded to the situation that hardware or software upgrades would be required. Load testing is a way to test performance.

Areas where Automation Testing should be prioritized:

1. Highly redundant tasks or scenarios 
2. Repetitive tasks that are boring or tend to cause human error 
3. Well-developed and well-understood use cases or scenarios first 
4. Relatively stable areas of the application over volatile ones must be automated.

 Advantages of Manual Testing

 1. Manual testing can be use in both small and big projects

 2. Number of test cases can be easily added/reduced according to change in requirement.

 3. Manual Testing is cheap as compared to Automated testing

 4. Easy to learn for new people who are entered in Software testing.

 5. Manual is more reliable then automated (in many cases automated not cover all cases)

 6. More bugs are found via Exploratory Testing. As, the more time a tester spends playing with the feature of the software the more bugs are found, which is not possible in case of Automation.

 Disadvantages of Manual Testing:

 1. User Interface testing is difficult to perform, especially when to test the color combinations.

 2. Actual load and performance is not possible to cover in manual testing for large number of users.

 3. Executing test cases manually is very time consuming.

 Advantages of Automation Testing:

 1. Test cases can be executed in limited time period.

 2. Load and Performance testing can be performed with no hurdles.

 3. It is best used in case of Regression testing (Repeatedly testing), where few test cases need to be executed repeatedly.

 4. Automated testing can be performed on different operating systems simultaneously.

 5. Automated tools are easy for volume testing. Volume testing is a way to test functionality.

 Disadvantages of Automation Testing:

 1. Every test case is to be converted to test script, so it is a time consuming task.

 2. Automation testing is more expensive as comparing with manual testing. Automated tools are very expensive that cannot be used by small organizations.

 3. Test Scripts are written in different languages and also all tools do not support all language of software. And moreover, the test script writer should be familiar with the all the languages (in which test scripts are to be written)

 4. The method/way of displaying the result is not user friendly. Information that is conveyed is hard to figure out.

 5. Automated tool may not identify missing heading tags since it cannot read the content.

 6. If there is a minor change in the requirement, or minor change in requirement, then whole script need to be changed and need to re-do all the things, whereas in case of Manual testing, only the required test case is modified.

 7. Not all tests can be automated and most times is difficult to decide what to automate and what to manually test.

 Some of the above disadvantages often cause damage to the benefit gained from the automated scripts. Though the automation testing has pros and corns, it is adapted widely all over the world.

preload preload preload