LR Controller online Graphs Directory to modify
1. By default, the Controller online monitor shows a maximum of 20 measurements for each graph. To increase it, go to the LoadRunner\dat\online_graphs directory to modify the value ofMaxDispMeasurments= in the file controlling each type of graph: 2. Default counters for the System Resource, Microsoft IIS, Microsoft ASP, or SQL Server monitors are defined in the res_mon.dft file within the LoadRunner/dat folder. Its values can be pasted from the [MonItemPlus] section within scenario .lrs files.
DescriptionFile Name
Allgeneralsettings.ini
System Resource Graphsonline_resource_graphs.rmd
Runtime Graphsonline_runtime_graphs.def
Transaction Graphsonline_transaction_graphs.def
Web Resource Graphsonline_web_graphs.def
Streaming Mediaonline_web_graphs_mms.def
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: Analysis
What is Transaction Response Time?How it is calculated by the tool Load runner
Transaction Response Time represents the time taken for the application to complete a defined transaction or business process. Why is important to measure Transaction Response Time? The objective of a performance test is to ensure that the application is working perfectly under load. However, the definition of “perfectly” under load may vary with different systems. By defining an initial acceptable response time, we can benchmark the application if it is performing as anticipated. The importance of Transaction Response Time is that it gives the project team/ application team an idea of how the application is performing in the measurement of time. With this information, they can relate to the users/customers on the expected time when processing request or understanding how their application performed. What does Transaction Response Time encompass?
The Transaction Response Time encompasses the time taken for the request made to the web server, there after being process by the Web Server and sent to the Application Server. Which in most instances will make a request to the Database Server. All this will then be repeated again backward from the Database Server, Application Server, Web Server and back to the user. Take note that the time taken for the request or data in the network transmission is also factored in. To simplify, the Transaction Response Time comprises of the following:
Processing time on Web Server
Processing time on Application Server
Processing time on Database Server
Network latency between the servers, and the client
The following diagram illustrates Transaction Response Time. Figure 1
Transaction Response Time = (t1 + t2 + t3 + t4 + t5 + t6 + t7 + t8 + t9) X 2 Note:Factoring the time taken for the data to return to the client. How do we measureransation Response Time?Measuring of the Transaction Response Time begins when the defined transaction makes a request to the application. From here, till the transaction completes before proceeding with the next subsequent request (in terms of transaction), the time is been measured and will stop when the transaction completes. Differences with Hits Per Seconds and Transaction Response timesHits per Seconds measures the number of “hits” made to a web server. These “hits” could be a request made to the web server for data or graphics. However, this counter does not represent well to users on how well their applications is performing as it measures the number of times the web server is being accessed. How can we use Transaction Response Time to analyze performance issue? Transaction Response Time allows us to identify abnormalities when performance issues surface. This will be represented as slow response of the transaction, which differs significantly (or slightly) from the average of the Transaction Response Time. With this, we can further drill down by correlation using other measurements such as the number of virtual users that is accessing the application at the point of time and the system-related metrics (e.g. CPU Utilization) to identify the root cause. Bringing all the data that have been collected during the load test, we can correlate the measurements to find trends and bottlenecks between the response time, the amount of load that was generated and the payload of all the components of the application. How is it beneficial to the Project Team? Using Transaction Response Time, Project Team can better relate to their users using transactions as a form of language protocol that their users can comprehend. Users will be able to know that transactions (or business processes) are performing at an acceptable level in terms of time. Users may be unable to understand the meaning of CPU utilization or Memory usage and thus using a common language of time is ideal to convey performance-related issues.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: Analysis
How transaction response time effects the performance of application?
How can we use Transaction Response Time to analyze performance issue? Transaction Response Time allows us to identify abnormalities when performance issues surface. This will be represented as slow response of the transaction, which differs significantly (or slightly) from the average of the Transaction Response Time. With this, we can further drill down by correlation using other measurements such as the number of virtual users that is accessing the application at the point of time and the system-related metrics (e.g. CPU Utilization) to identify the root cause. Bringing all the data that have been collected during the load test, we can correlate the measurements to find trends and bottlenecks between the response time, the amount of load that was generated and the payload of all the components of the application. How is it beneficial to the Project Team? Using Transaction Response Time, Project Team can better relate to their users using transactions as a form of language protocol that their users can comprehend. Users will be able to know that transactions (or business processes) are performing at an acceptable level in terms of time. Users may be unable to understand the meaning of CPU utilization or Memory usage and thus using a common language of time is ideal to convey performance-related issues.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: Analysis
How transaction response time is different from Hits Per Seconds in load runnner
Hits per Seconds is the number of hits made to the web server. These hits can be requests made to the web server for data or graphics. Hits per second measures the number of times the web server is being accessed. So, it does not represent well to users on how well their applications is performing. Transaction Response Time helps us to find performance bottlenecks in the application. We can further drill down by correlation using other measurements such as the number of virtual users that is accessing the application at the time of measurement. Other system related metrics like CPU Utilization etc. can be used to identify the root cause. In LoadRuner, after running a performance test, various measurements can be correlated to find trends and bottlenecks. Correlation can be done between the response time, the amount of load that was generated and the payload of all the components of the application.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: Analysis
How transaction response time is different from Hits Per Seconds in load runnner
Hits per Seconds is the number of hits made to the web server. These hits can be requests made to the web server for data or graphics. Hits per second measures the number of times the web server is being accessed. So, it does not represent well to users on how well their applications is performing. Transaction Response Time helps us to find performance bottlenecks in the application. We can further drill down by correlation using other measurements such as the number of virtual users that is accessing the application at the time of measurement. Other system related metrics like CPU Utilization etc. can be used to identify the root cause. In LoadRuner, after running a performance test, various measurements can be correlated to find trends and bottlenecks. Correlation can be done between the response time, the amount of load that was generated and the payload of all the components of the application.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: Analysis
How transaction response time is different from Hits Per Seconds in load runnner
Hits per Seconds is the number of hits made to the web server. These hits can be requests made to the web server for data or graphics. Hits per second measures the number of times the web server is being accessed. So, it does not represent well to users on how well their applications is performing. Transaction Response Time helps us to find performance bottlenecks in the application. We can further drill down by correlation using other measurements such as the number of virtual users that is accessing the application at the time of measurement. Other system related metrics like CPU Utilization etc. can be used to identify the root cause. In LoadRuner, after running a performance test, various measurements can be correlated to find trends and bottlenecks. Correlation can be done between the response time, the amount of load that was generated and the payload of all the components of the application.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: Analysis
Transaction Response Time in Load runner
Transaction Response Time :Time taken for the application to complete a transaction or business process. Objective of measuring the Transaction Response Time – to ensure that the application is working perfectly under heavy load. However, the definition of heavy load may vary from system to system. In Loadrunner by defining an initial acceptable response time, we can benchmark the application if it is performing as anticipated. Measuring transaction response time is important as it gives an idea of how the aplication is performing in terms of time. This performance parameter can be related to the end users at the time of processing request. It helps in understanding how their application performed. Transaction Response Time is the sum of
Time taken for the request made to the web server
Time taken by the web server to process
Time taken by the application server to process
Time taken by the database server to process
Time taken for the procesed request to come back to the client / user through various routes
Time taken for the request or data in the network transmission is also considered in this – Network latency between the servers and the client
Measuring Transaction Response Time in LoadRuner:
It begins when the defined transaction makes a request to the application. From here, till the transaction completes before proceeding with the next subsequent request (in terms of transaction), the time is been measured and will stop when the transaction completes.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: Analysis
What is Little's law?
Little's Law: Little's law is quite simple and intuitively appealing. The law states that the average number of customers in a system (over some time interval), N, is equal to their average arrival rate, X, multiplied by their average time in the system, R. N = X . R (or) for easy remembrance use L = A . W This law is very important to check whether the load testing tool is not a bottleneck. For Example, in a shop , if there are always 2 customers available at the counter queue , wherein the customers are entering the shop at the rate of 4 per second , then time taken for the customers to leave from the shop can be calculated as N = X. R R = 2/4 = 0.5 seconds A Simple example of how to use this law to know how many virtual users licenses are required: Web system that has peak user load per hour = 2000 users Expected Response time per transaction = 4 seconds The peak page hits/sec = 80 hits/sec For carrying out Performance tests for the above web system, we need to calculate how many number of Virtual user licenses we need to purchase. N = X . R N = 80 . 4 = 320 Therefore 320 virtual user licenses are enough to carry out the Load Test.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Enable remote desktop connection from remote registry
Machine A : PC which you are using to active remote desktop connection in another PC Machine B : PC for which remote desktop connection has to be activated. Step 1: From Machine A : Goto Start -> Run -> regedit. Open the Registry window of Machine A. (Note: User ID logged in Machine A should have ADMIN rights in Machine B) Step 2: Click “Connect Network Registry“ Step 3:Enter the computer name of Machine B, for which we have to change the settings and Click “OK” Step4:The destination machine (Machine B) will be connected with our current logged in machine (Machine A).
Step 5:Goto the below mentioned path and change the fDenyTSConnections to “0” and click “OK”. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services Step 6:After doing the following changes, right-click the remote machine (Machine B) name and click Disconnect.
NOTE: If suppose the above mentioned registry value is not found, then create a DWORD parameter with a value ’0′.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: General
How to check vuser status per script in Analysis in Load runner
To check vuser status (i.e. passed/failed/stopped) per script perform the following: 1. Analyse the results in Analysis 2. Open the Vuser Summary graph. 3. Right-click on the graph, select "Set filter," then select the script name. 4. Select the Vuser End Status that you are interested in (Passed/Failed/Error/Stopped).
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: Vugen
Prerequisites for load Testing in load runner
Important Set up before load testing includes: A suitable time to load-test the application, for instance when no development work is taking place on the server (load testing may cause the server to crash) and/or no other users are accessing the server (else the testing results would not yield the correct measures)
The performance metrics, accepted levels, or SLAs and goals
Objectives of the test
The Internet protocol(s) the application is(are) using (HTTPS, HTTP, FTP, etc.)
If your application has a state, the method used to manage it (URL rewriting, cookies, etc.)
The workload at normal time and at peak time
Before load or performance testing application should be stable at least tested at once through the manual method
Before load testing, do not record any page, which has 404 or server errors
During load test don’t do real transaction or any money ready work.
During load testing try to maintain the real scenario as user experience
Use meaningful test scenarios (use cases are helpful) to construct test plans with 'real-life' test cases.
Make sure that the machine running Load testing tool has sufficient network bandwidth, so the network connection has little to no impact on the results.
Let Load Test run for long time periods, hours or days, or for a large number of iterations. This may yield a smaller standard deviation, giving better average results. In addition, this practice may test system availability rate and may highlight any decay in server performance.
Ensure that the application is stable and optimized for one user before testing it for concurrent users.
Incorporate 'thinking time' or delays using Timers in your Load testing scenario Test Plan.
Keep a close watch on the four main things: processor, memory, disk, and network.
Only run Load testing tool against servers that you are assigned to test, else you may be accused of causing DoS attacks.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
"Object reference not set to an instance of an object" analysis error in load runner
This error comes when the following files in (Analysis Install dir)\bin\dat\ are corrupted: loader2.mdb, loader.mdf, loader.ldf Solution: You need to restore these files; take these files from another machine where you have installed the Analysis. Or Re-install the Analysis on your machine after a clean uninstall.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
What is Security Testing?
Security testing is related to the security of data and the functionality of the application. You should be aware of the following concepts while performing security testing:
1. Confidentiality - The application should only provide the data to the relevant party e.g. one customer's transactional data should not be visible to another customer; the irrelevant personal details of the customer should not be visible to the administrator and so on. 2. Integrity - The data stored and displayed by the application should be correct e.g. after a withdrawal, the customer's account should be debited by the correct amount. 3. Authentication - It should be possible to attribute the data transmitted in the application to either the application or the customer. In other words, no one other than the customer or the bank should be able to create or modify any data. 4. Authorization - The application or a user should only be able to perform the tasks which they are respectively authorized to perform e.g. a customer should not be able to withdraw more than the balance in their account without having an overdraft facility, the application should not be able to levy charges on a customer account without prior customer approval.
5. Availability - The data and functionality should be available to the users throughout the working period e.g. if the bank's operating times are from 8 a.m. to 8 p.m. on all working days, it should be possible for a customer to access their account and make the necessary transactions on their account. 6. Non-repudiation - At a later date, it should not be possible for a party to deny that a particular transaction or data change took place e.g. if a customer withdraws an amount from their account, this should trigger the relevant actions (posting to their transaction records, debiting their account and sending them a notification etc.). In your question, you mentioned that you wish to avoid any data breach by hackers. You should understand that hackers are not the only people from whom the application functionality and data need to be protected. There are other people that you need to consider as well: 1. Disgruntled customers 2. Unhappy or malicious employees of the bank 3. Unprofessional service providers e.g. an unprofessional hosting company that may have access to the application and the data 4. Unprofessional auditors Further, since financial data is so important, banking applications in certain countries have to be compliant to the relevant financial standards. Research the relevant standards that your application needs to follow.
Creating a secure application involves a lot of work in designing a secure application and designing a secure data store. Even after deployment, the application should be closely monitored to ensure that the data is being accessed by only the authorized people. If any security breach is reported, it should be analyzed carefully and the loopholes plugged.
Now, let us discuss the actual security testing. You should design security tests based on at least the following: 1. Stated security requirements 2. Security-related standards that the application should follow Assuming that it is a web application, 3. Common vulnerabilities found in web applications 4. Different browser versions on different operating systems (here you should note that implementing security only on the client-side may not suffice)
In your initial tests, you may want to use automated testing tools e.g. web vulnerability scanners, password crackers, web proxy tools etc. Based on your learning, you may want to execute the more complex security tests by hand. Keep yourself updated about the latest hacks and test them on your application before every release. As you might now appreciate, security testing is a vast area of knowledge and practice. In order to do justice to security testing, it is better to have a dedicated team for security testing.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Creating Dynamic itemdata in Web_submit_data in load runner
I was working on a project where there was a scenario of searching for a course by its first name or wild cards. The search returns different number of results for different queries. This request is followed by another web_submit_data where the ITEMDATA depends on number of search results. There are three dynamic variables here – length of the course variable, results returned and the third one being the ITEMDATA of the request itself. The solution is to build dynamic data on the fly and create a web_custom_request for the web_submit_data The following function will build the item data - const char* BuildItemData(int pageSize,const char*ItemDataStart,const char*ItemDataEnd, int lengthOfCourseVar, const char* ArrayName){ char* ItemData;
int iCount,lengthOfOneItem; lengthOfOneItem = lengthOfCourseVar+strlen(ItemDataStart)+strlen(ItemDataEnd); ItemData = (char*)malloc((pageSize*(lengthOfOneItem))*sizeof(char)); ItemData[0]=”; for(iCount=1;iCount<=(pageSize);iCount++) { strcat(ItemData,ItemDataStart); strcat(ItemData,lr_paramarr_idx(ArrayName,iCount)); strcat(ItemData,ItemDataEnd); } return ItemData; } I changed the web_submit_data to web_custom_request and used the string output as a parameter like this: lr_save_string( BuildItemData(lr_paramarr_len(“cor_Arr_ClassIDs”),”offId[]=class”,”&”,strlen(lr_paramarr_idx(“cor_Arr_ClassIDs”,1)),”cor_Arr_ClassIDs”), “ItemDataOnTheFly” ); web_custom_request(“xxxxxxx.aspx”, “URL=https://{par_Environment_URL}/xxxxxx/xxxxxx.aspx”, “Method=POST”, “TargetFrame=”, “Resource=0″, “RecContentType=text/html”, “Referer=https://{par_Environment_URL}/xxx/xxx/xxx/xxxx/xx/xxxxxx?xxxxxx={par_Location}”, “Snapshot=t12.inf”, “Mode=HTML”, “EncType=application/x-www-form-urlencoded; charset=utf-8″, “Body={ItemDataOnTheFly}&lrnid=xxxxx/{par_UserID}”, LAST);
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: Vugen
Run Time Settings in Load runner
In general load runner runtime settings plays a crucial role in Vugen scripting and scenario running.This is the heart of the Load runner.The run time settings are
General
Run Logic
Pacing
Log
Think Time
Additional attributes
Miscellaneous
Network
Speed simulation
Browser
Browser emulation
Internet Protocol
Proxy
preferences
download filters
Content check
Data Format Extension
Configuration
General
Run Logic
Whenever I am using a Vuser type that allows multiple actions in a single script, I will create a separate action for each business process and put appropriate percentage weightings on each action. It is very unusual to have to do anything more complicated than this. I don’t usually use the “sequential” option or create blocks unless I need to have fractional percentage weightings for a business process. Percentages must be integer values, so to run a business process 0.1% of the time you could create a block that runs 1% of the time, and put an action in the block that runs 10% of the time. It’s also rare to set a script in a scenario to run for a specified number of iterations (mostly done by time or set to run indefinitely). Generally “number of iterations” is only used when running the script in VuGen.
Pacing :
“As soon as the previous iteration ends” is used when running in VuGen or when loading/verifying data. Do not use this for load testing I have never seen the point of the “After the previous iteration ends” option. Why would you want to run an unknown number of transactions per hour against the system? Don’t use the “At fixed intervals”. If something causes your users to become “in step”, they will tend to stay that way and continue to all hit the server at the same time. “At random intervals” is definitely the way to go. Obviously for your users to create a certain number of orders per hour the iteration time must average to 3600/num iterations in an hour. Do not make the lower boundary value any bigger than the maximum time it takes to complete the business process, or you will end up creating less transactions per hour than you intend to. you can check how to calculate pacing according to the requirement from this blog also.
Log :
Enable logging:once you verify that your script is functional,disable logging to conserve resources. Logging creates additional overhead on your load generators, and can create huge log files. log absolutely everything when debugging in VuGen. When running the script as part of a scenario, I leave extended logging on but change the logging to “Send messages only when an error occurs”. This gives a little more information than turning logging off entirely, and won’t create any additional overhead while everything is running smoothly (and if the system is not running smoothly you are going to need to stop the test and investigate anyway). Standard log:sends a subset of functions and messages sent during script execution to a log.The subset depends on the Vuser type. Think Time: Just like the pacing setting, I think that it is a good idea to put some randomness in your think times. I use a random percentage of 50-150% of recorded think times. Use “Ignore think time” if you are debugging in VuGen or if you are loading/verifying data. Additional attributes :
This option is ignored by most people. It is used to create a parameter with a given value without having to edit the script (as runtime settings can be overridden in the Controller). In the screenshot I have created a parameter of ServerName with the address of the test envioronment. If you were testing in more than one test environment at a time, this would make save some time. Miscellaneous : Continue on error is generally only going to be used if you have written code to do something when you encounter an error. Usually the default behaviour of ending the current iteration and then starting the next one is sufficient). I don’t advise anyone to try to write a script that handles errors in the same way as a real user because it will create a lot of additional work for very little benefit, but doing something simple like writing some useful information to the logs and then calling lr_exit(LR_EXIT_ACTION_AND_CONTINUE , LR_FAIL) can be useful. “Fail open transactions on lr_error_message” should always be ticked. If you are raising an error, you should fail the transaction step that you are performing. “Generate snapshot on error” is useful. If it is a web script, any error messages should be added to your content check rules. Run your virtual user as a thread unless you have code that is not threadsafe or there is some other reason to run your virtual users as a process. The overall memory footprint on your load generators will be higher if you run as a process. I never use the “Define each action as a transaction” option. If I want a transaction in my script I will add it myself with lr_start_transaction. I never use “Define each step as a transaction” either. If it is a web script, I can use the transaction breakdown graph to get this information, otherwise I will add the transactions myself. Network :
Not all vuser types have this option available. Most of the time my virtual users will use the maximum bandwidth. If I want to emulate users with bandwidth constraints, I will do this in a separate scenario. Google calculator is handy to calculate bitrates if your bitrate is not available from the drop-down list e.g./ “256 Kbps in bps” All of the following settings only apply to web-based scripts. Each vuser type will have its own runtime setting options. It is important to know what they mean and how they will influence your test results before running any tests that you plan to report on. Browser Browser Emulation: Some people get confused by the User-Agent (browser to be emulated) setting. If 90% of your users use Internet Explorer 6.0 and the rest use Firefox 1.5, you don’t have to change the runtime settings for your users to match this. All it changes is the string that is sent in the “User-Agent” field of your HTTP requests. This is completely pointless unless your application has been written to serve different content to different browsers based on the User-Agent field. Internet Protocol: Proxy : Generally people won’t be using your web applications through your proxy server, so it shouldn't be part of your test either. If you start getting errors that are due a proxy server rather than the system under test, it will just confuse the people who have to fix the problem. A proxy server will also make IP-based load balancing ineffective. If it’s an intranet application and everyone will be using the application through the company’s proxy, then the proxy server should be explicity declared to be in scope for your load test. You should make sure that you have an identical proxy server for your test environment, or that you have permission to be generating load on a piece of Production infrastructure. Preferences : These settings are default values specified by HP, rather than being inherited from the web browser that is installed on your workstation. Generally you will not need to change them, but be aware that they are here. Download Filters: Download filters are a quick way of preventing your scripts from downloading content from certain URLs or hosts/domains. I generally use this feature when the web application in the test environment contains third-party images used for tracking website usage (e.g. images from Webtrends or Red Sheriff etc). I think it is better to specify which hosts your script is allowed connect to, rather than which hosts your script can’t connect to (because it’s easy to miss one accidentally, or the application may change and refer to a new third-party domain). Use web_add_auto_filter if you want to specify this in your script rather than your runtime settings. Data Format Extension: Configuration : A LoadRunner feature that has made my life a lot easier has been ContentCheck rules, which are available in the script runtime settings. If you are using a web-based vuser type, you can configure your Load Runner script to search through all returned pages for strings that match known error messages. Using web_reg_find functions is fine, but when you get an error LoadRunner reports it as “failed to find text” instead of something more descriptive. I will always create rules for any error messages I find during scripting and, if I receive an error while running a scenario, I will add the error message from the snapshot in the scenario results directory (the snapshot on error feature is very useful). All this is pretty obvious if you have taken the time to explore LoadRunner’s features or you have attended a Mercury training session, but I recommend taking things a step further. Ask your developers for a list of all the error messages that the application can throw. This should be easy for them to provide if the application is well designed and stores all the message in some kind of message repository instead of sprinkling them throughout the source code. Include error message for functional errors that you are likely to encounter. Creating a rule for “incorrect username or password” may save someone 20 minutes of investigation when they first run the script after the database has been refreshed. If you prefer to have error message you are checking for in the script (where you can add comments to them) instead of the runtime settings, you can use the web_global_verification function instead. The only difference between the two is the error message that LoadRunner will include in its log: Action.c(737): Error -26368: “Text=A runtime error occurred” found for web_global_verification (“ARuntimeErrorOccurred”) (count=1), Snapshot Info [MSH 0 21] …compared to: Action.c(737): Error -26372: ContentCheck Rule “ARuntimeErrorOccurred” in Application “Webshop” triggered. Text “A runtime error occurred” matched (count=1), Snapshot Info [MSH 0 21]
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: LoadRunner, PerformanceCenter, Scripting, Tips n Tricks, Vugen
JVM garbage collection in young generation
It comes from Java SE 6 HotSpot[tm] Virtual Machine Garbage Collection Tuning , one stop place to learn everything about GC internals. But to address your immediate questions:
Allocating new objects using new operator (almost) always happens in Eden space. But Eden is actually a stack. When you create new object needing N bytes, single pointer advances by N bytes on that stack and that's it. Allocating is that fast, no searching for free spot, compacting, whatever. Of course this stack is not infinite, at some point we'll reach its end, triggering minor GC. Also most likely multiple objects are already garbage. So what JVM does in minor GC is the following: traverse graph of objects starting from GC roots copy all objects reachable from GC roots to one of survivor spaces (no gaps, we know all of them and this is a single process) wipe out eden space (basically just moving this stack pointer back to 0) In subsequent minor collections there are additional steps: one of survivor spaces is examined as well. Live objects from both eden and one of survivor spaces are copied to second survivor space. This means there is always exactly one free survivor space. So how are objects ending in tenured generation? First young objects are copied to one of survivor spaces. Then they are copied to the other and again and again. Once given object jumps back and forth too many times (configurable, 8 by default), it is promoted to tenured space. Major GC runs when tenured space is full.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Java (Java Virtual Memory) Memory Types
Java (JVM) Memory Types: JVM memory area related jargons are key to understand the JVM on the whole. In this article let us discuss about the important memory areas in JVM. Heap Memory Class instances and arrays are stored in heap memory. Heap memory is also called as shared memory. As this is the place where multiple threads will share the same data. Non-heap Memory It comprises of ‘Method Area’ and other memory required for internal processing. So here the major player is ‘Method Area’. Method Area As given in the last line, method area is part of non-heap memory. It stores per-class structures, code for methods and constructors. Per-class structure means runtime constants and static fields.
Memory Pool: Memory pools are created by JVM memory managers during runtime. Memory pool may belong to either heap or non-heap memory. Runtime Constant Pool: A run time constant pool is a per-class or per-interface run time representation of the constant_pool table in a class file. Each runtime constant pool is allocated from the Java
virtual machine’s method area.
Java Stacks or Frames: Java stacks are created private to a thread. Every thread will have a program counter (PC) and a java stack. PC will use the java stack to store the intermediate values, dynamic linking, return values for methods and dispatch exceptions. This is used in the place of registers. Memory Generations HotSpot VM’s garbage collector uses generational garbage collection. It separates the JVM’s memory into and they are called young generation and old generation.
Young Generation Young generation memory consists of two parts, Eden space and survivor space. Shortlived objects will be available in Eden space. Every object starts its life from Eden space. When GC happens, if an object is still alive and it will be moved to survivor space and other dereferenced objects will be removed.
Old Generation – Tenured and PermGen Old generation memory has two parts, tenured generation and permanent generation (PermGen). PermGen is a popular term. We used to error like PermGen space not sufficient. GC moves live objects from survivor space to tenured generation. The permanent generation contains meta data of the virtual machine, class and method objects.
Discussion: Java specification doesn’t give hard and fast rules about the design of JVM heap data area. So it is left to the JVM implementers and they can decide on things like whether to allocate fixed memory size or dynamic.
Key Takeaways
Local Variables are stored in Frames during runtime.
Static Variables are stored in Method Area.
Arrays are stored in heap memory.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
JVM garbage collection in young generation?
It comes from Java SE 6 HotSpot[tm] Virtual Machine Garbage Collection Tuning , one stop place to learn everything about GC internals. But to address your immediate questions: Allocating new objects using new operator (almost) always happens in Eden space. But Eden is actually a stack. When you create new object needing N bytes, single pointer advances by N bytes on that stack and that's it. Allocating is that fast, no searching for free spot, compacting, whatever. Of course this stack is not infinite, at some point we'll reach its end, triggering minor GC. Also most likely multiple objects are already garbage. So what JVM does in minor GC is the following:
traverse graph of objects starting from GC roots copy all objects reachable from GC roots to one of survivor spaces (no gaps, we know all of them and this is a single process) wipe out eden space (basically just moving this stack pointer back to 0) In subsequent minor collections there are additional steps: one of survivor spaces is examined as well. Live objects from both eden and one of survivor spaces are copied to second survivor space. This means there is always exactly one free survivor space. So how are objects ending in tenured generation? First young objects are copied to one of survivor spaces. Then they are copied to the other and again and again. Once given object jumps back and forth too many times (configurable, 8 by default), it is promoted to tenured space. Major GC runs when tenured space is full.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
What is running Vuser scripts in Stand-Alone Mode in load runner
Running Vuser Scripts in Stand-Alone Mode:
In order to perform load testing with a Vuser script, you need to incorporate the script into a LoadRunner scenario. Before integrating the script into a scenario, you should check its functionality by running the script in stand-alone mode.
Stand-alone mode:
Running a script in stand-alone mode means running the script without using a LoadRunner Controller. This is done to establish how the script will execute when run from the Controller. If the script is Windows-based, you use VuGen to run it in stand-alone mode. If the script is UNIX-based, you must run it from a UNIX command line.
If the stand-alone execution is successful, then incorporate the script into a scenario.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: Vugen
What are EdenSpace,Survivor Space,Young Generation,Java Heap Area,Virtual Space,PermGenSpace,Native Area in JVM?
Eden Space:
Eden Space is a Part of Java Heap where the JVM initially creates any objects, where most objects die and quickly are cleaned up by the minor Garbage Collectors (Note: Full Garbage Collection is different from Minor Garbage Collection). Usually any new objects created inside a Java Method go into Eden space and the objects space is reclaimed once the method execution completes. Where as the Instance Variables of a Class usually lives longer until the Object based on that class gets destroyed. When Eden fills up it causes a minor collection, in which some surviving objects are moved to an older generation.
Survivor Spaces:
Eden Sapce has two Survivor spaces. One survivor space is empty at any given time. These Survivor Spaces serves as the destination of the next copying collection of any living objects in eden and the other survivor space. The parameter SurvivorRatio can be used to tune the size of the survivor spaces. -XX:SurvivorRatio=6 sets the ratio between each survivor space and eden to be 1:6 If survivor spaces are too small copying collection overflows directly into the tenured generation.
Young Generation: (-XX:MaxNewSize)
Till JDK1.3 and 1.4 we used to set the Young Generation Size using -XX:MaxNewSize. But from JDK1.4 onwards we set the YoungGeneration size using (-Xmn) JVM option. Young Generation size is controlled by NewRatio. It means setting -XX:NewRatio=3 means that the ratio between the Old Generation and the Young Generation is 1:3 . Similarly -XX:NewRatio=8 means that 8:1 ratio of tenured and young generation. NewRatio: NewRatio is actually the ratio between the (YoungGenaration/Old Generations) has default values of 2 on Sparc , 12 on client Intel, and 8 everywhere else. NOTE: After JDK 1.4 The Young Generation Size can be set using (-Xmn) as well.
Virtual Space-1: (MaxNewSize – NewSize)
The First Virtual Space is actually shows the difference between the -XX:NewSize and -XX:MaxNewSize. Or we can say that it is basically a difference between the Initial Young Size and the Maximum Young Size.
Java Heap Area: (-Xmx and -Xms)
Java Heap is a Memory area inside the Java Process which holds the java objects. Java Heap is a combination of Young Generation Heap and Old Generation Heap. We can set the Initial Java Heap Size using -Xms JVM parameter similarly if we want to set the Maximum Heap Size then we can use -Xmx JVM parameter to define it. Example: -Xmx1024m —> Means Setting the Maximum limit of Heap as 1 GB -Xms512m —> Means setting Java Heap Initial Size as 512m NOTE: It is always recommended to set the Initial and the Maximum Heap size values as same for better performance.
NOTE: The Theoretical limitation of Maximum Heap size for a 32 bit JVM is upto 4GB. Because of the Memory Fragmentation, Kernel Space Addressing, Swap memory usages and the Virtual Machine Overheads are some factors JVM does not allow us to allocate whole 4GB memory for Heap in a 32 bit JVM. So usually on 32-bit Windows Operating Systems the Maximum can be from 1.4 GB to 1.6 GB. . If we want a Larger memory allocation according to our application requirement then we must choose the 64-bit operating systems with 64 bit JVM. 64-bit JVM provides us a larger address space. So we can have much larger Java Heap with the increased number of Threads allocation area. Based on the Nature of your Operating system in a 64 bit JVM you can even set the Maximum Heap size upto 32GB. Example: -Xms32g -Xmx32g -Xmn4g
Virtual Space-2: (MaxHeapSize – InitialHeapSize)
The Second Virtual Space is actually the Difference between the Maximum Heap size (-Xmx)and the Initial Heap Size(-Xms). This is called as virtual space because initially the JVM will allocate the Initial Heap Size and then according to the requirement the Heap size can grow till the MaxHeapSize. PermGen Space: (-XX:MaxPermSize)
PermGen is a non-heap memory area where the Class Loading happens and the JVM allocates spaces for classes, class meta data, java methods and the reference Objects here. The PermGen is independent from the Heap Area. It can be resized according to the requirement using -XX:MaxPermSize and -XX:PermSize JVM Options. The Garbage collection happens in this area of JVM Memory as well. The Garbage collection in this area is called as “Class GC”. We can disable the Class Garbage Collection using the JVM Option -noclassgc. if ”-noclassgc” Java Option is added while starting the Server. In that case the Classes instances which are not required will not be Garbage collected.
Native Area:
Native Memory is an area which is usually used by the JVM for it’s internal operations and to execute the JNI codes. The JVM Uses Native Memory for Code Optimization and for loading the classes and libraries along with the intermediate code generation. The Size of the Native Memory depends on the Architecture of the Operating System and the amount of memory which is already commited to the Java Heap. Native memory is an Process Area where the JNI codes gets loaded or JVM Libraries gets loaded or the native Performance packs and the Proxy Modules gets loaded.
There is no JVM Option available to size the Native Area. but we can calculate it approximately using the following formula:
NativeMemory = (ProcessSize – MaxHeapSize – MaxPermSize)
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
What are the parts Of JVM And JVM Architecture Diagram?
Architecture of JVM:
JVM is the heart of any Java based Application Server. We face most of the issues deu to incorrect JVM tuning. It is very important to understand the Overall architecture of the JVM in order to trouble shoot different JVM tuning related issues.Here we are going to discuss the Architecture and the Major parts of a Java Process and the Java Heap Division.
The Following Diagram is just a basic overview of a Java Process in a 2 GB process Size Machine. Usually in 32 bit Windows Operating Systems the default process size will be 2 GB (In Unix based 64 bit operating Systems it can be 4GB or more). So i draw the following Diagram of Java Process to explain the Java Process partitions in a 2Gb process size machine. Java Process Architecture Diagram:
In the above diagram we will find different partitions of a Java Process. Please compare the above diagram with below descriptions. 1)Just for Example we can see that Process Size is 2048 MB (2GB)
2) The Java Heap Size is 1024MB (means 1GB) -Xmx1024m
3) Native Space = ( ProcessSize – MaxHeapSize – MaxPermSize) It means around 768 MB of Native Space.
4) MaxPermSpace is around -XX:MaxPermSize=256m
5)Young Generation Space is around 40% of Maximum Java Heap.
What Are these Different Parts in JVM?
Eden Space:
Eden Space is a Part of Java Heap where the JVM initially creates any objects, where most objects die and quickly are cleaned up by the minor Garbage Collectors (Note: Full Garbage Collection is different from Minor Garbage Collection). Usually any new objects created inside a Java Method go into Eden space and the objects space is reclaimed once the method execution completes. Where as the Instance Variables of a Class usually lives longer until the Object based on that class gets destroyed. When Eden fills up it causes a minor collection, in which some surviving objects are moved to an older generation.
Survivor Spaces:
Eden Sapce has two Survivor spaces. One survivor space is empty at any given time. These Survivor Spaces serves as the destination of the next copying collection of any living objects in eden and the other survivor space. The parameter SurvivorRatio can be used to tune the size of the survivor spaces. -XX:SurvivorRatio=6 sets the ratio between each survivor space and eden to be 1:6 If survivor spaces are too small copying collection overflows directly into the tenured generation.
Young Generation: (-XX:MaxNewSize)
Till JDK1.3 and 1.4 we used to set the Young Generation Size using -XX:MaxNewSize. But from JDK1.4 onwards we set the YoungGeneration size using (-Xmn) JVM option. Young Generation size is controlled by NewRatio. It means setting -XX:NewRatio=3 means that the ratio between the Old Generation and the Young Generation is 1:3 . Similarly -XX:NewRatio=8 means that 8:1 ratio of tenured and young generation. NewRatio: NewRatio is actually the ratio between the (YoungGenaration/Old Generations) has default values of 2 on Sparc , 12 on client Intel, and 8 everywhere else. NOTE: After JDK 1.4 The Young Generation Size can be set using (-Xmn) as well.
Virtual Space-1: (MaxNewSize – NewSize)
The First Virtual Space is actually shows the difference between the -XX:NewSize and -XX:MaxNewSize. Or we can say that it is basically a difference between the Initial Young Size and the Maximum Young Size.
Java Heap Area: (-Xmx and -Xms)
Java Heap is a Memory area inside the Java Process which holds the java objects. Java Heap is a combination of Young Generation Heap and Old Generation Heap. We can set the Initial Java Heap Size using -Xms JVM parameter similarly if we want to set the Maximum Heap Size then we can use -Xmx JVM parameter to define it. Example: -Xmx1024m —> Means Setting the Maximum limit of Heap as 1 GB -Xms512m —> Means setting Java Heap Initial Size as 512m NOTE: It is always recommended to set the Initial and the Maximum Heap size values as same for better performance.
NOTE: The Theoretical limitation of Maximum Heap size for a 32 bit JVM is upto 4GB. Because of the Memory Fragmentation, Kernel Space Addressing, Swap memory usages and the Virtual Machine Overheads are some factors JVM does not allow us to allocate whole 4GB memory for Heap in a 32 bit JVM. So usually on 32-bit Windows Operating Systems the Maximum can be from 1.4 GB to 1.6 GB. . If we want a Larger memory allocation according to our application requirement then we must choose the 64-bit operating systems with 64 bit JVM. 64-bit JVM provides us a larger address space. So we can have much larger Java Heap with the increased number of Threads allocation area. Based on the Nature of your Operating system in a 64 bit JVM you can even set the Maximum Heap size upto 32GB. Example: -Xms32g -Xmx32g -Xmn4g
Virtual Space-2: (MaxHeapSize – InitialHeapSize)
The Second Virtual Space is actually the Difference between the Maximum Heap size (-Xmx)and the Initial Heap Size(-Xms). This is called as virtual space because initially the JVM will allocate the Initial Heap Size and then according to the requirement the Heap size can grow till the MaxHeapSize. PermGen Space: (-XX:MaxPermSize)
PermGen is a non-heap memory area where the Class Loading happens and the JVM allocates spaces for classes, class meta data, java methods and the reference Objects here. The PermGen is independent from the Heap Area. It can be resized according to the requirement using -XX:MaxPermSize and -XX:PermSize JVM Options. The Garbage collection happens in this area of JVM Memory as well. The Garbage collection in this area is called as “Class GC”. We can disable the Class Garbage Collection using the JVM Option -noclassgc. if ”-noclassgc” Java Option is added while starting the Server. In that case the Classes instances which are not required will not be Garbage collected.
Native Area:
Native Memory is an area which is usually used by the JVM for it’s internal operations and to execute the JNI codes. The JVM Uses Native Memory for Code Optimization and for loading the classes and libraries along with the intermediate code generation. The Size of the Native Memory depends on the Architecture of the Operating System and the amount of memory which is already commited to the Java Heap. Native memory is an Process Area where the JNI codes gets loaded or JVM Libraries gets loaded or the native Performance packs and the Proxy Modules gets loaded.
There is no JVM Option available to size the Native Area. but we can calculate it approximately using the following formula:
NativeMemory = (ProcessSize – MaxHeapSize – MaxPermSize)
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
How to Understand LoadRunner Report Web Resource Monitors
Here you get the information about the performance of your Web server using Load Runner’s Web Resource monitor.
Hits per Second Graph on page
Throughput Graph on page
HTTP Responses per Second Graph on page
Pages Downloaded per Second Graph on page
Retries per Second Graph on page
Connections Graph on page
Connections per Second Graph on page
SSLs per Second Graph on page
About Web Resource Monitoring
The Web Resource monitor enables you to analyze the throughput on the web server, the number of hits per second that occurred during the scenario, the number of HTTP responses per second, the HTTP status codes (which indicate the status of HTTP requests, for example, the request was successful, the page was not found) returned from the Web server, the number of downloaded pages per second, the number of server retries per second, the number of open TCP/IP connections, the number of new TCP/IP= connections per second, and the number of SSL Connections per second.
Hits per Second Graph
The Hits Per Second graph shows the number of hits (HTTP requests) to the Web server (y-axis) as a function of the elapsed time in the scenario (x-axis). This graph can display the whole step, or the last 60, 180, 600, or 3600seconds. You can compare this graph to the Transaction Response Time graph to see how the number of hits affects transaction performance.
Throughput Graph
The Throughput graph shows the amount of throughput on the Web server (y-axis) during each second of the scenario run (x-axis). Throughput is measured in bytes and represents the amount of data that the Vusers received from the server at any given second. You can compare this graph to the Transaction Response Time graph to see how the throughput affects transaction performance. In the following example, the Transaction Response time graph is compared with the Throughput graph. It is apparent from the graph that as the throughput decreases, the transaction response time also decreases. The peak throughput occurred at approximately 1 minute into the step. The highest response time also occurred at this time. Retries per Second Graph The Retries Per Second graph shows the number of attempted Web server connections (y-axis) as a function of the elapsed time in the scenario (x-axis). A server connection is retried when the initial connection was unauthorized, when proxy authentication is required, when the initial connection was closed by the server, when the initial connection to the server could not be made, or when the server was initially unable to resolve the load generator’s IP address.
Connections Graph
The Connections graph shows the number of open TCP/IP connections (y-axis) at each point in time of the scenario (x-axis). One HTML page may cause the browser to open several connections, when links on the page go to different Web addresses. Two connections are opened for each Web server. This graph is useful in indicating when additional connections are needed.
For example, if the number of connections reaches a plateau, and the transaction response time increases sharply, adding connections would probably cause a dramatic improvement in performance (reduction in the transaction response time) Connections per Second Graph The Connections Per Second graph shows the number of new TCP/IP connections (y-axis) opened and the number of connections that are shut down each second of the scenario (x-axis). This number should be a small fraction of the number of hits per second, because new TCP/IP connections are very expensive in terms of server,router and network resource consumption. Ideally, many HTTP requests should use the same connection, instead of opening a new connection for each request.
SSL connections per second No of SSL connections opened per second After TCP/IP connection SSL connection is opened. SSL connection has heavy resource consumption. If we select simulate new user at each iteration then there should not be more than one SSL connection per sec
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: Monitoring
Common Errors with Java Scripts in Controller / Replay
Below are some common issues / errors that are faced while replaying / running java vuser scripts in load runner: "22994 : Error:Unable to create Java VM" This error usually means that the PATH and the CLASSPATH environment variables are not set properly. Delete ini file for Vugen: a. Shutdown Vugen. b. Go to "Window install directory" and look for vugen.ini. Delete the file. LoadRunner will recreate new configuration file when VuGen is launched. c. Launch VuGen and run the script again. "22997 Java Vuser Compilation failed" Make sure that LoadRunner and the Java environment are set up properly. 1. Ensure the "jdk install dir"\bin and "LoadRunner Install dir"\bin directories are in the PATH environment variable. 2. Ensure "LoadRunner Install dir"\classes, "LoadRunner Install dir"\classes\srv, and "JDK install dir"\jre\lib\rt.jar are in the CLASSPATH environment variable (preferably at the beginning of the entry). 3. Add any necessary application CLASSPATHs to the environment variable. 4. Remove potentially conflicting entries in both the PATH and CLASSPATH entries (sometimes having the Java environment set up for QTP will conflict with the LoadRunner compilation process). 5. If you are running the script on a remote load generator, make sure that you apply the same update (1-4 ) there, and then, restart the LoadRunner agent. 6. Make sure that any classpaths specified in the Run-Time settings of the script are valid on the load generator machine as well. "22992 Can't find Actions.class" Make sure that there is a file named "Actions.class" within the script directory. This is the Java bytecode file generated from compiling Actions.java. If this file does not exits, there could be a compilation problem. If the file exits, verify the folllowing: 1. Ensure the "JDK Install dir"\bin and "LoadRunner Install dir"\bin directories are in the PATH environment variable. 2. Ensure "LoadRunner Install dir"\classes, "LoadRunner Install dir"\classes\srv, and "JDK Install dir"\jre\lib\rt.jar are in the CLASSPATH environment variable (preferably at the beginning of the entry). 3. Add any necessary application CLASSPATHs to environment variable. 4. Remove potentially conflicting entries in both the PATH and CLASSPATH entries (sometimes having the Java environment set up for QTP will conflict with the LoadRunner compilation process). 5. Make sure to restart the Controller or VuGen in order to have changes in steps 1-4 registered. 6. If the script was working and started to throw the above error for no obvious reasons, try to clean up the initilization files and reregister LoadRunner's DLL. To reregister the DLLs and recreate the configuration files: a. Make sure all the LoadRunner applications are closed. b. Go to "LoadRunner Install dir"\bin directory and execute register_vugen.bat. c. Go to C:\Winnt, back up vugen.ini, then delete the file. LoadRunner will recreate new configuration files when VuGen is launched. 7. The JVM is limited to 56 KB per function. If one or more of the functions within the Actions class is too big, you will get the above error. To resolve this, break the function up into multiple functions.
Posted by raviteja gorentla
Email ThisBlogThis!Share to TwitterShare to FacebookShare to Pinterest
Labels: Vugen
What is ecommerce? Why testing required for e-commerce application?
Electronic commerce or E-Commerce Application:
Electronic commerce (e-commerce or e-comm), is the buying and selling of products or services over electronic systems such as the Internet and other computer networks. It provides technologies such as electronic funds transfer, supply chain management, Internet marketing, online transaction processing, inventory management systems, and automated data collection systems. E-Commerce Software Testing
It ensures that every page is tested, e-commerce transactions are validated and application is ready for customer use.
The software’s Functionality, compatibility, security, performance and usability are checked
Testing is crucial to e-commerce because e-commerce sites are both business critical and highly visible to their users;
Yet the time pressures in the e-commerce world militate against the thorough testing usually associated with business criticality, so a new approach is needed to enable testing to be integrated into the development process and to ensure that testing does not present a significant time burden.
Rapid Applications Development (RAD), in particular, suggests some promising approaches. Like most new ventures, though, e-commerce must find its own way and establish its own methods.
Growth of E-commerce:
E-commerce continues its double-digit year-over-year growth rate, because sales are shifting away from stores and because online shoppers are less sensitive to adverse economic conditions than the average consumer.
32 million people (66% of all adults) purchased goods or services over the internet in 2011, according to the Office for National Statistics. This was an increase from 62% in 2010.
Despite the continued growth of the channel, online retailers face several challenges to growth: Online stores are broadly perceived as a second choice for shoppers, online retail is becoming increasingly seasonal, and online shoppers rarely admit to browsing, which can drive valuable incremental money during their Web shopping experiences.
Key Principles of Effective E-Commerce Testing: Principle 1.:Testing is a risk management process: Effective testing adopts a strategy that is tailored to the type of application or service being tested, the business value of the application or service, and the risks that would accompany its failure. The detailed planning of the testing and the design of the tests can then is confirmed by the strategy into a business-focused activity that adds real business value and provides some objective assessment of risk at each stage of the development process. Plans should include measures of risk and value and incorporate testing and other quality-related activities that ensure development is properly focused on achieving maximum value with minimum risk. Principle 2: Know the value of the applications being tested: To manage risk effectively, we must know the business value of success as well as the cost of failure. The business community must be involved in setting values on which the risk assessment can be based and committed to delivering an agreed level of quality Principle 3: Set clear testing objectives and criteria for successful completion (including test coverage measures): The test programmed must be properly planned, with test scripts giving precise instructions and expected results. Some assessment can be made of how many of the requirements have been tested at any given time. Criteria for successful completion are based on delivering enough business value, testing enough of the requirements to be confident of the most important behavior of the site, and minimizing the risk of a significant failure. Principle 4: Create an effective test environment. It would be very expensive to create a completely representative test environment for e-commerce, given the variety of platforms and the use of the Internet as a communications medium. Cross-platform testing is, naturally, an important part of testing any multi-platform software application. In the case of e-commerce, the term ‘cross-platform’ must also extend to include ‘cross-browser’. In order to ensure that a site loads and functions properly from all supported platforms, as much stress and load testing as possible should be performed. As an absolute minimum, several people should be able to log into the site and access it concurrently, from a mixture of the browsers and platforms supported. It would, therefore, be beneficial to use automated tools, such as Segue’s Silk Performer or Mercury Interactive Load Runner, for performance/load testing. Principle 5: Test as early as possible in the development cycle: The earlier faults are detected, the cheaper the cost of rectification. In the case of an e-commerce site, a fault found after shipping will have been detected as a failure of the site by the marketplace, which is potentially as large as the number of Internet users. This has the added complication of loss of interest and possibly the loss of customer loyalty, as well as the immediate cost of fixing the fault. The fact that e-commerce development is rapid and often based on changing requirements makes early testing difficult, but testing strategies have been developed by the RAD community, and these can be mobilized for support. Principle 6: User Acceptance Testing (UAT): The client or ultimate owner of the e-commerce site should perform field testing and acceptance testing, with involvement from the provider where needed, at the end of the development process. Even if RAD is used with its continuous user testing approach, there are some attributes of an e-commerce site that will not be easy (or even possible, in some cases) to validate in this way. Some form of final testing that can address issues such as performance and security needs to be included as a final confirmation that the site will perform well with typical user interactions. E-commerce users are becoming increasingly intolerant of poor sites, and technical issues related to functionality, performance or reliability have been cited as primary reasons why customers have abandoned sites. Early exposure of users to sites with problems increases the probability that they will find the site unacceptable, even if developers continue to improve the site during beta testing. Principle 7. Regression testing: Regression testing confirms that changes did not have unintended effects, so this must be a major feature of any e-commerce testing strategy. Web-based applications that reference external links need regular regression testing, even if their functionality does not change, because the environment is changing continuously. Wherever possible, regression testing should be automated, in order to minimize the impact on the test schedule. Principle 8: Automate as much as possible: This is a risky principle because test automation is fraught with difficulties. The key is to take testing processes sufficiently seriously that you document them and control them so that automation becomes a feasible option – then you select, purchase and install the tools. It will not be quick or cheap – but it might just avoid a very expensive failure. Principle 9.:Capture test incidents and use them to manage risk at release time: A test incident is any discrepancy between the expected and actual results of a test. Only some test incidents will relate to actual faults; some will be caused by incorrect test scripts, misunderstandings or deliberate changes to system functionality. All incidents found must be recorded via an incident management system (IMS), which can then be used to ascertain what faults are outstanding in the system and what the risks of release might be. Principle: Manage change properly to avoid undoing all the testing effort. Things change quickly and often in an e-commerce development and management of change can be a bottleneck, but there is little point in testing one version of a software application and then shipping a different version; not only is the testing effort wasted, but the risk is not reduced either. Configuration Management tools, such as PVCS and ClearCase, can help to minimize the overheads of change management, but the discipline is the most important thing. Why is testing important in the e-commerce Application?
The first and primary reason is because e-commerce is, by its very nature, business critical and highly visible to its user’s.
Any failure can be immediately expensive in terms of lost revenue and even more expensive in the longer term if disaffected users seek alternative sites.
E-commerce is a massive and growing market place but one which requires large up-front investment to enter successfully.
The history of e-commerce development has been littered with expensive failures, at least some of which could have been avoided by better testing before the site was opened to the general public.
Quality Assurance of the software or application developed
Verification and validating the product/application before it goes live in the market to prevent it from intruders and hackers.
Defect free and user friendly application.