Thursday, March 18, 2010

Sample bug report

Lets assume in your application under test you want to create a new user with user information, for that you need to log on into the application and navigate to USERS menu > New User, then enter all the details in the ‘User form’ like, First Name, Last Name, Age, Address, Phone etc. Once you enter all these information, you need to click on ‘SAVE’ button in order to save the user. Now you can see a success message saying, “New User has been created successfully”.

But when you entered into your application by logging in and navigated to USERS menu > New user, entered all the required information to create new user and clicked on SAVE button. BANG! The application crashed and you got one error page on screen. (Capture this error message window and save as a Microsoft paint file)

Now this is the bug scenario and you would like to report this as a BUG in your bug-tracking tool.

How will you report this bug effectively?

Here is the sample bug report for above mentioned example:
(Note that some ‘bug report’ fields might differ depending on your bug tracking system)

SAMPLE BUG REPORT:

Bug Name: Application crash on clicking the SAVE button while creating a new user.
Bug ID: (It will be automatically created by the BUG Tracking tool once you save this bug)
Area Path: USERS menu > New Users
Build Number: Version Number 5.0.1
Severity: HIGH (High/Medium/Low) or 1
Priority: HIGH (High/Medium/Low) or 1
Assigned to: Developer-X
Reported By: Your Name
Reported On: Date
Reason: Defect
Status: New/Open/Active (Depends on the Tool you are using)
Environment: Windows 2003/SQL Server 2005

Description:
Application crash on clicking the SAVE button while creating a new
user, hence unable to create a new user in the application.

Steps To Reproduce:
1) Logon into the application
2) Navigate to the Users Menu > New User
3) Filled all the user information fields
4) Clicked on ‘Save’ button
5) Seen an error page “ORA1090 Exception: Insert values Error…”
6) See the attached logs for more information (Attach more logs related to bug..IF any)
7) And also see the attached screenshot of the error page.

Expected result: On clicking SAVE button, should be prompted to a success message “New User has been created successfully”.

(Attach ‘application crash’ screen shot.. IF any)

Save the defect/bug in the BUG TRACKING TOOL. You will get a bug id, which you can use for further bug reference.
Default ‘New bug’ mail will go to respective developer and the default module owner (Team leader or manager) for further action.

Wednesday, March 17, 2010

How Domain knowledge is Important for testers?

First of all I would like to introduce three dimensional testing career mentioned by Danny R. Faught. There are three categories of skill that need to be judged before hiring any software tester. What are those three skill categories?
1) Testing skill
2) Domain knowledge
3) Technical expertise.

No doubt that any tester should have the basic testing skills like Manual testing and Automation testing. Tester having the common sense can even find most of the obvious bugs in the software. Then would you say that this much testing is sufficient? Would you release the product on the basis of this much testing done? Certainly not. You will certainly have a product look by the domain expert before the product goes into the market.

While testing any application you should think like a end-user. But every human being has the limitations and one can’t be the expert in all of the three dimensions mentioned above. (If you are the experts in all of the above skills then please let me know ;-)) So you can’t assure that you can think 100% like how the end-user going to use your application. User who is going to use your application may be having a good understanding of the domain he is working on. You need to balance all these skill activities so that all product aspects will get addressed.

Nowadays you can see the professional being hired in different companies are more domain experts than having technical skills. Current software industry is also seeing a good trend that many professional developers and domain experts are moving into software testing.

We can observe one more reason why domain experts are most wanted! When you hire fresh engineers who are just out of college you cannot expect them to compete with the experienced professionals. Why? Because experienced professional certainly have the advantage of domain and testing experience and they have better understandings of different issues and can deliver the application better and faster.

Here are some of the examples where you can see the distinct edge of domain knowledge:
1) Mobile application testing.
2) Wireless application testing
3) VoIP applications
4) Protocol testing
5) Banking applications
6) Network testing

How will you test such applications without knowledge of specific domain? Are you going to test the BFSI applications (Banking, Financial Services and Insurance) just for UI or functionality or security or load or stress? You should know what are the user requirements in banking, working procedures, commerce background, exposure to brokerage etc and should test application accordingly, then only you can say that your testing is enough - Here comes the need of subject-matter experts.

Let’s take example of my current project: I am currently working on search engine application. Where I need to know the basic of search engine terminologies and concepts. Many times I see some other team tester’s asking me questions like what is ‘publishers’ and ‘advertisers’, what is the difference and what they do? Do you think they can test the application based on current online advertising and SEO? Certainly not. Unless and until they get well familiar with these terminologies and functionalities.

When I know the functional domain better I can better write and execute more test cases and can effectively simulate the end user actions which is distinctly a big advantage.

Here is the big list of the required testing knowledge:

  • Testing skill
  • Bug hunting skill
  • Technical skill
  • Domain knowledge
  • Communication skill
  • Automation skill
  • Some programming skill
  • Quick grasping
  • Ability to Work under pressure …

That is going to be a huge list. So you will certainly say, do I need to have these many skills? Its’ depends on you. You can stick to one skill or can be expert in one skill and have good understanding of other skills or balanced approach of all the skills. This is the competitive market and you should definitely take advantage of it. Make sure to be expert in at least one domain before making any move.

What if you don’t have enough domain knowledge?
You will be posted on any project and company can assign any work to you. Then what if you don’t have enough domain knowledge of that project? You need to quickly grasp as many concepts as you can. Try to understand the product as if you are the customer and what customer will do with application. Visit the customer site if possible know how they work with the product, Read online resources about the domain you want to test the application, participate in events addressing on such domain, meet the domain experts. Or either company will provide all this in-house training before assigning any domain specific task to testers.

There is no specific stage where you need this domain knowledge. You need to apply your domain knowledge in each and every software testing life cycle.

Types of Risks in Software Projects

As testing is the last part of the project, it’s always under pressure and time constraint. To save time and money you should be able to prioritize your testing work. How will prioritize testing work? For this you should be able to judge more important and less important testing work. How will you decide which work is more or less important? Here comes need of risk-based testing.

What is Risk?
“Risk are future uncertain events with a probability of occurrence and a potential for loss”

Risk identification and management are the main concerns in every software project. Effective analysis of software risks will help to effective planning and assignments of work.

In this article I will cover what are the “types of risks”. In next articles I will try to focus on risk identification, risk management and mitigation.

Risks are identified, classified and managed before actual execution of program. These risks are classified in different categories.

Categories of risks:

Schedule Risk:
Project schedule get slip when project tasks and schedule release risks are not addressed properly.
Schedule risks mainly affect on project and finally on company economy and may lead to project failure.
Schedules often slip due to following reasons:

  • Wrong time estimation
  • Resources are not tracked properly. All resources like staff, systems, skills of individuals etc.
  • Failure to identify complex functionalities and time required to develop those functionalities.
  • Unexpected project scope expansions.

Budget Risk:

  • Wrong budget estimation.
  • Cost overruns
  • Project scope expansion

Operational Risks:
Risks of loss due to improper process implementation, failed system or some external events risks.
Causes of Operational risks:

  • Failure to address priority conflicts
  • Failure to resolve the responsibilities
  • Insufficient resources
  • No proper subject training
  • No resource planning
  • No communication in team.

Technical risks:
Technical risks generally leads to failure of functionality and performance.
Causes of technical risks are:

  • Continuous changing requirements
  • No advanced technology available or the existing technology is in initial stages.
  • Product is complex to implement.
  • Difficult project modules integration.

Programmatic Risks:
These are the external risks beyond the operational limits. These are all uncertain risks are outside the control of the program.
These external events can be:

  • Running out of fund.
  • Market development
  • Changing customer product strategy and priority
  • Government rule changes.

What is client-server and web based testing and how to test these applications

Projects are broadly divided into two types of:

  • 2 tier applications
  • 3 tier applications

CLIENT / SERVER TESTING
This type of testing usually done for 2 tier applications (usually developed for LAN)
Here we will be having front-end and backend.

The application launched on front-end will be having forms and reports which will be monitoring and manipulating data

E.g: applications developed in VB, VC++, Core Java, C, C++, D2K, PowerBuilder etc.,
The backend for these applications would be MS Access, SQL Server, Oracle, Sybase, Mysql, Quadbase

The tests performed on these types of applications would be
- User interface testing
- Manual support testing
- Functionality testing
- Compatibility testing & configuration testing
- Intersystem testing

WEB TESTING
This is done for 3 tier applications (developed for Internet / intranet / xtranet)
Here we will be having Browser, web server and DB server.

The applications accessible in browser would be developed in HTML, DHTML, XML, JavaScript etc. (We can monitor through these applications)

Applications for the web server would be developed in Java, ASP, JSP, VBScript, JavaScript, Perl, Cold Fusion, PHP etc. (All the manipulations are done on the web server with the help of these programs developed)

The DBserver would be having oracle, sql server, sybase, mysql etc. (All data is stored in the database available on the DB server)

The tests performed on these types of applications would be
- User interface testing
- Functionality testing
- Security testing
- Browser compatibility testing
- Load / stress testing
- Interoperability testing/intersystem testing
- Storage and data volume testing

A web-application is a three-tier application.
This has a browser (monitors data) [monitoring is done using html, dhtml, xml, javascript]-> webserver (manipulates data) [manipulations are done using programming languages or scripts like adv java, asp, jsp, vbscript, javascript, perl, coldfusion, php] -> database server (stores data) [data storage and retrieval is done using databases like oracle, sql server, sybase, mysql].

The types of tests, which can be applied on this type of applications, are:
1. User interface testing for validation & user friendliness
2. Functionality testing to validate behaviors, i/p, error handling, o/p, manipulations, services levels, order of functionality, links, content of web page & backend coverage’s
3. Security testing
4. Browser compatibility
5. Load / stress testing
6. Interoperability testing
7. Storage & data volume testing

A client-server application is a two tier application.
This has forms & reporting at front-end (monitoring & manipulations are done) [using vb, vc++, core java, c, c++, d2k, power builder etc.,] -> database server at the backend [data storage & retrieval) [using ms access, sql server, oracle, sybase, mysql, quadbase etc.,]

The tests performed on these applications would be
1. User interface testing
2. Manual support testing
3. Functionality testing
4. Compatibility testing
5. Intersystem testing
Some more points to clear the difference between client server, web and desktop applications:

Desktop application:
1. Application runs in single memory (Front end and Back end in one place)
2. Single user only

Client/Server application:
1. Application runs in two or more machines
2. Application is a menu-driven
3. Connected mode (connection exists always until logout)
4. Limited number of users
5. Less number of network issues when compared to web app.

Web application:
1. Application runs in two or more machines
2. URL-driven
3. Disconnected mode (state less)
4. Unlimited number of users
5. Many issues like hardware compatibility, browser compatibility, version compatibility, security issues, performance issues etc.

As per difference in both the applications come where, how to access the resources. In client server once connection is made it will be in state on connected, whereas in case of web testing http protocol is stateless, then there comes logic of cookies, which is not in client server.

For client server application users are well known, whereas for web application any user can login and access the content, he/she will use it as per his intentions.

So, there are always issues of security and compatibility for web application.

Smoke testing and sanity testing - Quick and simple differences

SMOKE TESTING:

  • Smoke testing originated in the hardware testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch fire and smoke. In software industry, smoke testing is a shallow and wide approach whereby all areas of the application without getting into too deep, is tested.
  • A smoke test is scripted, either using a written set of tests or an automated test
  • A Smoke test is designed to touch every part of the application in a cursory way. It’s shallow and wide.
  • Smoke testing is conducted to ensure whether the most crucial functions of a program are working, but not bothering with finer details. (Such as build verification).
  • Smoke testing is normal health check up to a build of an application before taking it to testing in depth.

SANITY TESTING:

  • A sanity test is a narrow regression test that focuses on one or a few areas of functionality. Sanity testing is usually narrow and deep.
  • A sanity test is usually unscripted.
  • A Sanity test is used to determine a small section of the application is still working after a minor change.
  • Sanity testing is a cursory testing, it is performed whenever a cursory testing is sufficient to prove the application is functioning according to specifications. This level of testing is a subset of regression testing.
  • Sanity testing is to verify whether requirements are met or not, checking all features breadth-first.

Thursday, March 11, 2010

Software Quality Assurance

Software quality assurance (SQA) consists of a means of monitoring the software engineering processes and methods used to ensure quality. The methods by which this is accomplished are many and varied, and may include ensuring conformance to one or more standards, such as ISO 9000 or CMMI. SQA encompasses the entire software development process, which includes processes such as software design, coding, source code control, code reviews, change management, configuration management, and release management.