Wednesday, November 28, 2007

Build Verification Testing (BVT)

Build Verification Testing (BVT)
Have you ever received a build into testing which was missing files, had the wrong file, or a file with the incorrect language version? These and other problems associated with the file properties of each new build usually result from a improperly designed build verification test (BVT) suite. The primary purpose of the BVT is to validate the integrity of each new build of a project. Some teams combine the BVT along with the Build Acceptance Test (BAT) which is not necessarily a bad approach if the test team owns the BVT. However, I have found in these situations the test team usually places more emphasis on the BAT to verify basic functionality of the build rather than the BVT which actually validates the build and thus misses critical problems with the build itself.
Ideally, the person or team that builds the project should be responsible for validating each new build before releasing it to build acceptance testing, but the test team should have oversight into what is being checked. It does no good to run a new build through a BAT process only to discover a missing file, or a German help file in a Japanese language version of the project. At a minimum the BVT should check every file in each new build for:
 Correct version information
 Correct time and date stamps
 Appropriate file flags
 Correct cyclic redundancy check (CRC) keys
 Correct language information
 ISO 9660 file naming conventions based on standards
 Viruses
Changes in the build can help the test team focus on critical areas, or identify areas of the project that should be revisited. Other information a BVT should provide to the test team includes:
 New files added to the project build
 Files removed from the project build
 Files with binary changes
It might also be a good idea to scan file names for public acceptability. For example, I remember one project in which a clever developer decided to call his new library 'sexme.dll.' This file name was released in the shipped product and several customers objected.
Even on large projects with several hundred files the tests to validate the integrity of a build would not take more than a few minutes using automated processes. The few minutes spent validating each new build is well justified considering the cost of releasing an build into testing and discovering the build is invalid (even though it may have passed the BAT) because then you have to question the validity of the test effort on that build.

Definition Here’s the definition we use: “A build acceptance test (sometimes also called build verification test a.k.a. BVT, smoke test, quick check, or the like) is a set of tests run on each new build of a product to verify that the build is testable before the build is released into the hands of the test team. The build acceptance test is generally a short set of tests, which exercises the mainstream functionality of the application. Any build that fails the build verification test is rejected, and testing continues on the previous build (provided there has been at least one build that has passed the acceptance test). So build acceptance tests are a type of regression testing that is done every time a new build is taken. Build acceptance tests are important because they let developers know right away if there is a serious problem with the build, and they save the test team wasted time and frustration.”

In testing, a Build Verification Test (BVT), also known as Build Acceptance Test, is a set of tests run on each new build of a product to verify that the build is testable before the build is released into the hands of the test team. The build acceptance test is generally a short set of tests, which exercises the mainstream functionality of the app s/w Any build that fails the build verification test is rejected, and testing continues on the previous build (provided there has been at least one build that has passed the acceptance test).
BVT is important because it lets developers know right away if there is a serious problem with the build, and they save the test team wasted time and frustration by avoiding test of an unstable build.

No comments: