Installation Testing
by Erik Feigin
assignment for course
"Software Quality Evaluation"
Sami Shamoon College of Engineering
Israel, Ashdod
8 June 2017
Be the guardian of great amounts
of important information - Let's learn from examples:
********Testing tools*********
***Implementation Examples***
List of installation software
Remote Install Mac OS X
Installer
Windows
InstallShield
InstallCore
Inno Setup
EMCO MSI Package Builder
Cross Platform
jPort
InstallAnywhere
Installer VISE
InstallJammer
Instalation Tests Examples
Intelligence Fusion Center
C-Registration System
The Chromium Projects
What is Software Installation Testing?
Objectives of Software Installation Testing
Does the software demonstrate that its installation functions and behaviors behave correctly?
On the specified system configuration environment, can the software be successfully installed under each of the validated running conditions?
-Can the software be properly installed on all specified system configurations?
The objective of the software installation testing is not to find software errors but to find installation errors.
Why Software Installation Testing?
In the wider sense, installation is the very first thing the customer will do. Any faults found here will immediately give the wrong impression about the product.
A trouble free installation creates a sense of confidence, while a faulty installation marks the customer’s dreadful notion of the product before they have even looked at it. In order to be confident about user’s initial experience with the software the installation testing should be of the highest possible quality.
Software Installation Testing plays a vital role in the testing activities and its first interaction of user with any product. The experience that the customer has while installing a software product can either create a sense of confidence or forewarning in the product.
Its main purpose is to validate the given software product to see if it can be correctly installed in a specified system environment with proper system configurations and running conditions.
*This testing is accomplished through either actual or simulated use of the software being tested within the environment in which it is intended to function.
This type of testing is performed to ensure that all Installed features and functions properly. It is also verify that all necessary components of the application are, indeed, installed.
We also test all the user setup options (full, typical and custom), navigational buttons (Next, Back, Cancel, etc), and user input fields to ensure that they function properly and yield the expected result.
Software Installation Testing is one of the most complicated types of testing to plan and execute.
Software Installation Testing is one of the important types of system testing.
Types of
Installation
Automated Installation
In this administrator schedules the installation of product on certain prerequisite conditions.
#5: You can release more frequently
A single deployment performed by an automated deployment mechanism has a low overhead. A release process with a low overhead is one that can be repeated frequently. As I’ve discussed on this blog in the past, frequent releases are desirable for many reasons, but the crux of the matter is that frequent releases promote truly agile software development.
Teams that release frequently can deliver valuable features to their users more often and in incremental steps. In doing so they can gather continuous feedback from these users on the software they are creating and adapt their approach as a result. This feedback can be the difference between a product delighting its target audience or missing them completely.
#4: Deploying to somewhere new is not a headache
Automated deployments are not only repeatable, but they are configurable too. Although the underlying release process is permanent, the target environments and machines can be changed easily.
This means that if software needs to be deployed to a new test environment or if a new client installation needs to be created, the overhead of deploying to that additional target is negligible. It is simply a case of configuring your existing set-up and relying on your tried and tested release automation to get the job done.
#3: Engineers spend their time developing software
Performing and validating a manual deployment process is often a time-consuming, thankless task. This job can fall to developers and testers in a development team who would otherwise be spending their time producing the next set of awesome features and improvements to the software.
The time taken to initiate a fully automated deployment is counted in seconds. Validation of those deployments happens behind the scenes and team members may only need to spend further time on a deployment if something has actually gone wrong. As a result, the team get to spend more time doing what they enjoy and have been assembled to do; create great software.
#2: Anyone in the team can deploy software
With an automated deployment process, the knowledge of how to release your software is captured in the system, not in an individual’s brain.
Performing manual or partially-automated deployments are often the responsibility of a small subset of people in an organisation. In fact, it’s not uncommon for this duty to fall to a single person in given project team. If that person is off ill or otherwise unavailable at short notice, releasing can become a nightmare. Now, anyone who has access to the “Deploy” button can initiate a release.
#1: Deployments become much less error-prone and much more repeatable
Manual deployments are error prone. This is because it involves humans doing stuff, so is governed by Murphy’s Law: if anything can go wrong, it will go wrong. Important steps in a release can be accidentally missed, faults that occur during release may not be spotted, the incorrect versions of software can be shipped and broken software ends up going live. If you are lucky, you’ll be able to recover from this quickly. If you are unlucky, well… it’s pretty embarrassing at best.
Automated deployments don’t suffer from variability. Once configured, the process is set, and will be the same every time a release is initiated. If it works the first time you deploy your software into a given environment, it’ll work the 100th time.
Clean Installation
This is the installation where old version of product is not installed on system and product is being installed for the first time. Installation of OS is example of clean installation
Have to save your data or the data will be lost during the procedure.
Have to load more drivers
Takes longer and more involved
I believe it runs better.
Less chances of an incompatible driver.
Less chances of problems down the road.
Better chance of a good install.
The entire drive is wiped out, so there is no chance of a corruption (unless the install has problems)
Headless installation/Network Installation
In this kind of installation, Monitor or console is not needed and installation is made on targeted computers that are connected with a machine on network and in this user involvement is not needed at targeted computers for installation of product on each system. This kind of installation takes place in big organizations where product is to be installed on thousands of system all together.
Unattended installation
User involvement is not needed in the installation of product and sometimes if user intervention is involved then whole installation is done through answer file in which user mentions all the parameters that are needed for installation. Installation of xp operation system is example of this.
Lower support costs
By minimizing errors during the setup process, increasing the consistency of the computers in your organization, and reducing the amount of time a technician needs to spend setting up a computer, you can reduce the overall support costs in your organization.
Shorter installation times
Unattended installation is faster than interactive setup because Setup does not have to prompt administrators or technicians for configuration information and wait for them to enter it; instead, Setup reads configuration settings from an answer file.
Greater consistency during a rollout
By using the same answer file to install and configure the operating system, you can ensure that all of the computers in your organization are set up exactly the same way.
Fewer errors during installation
Because unattended installation uses an answer file to install and configure the operating system, there is minimal interaction by administrators or technicians during the setup process. This reduces the number of errors that are introduced during the setup process.
Attended Installation
This is the most common form of installation in which involvement of user is necessary to give its input/selection of choice. Tasks that user performs in an attended installation:
If installer is downloading its packages from internet during its installation and LAN is disconnected then it also gives error message “Check you internet connection “, for example.
User also helps in mitigating certain error like memory is not enough then installer asks for a new place for its installation.
Entering password if software is asking for it. For example when we install mysql, during installation of its configuration it asks for the password.
Selecting directory place to install
Accepting end-user license agreement that is in short called EULA
Silent Installation
Installation that does not show messages during its installation on console. All messages of installation are normally saved in a log file in case of silent install.
These types of programs may even run without you initiating the installation process. They may be tacked on to another installer or executed by a malicious file, such as virus.
While silent installations are used for many good reasons, some programs, such as spyware and adware use silent installers to run without your knowledge.
Even though silent installers run without any user interaction, legitimate programs typically require you to manually initiate the installation process.
For example, a network administrator may prefer to distribute a software program via a silent installer to ensure all users within the network have the same installation settings.
Silent installs are useful for simple programs that have limited installation options.
They are also helpful for installing software on several machines at once.
Summary and Conclusions
Conclusions
It would be difficult, expensive, and exorbitantly time consuming for a company to set up and perform these kind of tests on their own!
The goal of installation testing is to make sure the software is installed and working as expected after installation. Sometimes called implementation testing, it’s one of the most important tests of distributed software.
Summary
If you think about the last application you installed, it likely involved allowing you to choose an installation directory, installation options you could select or deselect, such as if you wanted a shortcut placed on the desktop. In the background, registry changes were also made, and an uninstallation procedure was put in place. Each of these things are also affected by operating environments and other software already installed on the system. All of these steps are testing during installation testing, and verified in as many end-user configurations as possible.
In the wider sense, installation is the very first thing the customer will do. Any faults found here will immediately give the wrong impression about the product.
Installation testing is specific to distributed software that end-users must install on their computer or mobile device. These could be browser extensions, apps, server software, database applications, or independent software applications. It focuses on what customers need to do in order to successfully install, or uninstall, your application. The process involves full installations, partial installations, installation configuration options, patching processes, and uninstall processes.
Challenges of Software Installation Testing
What are major problems and needs
you won't forget it
The Major needs are:
* A well-defined software installation test process with cost-effective test criteria and standards.
* Well-defined test models for software installation planning, testing and analysis to model and present.
* Diverse system environment and configurations
* Various system running conditions
* Various system installation functions
* Well-defined metrics measuring and predicting test complexity, costs, and coverage.
* Cost-effective systematic test generation methods and tools to support automatic software installation testing.
The Major Problem is: Many software makers experience their highest abandonment rates during the installation phase where users are giving up before even using the product.
The current practice in the real world is summarized below:
Installation Testing is done just using various ad-hoc approaches.
Lack well-defined test models, standards, and test coverage criteria.
Test Cases are designed manually using an ad-hoc approach without models and criteria.
Most test executions are done manually including file checks.
Only GUI-based executable test scripts are generated in ad-hoc approach.
Installation test is just carried out using an inhabited or jumbled plan.
Though Installation Testing observes its own importance, industry identifies the following challenges in software installation validation
Major concern of installation testing is Time! It requires lot of to even execute a single test case. The complexity increases when it’s a life-size application to be tested with many test cases on different configurations.
Software Installation Testing needs to be performed under a very tight schedule without well-defined test models, test criteria, and tools.
The software product installation should be validated under various system running conditions
The software product should be installed on a diverse system environment with many different configurations.
Installation failures leave the user’s system badly damaged. User might require reinstalling the full operating system.
Typical test cases should be taken in to account for Installation testing
* Running setup of product that is pre-installed on system should show GUI that should ask for Repair/Remove option or un-install and reinstallation option of same product.
* Test cases should be written to test how uninstallation behave when we uninstall after stopping uninstallation in midway.
* Testing should be done to show proper messages when recent version is installed on system and we are trying to install old version of same product.
* Installer should be tested without having system administrator privileges.
* A test case should also be written for installation of product in a folder where write option is disabled.
* A test case should be written for successful uninstallation and all files should be removed after uninstallation of the product.
* Concurrent installation of multiple product test cases is also most important.
* Time taken for installation and extraction.
* Testing should be to see the path of extraction of software and in general extraction of software takes place in temp file.
* Test cases should be written for log files, that includes all the activities of installation.
* Uninstallation of product should be tested through add/remove programs and also from the path of the installed program, where we could find a un-installation file.
* On double clicking the setup file, installation should start.
* Installation testing should also be done when some other software of same kind is running and this testing should also be done with the software that is using a lot of RAM for its operation.
* Test cases should also be written for insufficient memory, here I am talking about RAM.
* Testing should be done for insufficient disk space and a proper error message should flash for insufficient disk space or some error number should flash.
* Testing of installer should be done when Firewall is on and security of system is high.
* Network speed / LAN Connectivity should also be tested if installer is using net to download its package file for installation.
* Update installation and patch installation should also be tested.
* Test cases should be written to compare all the files and packages installed on system with the previous version of installer packages.
* Installation on network, on multiple machines through Master machine.
* Test script should be written to check the changes in registry.
* Test cases should be written for forced stop of installation.
* Default installation and custom installation path should be included in test cases for testing.
* Test cases should be written for all work flows and this type of test cases come from requirements.
* Test cases should be written that check the old version of product installed. If old version is installed, then test case should be written to install the product on the same path.
* Test script should be written to check the required Disk Space for installation.
* Test cases should be written for proper error message during installation.
* Test cases should be written to check the disk space before installation and just after installation of disk space.
Try to come up with original ideas
Few typical things that is taken in to account before starting testing
Operating system on which testing would be done.
Distribution of software like through CD or directly from Internet.
Supported Operating system for which installer is going to be launched.
