Automation Testing Projects - Estimation Guidelines

 

Background:

Estimation helps you to estimate or guess. It provides you an approximate in numeric value; it could be number of person days or hour’s etc. It is a pre-planning activity on any type of automation projects, which helps or puts Manager or Lead in ease to manage the project and complete the activities on time as per the agreed estimation.

It helps project management if there are any deviations; revisit the estimation at any point in time during project management life cycle as per the agreed frequencies.

Project is typically has the major phases like requirements, planning, coding , testing and production; in these lines we will have scope changes, issues /risks, delay in time and resources unplanned leaves,  infrastructure/Environment outage etc.,. The outcome of the unexpected changes in any of these lines will have an impact on project. Hence, making good estimates is very crucial. Decision making on the estimations varies from the project to project. One should not do overestimate to overcome the obstacles, in fact it is a wrong approach.

In my experience, making estimation should be closure to actual means genuine / realistic and accurate. We cannot predict any issues that comes in between the project execution so one should have frequency parameters to revisit the estimations defined or agreed; it increases the transparency between the Client and the Vendor. Hence, we will produce estimates with highly optimistic mindset. Adhering to agreed estimation is very important to build a good reputation and relationship with the client and results to longer and stronger bonding.

Agreed frequency parameters, few examples:

1.       Change in scope

2.       Resource ramp up & down in size

3.       Change in Regression size

4.       Decrease in dependency on the teams

Introduction:

There are so many estimation techniques like Agile COCOMO, Poker, Three Point Estimation, Proxy Based Estimation; Wide band Delphi, Function Point Analysis and popular FP Variants in software testing but it is true that no estimation model exactly fits to the similar requirements too.

This document outlines and highlights to collect the basic information that is required to estimate the automation testing efforts / automation projects.

Here with summarizing the ways to collect the raw data for the estimation, the ways to map the requirement with the available data and estimating the efforts for the project.

NOTE: Do not go by the historical data and provide that estimation, as each project is unique. Historical data gives you an idea but do not rely on them. Always, suggest you to understand your project and strength of your team and other supporting teams (Example: infrastructure including tools and management support)

Touchpoints to consider while doing Estimation:

1. Define the Regression Size (Ex: Number of Manual Test Cases)
2. Define the size
1. Manual Test Cases Count (Manual Test Cases Count = Regression Size - Automation Not Feasible)
2. Automation Test Case Count (Automation Test Case Count = Regression Size – Automation Feasible Test Cases)
3. Define the complexity for derived automation test cases

                                                               i.      Functionality Complexity (Complex, High, Medium & low)

                                                             ii.      Automation development Complexity (Complex, High, Medium & low)

4. Automation Skillset (Resource Capability) (Expert, intermediate, novice & beginner)
5. Automation code Re-usability percentage (NOTE: if the re-usability is less than 30%, I suggest not to automate such feature or module or project, which won’t give best ROI )

                                                               i.      Derive Existing Automation code Re-usability percentage

                                                             ii.      Derive Newly Automation code Re-usability percentage

6. Development Effort – derive the development effort for one test case based on the following factors

                                                               i.      Code complexity

                                                             ii.      Functionality complexity

                                                           iii.      Resource skillset / experience

                                                           iv.      Technology of the application – efforts might varies again each of the following and percentage of code re-usage

1.       Web Services

2.       Web applications

3.       Mainframes

4.       Windows

5.       .Net

6.       Angular JS etc.

7. Also, including the efforts for the following automation code dependencies

                                                               i.      Functionality understanding (KT)

                                                             ii.      Test Data creation

                                                           iii.      Automation Feasibility study (Analysis)

                                                           iv.      Clarifications delays

                                                             v.      Code Debugging & Dry run for few test runs (to validate code robustness)

                                                           vi.      Code reviews & corrections

                                                          vii.      Any other dependencies

                                                        viii.      Technical challenges if any

8. Execution – if the automation execution time is less than manual execution then only consider the automation feasibility study prior to automation development.

Approach:

Automation Estimation - Provided simple 3 Points for easy understanding

Point 1: 

Grade the efforts required in hours to develop the single automation script among the resource capability; that you have in your team (Refer the below table). The Efforts required/script may vary from application to application. The given efforts are not industry standards, it should be derived from your historical information or experience

NOTE: Similarly, you can grade the efforts for single automation script execution to calculate the efforts

 Point 2: 

Now, you Grade the complexity percentage from your Regression suite size and derive the test cases count by calculating rightly (Refer the below table). Here, the percentage may vary from organization to organization or application being tested. These percentages helps in deriving the test cases count itself.

Point 3: 

The above two table provides you required inputs to calculate the development efforts against resource capability. 

Here, given 5% as Buffer that includes technical challenges only. The Total Days required here is purely the development efforts. Additionally, you have to include the efforts required for the other automation activities listed below.

NOTE: Similarly, include the additional activities while doing calculation efforts for execution effort estimation as well.

     Calculation for your understanding

            Complex (Hrs.) = Test Cases Count [12] * Complex (Hrs.) [ 4 ] 

                                     = 48 (Complex (Hrs.)) for Expert Resource

A Typical Automation contains the following activities – Work Breakdown Structure (WBS)

1.       Feasibility Study (Analysis) – Here executing the manual test case at least once and verify the data management is easy and have no dependencies.

2.       Framework design and development

3.       Script Development

4.       Integration of re-usable components

5.       Self execution / Dry Run – to make sure functionality is working fine

6.       Code Review

7.       Execution Review

8.       Sign – Off

 

Benefits:

1. Now we are able to communicate facts clearly with proper estimation model.
2. Able to track and monitor the activities well.
3. Finishing the activities on time with transparency.
4. Able to display clearly and confidently to the Client if there is any deviation from the given timelines well in advance.

Summary:

Initially, identify the feasible test cases for automation. Understand the project requirement correctly and map with the estimation experience that you have and with the existing techniques that are available. Create your own estimation framework that makes you comfortable in tracking and monitoring the activities.

If we encounter any issues due to any cause in between the project execution, you can communicate and give clarity on the delays to direct & indirect stakeholders with estimation deviations with valid evidences.

This will help you to improve the existing approach and gives add value to the customer in the long run too.