Background:
Estimation helps you to estimate or guess. It provides you
an approximate in numeric value; it could be number of person days or hour’s
etc. It is a pre-planning activity on any type of automation projects, which helps or puts Manager or Lead in ease
to manage the project and complete the activities on time as per the agreed estimation.
It helps project management if there are any deviations; revisit
the estimation at any point in time during project management life cycle as per
the agreed frequencies.
Project
is typically has the major phases like requirements, planning, coding , testing
and production; in these lines we will have scope changes, issues /risks, delay
in time and resources unplanned leaves, infrastructure/Environment outage etc.,. The
outcome of the unexpected changes in any of these lines will have an impact on
project. Hence, making good estimates is very crucial. Decision making on the
estimations varies from the project to project. One should not do overestimate to
overcome the obstacles, in fact it is a wrong approach.
In my
experience, making estimation should be closure to actual means genuine / realistic and accurate.
We cannot predict any issues that comes in between the project execution so one
should have frequency parameters to revisit the estimations defined or agreed;
it increases the transparency between the Client and the Vendor. Hence, we will
produce estimates with highly optimistic mindset. Adhering to agreed estimation is very important to build a good
reputation and relationship with the client and results to longer and stronger
bonding.
Agreed frequency parameters,
few examples:
1.
Change
in scope
2.
Resource
ramp up & down in size
3.
Change
in Regression size
4.
Decrease
in dependency on the teams
Introduction:
There are so many estimation techniques like Agile COCOMO, Poker,
Three Point Estimation, Proxy Based Estimation; Wide band Delphi, Function
Point Analysis and popular FP Variants in software testing but it is true that
no estimation model exactly fits to the similar requirements too.
This document outlines and highlights to collect the basic
information that is required to estimate the automation testing efforts /
automation projects.
Here with summarizing the ways to collect the raw data for
the estimation, the ways to map the requirement with the available data and
estimating the efforts for the project.
NOTE: Do not go by the historical data and provide that estimation,
as each project is unique. Historical data gives you an idea but do not rely on
them. Always, suggest you to understand your project and strength of your team
and other supporting teams (Example: infrastructure including tools and
management support)
Touchpoints to
consider while doing Estimation:
- Define the Regression Size
(Ex: Number of Manual Test Cases)
- Define the size
- Manual Test Cases Count (Manual
Test Cases Count = Regression Size - Automation Not Feasible)
- Automation Test Case
Count (Automation Test Case Count = Regression Size – Automation Feasible
Test Cases)
- Define the complexity for
derived automation test cases
i.
Functionality Complexity (Complex, High, Medium
& low)
ii.
Automation development Complexity (Complex,
High, Medium & low)
- Automation Skillset
(Resource Capability) (Expert, intermediate, novice & beginner)
- Automation code Re-usability
percentage (NOTE: if the re-usability is less than 30%, I suggest not to
automate such feature or module or project, which won’t give best ROI )
i.
Derive Existing Automation code Re-usability
percentage
ii.
Derive Newly Automation code Re-usability
percentage
- Development Effort –
derive the development effort for one test case based on the following
factors
i.
Code complexity
ii.
Functionality complexity
iii.
Resource skillset / experience
iv.
Technology of the application – efforts might
varies again each of the following and percentage of code re-usage
1. Web
Services
2. Web
applications
3. Mainframes
4. Windows
5. .Net
6. Angular
JS etc.
- Also, including the
efforts for the following automation code dependencies
i.
Functionality understanding (KT)
ii.
Test Data creation
iii.
Automation Feasibility study (Analysis)
iv.
Clarifications delays
v.
Code Debugging & Dry run for few test runs
(to validate code robustness)
vi.
Code reviews & corrections
vii.
Any other dependencies
viii.
Technical challenges if any
- Execution – if the
automation execution time is less than manual execution then only
consider the automation feasibility study prior to automation development.
Approach:
Automation Estimation - Provided simple 3 Points for easy understanding
Point 1:
Grade
the efforts required in hours to develop the single automation script among the
resource capability; that you have in your team (Refer the below table). The Efforts required/script may vary from application to application. The given efforts are not industry standards, it should be derived from your historical information or experience
NOTE: Similarly, you can grade the efforts for single automation script execution to calculate the efforts
Now, you
Grade the complexity percentage from your Regression suite size and derive the
test cases count by calculating rightly (Refer the below table). Here, the percentage may vary from organization to organization or application being tested. These percentages helps in deriving the test cases count itself.
Point 3:
The above two table provides you required inputs to calculate the development efforts against resource capability.
Here, given 5% as Buffer that includes technical
challenges only. The Total Days required here is purely the development
efforts. Additionally, you have to include the efforts required for the other
automation activities listed below.
NOTE: Similarly, include
the additional activities while doing calculation efforts for execution effort
estimation as well.
Calculation for your understanding
Complex (Hrs.) = Test Cases Count [12] * Complex (Hrs.) [ 4 ]
= 48 (Complex (Hrs.)) for Expert Resource
A Typical Automation
contains the following activities – Work Breakdown Structure (WBS)
1.
Feasibility Study
(Analysis) – Here executing the manual test case at least once and verify the
data management is easy and have no dependencies.
2.
Framework design and
development
3.
Script Development
4.
Integration of
re-usable components
5.
Self execution / Dry
Run – to make sure functionality is working fine
6.
Code Review
7.
Execution Review
8.
Sign – Off
Benefits:
- Now we are able to
communicate facts clearly with proper estimation model.
- Able to track and monitor
the activities well.
- Finishing the activities
on time with transparency.
- Able to display clearly and confidently to the Client if there is any deviation from the given timelines well in advance.
Summary:
Initially, identify the feasible test cases for automation. Understand
the project requirement correctly and map with the estimation experience that
you have and with the existing techniques that are available. Create your own
estimation framework that makes you comfortable in tracking and monitoring the
activities.
If we encounter any issues due to any cause in between the project
execution, you can communicate and give clarity on the delays to direct &
indirect stakeholders with estimation deviations with valid evidences.
This will help you to improve the existing approach and
gives add value to the customer in the long run too.
Congratulation Vasu.
ReplyDeleteThanks a lot
DeleteNeatly explained
ReplyDeleteThanks
DeleteThis comment has been removed by the author.
ReplyDeleteVery nice explanation with simple and general terminology. Thanks for provide nice article.
ReplyDelete