agile top banner

Metrics Management In Automation Projects-Deliver Value Endlessly In Your Agile Projects

Read it in 3 Mins

Last updated on
31st May, 2022
Published
26th Oct, 2018
Views
4,146
Metrics Management In Automation Projects-Deliver Value Endlessly In Your Agile Projects

Introduction

Projects have been executed for decades for many reasons including customer requirements, technological advancements, and compliance requirements etc. The success of the projects has been driven mostly by conformance to plan for plan-driven projects and the value of delivery for the Agile projects.

There are many factors that attribute to the success of the projects and had helped project managers steer projects in the right directions, take corrective and preventive actions.

Metrics is one of the most important aspects of project management which can assess if your existing project or program is doing enough to justify your existence. A metric, by definition, is any type of measurement used to measure some quantifiable component of performance. A metric can be collected through observation, such as delay in days, or number of defects; or the metric can be derived from directly observable quantities, such as defects per “x” lines of code, a cost performance index (CPI), or a schedule performance index (SPI) Metric is also called as an indicator, or a key performance indicator (KPI). We will see how metrics can manage automation projects effectively to realize its goals and objectives.

 
Why Automation? 

Why Automation?

We’ve witnessed so far application or product being built to solve a customers’ problem. Most of the applications including e-commerce are now being based on human-centered design or user-centric. These processes were in place and able to sustain when business productivity
trumped technology productivity.

Technology took mainstream lately and enterprises started identifying opportunities for automation to cut costs and invest more in the development of new business to grow their top line. RPA product companies took advantage of these opportunities and came up with new age automation suites to solve clients needs thereby saving millions of dollars in costs.


What to Automate?
Rule-based Automation
Rule-based Automation

There are applications or systems that involved repetitive actions from users. For example. invoice processing, purchase order approvals, invoice reconciliation etc. These opportunities became the low hanging fruits for the enterprises to invest on. If you see really, these are rule-based systems that work based on if-else conditions.

The rule-based automation products came to limelight in recent years and demonstrated the possibility of automating these user interactions with the system and in fact reduced the error or exception rates drastically down.

Intelligent Process Automation
Intelligent Process Automation

There are also applications that would require high-level behavior to do automation. For exp- to approve an invoice, the automation system should be able to read different formats of a document like pdf, doc, tiff, etc. It has to identify different fields in the invoice document like PO number, Vendor Name, Line item etc. But a lot of the times, these documents will not be structurally formatted and emphasizes the need for some kind of intelligence. This is where some NLP based systems can identify different structures, formats to effectively automate these systems.

 
Metrics for Improving the success of Automated Projects 

Every organization has its own approach for managing automation projects and have come up with assessments and frameworks to deliver benefits to their clients on their automation journey. There are multiple phases associated with the assessment framework and the below framework can help any automation projects reach its goal and measure the maturity level.
Metrics for Improving the success of Automated Projects

The phases are,

  • Kick-off
    • This is where the what part of the automation use cases are defined. Risks are identified along with mitigation plans.
  • Ideation 
    • The use cases for automation are brainstormed for BOTs development.
  • Assessment
    • Use cases are finalized based on different techniques.
  • Configuration
    • Architecture is finalized along with dependencies with related components.
  • Release & Transition
    • BOTs are certified, deployed and marked for the transition to the operations team.
  • Monitor & Support
    • Monitor logs, failures, exceptions and get customer feedback on implementation

  • Benefits Realization
    • ROI, Productivity improvement, FTE reductions details are gathered

How to measure Automation success in projects?

Metrics is one of the most important aspects of project management. Project managers use the metrics to track the project progress against the plan. It serves as a navigation system for them and can be used to drive towards the project goal.

The metrics can aid project managers to identify pitfalls and shortcomings ahead of time thereby providing enough time for the project managers to make proactive and corrective measures.

There are generally 6 factors that managers generally measure to create metrics that determine project success. Let’s see those metrics for implementing the Automated projects.

Metrics for implementing the Automated projects.

  • Value delivered to the client.
  • Time/Schedule to deliver project deliverables. (under / on / over)
  • Cost to deliver project (under / on / over)
  • Scope of work to be delivered. (addition / deletion)
  • Quality of deliverables and of the process. (Benchmarks met if any)
  • Risks to project success (Mitigation / Contingency plans)

Automation metrics are no different than traditional project metrics but have few variations on what metrics needs to be tracked to effectively manage the projects and deliver the desired outcomes.

Metric 
Operation Definition 
Outcomes 
Man to BOT RatioNumber of Bots VS Number
    of FTE in the  Team
Time of  Market,TCO
% of reusable components(Number of components reused/Total number of components)*100Time of market
% reduction in FTE(Number of FTE  released due to BOTS/Number of FTEs before  the BOTS release)*100Time of market
Thoughput(Time is taken before automation-time taken after automation/time taken before automation)*100Time of market
ROI
Total cost saving/ overall cost of implementation
TCO
 BOT resolutionNumber of tickets resolved byBOTS/Overall number of tickets
Time of market
Cycle timeAverage time to implement a bot(from requirement to production)Time of market

Automation metrics can be chosen based on the type of the projects being executed as listed above and are applicable to RPA/IPA projects as well. The major factors to be considered while executing the automation projects are,

  • What problem is your BOT solving for the customer?
  • Have you worked out the cost benefit for the BOT?
  • What is the right RPA tool for your business need?
  • Have you considered the scalability of BOTs, ease of implementation and robustness?

Conclusion

Automation is here to stay and will continue to evolve. McKinsey report says nearly half of the activities that people do can be automated theoretically using current technologies. As we progress to foreseeable future, more and more projects will embrace automation leaving humans to solve the most complex problems.

The metrics management approach and defined a set of metrics may be suitable for a particular automation team and not suitable for another team depending on the nature of the automation, but it has worked out pretty good for me so far.

Profile

Ramkumar Armugam

Blog Author

Ramkumar is an experienced Program Manager with 13+ years of success in leading all phases of diverse technology IT Projects in retail, e-commerce, insurance and pharma market research industries. He has more than 7+ years of experience in leading and executing projects and programs using agile and lean methodologies. He is currently working as Senior Manager in Cognizant Technology Solutions India Pvt Ltd and holds multiple certifications including PMP, PMI-ACP, CSM, CSPO, CSP and ICP-ACC. He has a zeal for project and program management and his current endeavor includes leading a large scale distributed product development team in delivering a world class product features in the area of Finance and HR domains for a large US retailer. He is a regular contributor to projectmanagement.com.