How to assess operations - part 1

Introduction

Business transformation is a process where an organisation changes from an existing to a target operational state. Organisations spend a significant amount of effort identifying these improved target states. Corporate strategy drives this at macro level while team, department, function and business area managers are responsible for the execution at a lower level.

Whether identifying, assessing, initiating or executing transformation it's critical to have a good understanding of the current operational state. This is the baseline against which everything else flows; feasibility, benefits, effort, time, cost, risk and more.

The current operational state can be investigated, documented and discussed by carrying out an operational assessment. This is an activity which is relevant to all forms of business transformation. An operational assessment can take many forms in terms of scope, approach and level of detail. This depends upon the nature of the transformation in question.

This article is a discussion of operational assessments and how they apply to different organisations and transformation programs. I've put this together based on personal experience as an analyst, consultant and project manager, however every organisation and every project is different. Some viewpoints may not be relevant in some situations, but hopefully still provide a useful basis for consideration.

To plan and execute an operational assessment in a quality way we need to be clear on the purpose of assessment, the structure or organisations, the way assessments vary for different transformation types and different operational areas. The discussion is organised in the following way:

For clarity, examples of business transformation programs are:

• The launch of a new product or service
• Entering a new market
• A process change
• Shift of operations to new locations
• A new IT system or an upgrade or change to an existing system
• Work process or technology outsourcing.

And examples of affected operational areas are:

• Corporate functions
• Operations (industry specific e.g. manufacturing, equities management)
• Research and development
• Marketing
• Finance
• Human resources
• Facilities
• Legal.

We will define these in more detail later.

Operational assessments can occur at different levels of detail. Small scale assessments may occur as part of daily operations. Events such as encountering issues, reviewing results or holding team meetings may lead to the documentation of an idea for transformation in the form of a simple document, e-mail or support request. This could be considered as an initial assessment of a business situation. Usually detail is minimal at this stage; only what is required to consider if further investigation is merited.

Small scale assessments can lead to more formalised assessments with the objective to validate the opportunity or problem in detail and capture baseline factors such as feasibility, benefit level and effort estimates. As more detail becomes available a decision may be taken to transform the idea into a formalised change request, project or program. This progression into an executional phase often starts with another form of assessment such as 'requirements gathering' or 'as is' analysis.

So, assessments can happen at different levels of details throughout the transformation process. Consider a generalised flow from idea to execution:

1. A need for change is identified with a 'mini assessment', for example:
   • During daily operations; an issue is encountered or opportunity discovered
   • As part of the strategy or planning process
   • As part of the execution of an existing transformation program
2. A more formalised and detailed assessment is initiated to assess feasibility, benefit and cost
   • This can help create a baseline for change
   • And a business case for change
3. If approved, a transformation program may be initiated, which may have in depth assessments such as:
   • 'as is' analysis, requirements gathering
   • Design, build, deploy etc.

There tends to be more formalisation around step 3; the execution of business transformation programs, than steps 1 and 2; the idea generation through to approval.

There are benefits to put more formality around the `idea to transformation project initiation process. I'd recommend tracking this in a centralised way:

• Create an idea management process
• Define the minimum requirements of what should be assessed to back-up an idea
• Include stage gates with formalised criteria to progress to next stages
• Capture key metrics (number of ideas, source etc.)
• Evaluate transformation program results against the original benefits associated with the idea.

This should improve the quality of ideas that progress to transformation programs and the likelihood of success of those.

The scale of assessment

At the simpler end of transformation the scale of an assessment can take the form of a detailed description of a business issue. This forms the basis of a case for change. Consider the following examples:

• A need to hold more stock could lead to a contract tender for warehouse space
• A need to for a report to meet a new statutory reporting requirement could lead to a systems development.

At the more complex end of the scale an assessment may need to analyse a comprehensive range of factors in depth. This could include external factors such as the economic forecast, market forecast and competitor analysis as well as a internal factors such as a detailed operational analysis. Consider the following examples:

• A mid or long-term planning exercise such as annual planning (external and internal factors)
• An investigation into potential acquisitions (external and internal factors)
• A shift into a new product segment or a new market (external and internal factors)
• A major process and technology program such as new business systems (more weighted towards internal factors).

The need for assessment

Learning from past transformation projects

Looking back at the issues and problems I've experienced while working on transformation programs, many were at least in part caused by an incorrect understanding of the current state. This can be traced back to either a lack of assessment or a poor quality assessment.

When encountering an issue during a change program there is a tendency to blame factors such as the methodology, the team capability, and the resource availability. However, while these are often contributing factors, the root cause is often a mistake or underestimation of scope including the work required to deal with that scope. This is a fundamental error in defining the benefits of transformation without due diligence on identifying the true 'volume' of transformation required to achieve those.

The project management institute defines key project management dimensions of cost, time, scope, quality, benefits and risk. Experienced project managers are well versed in the challenges of balancing these. However, projects are often in trouble from day one. The errors made in scoping prior to project initiation lead to unrealistic timelines, incorrect effort estimates and in the worst case project objectives which are not feasible. These issues also mean the project teams are not correctly sized or lack required capabilities. Often it's not only the case that cost, time, scope, quality, benefit and risk are mismanaged, it's the case that they are not correctly sized at the outset.

When it comes to a lack of assessment or a poor quality assessment some situations I commonly see include:

• A lack of any form of assessment of the current state. The project may have a clear target state, but no feasible or well planned path to get there
• An assessment was carried out but without the correct scope and depth
• An assessment was carried out, but was executed without the right expertise, leading to incorrect assumptions and misunderstandings
• As part of the assessment the feasibility of the target state was not validated. This would be more common on unique or so called 'bleeding edge' transformations when there is little experience or data to validate the target state benefits.

It's useful to consider why assessments are often incomplete and develop strategies to mitigate these. Some of the factors I've observed are:

• A tendency to want to get on with executing the change. The assessment is rushed through with minimal deliberation. Stakeholders may have already made up their mind.
  • Bias can be a big contributing factor here
• Financial factors such as the need to complete the change within the current fiscal years budget
• Overconfidence with the level of understanding of the current situation
• A lack of authority within the project team or project stakeholders, this may happen in transformations with broad scope that touch areas outside of the main sponsors area. In this case it may be difficult to access information and expertise to assess the full scope of impacts.

Challenges maintaining depth of knowledge

Managers have to balance time between managing strategy and people as well overseeing their operational areas. As managers take on more responsibilities it becomes difficult to retain a depth of operational knowledge.

Consider that operations are divided into business units and functions. These then break down into departments and teams which run processes. Each process may involve human actions, data, systems, reports and controls. They may also be affected by issues, problems, risks and other transformations. The level of detail to consider when analysing an operation is extensive.

The more responsibility a manager takes the harder it is for them to maintain depth of knowledge on the current operations. Managers risk using out of date knowledge or making incorrect assumptions when they are involved in the identification and planning of business transformations. I've observed that managers have a tendency to underestimate the effort to analyse, design and build solutions in complex areas. A formalised assessment process with the structured involvement of experts can help to mitigate this risk.

Poor knowledge management

A common issue organisations face is a lack of documentation or poor quality documentation. Creating and maintaining documentation can be challenging. Often this work is avoided. Even when good quality documentation is created it may not be kept up to date.

A lack of documentation compounds the difficulties that managers face staying up to date on detail. It also makes it hard to onboard new employees or utilise consultants or contractors. Furthermore it represents a control and audit risk. As time passes a lack of documentation usually leads to poor or inconsistent knowledge transfer. This is a major source of generating process variation.

An organisational issue that arises due to poor knowledge management is the creation of the 'lone wolf' expert. Teams may end up with individuals who keep knowledge to themselves. This can lead to process bottlenecks and business continuity risks. These individuals can also become major barriers to transformation. This can be difficult for managers to deal with.

A lack of knowledge management and documentation should be considered as a high risk for transformation. If the current operational situation is not well understood then the volume of change to achieve a transformations objectives cannot be well defined. During transformation programs it's very common for scope increases to occur due to discovering new processes, new issues or hidden data and systems complexities.

Misleading reports

Leaders and managers review and discuss a range of reports as part of the decision making process. These reports contain financial and operational metrics. While they are useful, they can only convey part of what is happening in operations. For example a report showing manufacturing variances per cost center may give a good overview of manufacturing performance, but it will rarely explain why performance varies. It's no replacement for observing the process in the factory.

One mechanism which helps is the preparation of accompanying commentary by experts. However it's not feasible to prepare commentary covering everything, neither is it possible to involve deep level experts in all decision making processes.

Bias

Managers have a tendency to make decisions with bias towards their previous experiences. For example a manager who has experienced successful outsourcing may have a tendency to suggest outsourcing as a solution when they move to a new organisation. Another manager who had a great experience with a certain brand of IT system may suggest that as a solution.

Managers vary in how they focus on key operational attributes such as strategy, people, process, data and systems. This is one reason why having a diverse management team helps maintain a balanced view.

The business book genre contains books which explain a range of different approaches as the key to success. This is in part due to a tendency to try to rationalise success after the fact from a biased or individual viewpoint.

Triggers for assessment

In order to identify and plan for operations assessment it's useful to consider the different triggers for business transformation.

Daily business monitoring may uncover an acute business issue or opportunity. Within the technology space there are formalised processes that help to manage this. Frameworks such as the IT infrastructure library (ITIL) describe how issues may be linked to more fundamental underlying problems which can lead to change requests and transformation projects. Within the process space approaches such as 'Lean' promote the idea of actively looking for inefficiencies at every level in an organisation. Lean organisations will often have a form of improvement request log which may lead to transformation programs. Effective teams often have a form of daily or weekly review which can act as a source to identify transformation opportunities.

Periodic performance reviews also lead to the identification of transformation opportunities. I'd consider these reviews to occur in three broad categories:

• Periodic financial review led by financial accounting with focus on financial performance via the balance sheet, profit and loss, cash flow and other more detailed reports
• Periodic business review led by management accounting/business leads with focus on key business metrics such as sales, costs, profitability etc. by management dimensions
• Operational reviews at the business area or functional level with detailed reviews of operational metrics. These take the form of separate reviews for each area including for example human resources, marketing, manufacturing etc.

Reviews usually occur on a weekly, monthly, quarterly and annual basis. The aforementioned change and opportunity logs and other 'idea' trackers may form a part of the standard agenda. Reviews are a key part of the decision making process on whether to investigate opportunities further. Given that it's important that any small-scale or basic assessment work that forms the basis for discussion is of an appropriate level of detail and quality.

If an organisation decides to centrally track ideas as part of transformation management it may be useful to identify the source of those ideas i.e which review process, meeting or activity they originated in as well as which team, individual etc.

Annual planning

The annual planning process is a major trigger for transformation programs. This is where the executive committee come together to discuss and make decisions about the future of the business. Ideas are translated into targets, which are then used to create detailed plans and budgets. Achieving the plans may mean identifying and initiating business transformation.

The approach to strategy and planning varies by organisation, usually there are detailed targets for a year ahead, higher level targets for a longer time-frame, say 3-5 years, and at the minimum directional statements for the longer term.

The annual planning process relies on three main inputs:

• An assessment of the current business situation including factors such as current performance, issues, risks and opportunities. This information may be gathered as:
  • Metrics from standard reports
  • Commentary from key experts and managers
  • Workshops, meetings and other discussions
  • One time analysis of specific topics
• An assessment of the external situation including factors such as economic forecast, market data and competitor insights
• Input from third party experts (auditors, consultants, contractors, research agencies etc.)

Gathering and processing these inputs is in itself a form of assessment. The quality of this assessment is critical as it provides the information executive members will base their decisions on.

Assessing the current situation is a complicated process. Individual teams, departments, functions and business areas must create management summaries which are combined and consolidated for the executive review. The following information is used:

• Business and operational performance metrics
• Scope of operational activities (i.e. processes)
• Organisation structure (i.e. org chart, key roles)
• Key systems (and state of the systems)
• Key data (and state of the data)
• Key controls and risks
• Current issues and problems.

Consider for example that a sales team manager may need to summarise sales performance for a regional manager who summarises for a national manager who then summarises for the head of sales. The head of sales is then responsible for representing sales within the executive committee discussions. They must present sales data including considerations for supporting areas such as IT, HR etc. as they relate to sales. This process will happen for all operational areas.

Within the annual planning process there is usually a cycle of assessment, discussion and feedback. In some organisations annual planning can take as long as six months, and involve multiple rounds of draft targets and plans. This is in part due to the interconnected nature of an organisation and how a change to a budget or transformation in one area may affect other areas.

In addition to the performance discussion there may be special topics such as:

• A deeper than normal analysis of a specific product or services
• A deeper analysis of a specific competitor
• A study into the impact of a location change
• A study into the impact of hiring or lay-offs.

When special topics relate to external areas or new business activities organisations often engage contractors, consultants or research agencies to help.

As targets become firm, each operational area must consider and plan how they will achieve the targets. This may result in the need to carry out a deeper assessment of the current operation. This is usually a step down into the next level of detail. The assessment will vary depending on the target:

• In the case of target cost reduction we may assess key performance indicators related to process volumes, errors and re-work
• In the case of target revenue growth we may assess product and customer performance, competitors and the market.

Employees and managers should keep in mind that whenever they report information they are taking part in the 'assessment' process that forms part of the strategy and planning process.

Types of assessment

Moving beyond the planning process the scope and approach for assessments will vary by factors such as:

• Which business areas are in scope of a transformation?
• What is the nature of the transformation?
• What is the target state of the transformed business?

These and other factors inform the type of assessment needed. Assessments will vary by factors such as:

• The resources and time required
• The information to access; such as data extracts, reports, presentations and other documents
• The people that need to be involved and the method to involve them such as interviews and workshops
• The level of detail required.

I would propose to classify assessments into two broad categories; those that investigate ideas prior to the initiation of formalised transformation projects are and those that form part of formalised transformation projects.

Prior to project initiation

After a potential transformation is identified the first assessment captures information needed to build a case for change and a project proposal.

It's this first assessment that can lack formality and depth which may leads to issues further down the line when trying to execute business transformation.

A case for change centres around financial investment required, benefits and the payback period. To build this financial case we need to assess key factors such as scope and approach which help to build a picture of the resources and time required. It's also important to clearly identify assumptions made and the main risks and opportunities. In addition to informing the financial case these are important feasibility considerations.

A project proposal accompanies a case for change and adds further detail on how the change can be executed. The information required in a proposal varies by the magnitude and type of change, but typically would include factors such as project plan, scope, team roles and other factors that are important from a project management perspective. Together with the financial case this allows stakeholders to consider everything that is needed to initiate a transformation project.

After project initiation

Transformation projects will usually start with a detailed assessment immediately after the project initiation. These assessment phases often have other names such as requirements gathering or 'as is' analysis. In the case of agile projects this may be done in parallel with design and build.

This assessment is at a lower level of detail and is aimed at uncovering the information needed to design, build and implement the transformation. The focus will vary with the type of transformation, some examples:

• Implementation of new business systems may have a requirements gathering phase focussing on existing processes, data and systems
• Work process outsourcing may have an analysis phase focussing on documenting processes, training and work-shadowing
• Process re-engineering may have a phase focussing on process mapping of existing processes
• Investigation into potential acquisition targets may focus on profiling of the target acquisitions
• A capability building initiative may focus on skills analysis.

Let's discuss a few of these in more detail.

New or upgraded business systems

Business systems enable processes by providing a level of automation of information capture and activity execution while also providing measures as a function of reporting. For example:

• Production planning systems manage the planning process, capture planning data and provide plan reports
• Equity systems such as online trading systems execute trades, hold trade position information and provide equity reporting
• Accounting systems capture financial transactions and generate financial and management reports.

Projects to implement new or upgraded business systems are difficult. They rely on a good understanding of business processes, business decisions making, organisation roles, data and technology. These projects often encounter problems due to failures of assessment, two common cases are:

• New systems are designed based on current processes without assessing the current and future suitability of those processes. An opportunity to resolve issues and workarounds and take a step towards good practice may be missed.
• New systems with reporting capabilities are often built to generate the existing reports in the new system. For example several consulting studies I've encountered in the past indicate that a lot of the information provided in reports is not utilised in the decision making process. I've personally encountered this on several projects. Projects which touch on reporting should assess each element of information provided on a report and question it's usage and value.

In both of these cases the requirements are based on the current process, without taking the step to fully assess the value or suitability of it. Projects often identify incremental improvements to the current state based on existing pain-points, but less commonly challenge the value of existing processes and their outputs with a view to a more complete re-design.

This is understandable for a few reasons including:

• A primary output of business systems is the reports that provide information for senior decision makers. The desired content of these reports can drive a lot of the process and data complexity. Unfortunately it can be very difficult for project teams to access senior decision makers. Mid and lower level experts assigned to projects often lack the perspective, authority and confidence to make decisions on changes to the outputs of systems.
• Organisations are often trapped in historical ways of working; executing a process, utilising data, reviewing results, making decisions all in a certain style. It can be difficult to challenge this. An example from finance is the structure of the chart of accounts, a poorly designed chart can create a lot of extra work and lead to poorly structured reports, however a re-design would require involvement of the CFO and all the finance directors of a company as well as many experts both inside and outside of finance.

Not every business systems project should aim for a complete re-design. However, each project should have a formalised assessment to consider the state of the current situation and question carefully the extent to which they want to replicate existing ways of workings or create something new. It's good practice to run a cost / benefit analysis on scope elements or design points of a transformation project. This helps to focus work towards value.

For business systems, the main categories to consider during assessment are:

1. Value and suitability of the current solution:
   • Estimated level of replication vs. re-design required for future solution. This can be considered at a summary level and for each of the individual factors below
2. Data:
   • Input data volume, quality
   • Details of automated or manual data conversions prior to load to the systems
   • Manipulations to the data in system
3. End to end process flows and task level instructions with details including:
   • Level of automation or manual work
   • Process performance metrics
   • Task effort vs. elapsed times
   • Complexity measures
   • Risk measures
   • Issues and workarounds
4. Structure, content and format of outputs:
   • Reports
   • Interfaces
5. User experience, roles and security:
6. Internal controls

When assessing the above it's useful to work through a checklist or questionnaire. The content of these will vary by transformation type and business are. The type of questions to consider include:

1. Process
   • Why does this process exist?
   • Is this process manual or automated
   • Is this process standardised or are there variations?
2. Information - why does this data element exist?
3. Are there any issues, problems or workarounds?
4. Have there been any control exceptions - are suitable controls in place for this activity, data element etc.

Outsourcing and shared services

With outsourcing and shared services the focus is on work migration across organisations. Consider this as shifting work from a sending organisation to a receiving organisation. The assessment here should focus on scope, and state of the work to be migrated. This could also be applied to a shift of work between teams or across locations within the same organisation.

It's critical to assess the process performance and the issues present in the current operation including issues, exceptions, variations, deviations and workarounds. These should be documented as a performance baseline which will be useful in quantifying post migration success. Success should be consistency with or improvement above this baseline.

One common issue with outsourcing and shared service space is unrealistic expectations of immediately improved performance. In some cases organisations considered a move to shared services or outsourcing a failure, where in reality the issue was a shift of broken processes and systems to a new organisation with an expectation of the performance to be improved at reduced cost.

Cost savings may be possible due to labour arbitrage by shifting existing processes to a new organisation. In this case a short term dip in performance should be expected due to challenges with knowledge transfer, and other transition related issues. If there is a desire to improve processes some investment is usually required. Note this is often more cost-efficient in the receiving organisation.

In addition to the baseline analysis, details of the current process are needed to manage the migration of work from the sending to receiving organisation. These details should include:

• Existing process documentation; process flows, task instructions, operating procedures
• Actual process steps & variation from the documentation, this can be identified during interviews and work shadowing
• Organisation chart and roles and responsibilities and the mapping of roles to process steps
• Data related documentation including field standards describing any data elements entered in systems or retrieved in reports
• Systems documentation accompanying the operating procedures and task instructions explaining how the systems work and any relevant settings that operational teams need to be aware of
• Any physical dependencies, such as locations, documents and equipment.

The assessment of these factors can be used to design and build the receiving organisation and how it interacts with the receiving organisation.

Acquisition integration

When a company is considering an acquisition it starts by identifying targets. If potential targets are found the next step is to assess them in more detail. This may be followed by an offer, a period of transacting and contracting and a post acquisition integration.

The assessment for potential acquisition targets is usually focussed on validating the following:

• The scope of the operations of the organisation
• The detail behind the published results
• Quality of plans and forecasts
• Issues, risks and opportunities.

One of the main challenges with this kind of initiative is a lack of access to data. While assessing an acquisition there may be limited access to data and limited communication channels. Sources of information to target include:

• Strategy documents
• Financial and management reports
  • Published annual accounts
  • Internal business reports (cost, profit, cash, assets etc.)
• Organisation model
  • Functions, teams, employees (org charts, policies, role descriptions)
  • Employee list by role with details (location, benefits etc.)
• Process information (flows, operating procedures, issue lists etc.):
• IT architecture and system inventories
• Locations details and physical assets lists
• Any issues or risks which may affect the stability of the operation
• Opportunities.

New product or service launch

In the case of a new product or service assessment will focus on:

• Economic outlook for the relevant markets
• Market outlook for the relevant product or service category
• Competitor analysis for the relevant product or service category
• New operational capabilities required to design, build and deliver the new product or service, for example changes to:
  • Manufacturing, supply chain and distribution
  • Sales teams
• Impact of the new operations on the existing business support functions IT, HR, Finance, Legal etc.

Assessments for new products and services are usually highly tailored to the product or service in question.

Scoping an assessment

When planning an assessment it's useful to think carefully about the desired result of the transformation being considered. This can help inform what to assess. Consider a few examples:

If the objective is to grow revenue or improve profitability focus areas would include:

• Sales and profitability by category, product, region, customer
• External factors; economic outlook, market, competitors analysis
• Product or service costs

If the objective is to reduce cost focus areas may include:

• Policy review:
  • Spending limits
  • Approved suppliers, payment terms and contracts
• In depth process analysis
  • Can work be stopped
  • Can work be harmonised and simplified
  • Can work be re-located or outsourced
• Asset optimisation
  • Facility cost reviews
  • On premise IT reviews.

If the objective is to increase innovation and launch new products focus areas may include:

• Skills and capability reviews
• Recruitment of top talent
• Workplace culture, ways of working, and processes surrounding idea generation, collection and review
• The research and development function structure, budget and effectiveness
• Use of external agencies and consultants.

If the objective is to increase controls after an incident such as fraud, a data leak or incorrect statutory declaration, focus areas may include:

• In depth review of processes and their control steps and reports
• Roles & responsibilities, handovers of information and authority to review and approve
• Systems and data access controls.

In each case consideration of the desired transformation helps to identify the focus areas for assessment.

Interpreting results

It can be difficult to interpret the results of an assessment. Consider an assessment identifies how many manual journals an accounting department posts in a month, how do they gauge whether this is efficient or inefficient? Making comparisons can be useful. A few ways to do this include:

• Connecting with industry, market and professional bodies. Information and insights can be gained from activities such as attending events and reviewing journals
• Utilising research agencies or consultants
• Accessing benchmark data from other sources.

Benchmark data is often used by consultants. I've found both freely available and paid data to be useful in previous projects. Benchmarking can give an idea of the level of potential improvement that may exist without doing a deep analysis of roles, process, data and systems.

For example when assessing the efficiency of the finance function calculated metrics may include:

• The cost of the finance function as a % of revenue
• Average finance salaries
• Process metrics:
  • Number of manual journals posted
  • Number of reversals posted
  • Payment on time rates
  • Number of failed or rejected payments
  • Number of post close-adjustments
  • Number of calls, requests etc.

These can be benchmarked both internally and externally.

Internal benchmarking works well with multi category multinational organisations. Results may be compared by industry, business areas and locations.

In addition to commonly reviewed process metrics other factors such as process design, use of technology, organisation structure and data volumes can also be benchmarked on a similar basis. I have found it useful to look at factors such as number of employees per manager or number of business systems per business area or function, comparing these to what I've seen in organisations that perform well can give me a rough idea of how organisations differ and provide clues on where to assess in more detail.

When it comes to benchmarking it's important to adjust expectations based on the overall business situation. An organisation with a history of business issues, old systems and no process standardisation will rarely be able to transform isolated processes to reach a level in-line with a top performing organisation. This would require a broader transformation of culture, policy, processes and systems.