How we allocate our engineering efforts is arguably the most important component of success for an organization, especially in the current era:

• Work in the post-covid world is distributed, and engineers have come to expect a higher degree of autonomy.

• Resources are scarce and the ability to hire are limited when the economic context is less favorable.

• Separate product and engineering functions is the de-facto standard for modern organizations, and that collaboration is key to success.

Yet, measuring the allocation of engineering efforts remains a challenge many organizations struggle with. Our discussions with engineering leaders have taught us enough about engineering management practices to give an honest opinion on the various approaches to measuring engineering investments we've encountered on the field.

Traditional approaches

We evaluate several approaches to measuring engineering investments. Each approach is rated (1, 2, 3) on several axes:

• Developer experience: how the solution aligns with the developers best interests.

• Data quality: how exhaustive and how accurate the solution is at measuring engineering investments.

• Report quality: whether the solution produces flexible and readily-usable management reports.

• Auditability: whether the solution allows to trace back investment buckets to individual work items.

Linking commits to tickets

A single ticketing system is adopted throughout the organization, and management requests for commits to be linked to a ticket. For example, the organization may adopt JIRA globally, and expect GitHub pull requests titles to contain a JIRA ticket reference. The goal is to make the ticketing system the source of truth for engineering work, making the data it contains appropriate to measure engineering investments.

This approach is by far the most common. It usually hits a limit as organizations reach several hundred developers or grow through acquisition, as agreeing on a single platform and process becomes difficult. However, a few tech giants still manage to apply this approach at scale (e.g., Apple with Radar).

Enforcing tickets for every commit

This is a variation of the first approach (linking commits to tickets) where a ticket reference is strictly enforced, for example by preventing GitHub pull requests being merged unless a JIRA ticket is linked. The goal is here again to make the ticketing system the source of truth for engineering work. The additional constraint aims to ensure that the coverage of engineering efforts through tickets is exhaustive, therefore mitigating the "data quality" weakness of the first approach.

This approach is most often seen in traditional companies where developer experience isn’t a primary concern, such as heavily regulated environments (e.g., investment banking, healthcare) where process overhead is naturally high. It however also found its place in some high-growth tech-companies with the hope of addressing the data quality issues of the first approach.

Manual reporting in spreadsheets

The organizations adopts a reporting format entirely disconnected from the development flow, such as a spreadsheet to be filled on a weekly basis. We have seen both variations where managers are responsible for filling the spreadsheet on a weekly basis, and others where IC would do this declaration.

In our experience, this approach is most often driven by the finance department’s need of measuring R&D expenditures for tax purposes, than by the engineering leadership’s need for visibility (as the data is often too unreliable to be used for steering purposes).

Measuring allocation with Echoes

It's naturally not surprise that Echoes shines in comparison with the traditional approaches covered here, as it was specifically designed to address their drawbacks. Echoes measures engineering allocation without the need to enforce a single way of working, and without compromising on developer experience.

Want to see Echoes in action? Give us a try or book a demo!