Engineering as a black box?

When starting at JobTeaser, the new CTO remembers being welcomed several times by this remark: “Very happy to meet you, unfortunately we won’t get to interact much as you work in Tech, and Tech is a black box”. It appeared that collaboration with the rest of the company was not so developed. Before long, it was clear that developing interactions and maximizing tech impact on business were the first challenges of the job. How to rebuild trust? 

Bring transparency into tech

What is the engineering department doing? Are they fast enough? 

The best way to overcome what the other departments perceived as opacity of tech was to bring data & facts to the table, to help them understand how tech is working.

This is why engineering leadership at JobTeaser wanted to measure how efforts were allocated on the multiple priorities defined by the company, and be able to communicate on it very easily. Displaying allocation of efforts so it can be understood by everyone, and give the possibility to dig into details, allows: 

• To check how engineering resources are invested, and ensure that the impact on business is maximized,

• To communicate on engineering teams’ advancement on monthly, quarterly and yearly basis, to the attention of the whole organization,

• To ensure that prioritization is respected by everyone, and that teams efforts are not spread on new priorities or shadow work like technical debt,

• To follow teams’ advancement week after week, and dig into difficulties when needed, with the help of DORA metrics, for instance.

These factors all together helped to better understand, and thus improve teams’ velocity, which was a critical issue to both tech and product departments. 

JT engineering leadership chose Echoes to answer these challenges, after an extensive market survey. They chose Echoes for its usability: information is straightforward, user interfaces are easy to understand for everyone, incl. Leadership and not only IT. And last but not least, JT chose Echoes as it is resilient to differentiation between squads organizations, such as JIRA processes.

How to customize Echoes, to match JobTeaser reporting structure?

JobTeaser has the singularity to have 3 different kinds of customers with different context & problems: students, universities and recruiters. Therefore the company has to face multiple challenges, and thus priorities. 

At JobTeaser, Echoes was configured in order to follow efforts invested in 200 initiatives, distributed in several pillars. 

So engineering leadership invested time in testing Echoes configuration, in order to map their reporting structure in Echoes. Today, they are following the allocation of engineering efforts distributed on their four pillars, and are able to drill-down into the details of each pillar, to the initiative level and to each action that is composing the initiative. At each level of this structure, they can visualize the allocation of efforts between what would have an impact on final customers, maintenance & risk mitigation.

Echoes to follow product teams advancement, and keep an eye on priorities

Echoes is used at different level of engineering leadership: 

By the CTO 

• Uses data to aggregate information on huge dashboard to give an overview of advancement on JT initiatives

• Uses Echoes to deep dive into a specific topic or problems

• Granularity is very valuable for exec team: it allows them to get the big picture of the engineering organization advancement, and also to dig into details on one particular topic when needed

By the Tribe engineering leads

• Extract data for monthly, quarterly or yearly reports,

• Extract data to feed notion pages, dedicated to each initiatives with context, description, involved team, status, effort allocation, lead time,

• Check on a weekly basis, to ensure that teams are on the right track.

• Use sometimes Echoes as a starter for 1:1 meetings.

By the engineering managers 

• Check on teams progress each week, through alignment & DORA metrics

How to break the black box, and become an outcome-first organization

Engineering impact on business is now visible, and trackable.

At corporate level

• A huge dashboard where all initiatives are gathered, with collected information as “on-track/off-track”, who is contributing, and time invested. This dashboard is updated and displayed each quarter.

• Around 200 initiatives are also listed in Notion, so anyone in the company can follow the status of each initiative, in order to avoid meetings. Notion gathered context, description, addressed persona, expected outcomes etc. 

At engineering level

• Echoes leverages GitHub labels to express the purpose of work, directly on GitHub pull requests as part of the traditional development flow. This contributes to putting the “why” of every code change at the forefront.

• With the Slack integration, Echoes sends a weekly report of how engineering work was allocated across company outcomes directly on each team’s channel. This helps the engineering team take a step back and reflect on their priority choices, without ever needing to go to Echoes interface itself.

Perceived benefits

The CTO’s point of view

• Echoes helps visualize where efforts and time are going 

• Echoes helped value engineering efforts at JobTeaser.

• Echoes helped shine a light on shadow work. CTO learned a lot about shadow work by digging gathered information around platform instabilities, recurring incidents.. But also unexpected reprioritization. 

The Tribe leaders’ point of view

• Echoes help to rationalize and explain something that is not obvious to non-tech people

• “I am happy to visualize where I am, and if I am heading in the right direction”

• “I can visualize how my teams are improving”

• “I am able to differentiate team performance from project & context. I have the right estimate of a team workload, or nb of priorities.”

• “We get visibility that we can’t have through perception, or feeling etc. we normalize what is done, now we get a reference, there’s a context, teams are analyzed always with the same parameters.”

The stakeholders’ point of view

• JobTeaser CMO stated that Echoes greatly helped to follow-up and challenge the past allocation of resources, which was previously not feasible and most importantly to realign and refocus on strategic priorities and revenues, defined at company level.

What's next?

• To go further on transparency, Engineering leaders want to enrich the 200 initiatives listed in Notion, with data extracted from Echoes as status, percentage of efforts invested by each team on the initiative, or DORA metrics. The objective here is to maximize visibility on engineering efforts to the attention of the whole company. (work in progress)

  
  

• To learn more on their effort allocation, JobTeaser is considering going further with Echoes configuration, driven by a push from the COO, to report engineering activity across more axes than the goal alone (e.g., persona, country, ...).

  
  

• For the time being, only a few PMs are using Echoes. Some engineering leaders think that sharing Echoes information with product managers would benefit product teams, and help the organization be more impact-oriented.