Methodology

Reading Domain Expertise Signals Without a Master Skills Database

By Crewpath Team  · 

Domain expertise signal extraction concept

Most professional services firms do not have a clean, current, well-maintained skills database. They may have something: a LinkedIn-connected profile system, a manually-updated CV repository, a self-reported competency matrix that was last touched during the most recent performance review cycle. But a skills database in the sense of a structured, consistently tagged, regularly validated record of each consultant's capabilities and experience depth — that is uncommon outside of the largest professional services organizations, and even they often have patchy data in practice.

Building a master skills database from scratch, as a prerequisite for deploying an allocation scoring model, is a project that tends to die before completion. The data entry burden is high, the maintenance discipline required is significant, and the result is a structured asset that depreciates quickly as consultants complete new projects and the database falls behind reality.

There is a better source for domain expertise signals than a self-reported skills database: the project history records that every PSA platform generates automatically.

Why project history outperforms self-reported skills

Self-reported skills have several well-known reliability problems. Consultants tend to over-claim expertise in areas where they have limited experience, particularly when they are motivated by career development considerations. They tend to under-claim in areas where they have deep expertise that does not feel notable to them. The rating scales are typically ordinal (1-5, beginner-expert) but are applied inconsistently across the organization. And the data ages immediately upon entry — a skills profile completed in March does not reflect the expertise gained in an April-to-August engagement.

Project history records are different. They are not self-reported; they are generated by the completion of actual work. They are time-stamped, so their recency is inherent in the data. And they capture the full depth of a consultant's engagement history, not their self-assessment of that history. The limitation is that raw project history — a list of project names, clients, and dates — is not a skills signal until it is classified against a consistent taxonomy of engagement types.

The engagement taxonomy as the enabling structure

The classification step is where the value is created. An engagement taxonomy is a structured list of consulting work types that can be applied consistently across all project records: M&A integration, operational restructuring, post-merger integration, cost reduction program, financial modeling and forecasting, change management and organization design, technology implementation and PMO, regulatory response, and so on. Each completed project in the PSA system gets tagged to the relevant taxonomy category or categories.

Once tagged, the project history becomes a structured expertise signal. A consultant with six tagged engagements in "M&A integration" over the last 36 months has demonstrably more domain expertise in that category than a consultant with one engagement 28 months ago. The model can calculate this systematically across the entire roster against any incoming engagement brief — in a way that self-reported skills ratings cannot.

Crewpath ships with a default taxonomy of 28 engagement types that maps well to generalist management consulting, advisory, and operational consulting practices. For firms with specialized service lines, the taxonomy is configurable. The implementation step for most firms is a classification exercise: reviewing the last 36 months of project records in the PSA, assigning taxonomy tags to each, and establishing a process for tagging new projects when they are created. For a 100-consultant firm with 3-5 active projects per consultant per year, this is typically a two-to-three week effort, not a multi-month data project.

Certification records as the supplementary signal

Formal certifications and qualifications provide a secondary expertise signal that complements project history. The value of a certification signal is strongest in areas where the certification is a meaningful proxy for domain knowledge: a certified public accountant designation on a financial restructuring engagement, a Project Management Professional certification on a program management engagement, a CIPP/US designation on a data privacy and compliance engagement. In these cases, the certification reduces uncertainty about domain competence in a way that a single project tag does not.

The certification signal is treated as additive rather than substitutive. A consultant with recent relevant project history and a relevant certification scores higher than one with only the project history. A consultant with a certification and no recent relevant project experience scores lower than one with recent project history and no certification. The reason is that project history is a direct record of recent work; a certification validates knowledge at a point in time but doesn't confirm that the knowledge is being actively applied.

Handling sparse and uneven data

New consultants have short project histories. Consultants who have moved between firms bring history that may not be in the PSA system. Consultants who worked primarily in practice areas that weren't well-tracked in older versions of the firm's PSA setup may have incomplete records for work done three or four years ago. Sparse data is a real and normal condition for a significant portion of most firms' rosters.

The model handles sparse data conservatively: limited project history contributes low signal rather than negative signal. A consultant with two years of project history scores as low-certainty on domain expertise, not as incompetent. This is the appropriate treatment: the model does not know what a consultant without project records cannot do. It knows what a consultant with a rich project record has done. The distinction matters for a firm that wants to avoid systematically disadvantaging newer consultants in the allocation process.

The data quality feedback loop

The retroactive pilot — running the scoring model against 12 to 24 months of historical engagement decisions — is valuable partly as a scoring exercise and partly as a data quality diagnostic. Firms consistently discover, when they first run the scoring pass, that their project records are tagged inconsistently across practices, that certain engagement types are systematically under-represented in the taxonomy, and that some consultants' project histories have significant gaps because historical projects were logged with minimal metadata. The pilot surfaces these gaps in a context where fixing them has immediate operational value rather than being an abstract data hygiene exercise.