Why salary surveys are an unreliable source for competitive pay
A lot can happen in a year.
That’s as true in the realm of talent and compensation as in any other part of life.
Which is one of many reasons that annual salary surveys are an unreliable source for the market insights needed to ensure fair and competitive pay.
This article will dive into this data reliability issue in more detail (as well as all of the other problems with traditional salary surveys as a data source) and explore alternative options for salary benchmarking data which offer increased accuracy.
Here’s a brief run-down of what will be covered:
- What are salary surveys?
- Why do companies use salary surveys?
- Who are the main salary survey providers?
- The 6 key limitations of salary surveys
- How modern benchmarking software overcomes these limitations
- What about using Glassdoor or LinkedIn job adverts?
- Can you use both salary surveys and benchmarking software together?
What are salary surveys?
Salary surveys are the traditional format for benchmarking data, giving HR teams insight into what compensation looks like across different job roles, levels, locations, industries, and so on.
The benchmarking data is gathered through a ‘salary survey’ which gathers information from a range of companies about employee compensation. The survey results are aggregated and sold to HR and People Teams as benchmark – typically covering base salary, variable pay, equity, and benefits.
These salary surveys are typically conducted annually by large global HR consultancies like Radford or Willis Towers Watson.
The data can be purchased either as a full dataset or a custom peer-group report which reduces the data to smaller, more relevant subsets such as specific locations or industries. Historically salary surveys were provided as a static spreadsheet, but today many providers have a software platform (e.g. The Radford Platform) so that the data can be viewed and analysed online.
Salary surveys used to be the only way to access salary benchmarking data and understand market trends in pay for different roles – but that is no longer the case.
Why do companies use salary surveys?
The core reason that companies use salary surveys is for salary benchmarking i.e. to align employee salaries with the wider market to ensure fair and competitive pay.
There are several instances during which salary benchmarking data will be used, with the main ones being:
- Creating (or editing) a salary band structure. Benchmarking data is used to determine the midpoint of each salary band, based on the desired target percentile.
- Defining the salary range for a new job position. In an ideal world, salary bands should be implemented for all possible roles that will ever exist within a company’s org structure, but this often isn’t the reality. This means that when a new role is added to the company, benchmarking data is required to help define the salary range that will be advertised.
- Planning compensation adjustments during a compensation review. Most companies conduct an annual compensation review which should always include market adjustments to employee salaries (and the overall salary bands) to ensure their pay remains competitive as the market changes.
These are all fundamental pieces of work for HR and People teams – so regular access to reliable salary benchmarking data is absolutely vital.
Salary surveys used to be the primary way to access that salary benchmarking data. This is no longer the case, with real-time benchmarking tools like Ravio now on the market and offering a much more reliable source of market data.
So why do some companies still choose to use salary surveys? There are two main reasons:
- Brand trust and credibility. Salary surveys are provided by large traditional HR consultancies like Radford, Mercer, or Willis Towers Watson. These are well-known and respected brands which can hold a lot of weight and credibility for business leaders, making it easier to win stakeholder buy-in and budget approval for them as a supplier compared to modern benchmarking software providers.
- Data breadth. The consultancies typically collect compensation data from some of the largest corporations across the globe and so the resulting data spans a huge range of job roles and job levels. However, this is also a limitation (as we’ll see later) because it results in a complex dataset that can be irrelevant for smaller companies and is often impossible to align with your organisation structure.
Who are the main salary survey providers?
Salary surveys are typically conducted by large global HR consultancy firms, with the top providers including:
There are also third-party benchmarking data suppliers (such as Payscale) who aggregate the salary survey data from several of the consultancies above to enable the use and comparison of several datasets in one place.
The 6 key limitations of salary surveys
Why are salary surveys an unreliable source for salary benchmarking data?
There are 6 key limitations:
- Out-of-date data
- Error-prone manual data
- Broad peer group
- Manual and time-consuming to use
- Inaccurate job matching
- Incomplete compensation management solution.
Let’s take a closer look.
1. Out-of-date data
The salary survey approach is for a large-scale survey to be conducted to gather compensation data, typically once a year. The data is then aggregated and released for sale, typically months after the survey collection.
The resulting dataset, therefore, only ever reflects the single point in time when survey responses were submitted, so is already out-of-date by the time HR and People teams receive it. It’s then another 12 months before new benchmarking data is released.
Talent and compensation markets move quickly, so it’s critical that benchmarking data always gives an up-to-date view of market salaries.
Benchmarking data that is only updated once a year (as with salary surveys) will never be current enough to stay competitive.
To give just one example to illustrate this point, we can see from Ravio’s latest Compensation Trends report using real-time benchmarking data that software engineering salaries in Europe have been volatile in the past year (2023-4), with significant increases in competitive pay at the P3 level, and decreases at the M3 level.
It’s clear from this just how much market fluctuations impact compensation.
To stay ahead of those changes and keep compensation competitive and fair, companies need access to benchmarks that show more than just a once-a-year snapshot.
2. Error-prone manual data
Salary surveys typically operate on a give-to-get model. This means that if you want to purchase the resulting dataset you need to submit information on your own employee compensation to the survey.
The salary survey submission requires HR and People teams to compile a significant amount of information and data on the organisation structure and employee compensation packages, and then input it into the format provided.
Typically either a blank template is provided (usually an Excel spreadsheet) which must be populated and submitted or an online submission is done through the provider’s platform.
This example from Mercer’s data collection platform highlights how much information must be submitted each time:
Additional complexity arises due to the need to align with the salary survey provider’s level framework and job catalogue (see section below on inaccurate job matching).
Human error is inevitable with such a manual and lengthy process which involves moving data from HR systems to survey submission documents – raising huge questions about the accuracy and reliability of the resulting compensation benchmarking data.
3. Broad peer group
The HR consultancies that conduct salary surveys typically work primarily with the largest companies all over the world, often from legacy industries like manufacturing or oil and gas.
One of the most important factors in accurate salary benchmarking is ensuring that the data reflects a peer group of companies who you actually compete with for talent – typically companies in a similar industry, at a similar stage of growth, with a similar headcount, or in the same HQ location. These are the companies that you want to benchmark employee salaries against to ensure competitive compensation packages that will attract and retain talent for your company.
For most companies, the typical salary survey data pool is way too broad to give relevant benchmarking information. This is especially true for private companies because these traditional salary survey providers tend to work primarily with large public companies, so their data for private companies and startups is relatively weak.
Most salary survey providers do offer custom peer group reports to narrow the data down to specific organisations – but at an additional cost. However, this can only ever include companies that have submitted a response to the salary survey which, as we’ve seen, tends to be those large, global, legacy corporations – it therefore will never be relevant data for companies that are very different to this profile e.g. a SaaS scale up.
4. Manual and time-consuming to use
Salary surveys require a lot of effort from HR and People teams to use.
To start with, there’s the manual salary survey submissions.
We’ve already seen that these lead to error-prone data. They’re also very time-consuming, often taking days (or even weeks in some cases) to complete, as employee data needs to be dragged from HR systems and inputted into the right format for the submission.
Then there’s the way that the data is provided.
The default is for salary survey datasets to be provided as spreadsheets. These spreadsheets are very difficult to use as they’re essentially a vast list of every single job role and job level and the corresponding average compensation package. Interpreting the data to find the right benchmarks to apply to your company is an incredibly painful process (especially because of the job matching issue in the next section).
In fact, we’ve heard it described as ‘spreadsheet hell’ on more than one occasion.
What makes it even worse is that this process needs to be repeated every single year: it’s a recurring effort rather than a one-time investment.
Today many of the salary survey providers do have the option to receive benchmarking data via a software platform instead of as spreadsheets, such as The Radford Platform or Korn Ferry Pay.
However, these providers specialise in consultancy, not software. Their platforms are missing much of the functionality that you’d expect from a compensation tool. User reviews also suggest that they are unintuitive and difficult to use – indicating that the problems with the spreadsheets are simply transferred to a platform format, rather than being resolved.
5. Inaccurate job matching
The large, legacy companies that make up the majority of the data pool for most salary surveys also typically have hugely complicated organisation structures.
Every company that submits a response to the salary survey has their job roles and levels added to the dataset, resulting in a huge amount of roles and levels being captured. Radford’s technology compensation survey, for instance, contains 899 unique job roles.
When compensation data is submitted to a salary survey, it should be formatted in alignment with the salary survey provider’s level framework and job role catalogue to ensure consistency. In reality, different users interpret the level framework and job catalogue differently, which leads to huge inconsistencies in the data. Because the data is manually submitted, there is no way for the salary survey provider to check and verify the roles and levels submitted, or mitigate the inconsistencies.
In terms of People Teams using the data for benchmarking, this means that it’s impossible to ensure an accurate mapping between internal salary bands and employee data, and the salary survey benchmarking data.
6. Incomplete compensation management solution
Salary surveys provide a one-off benchmarking dataset.
Today it’s common for salary survey providers to offer a platform (like The Radford Platform, for instance) which enables users to view and analyse the benchmarking data through an online tool rather than a spreadsheet.
Some of these platforms have additional features – The Radford Platform can also be used to build a job architecture structure, for instance.
However, they are never comprehensive compensation management solutions.
The salary survey providers are large HR consultancies that specialise in supporting People Teams with the design and implementation of various compensation structures or processes, as consultancy projects.
If you’re looking for an end-to-end compensation management solution, salary surveys are not the answer. Vital features like salary bands, pay equity analysis, compensation review planning (e.g. budgeting scenarios) that would be required in order to make full use of the benchmarking data, are not provided. Some of these areas could be covered through (expensive) one-off consultancy projects, but this doesn’t meet ongoing compensation management needs for most teams.
How does compensation benchmarking software overcome these limitations?
Thankfully, gone are the days when salary surveys were the only option.
Today, salary benchmarking tools like Ravio offer up-to-date and accurate compensation data, automatically mapped to align with your job architecture and levels, and accessible anytime through a modern and user-friendly platform.
Overall, switching from salary surveys to software is confidence that your compensation stays competitive with the market, enabling you to attract and retain the best talent and hit People Team goals.
Let’s take a closer look at how providers like Ravio are overcoming the limitations of traditional salary surveys:
- Out-of-date data ➡️ real-time data from HRIS integrations
- Error-prone manual data ➡️ data direct from source
- Broad peer group ➡️ relevant data pool and filtering functionalities
- Manual and time-consuming ➡️ intuitive software with instant data insights
- Inaccurate job matching ➡️ job level mapping done for you
- Incomplete solution ➡️ comprehensive compensation management solution
Out-of-date data ➡️ real-time data from HRIS integrations
Compensation benchmarking software typically gathers compensation data through integrations with company HRIS systems.
Through the integration, the benchmarking data is automatically updated to reflect salary changes or additional new hire salaries added to the HRIS. This means that the benchmarks shown are always up-to-date and reflective of the quickly-changing talent market, with no more static and outdated spreadsheets in sight.
Error-prone manual data ➡️ data direct from source
Ravio's platform eliminates human error from the process, because compensation data is automatically gathered from the HRIS system, with no manual input required.
Plus, all Ravio data passes verification by our in-house benchmarking experts and data science team. This ensures a high degree of accuracy, as well as a high degree of consistency in how data is labelled and interpreted – key to producing reliable benchmarks.
Broad peer group ➡️ relevant data pool and filtering functionalities
Modern benchmarking tools tend to have a more targeted data pool due to the nature of the companies that currently utilise these modern tools e.g. high-growth tech scale ups.
This minimises the data skew caused by the majority of salary survey data points coming from a handful of the world’s largest (often legacy) businesses, meaning the benchmarks are much more relevant from the word go.
Benchmarking platforms also include the ability to easily filter data for different locations, industries, company sizes, and company stages. This ensures the benchmarks and job catalogue are always highly relevant and tailored to each company, without the need to pay extra for peer group reports.
It’s important to check that these software solutions also maintain enough data breadth though, as the sample size is an important factor in data confidence and accuracy.
Ravio’s dataset is sourced from over 1,000 companies with over 300,000 individual employee data points. Plus, Ravio's expert data team employs advanced statistical analysis to ensure high-quality benchmarks for even the most niche roles.
Manual and time-consuming ➡️ intuitive software with instant data insights
Compensation software eliminates the manual wrangling required when using salary surveys for benchmarking. With software solutions there’s none of the waiting around for a spreadsheet of data to be delivered. Instead, you can onboard and access benchmarking data within minutes, and the HRIS integration means that no recurring manual submissions are required to use the platform.
Inaccurate job matching ➡️ job level mapping done for you
The very first step for every Ravio customer is for their job levels and/or individual employees (depending if they have an existing level framework or not) to be mapped against the best practice Ravio level framework.
This removes the manual job matching and ensures consistency in the interpretation of job roles and levels. Ultimately, this ensures like-for-like comparisons between company roles and market benchmarks.
If you’re evaluating different salary benchmarking tools, we’d always recommend checking how each provider manages this job level mapping – it’s critical for accurate benchmarking, and different platforms may take different approaches to the process outlined here for Ravio.
Incomplete solution ➡️ comprehensive compensation management solution
Modern compensation management platforms offer accurate and up-to-date benchmarking data than salary surveys – but they also offer so much more.
Providers like Ravio also include additional benchmarking features such as workforce analytics to automatically compare your employees to the market benchmarks, or market trends to access real-time insights on salary increases, hiring rates, attrition rates, and more.
Plus, Ravio isn’t just for benchmarking. It’s a comprehensive compensation management platform with a whole host of additional features compared to a salary survey dataset – from comparing benchmarking data to employee compensation, to creating a salary band structure aligned with the market, to instantly identifying band outliers or pay equity issues, to communicating compensation packages to employees, and much more.
What about Glassdoor or LinkedIn?
The other alternative data source that some companies use for salary benchmarking is desk research: finding anonymous employee-reported salary data from company review sites like Glassdoor, or from the salary range listed in job adverts on sites like LinkedIn or Indeed.
They’re free and easy to access which makes them appealing, but these are both highly inaccurate sources of data.
In the case of Glassdoor, the data is unverified, crowdsourced from users of the website with no checks to ensure that the user is submitting accurate data.
There’s typically a huge amount of variance in the salaries submitted and the salary range shown to you simply represents the average of all salaries ever submitted for that role: it doesn’t account for outliers or identify up-to-date shifts in the salary based on market trends.
Here’s an example of a Senior Software Engineer salary range from Glassdoor’s salary calculator to illustrate this point – there’s no descriptive information to anchor their definition of ‘senior software engineer’ on (aside from the 1-3 years of experience, which is confusing in itself for a ‘senior’ role), and the salary range is too broad to be useful.
In terms of job adverts, they only ever tell you how a handful of companies benchmark a role. You have no idea what their compensation philosophy and target percentile is or where they got their benchmarking data from, and you’re making a best guess that their interpretation of the job role and level is the same as what you’re looking for.
💡 How switching from Glassdoor to Ravio brought confidence to Tiqets’ compensation strategy
Tiqets is an online booking platform for museums and attractions around the world. The company has around 230 employees, mostly based in their Amsterdam HQ but with some working remotely across different locations in Europe.
For a long time Tiqets relied on sources like Glassdoor and LinkedIn to collate salary data on which to base their employees’ compensation packages.
This was time-consuming for the People Team and led to a lack of confidence in the data, especially during compensation review conversations. As Maartje Koopman, Head of People and Culture at Tiqets, put it:
“It created frustrations with line managers. They didn’t trust the data that we came with, which made it difficult for us to discuss compensation and talk about what people’s pay should be. Were we paying the right amount to people, or were we way off in the market?”
The team has since made the switch to using Ravio, accessing reliable compensation data for relevant roles that ensures managers and employees have trust in the process.
“I think it’s so much better now. If managers look at Ravio, they see that the right comparisons are being made.”
Can you use both salary surveys and compensation benchmarking software together?
Salary surveys and compensation benchmarking software don’t have to be mutually exclusive benchmarking solutions.
Some companies find it beneficial to have multiple data sources for salary benchmarking, and so opt to combine both salary survey data from a provider like Radford and a salary benchmarking tool like Ravio’s.
This is especially true for:
- Established scale ups or public companies with a large headcount and employees spread across a high number of locations, job roles, and job levels. Different data providers have different market focuses and so will have stronger data for some locations or roles, so combining multiple data sources ensures full coverage.
- Companies that are highly reliant on niche roles within their industry. There is inevitably a smaller sample size of benchmarking data available for niche or new roles, so it’s worth comparing across providers to increase confidence.
- Combining salary surveys and software also means that you gain access to all of the additional compensation management features that are typically included in a modern software solution such as a salary band tool or automated pay equity analysis.
So, are salary surveys still the best data source for salary benchmarking?
No. Salary surveys are no longer the best data source for salary benchmarking, especially if they’re the only data source used.
The old school, manual and annual salary survey approach results in benchmarking data that is quickly outdated, full of inaccuracies, and cumbersome for People Teams to translate for their needs – all of which is extra frustrating, given the high price tag.
Salary benchmarking has evolved, and real-time tools like Ravio bring confidence to compensation benchmarking.
Try out Ravio with 3 free benchmarks
Explore the most comprehensive real-time total reward data set with 3 benchmarks for free, or book a demo if you’d like a full tour of the Ravio platform.