(Printed version available free to subscribers.)

1. Introduction

Just how important external salary data is to modern HR practice was underlined by a recent CIPD report showing that nearly three- quarters of organisations either purchased or participated in pay surveys. As these findings indicate, more and more companies are now relying on such surveys to set competitive market salaries. But while they are one of the most important tools of the trade in salary administration, little has been written to guide the personnel specialist through the pay survey minefield. No recognised standards for the production of acceptable salary surveys exist and excellent, well researched and presented surveys coexist alongside collections of figures or opinions that are hardly worth calling surveys at all.

Given the general lack of helpful literature, back in 1983 the IDS Executive Compensation Review (ECR) thought that a user’s guide to understanding salary surveys was called for. The need is just as great today and to meet that need, the ECR has decided to re-issue a revised and updated version of Understanding Salary Surveys.

Since the first edition was published, the widespread availability of computers and the Internet has allowed producers to supply more varied and sophisticated surveys. Added to the traditional range of surveys has been the development of online services that can give up- to-date tailored information on demand. In addition to such ‘designer’ surveys, this publication is also relevant to those wanting to use the following:

• general commercial surveys produced and sold either by specialist producers or by management consultants. Sale may be limited to participants or available to non-participants at a higher price;
• club surveys which collect information from a small number of employers in a salary club. Club surveys are conducted by a member company or by a consultant and often provide more detailed information than commercial surveys;
• membership surveys conducted by professional associations. The information is collected from individuals rather than employers and are usually less detailed and less expensive than commercially-produced surveys;
• recruitment surveys published by recruitment consultants based on records of salaries offered and accepted. Such surveys normally cover specialist groups such as lawyers or accountants.

Although salary surveys are an indispensable aid to making informed pay decisions, they are no substitute for judgement. Even an excellent survey cannot decide what to pay. In the end, that is a matter of choice and company policy. Surveys provide information, which may be good or bad, but companies make decisions. This user's guide is designed to help with those decisions.

The report aims to give an insight into how salary surveys are put together and how they are used. It explains some basic terms and points out some of the pitfalls. At each stage of survey construction problems are involved and no report can provide all the answers. The best that can be hoped for in an imperfect world is for a reasonable guide to the salary market. For this reason, most firms use a number of surveys to check the competitiveness of their salary policies.

Chapter 2 explains the main statistical and salary definitions used and Chapter 3 covers some of the most important concepts in salary surveys – sampling and job matching. In Chapter 4 we look at the importance of timing and describe collection methods, while Chapter 5 focuses on the different ways of presenting survey results. Chapter 6 concentrates on club surveys and Chapter 7 provides a brief user's guide. At the end we summarise the important points as a checklist for assessing survey data.

Details of all surveys mentioned in the text are included in the MPR’s Directory of Salary Surveys 2001/02. The Directory is available to ECR subscribers and online at www.incomesdata.co.uk

Survey users may also find the following book helpful: Reward Management: Handbook of Salary Administration by Michael Armstrong and Helen Murlis. Published by Kogan Page. Price £29.95. Website: www.kogan-page.co.uk

Salary surveys are designed to tell us about annual pay or total earnings levels at a given point in time. Such surveys are relatively simple to understand, but before picking one up, users should be familiar with some rudimentary statistical terms and also be aware of some of the hidden pitfalls. In this chapter we:

• outline some basic statistical tools;
• stress the importance of clear definitions;
• look at the analysis of data.

To fully understand salary surveys a little statistical knowledge is unfortunately needed. Few personnel specialists, when faced with an array of salary figures, could easily identify the relevant information. For this reason, survey data is reduced to a single, easily understood figure, called an average. An average is technically defined as ‘a point within a group of data which is central to the group and around which other values are distributed’.

Confusion is often increased, however, by the existence of three different types of average – mean, median and mode. Most salary surveys use at least one of these averages to give some idea of the middle or ‘typical’ salary for a job. The three averages, defined below, are illustrated using the salaries of the 13 utility company chief executives shown in Table 2.1. These are the actual salaries of the chief executives as given in the latest available annual accounts.

The arithmetic mean, which is most commonly referred to as the average, is simply the sum of the individual figures divided by the number of items. In Table 2.1, the mean salary of the 13 lead executive is £348,817. This is calculated by adding up all the individual salaries to reach a total of £4,534,619 and then dividing

this by the number of executives. For illustrative purposes, the full calculation is shown in Box 2.1.

The median is the middle number of a particular range of data placed in rank order, with half the figures falling on one side and half on the other. The median salary of the 13 utility executives is £342,500. This is found by arranging all the salaries in consecutive order, running from the lowest of £143,000 for the two chief executives of Pennon, to the highest of £522,668 for the chief executive of Scottish Power. The number of values are then numbered and the median is calculated by adding one to the total and dividing by two. In summary, this would be:

number of salaries (13) + 1 divided by 2 = 7.

The median is then the seventh value in the range of data in the table, as Graph 2.1 shows.

It is important to note that the point where the median cuts the data in half does not always fall neatly on an existing figure in the range. This only happens where there is an odd number, as in the table. Where there are an even number of salaries, the median is normally the mean average of the two middle figures. If there were only 12 salaries, for example, instead of 13 as in Table 2.1, the formula would be 12 + 1 divided by 2, which would equal 6.5. In this case, the median would fall half way between the sixth and the seventh salary.

The mode is the most popular salary, that is, the salary received by most employees in the sample. If 20 staff received £110,000, five received £100,000 and 10 received £120,000, then the modal average would be £110,000. Sometimes, rather than produce a single salary, surveys identify a ‘modal range’ such as £10,000 to £15,000 which contains the largest concentration of salaries. In Table 2.1, the modal average is £143,000 because two of the 13 chief executives received this amount, while the salaries of all the others were different. This example illustrates some of the drawbacks associated with modal averages, particularly when they are based on small samples. Another is that when there is a large number of returns there can be more than one modal average.

In some cases, more complicated statistical procedures, such as weighted averages, are adopted to give a more representative picture. Weighting is used where there are large numbers of employees. In such situations, a simple average can distort the true picture because it may understate the impact of large number of employees receiving the same salary. Consider the example below using a hypothetical management grade structure:

The simple mean average of these salaries, as defined above, is £41,667. Contrast this with the weighted average which is £31,481, some £10,000 lower. To calculate the weighted average, each salary rate is multiplied by the number of employees receiving that salary to give a total for each rate. In the example, the £20,000 rate for grade M1 is multiplied by the number of managers on the grade, 50, to produce a total of £1,000,000. This procedure is reproduced for all the other grades and the figures are then added up to arrive at a total paybill cost of £4,250,000. This amount is then divided by the total number of managers, 135, to derive the final weighted average of £31,481. A tabular summary of this calculation is given in Box 2.4.

The difference between the two averages is due to the fact that the simple mean gives equal weight to all the salary rates and ignores the number of employees receiving that salary. In this example, the 50 managers paid £20,000 are accorded equal significance in the average calculation as the five paid £75,000. This higher rate will push up the overall simple mean average. Weighting adjusts for the relative importance each salary rate plays in the overall paybill.

In themselves, averages do not always give a full picture and many practitioners are equally interested in the distribution of salaries. This is because employers are not normally concerned with paying all postholders the same average salary, rather they want to fix a range that is appropriate for a particular job. Individual employees are then slotted into the range on the basis of various measures such as experience or performance. Consequently, an understanding of the distribution of salaries gives employers the scope to make judgements about what salary levels suit their own circumstances.

Typically, a range of salaries is represented by distribution curves or bar charts. The standard type of distribution curve is called a normal curve which is bell shaped. But not every distribution curve has this shape, some have the bulk of salaries to the left and others to the right. These are ‘skewed’ curves and are classed as either positive when pushed to the left or negative when pushed to the right. An indication of the pattern of distribution can be gained from the relationship between the three averages. When salaries are symmetrically dispersed, as in a normal or bell shaped curve, the three averages, mean, median and mode, are the same. In contrast, when the distribution curve is positively shaped the mode is lower than the median, which in turn is lower than the mean. The order is reversed when a curve is negatively shaped, as can be seen in Graph 2.2 below. In reality, salary distribution curves tend to be positive as only a small number of employees usually receive the highest salaries.

A more numerical way of looking at the distribution of salaries is to calculate quartiles and deciles. These figures, in particular quartiles, are the most common way distributions are described in salary surveys. Just as the median cuts the distribution of a group of salaries into two equal parts, so data can be further broken down into quarters and tenths. Quartiles divide data into four equal parts and deciles into 10. In summary, the:

• upper quartile (UQ) cuts off the top quarter of the distribution;
• lower quartile (LQ) cuts off the lower quarter;
• highest decile (HD) cuts off the top tenth;
• lowest decile (LD) cuts off the bottom tenth.

These figures are found in a similar way to the median. Salaries are ranked from the lowest to the highest and formulae are applied that indicate the place where the relevant quartile or decile value can be found. Looking at Graph 2.1, the lower and upper quartile formulae show that the relevant figures are the fourth and tenth values, £283,500 and £480,000 respectively. The lower and upper quartile calculations are as follows:

• lower quartile position = (number of salaries (13) + 3) ÷ 4 = 4
• upper quartile position = (3 x number of salaries (13) + 1) ÷ 4 = 10

Survey users should be aware, however, that these formulae are not universal and different statistics texbooks may give different presentations. Likewise, various computer software programmes also use different formulae. This is the case with the widely used Microsoft spreadsheet programme Excel and the equally widely used specialist statistical package SPSS. For this reason, the quartiles found by one programme can differ from those given by the other.

Although there are three types of average, the mode is rarely used and in practice most surveys concentrate on the mean, median and lower and upper quartiles. Given that salary distributions are usually positively skewed, for survey users there is often a conflict between which is the best average to adopt. An indication of the conflicting messages provided by means and medians is provided by Table 2.2, taken from Remuneration Economics’ annual national survey conducted on behalf of the Institute of Management. Table 2.2 illustrates that for some jobs, particularly at the highest levels, the gap between median and averages can be quite wide. Further, the gap is most often exaggerated when the figures are based on small numbers. The bigger the sample, the more likely the two averages will coincide.

The dilemma this table poses for survey users is which average best describes the market rate for the job in question. Both the mean and the median have their uses and limitations, but in the end it depends on the purpose to which the survey is to be put. Mean averages, for example, are readily understood and employees often feel it is fair to be paid the average salary. In addition, mean averages have some practical advantages. They can be used as the basis for further calculations whereas medians cannot. It is relatively simple to combine separate averages, duly weighted, to find a mean for a whole sample. Such a procedure is not possible with medians as the mid-point is highly specific to a particular data range. When several data ranges are being compared there is be no way of telling in advance where the new mid-point will fall without undertaking the whole calculation again.

On the other hand, consultants often argue that the median is a more representative figure because it reduces the impact of the more extreme salaries. Many firms also prefer to use the median, along with the inter-quartile range, as the basis for salary comparison because it says more about the distribution of pay levels.

Alongside an understanding of what the various statistical measures are conveying, a clear definition of what is being surveyed is also essential. The meaning of basic salary or total earnings may seem obvious, but a close inspection of surveys soon shows that they are often used in different ways. For this reason, a good survey will always provide definitions of the terms used and the user should always check them before analyzing the data. The main earnings definitions are as follows:

Basic or base salary is gross annual salary before deducting tax, National Insurance and pension contributions. It normally includes merit increases or incremental payments and usually excludes bonuses, overtime pay, and most other allowances. Elements like London weighting or part-time directors' fees may or may not be included. Salaries are either annual, monthly or weekly and may be valid for a single date or collected over a 12-month period.

Sometimes called total cash or total remuneration, this is normally basic salary plus all the extra allowances described above. The various total cash elements, however, may not be paid on the same date. Basic salary information collected in April can be added to bonus payments paid in October. Differences in time-scale between the various components of total cash have to be accepted as inevitable.

With the growth of non-cash fringe benefits, attention is increasingly focused on providing estimates of the ‘total remuneration package’. Total remuneration figures combine cash payments and cash valuations of non-cash fringe benefits. Benefit valuation is notoriously difficult and any survey that includes a figure for total remuneration should fully describe the methods adopted. Such estimates are still relatively rare, but the Executive Remuneration Review includes total remuneration figures that add in the total cost of the provision of benefits.

In most surveys, information is normally given on bonuses paid in the past year, although some, such as Monks Partnership’s management pay surveys, also provide target payments. Sometimes, the value of bonus payments are hidden because they are spread over the whole sample rather than limited to those who actually receive them. If there are two employees, for example, each earning £10,000, but only one receives a bonus of 10 per cent, when the payment is distributed over both, the size of the bonus for the sample is only 5 per cent.

Surveys of professional earnings often combine salaries for those who are employed with those who are self-employed. Earnings received by self-employed professionals include the taxable value of perquisites and are based on gross income reduced by tax-allowable expenses such as the depreciation of capital assets. Aggregating data in this way, especially when earnings apply to different time periods, ensures that the findings are difficult to interpret. It is often more helpful if surveys provide separate analyses for employees and the self-employed.

Information on benefit provision is relatively straightforward, although it should be made clear at what job or salary level particular benefits are available. Many salary surveys collect at least some information on benefits, although few details are given about how they operate. For more extensive information it is necessary to consult a specialist benefit survey.

Salary information is usually analyzed according to a number of different variables, such as location, industry and company size, which can make a considerable difference to salary levels. A small manufacturing company in the East Midlands, for example, would not expect to pay the same as a merchant bank in the City, whether the salary in question is for a finance director or a filing clerk. Once again, clear definitions of each variable should be provided in every survey.

There is often a strong correlation between salaries and company size. Surveys may use several indicators of company size, but the most common are annual sales turnover and the number of employees. In general, the bigger the company, the higher the salaries. Pinpoint accuracy in measuring size is not necessary because surveys usually analyze salaries within a specified range. Industry is often analysed according to the Standard Industrial Classification, although some surveys adopt their own sub-divisions.

3. Survey sampling and job matching

Ideally, compensation managers would want the surveys they use to be so comprehensive that they are based on returns for every single individual in the market they are looking to match. In practice, they have to settle for much less than this, with the quality of the samples in the surveys actually available ranging from the haphazard to the broadly representative. This variation in coverage is not without its importance to compensation specialists as it has a big impact on the quality of the pay information available. Surveys based on unrepresentative or small samples, for example, will produce distorted results. Before employing a survey, therefore, users should take a careful look at the way the information was collected.

But even if there are no sample problems, a survey will be of little value to users unless they can relate its findings to the jobs inside their own organisations. For valid results, like must be compared with like. Consequently, how a survey matches and describes the jobs it aggregates in its tables is just as important to its value as the number of participants it attracts. In this chapter, therefore, we:

• outline some of the problems with sample construction;
• explain the way jobs are matched.

At first sight, information gathering appears to be a rather simple operation, as all it seems to require is a few administrative procedures. Get hold of an appropriate list of the target group, send out questionnaires asking for the required information and then encourage the recipients to fill them in and return them for analysis. In reality, however, the whole process is fraught with difficulties at every stage. Up-to-date lists of the right person to contact are not always available, while even the best designed questionnaires are liable to be misinterpreted by those on the receiving end. Finally, recipients are rarely keen to spend time and effort completing them. The consequence of these practical difficulties is that surveys are always based on only part or a sample of the target group. This is not always a problem, but for survey results to be valid it is important to ensure that they are representative of the whole population.

Given these practical drawbacks, most salary surveys are unlikely to be strictly representative. An illustration of the influence that unrepresentative samples can have on survey findings can be seen from Graph 3.1. This gives example salaries of a target group of employees that an employer wishes to use as a benchmark for its own staff. The range of salaries is relatively narrow, just £10,000, and the average for all of them is £24,300. If the survey collects information from the bottom five employees (shown as white bars), however, the average will be £2,600 lower at £21,700, while if the top five were surveyed (shown as black bars), the average would be £2,600 higher at £26,900. On the basis of such small numbers the differences may not appear significant, but when the target group is much bigger the potential divergence between the actual all-market average and the survey average can be magnified. Further, if an employer applies the findings to its own employees then the cost implications of a £2,600 difference could be substantial if its own workforce is large. In these circumstances, the best that can be hoped for is that any results are ‘reasonable’ approximations, with reasonable in this case depending as much on judgement as on statistical theory.

The lesson to be taken from this example is that the more that is known about the selection and composition of a survey’s sample, the easier it is to assess its quality. In summary, salary survey samples often fall short of strict statistical purity because:

• sample frames are often not comprehensive;
• sub-samples may be small;
• samples may be clustered;
• response rates may be low.

Surveys start with the identification of a target group such as directors or computer specialists. Other variables such as location or industry are also often decided at an early stage. These initial steps are followed by the specification of a ‘sample frame’, that is a list of companies or individuals from which a random sample of the target group will be drawn. Although the aim is to include all members of the target population, this is not always possible because comprehensive lists are rare.

The membership lists of professional organisations, such as those for engineers, chemists and architects, are reasonably good sample frames, although not all chemists or engineers would necessarily be members. Commercial survey organisations have access to publicly available lists of companies such as the FTSE 500 or members of trade and employers’ organisations. Producers generally contact as large a group as possible and hope there are enough responses to produce a usable analysis. The selection of participants, however, remains fairly haphazard and the resulting sample is more likely to be self-selecting rather than truly random.

Some consultants, such as the Hay Group, restrict their sample sizes by limiting participation to those who subscribe to its propriety job evaluation scheme. Others limit both the sample and the sale of their surveys to those who participated. In circumstances like this, small samples are not always viewed as a disadvantage. Surveys conducted by a number of firms in a pay club, for example, are valued precisely because they deliberately limit information to a common interest group. Although the club members may change from year to year, all participants have an incentive to respond.

Employers are often only interested in the salary levels for a small sub-class of individuals such as financial accountants employed by firms with £1 million turnovers. For this reason, most surveys divide its overall returns into smaller groups for further analysis. Typical breakdowns include company size, measured by either turnover or employee numbers, region and job function. The more the overall sample is divided, however, the smaller the sub-groups are likely to be and consequently less representative. It is important, therefore, to look at the sample numbers falling into sub-classes and check whether they are large enough to be acceptable. Most survey producers have thresholds below which they will not analyse data and this should be defined in the published report.

Sample distortion can also result from ‘clustering’. In theory, the more individuals or companies there are in a sample the more representative it is. Yet, when large numbers of individuals are drawn from a small number of companies, the sample of jobholders is unlikely to represent the whole population.

Take as an example a survey of 100 chief executives and 1,000 data processing staff drawn from 100 companies. The simple fact that the numbers of data processing staff are far greater than chief executives is no guarantee that the data processing sample is more representative. On the contrary, what is most relevant is the relationship of the sample to the target population as a whole, and 1,000 data processing staff working for 100 companies are more likely to share similar conditions than 1,000 staff working for 500 companies. The clustering of large numbers of individuals in this way can produce an unrepresentative sample. In this case a smaller number of chief executives could be more representative of a much smaller target population.

A low response rate from the target group can also raise questions about sample representation. Either too little information may be collected or the existence of a few returns may indicate that they have been completed by respondents with atypical attitudes or conditions.

The response rate for most surveys, even compulsory official ones, are unlikely to be 100 per cent. Salary surveys that depend on voluntary participation have to be satisfied with much lower response rates, especially when they are conducted by postal questionnaire. Even surveys conducted by professional bodies like the Institution of Civil Engineers, which attempt to gather information on the whole population, have response rates of less than 40 per cent. In contrast to the professional institute surveys, most commercial surveys do not indicate their response rate at all, so users have to rely on their own judgements about whether the make up of the sample matches their own requirements.

These judgements can be aided by an awareness of the most common sources of sample bias in many of the currently available surveys. The factors that push or pull a survey’s pay findings in one direction or another include: company size; industry; region; the time the data was collected; and self-selection. An example of how company size can affect results can be seen in Box 3.2 below, which compares the average salaries for chief executives given in the latest management surveys from Inbucon and Remuneration Economics (RE). As the box shows, the overall averages found by the two surveys, £247,666 according to Inbucon and £197,590 according to RE, differ by some £50,000. This difference can largely be explained by the distribution of the number of chief executives in each company turnover band. As smaller companies tend to pay lower salaries, the weight of RE’s returns from companies with less than £5 million will depress its average compared to Inbucon’s. Just as different sized companies can affect the overall results, so can the dominance of high or low paying industries or regions.

A more difficult influence to detect in survey findings is the self- selection participants. When looking at a survey, users need to ask themselves who would most likely respond. This is a particular problem with reports produced by recruitment consultants, which tend to draw their information from candidates looking for another job. Such job movers are less likely to be typical of the workforce as a whole and they can also have an incentive to exaggerate their current pay packages.

As important as sample construction is job matching. For valid results, like must be compared with like and survey producers must take steps to ensure that it is similar types of job that are combined. Job matching normally involves looking beyond job titles at the actual tasks and levels of responsibility involved. The different approaches are:

Job titles on their own or combined with some indication of qualifications or experience are suitable for surveys covering less complex jobs where tasks are clearly defined. The results of this approach, however, are not always dependable because job titles are generally an unreliable guide to actual work content and responsibilities. This problem is greatly increased when samples are drawn from a wide variety of companies and industries. On the other hand, where there is a good deal of similarity, as in the early stages of a professional career, the results may be more acceptable.

In some surveys, job titles are accompanied by a brief job description or indication of their job scope. This approach is popular among recruitment specialists and some consultants. Such short descriptions can be reasonably accurate for well-defined specialist groups where survey users know the operating context. Although, as with surveys that rely on job titles only, there is always a danger that definitions will be too general and thereby miss important distinctions.

Rather than using job titles, some survey producers, such as Reward, have defined their own responsibility definitions as the basis for job matching results. Consultants Monks Partnership have adopted a similar approach for its Management Pay UK surveys, but have added in plus and minus modifiers to allow participants to refine the scope of the jobs they are matching.

Job evaluation is probably the most accurate way of job matching, an approach taken by consultants such as the Hay Group and Watson Wyatt Worldwide. Their surveys are based on information provided by subscribers to their job evaluation schemes. But while consultants like Hay and Watson try to ensure as much consistency as possible, there can still be some variation between organisations. Further, while such surveys may be reasonably accurate, they can be of limited use to non-subscribers because there are few reference points to match jobs outside the sample.

4. Survey data: timing and collection

Nearly all statistical surveys and their value are affected by timing. Salary surveys suffer more than most because of the speed at which pay data becomes ‘stale’, a factor that can undermine their very purpose. Users need to know what the market rate is now – not what it was six months ago or last year. Yet, by its very nature, data collection takes time. Even during a period of low inflation, it still matters how frequently surveys appear, when data was collected, what period they covered and when the salaries reported were last increased. All these problems can be compounded by changes in sample composition from year to year, making comparisons over several years difficult. In this chapter we look at some of the pitfalls involved in survey timing, in particular looking at how producers deal with changes and the impact this has on the interpretation of results. The topics covered include:

• the impact of different collection dates;
• understanding increases in and between surveys;
• matched samples;
• methods of collection.

Despite the innovations of recent years such as the Internet, the overwhelming majority of survey producers still collect the data for their reports once a year. Of the reports listed in the IDS Executive Compensation Review’s biennial directory of salary surveys, around two-thirds are produced annually. But collection and production dates do vary, ranging from continuous updating to the irregular. More common collection periods include half-yearly, quarterly and every two years.

Most usually, surveys record the actual pay and earnings received by employees on a fixed date, such as the first week in April as with the officially produced New Earnings Survey or at 31 March as with Watson Wyatt’s investment staff report. Such surveys give a snapshot of practice at that time, but the results are sensitive to the pattern of settlements. Moving the collection date from the beginning of December to the end of January, for example, could have considerable impact on executive salary surveys because 1 January is a major review date.

Some producers, such as Watson Wyatt and Alan Jones, are constantly updating their pay information which are collected on ‘live’ databases. Rather than send questionnaires out once or twice a year, as in the traditional manner, new pay details are collected soon after they are reviewed and stored on a computer database. The up-to-date information can be analysed at any time or, as with Alan Jones, monthly reports are prepared. While these reports still give a 'snapshot' of salaries on a particular date, the information is likely to be fresher than data collected by annual questionnaires. These live databases also allow greater scope for the development of ‘tailor-made’ surveys for individual clients. Companies approaching their own review dates can be provided with the most current information available for a range of relevant jobs. As with any other survey, however, the system relies on the efficient and sufficient collection of information and unless enough data is collected they can still produce unrepresentative sub-samples.

Most reputable surveys describe the methods used to collect information in the introduction or in a special section. There are five main ways of acquiring salary data for analysis. The information may be collected by:

• completion of a postal questionnaire by employers;
• completion of a postal questionnaire by job holders;
• personal visits to employers by the survey producers;
• use of recruitment agency records;
• use of published information such as company annual reports.

Postal questionnaires to employers are used for most surveys. One drawback, however, is that they are not always completed by the most appropriate personnel. Alternatively, postal questionnaires may be sent to job holders, a method often adopted for surveys of members of professional bodies and trade unions. Although this approach can generate a good response because employees are keen to find out what others are earning, it often fails to produce key information on levels of responsibility and job definition. In addition, respondents may often feel there is an incentive to exaggerate or depress their earnings.

Questionnaire design can have an important impact on the response rate. High standards of design and layout are essential if accurate information is to be collected. Good practice suggests that questionnaires should be published along with the survey, but many commercial organisations are reluctant to do this because they are of value to competitors.

Personal collection of salary information from every employer or employee is both labour intensive and expensive. For these reasons, this method is sparingly used, although it is adopted for club surveys of highly specialised staff or by survey organisations with proprietary job evaluation schemes. While expensive, personal collection almost always produces better job matching than postal surveys.

Recruitment consultants maintain good coverage of the job markets they specialise in but salary information is limited to those searching for a new appointment. This can distort the figures as there is no information on current job holders. The information is also subject to market fluctuations so that sample sizes can vary greatly from one year to the next and salary movements can be exaggerated.

Before employing a recruitment agency survey, users should look at the period over which the salaries were collected and what they are based on. Pay data can vary between salaries in the current job, the required salary on changing job, actual salaries offered/accepted on changing job, employers' expected offers or the agency's view of the current market range.

A number of published sources such as job adverts, company annual reports and Government publications are also available. Job adverts have the same limitations as recruitment agency surveys and are often vague about the actual salary on offer. On the other hand, company accounts provide extensive details on boardroom pay and benefits, but the information is always dated.

When employers are approaching their annual pay reviews, their interest is not always confined to current salary levels, but also the ‘going-rate’ of increases. Yet, just as there are differences between how producers collect their information, so there are with how they calculate increases. Consequently, some care is needed to understand what the figures are actually saying as the question of how the increases are calculated can be vital. This is because even with the same set of data, the final result will vary according to the method used. The following, for example, are just two of the ways commonly adopted by survey producers to calculate pay increases:

• year-on-year comparisons between average or median pay levels;
• measures of actual average or median increases.

The difference between the two approaches may seem obscure at first sight, so the following example should illustrate its importance:

In this simple example, the two approaches produce very differEnt pay movement figures. The year-on-year comparison shows a rise of 1.8 per cent, while the average increase is notably higher at 5.5 per cent. The year-on-year figure is calculated by working out the percentage rise between the average ‘Year 1’ and ‘Year 2’ salaries of £55,000 and £56,000, while the average increase is the 10 per cent and 1 per cent rises added together and divided by two.

Although the current level of rises are generally the main focus of interest, the movements given in surveys are not always a good guide to present growth. Increases at the end of the survey period may be higher or lower than the average, a fact hidden by the evening out process of averaging. Neither will an average indicate the trend over the period, whether rises are moving upwards or downwards.

To deal with these and other problems of timing, it is helpful to know the salary review dates of the survey participants. Graph 4.1 compares the spread of the review dates of the pay data collected by Inbucon and Remuneration Economics (RE) for their major management surveys. As the graph shows, the pattern of the review dates in the two surveys differ, with Inbucon’s returns dominated by January and RE’s by April. This difference reinforces the importance of survey collection dates. Inbucon’s cut off date is July, which allows it to pick up January awards, while RE’s is January and consequently many January reviews will not have been completed.

The various approaches used to calculate pay movements is not the only problem faced by those wanting to track changes over time in the same survey or make comparisons between two surveys. Another is the difficulty of ensuring that like is being compared with like. Samples in surveys are rarely the same from one year to the next, which means that salary level comparisons over time may be unstable and that any reported increases can give a distorted picture. An illustration of these difficulties is provided by the Janet

Salmon survey of leisure industry executives. According to its 2000 survey, for example, the average salary of catering managers was £41,361. The following year, however, the average for these same managers was reported as £36,305 and the corresponding salary movement was shown as a 12.2 per cent cut. As salaries rarely go down, such a change should automatically sound a warning note that there has been a shift in the sample. This is acknowledged in the survey which states that: ‘A major difference in the average increases can reflect that different companies participate each year.’

One way around these difficulties is to identify and match the same participants in each sample, using this group as the basis for more accurate comparisons. Alternatively, participants may be asked to provide information for both the current and previous year. But while using matched samples is the best way of producing consistent figures, different matches will produce different results. Computer Economics regularly bases its results on two matched samples: a matched group of individuals performing at the same level of responsibility as before; and a matched group of companies, which compares all individuals at each level of responsibility last year with those currently performing at those levels.

Individual matching fully reflects general and merit increases and may also include increases in bonus payments for a static group of employees. A matched company sample, on the other hand, reflects movement between jobs and therefore tends to reduce average increases. Higher bonuses, merit or incremental payments for some in the same jobs may be offset by lower payments for new recruits since the previous survey. Table 4.1 illustrates how these different methods of matching can still produce different increase findings.

Computers have considerably improved the speed at which data can be analysed and put together, but surveys with extensive coverage can still take a long time to complete. Three months is a common time-lag between data collection and publication, although it can be less for small scale surveys or surveys from recruitment agencies. These time lags can be important, as users require up-to-date information. In these circumstances, the data may need to be adjusted by either the user or sometimes by the provider, although where this is the case any assumptions and judgements underlying the adjustment should be made clear. For users wishing to update its surveys, Reward regularly provides guidance on how much its figures should be adjusted by.

Once data has been collected, it needs to be presented in a way that is clear and easily understood by the user. The centrepiece of all surveys are tables that set out rows and columns of salary figures analysed by responsibility level, company size or location. All too easily, however, general trends can be lost among the detail of a survey’s tables and consequently pictorial representations of data are often more accessible and can give a clearer overall view. But while the widespread adoption of computers has increased the opportunities to produce charts and graphs, users should remain alert to the general rule – attractive presentation is no substitute for good results. In this chapter we look at:

• table interpretation;
• graphs, bar charts and pie charts;
• regression analysis.

The most common and comprehensive way to present survey results is in a table of figures. A typical example, taken from Mercer Human Resource Consultancy’s office administration survey, is shown in Table 5.1 below. To supplement tables such as these, Mercer and others, like SMCL, provide more detailed breakdowns of the pay information on each job position covered (see Table 5.2). But while it is easy to assume that such tables are objective, it is important to remember that they are no more than summaries of all the data collected. The rules of the previous chapters, therefore, should be applied. Samples need to be representative and definitions consistently applied.

A table should include enough information, such as sample size, to make judgements about the quality of data. A clear indication of what is being summarised, like basic pay in companies with a £200 million turnover, is also needed. Extra care is especially required when two separate surveys are compared and particular attention is needed to check that definitions are consistent.

Averages, medians, quartiles and deciles are the focus of most users’ attention and are, therefore, the salary information most frequently included in survey tables. Yet users need to be aware that although the presentation of most tables looks similar, the basis on which they are compiled can be quite different.

A pictorial representation of data is one way of making information more accessible, a practice which has spread with the growth of computers. Three types of presentation are normally adopted:

• line graphs;
• bar charts;
• pie charts.

Simple graphs show the relationship between two variables, such

as salary and company size, against two axes drawn at right angles to each other. Mostly there is a straightforward linear relationship between data points which can be simply connected by a line. For comparison purposes it is possible to put two or more lines on the same graph, distinguishing between them by using different patterns such as dots or dashes (see Graph 5.1).

Bar charts are often adopted to illustrate salary levels by job category or company size or industry sector (see Graph 5.2). Pie charts, as in Graph 5.3, are mostly used to illustrate the composition of survey samples.

Charts are generally clear and easy to understand, but sometimes to save space a bar is shortened or broken. This can exaggerate survey results, making some pay increases from one year to the next, look disproportionately large. Similar considerations apply to graphs which may have shortened vertical scales.

The relationship between salary information and another variable, such as company size, however, is often complicated. When plotted, the data can look like an array of unconnected dots which when left in this raw fashion is called a scatter diagram or scattergram.

One way of illuminating the relationship between the variables is to plot a line of ‘best fit’ between the data points. These are regression lines (usually straight) and are calculated by complex formulae.

More often than not, salaries are not simply related to a single variable such as company size, but are associated with more than two factors. The line of ‘best fit’ between a number of variables such as salary levels, company size and job responsibility is called a multiple regression line. Analytical sophistication, however, is no substitute for poor data and relating top pay to more than one variable may produce confusion rather than enlightenment. Variables like turnover, number of employees, age and job level are unlikely to be statistically independent. Neither is the relationship between variables always best expressed by a straight line.

While regression lines can be helpful in highlighting relationships between two or more variables, they can also mislead by giving results a spurious precision, making the real world seem simpler than it is.

Considering the level of analytical sophistication needed and the questionable value of some of the results, it is not surprising that regression is rarely used in most published surveys. Producers seem to prefer giving results in several tables such as salary level by company size or job responsibility. Regression lines are helpful in other ways, however, and can be used to set company salary policy lines or compare internal salaries with market levels.

For some companies a survey bought off-the-shelf does not always produce enough detailed information. The best way around the problem is to conduct a club survey. This involves a number of firms pooling pay information by joining together as a salary club. Such clubs have been around a long time and many employers place a high value on them. Several major companies say their club surveys are by far the most useful. In this chapter we:

• look at setting up a club;
• outline information collection methods;
• provide a checklist of main points.

Firms in the same industry sector often have a common interest in pooling information about salaries, particularly in shortage areas. A salary club frequently starts with an informal exchange of information between companies which employ similar types of staff. When the surveys become more formal they are either conducted by management consultants or by the companies themselves.

Clubs tend to operate in single industries, although some cover a range of sectors such as 'blue chip' companies or British subsidiaries of US-owned multinationals. All management and professional grades are normally covered, although some concentrate on one employee category within one industry. Where a single employee category is the focus of interest this is usually because there is strong competition.

Club members say there is an advantage in knowing who the other participants are, which also guarantees that the pay information is relevant. Members also have a say in the content of the survey, suggesting what questions should be asked and how the data should be analysed. Almost as important as the survey itself, according to some, are the regular meetings between club members. Useful contacts are established and information can be exchanged informally.

Unlike commercial surveys, a club survey can normally expect a very high response rate, especially from those with a small membership. Except in mitigating circumstances, failure to provide information usually means expulsion from the club.

One club with 24 members, for example, regularly achieves a 75 per cent response rate and says anything below 60 per cent is unacceptable. This is out of a maximum of 43 job categories and 1,032 salary observations.

Clubs can also produce more accurate job matches than off-the-shelf surveys. Great care is often taken over job matching. Typically, profile job descriptions of posts are circulated, containing the job title, a brief description of responsibilities, reporting level and supervisory role together with the expected age and/or experience. Regular audits are conducted to ensure that any regradings do not distort the results.

Although club surveys have a good response rate and are based on accurately matched jobs, they are prone to the problem of ‘clustering’. Samples can easily be distorted by the domination of a few large company divisions and the results are unlikely to be representative of the industry as a whole.

Where commercial rivalry is involved anonymity is important. To protect company identities, participants are given code numbers with only those collecting the data holding the key. Some clubs have decided that all participants should be given the key and in small clubs, which have been pooling information for a long time, each others' identity is likely to be known.

Club surveys, unlike most commercial surveys, not only collect information on actual salaries, but also on salary ranges. Aggregation of company data before the analysis is carried out is sometimes desirable, particularly when only national salary information is asked for (as is usual in surveys of managers and professionals).

Surveys are generally conducted by questionnaire and the information asked for normally includes all the company job codes along with the number of postholders. Basic pay information such as average, median, highest and lowest actual salaries will be collected along with the minima, maxima and mid-points of salary ranges.

There is greater variation over the additional data collected because it is often tailored to company need. One club survey, for example, collects information on 23 areas ranging from annual salary increases to travel and accident insurance.

Today, with the aid of computers, information can be collected and reports produced quickly, in around three to four weeks. Clubs can also be strict about enforcing deadlines, although the fewer the participants the quicker the results can be processed.

Results are typically listed by company code, ranked by total cash earnings and include actual basic salaries or salary range mid-points. Also included will be the number of employees in each job category. Additional information such as the minima, maxima and mid-points of the salary ranges, the highest and lowest actual salaries and bonus payments may also be provided.

7.Using salary surveys

Knowledge about survey construction has a practical value. Salary surveys may only be one of many factors which influence company pay decisions, but they still play an important role. With limited collective bargaining for managerial and professional staff the setting of competitive pay rates is often informed by relevant salary surveys.

In this chapter we:

• outline how surveys relate to internal pay ranges;
• describe the steps involved in salary comparisons;
• look at how surveys are used and chosen.

Practice varies. Some companies aim for a particular market position such as the upper-quartile, as with blue chip firms. Some fix their internal salary levels in relation to a single management consultant's job evaluated survey. Others take a more flexible approach, making judgements based on several surveys or respond to specific market needs.

Whichever approach is adopted, it is always useful to have as much information as possible about the marketplace. Employing several surveys is probably better than using just one, but selecting the right surveys is equally important. To help with this, the IDS Executive Compensation Review publishes a directory of salary surveys every two years giving details on nearly all the major surveys available in the UK. The directory can also be accessed on the Internet at www.incomesdata.co.uk

A number of the available surveys also include information about how best to use their data for comparison purposes. Prominent examples include Computer Economics/Remuneration Economics, Monks Partnership and Reward. An illustration of the advice given in these surveys is shown in Box 7.1, which reproduces a worked example from Reward.

Chosen surveys should fit the company profile and type of job. A look at the list of participants will show whether companies of a similar size and type were included. For some jobs – such as computer staff – there are a number of well established specialist surveys. And for others, a general survey may be preferred to a specialist one with only a few participants.

A few surveys only give broad salary bands. Although this is helpful for giving a general indication of market position, the detail is not fine enough to check the competitiveness of a salary structure. Most reports, however, do include tables of average or median salaries and often deciles and inter-quartile ranges.

Survey tables should be selected which provide information that matches the company profile. A profile will include factors such as:

• company size by sales turnover, number of employees or net assets;
• industry or product sector;
• location;
• whether a subsidiary or parent company.

A comparison of jobs of similar weight and responsibility is equally important. Job titles are normally insufficient for this, but a number of surveys provide more detailed descriptions of responsibility levels. For non-participants, accurate job matching can still be difficult and in all circumstances some judgement will be needed.

Particularly when several surveys are involved, users should take some care to check that definitions and measures are consistent. Some surveys, for example, use company averages rather than individual salaries, while others base their findings on range mid- points or on the earnings of the ‘median’ or ‘representative’ job holder.

A good look should also be taken at sample sizes. Salaries based on unrepresentative samples are a poor guide to market conditions. Further, ‘rogue’ figures should be weeded out. These can normally be identified by a comparison with the salaries given above and below or with the previous year’s survey.

Once the right surveys have been found and the best information selected, a salary policy has to be determined. This will include decisions about market position such as whether to be in the lower- quartile or the upper-quartile. Again, there is a need to decide whether the aim is to match basic salaries or the total remuneration package.

Companies wanting to pay the ‘going rate’ or the ‘average’, usually find that the median is more useful than the mean because it excludes atypical salaries. A median, along with the inter- quartile range, also gives a better idea of salary distribution. With most of the practical emphasis on the median, some surveys do not even include a mean average. Whichever is finally chosen, it should be remembered that medians cannot be compared to means.

Having identified the relevant pay information, a comparison of the survey figures with internal salaries needs to be carried out. Firms wanting simply to check that internal pay levels are in line with the market can work out a ‘compa-ratio’, which is short for

comparative ratio. Alternatively, firms wishing to construct their own salary ranges based on the survey can construct a salary policy line.

A compa-ratio, as the name suggests, compares internal salaries with survey salaries. A ratio of one shows the two are equal, a ratio less than one indicates internal salaries are below the survey rate and a ratio of more than one means they are above. A survey median basic salary of £20,000 compared to an internal job rate of £18,000, for example, gives a compa-ratio of 0.9. Such compa-ratios can be calculated for either individual postholders or the organisation as a whole and can be above or below one for a number of reasons. They can be affected by the length of time postholders have been in their jobs, for example, or by differences in aggregate performance ratings.

For those with more ambitious aims – the development of company wide pay ranges – a salary policy line can be derived. A policy line is a curve or straight line that relates internal mid-points to the chosen survey target figure. A company which wanted to pay in the upper- quartile range, for instance, may plot the upper-quartile salary figures for a number of comparative posts on a graph. A regression line or line of ‘best fit’ may be obtained which serves as the company’s policy line. Assuming that the policy line represents all the mid-points, salary ranges can be constructed around it. A graphical representation of how an organisation’s pay practice can be related to both the required policy line and market rates is shown above. The graph is taken from an established textbook on reward management written by leading practitioners Michael Armstrong and Helen Murlis.

Although the policy line often relates internal salaries with survey salaries, medians or means should not be confused with salary range mid-points. Both averages may be lower than mid-points because promotion, transfer or resignation leaves the remaining postholders concentrated at the bottom of a pay range. Alternatively, they can be higher because low turnover ensures that the majority of postholders reach scale maximums.

An average is technically defined as ‘a point within a group of data which is central to the group and around which other values are distributed’. Confusion is often increased, however, by the existence of three different types of average – the mean, median and mode.

Correlation measures how closely two variables are related and may be either negative or positive. If the observed values increase or decline together there is a positive correlation. If one increases and the other decreases the correlation is described as negative. Mathematically, the correlation is indicated by a number ranging from –1 (negative) to +1 (positive). If the number is zero or close to zero this is interpreted as indicating there is no relationship between the two variab