Section II: Methodology

• Benchmark Positions
• Sample Selection
• Questionnaire, Job Matching and Data Collection
• Data Validation
• Sample Weights

 

Benchmark Positions

The Health Services group in the federal public service includes approximately 2,600 employees in nine occupational groups. The parties jointly identified the positions to study and agreed to focus on the following 24 benchmark positions:

Table 1: Benchmark Positions

Occupational Group	Benchmark Position
DE - Dentistry	1 - Dentist
MD - Medicine	MD-MOF - Medical Officer 2 - Medical Officer - General Practice 3 - Medical Officer - Medical Expertise 4 - Medical Officer - Public Hygiene	MD-MSP - Medical Specialist 5 - Specialist Physician 6 - Department Head Physician
ND - Nutrition and Dietetics	7 - Dietitian (Clinical Nutrition) 8 - Community Nutritionist (Advisory)
NU - Nursing Sciences	NU-CHN - Community Health Nursing 9 - Community Health Nurse 10 - Occupational Health Nurse 11 - Nurse Practitioner	NU-HOS - Hospital Nursing 12 - Nurse 13 - Assistant Head Nurse 14 - Head Nurse 15 - Internal Nurse Educator
OP - Occupational and Physical Therapy	16 - Physical Therapist 17 - Occupational Therapist
PH - Pharmacy	18 - Pharmacist
PS - Psychology	19 - Clinical Psychologist 20 - Industrial Psychologist 21 - Chief or Department Head
SW - Social Work	22 - Social Worker
VM - Veterinary Medicine	23 - Public Health Veterinarian 24 - Veterinary Pharmacologist

A group made up of representatives of the employer (TBS), the bargaining agent (PIPSC) and the PSLRB (CARS) developed 24 job capsules (See Summaries of Job Capsules in Appendix B) based on the work descriptions of the various benchmark positions.

Sample Selection

Statistics Canada developed the survey sample for the requirements of this study. Health care establishments were selected from the Statistics Canada Business Register - a structured list of businesses engaged in the production of goods and services in Canada. At the beginning of the study, the parties identified a group of industries that they felt should be surveyed in order to provide sufficient coverage of the 24 benchmark jobs. Statistics Canada analyzed the proposed industries in terms of presence or not of the selected jobs and size of the enterprises. The analysis was based significantly on data obtained from the 2001 Census of population. Based on this analysis and the parameters of the study (quality standards, time available for collection activities and allocated budget), Statistics Canada concluded it was not possible and/or relevant to cover all industries of interest. Consequently, the following industries which had been initially selected were excluded from the study: Pharmaceutical & Medicine Manufacturing industries (NAICS 325410), Pharmacies and Drug Stores (NAICS 446110), Testing Laboratories (NAICS 541380), Universities (except for the 5 university veterinary schools) (NAICS 611310), Offices of Physicians (NAICS 621110) and Offices of all other Health Practitioners (NAICS 621390).

Table 2 below shows the remaining target establishments with their codes and descriptions as prescribed in the 2002 North American Industry Classification System (NAICS) and the threshold upon which the target population was established. The sampling unit is the establishment.

Table 2: Target Establishments

NAICS Code	NAICS Description	Threshold (Minimum Number of Employees)
621340	Offices of Physical, Occupational and Speech Therapists	100
621420	Out-patient Mental Health and Substance Abuse Centres	200
621494	Community Health Centres	200
622111	General (except Paediatric) Hospitals	200
622112	Paediatric Hospitals	200
622210	Psychiatric and Substance Abuse Hospitals	200
622310	Specialty (except Psychiatric and Substance Abuse) Hospitals	200
623110	Nursing Care Facilities	200
623222	Homes for the Psychiatrically Disabled	200
N/A	5 Veterinary Universities (Université de Montréal, University of Guelph, University of Prince Edward Island, University of Saskatchewan, and University of Calgary)	N/A
N/A	Workers’ Compensation Boards	N/A
N/A	Provincial/Territorial Governments	N/A

On the basis of these criteria, the target population of the study was determined to be 858 establishments. Taking into account the parameters of the study (time available for collection activities and allocated financial and human resources), Statistics Canada estimated that a minimum of 200 establishments needed to be visited. This initial sample of 200 establishments was then allocated by Statistics Canada to the sampling strata in an optimal fashion such that reliable estimates for the census regions could be produced without having significant adverse impact on the reliability of the national estimates.

Take-all Strata

There were five take-all strata (census) defined for the study. The expected response rate for each of these strata was 50 percent (100 percent in Quebec), leading to the expected number of respondents for the take-all strata as shown in the table below.

Table 3: Expected Number of Responding Establishments from the Take-all Strata

Stratum	Number of Establishments	Expected Number of Respondent Establishments
Quebec	264	5*
Territories	2	1
Workers Compensation Boards	11	6
University Veterinary Schools	5	3
Provincial/Territorial Governments	13	7
Total	295	22

*It should be noted that all Quebec establishments were considered to be equivalent to 5 establishments for data collection purposes because of “centralized” data administration.

Take-some Strata

After accounting for the establishments that were in the take-all strata, there were 563 establishments from which to select 178 establishments in the take-some strata in order to meet the minimum number of establishments required for this study (200 establishments). The sampling strata for the remaining units were defined using cross-classification of the census area by six‑digit NAICS codes.

Table 4: Sample Allocation Across Census Regions

Census Region	Establishments	Sample	Adjusted Sample Based on 50% Estimated Response Rate*
Newfoundland, Prince-Edward Island, Nova Scotia, New-Brunswick	73	33	66
Ontario	268	64	128
Manitoba, Saskatchewan, Alberta	135	45	90
British Columbia	87	36	72
Total	563	178	356

*Since the expected response rate is 50%, the required sample size is doubled.

Therefore, a total of 637 establishments (264 establishments from Quebec, 17 other establishments from the take-all strata plus 356 establishments from the take-some strata) constituted the sample for this study. Of these, 448 agreed to participate in the study, with 436 providing sufficient data to be included in the report.

Questionnaire, Job Matching and Data Collection

Questionnaire

A questionnaire was developed by the PSLRB and approved by the parties. The questionnaire comprised 23 sections. Twenty-one of those sections were meant to collect data on the incidence and characteristics of certain benefits and working conditions regrouped under the following topics:

• Basic Group Life Insurance and Related Plans;
• Sickness Indemnity Plans;
• Dental Care Plans;
• Supplementary Health Care Insurance;
• Paid Time Away from Duty Benefits;
• Hours of Work;
• Paid Rest and Meal Periods;
• Pay Supplements;
• Educational Leave/Educational Assistance Plans;
• Membership Fees (Professional Associations);
• Severance Pay Plans and Retirement Allowances;
• Pension Plans.

Another section was designed to gather data on initiatives that were put into place by employers to address the recruitment and retention issues whereas the last section was designed to elicit information on the costs incurred by employers for the different components of total compensation.

When it was possible to obtain the information prior to the interview, the questionnaires were pre-coded with data extracted from collective agreements and benefit brochures and confirmed with the respondent at the interview. Otherwise, the interviewer completed the questionnaire on-site with the respondent representative(s).

Job Matching

In order to meet the requirements of this study, benchmark job matching was used as a method to determine whether jobs in respondent organizations were comparable enough to the benchmark positions in the Federal Public Service to be declared equivalent in terms of the essential aspects of the work. The job capsules developed by the parties were used as a basis for job matching in face-to-face interviews with the respondents. Each job capsule covers a different job and contains a summary of the responsibilities carried out by the job, information pertaining to minimal education and employment requirements (membership in professional associations), a list of job matching criteria (typical duties) and other additional information which Survey Officers used to explain the nature of the job during survey visits and to make decisions on job matching. A number of the matching criteria were identified as mandatory for a match to be established.

In order to facilitate the job matching process at the interview and to ensure that the information was compiled in a standardized way, interviewers were provided with job matching sheets which were pre-filled when the work descriptions had been provided prior to the interview or completed on-site with the respondent representative(s). Questions or issues pertaining to the job match were noted on the job matching sheet and referred to the Project Manager for follow-up or resolution as applicable.

Data Collection

The study was launched by a letter sent to the deputy ministers of health in each province and territory describing the study and advising that the PSLRB would directly contact the participating establishments selected by Statistics Canada as part of the sample. The letter was followed immediately by a written communication to health care establishments to solicit their participation.

Concurrently, the PSLRB retained the services of the Ottawa firm Hackett Consulting Inc. to hire interviewers. The interviewers all had extensive experience in this field. A one-week training session was provided to them in February 2008 in order to familiarize them with the 24 job capsules, the questionnaire and the various software applications to be used to gather data.

Data collection began immediately after the training session. The first two weeks were spent contacting the establishments by telephone to confirm their participation in the study, to identify potential matches and to obtain documents (work descriptions, collective agreements, benefits brochures, etc.) necessary for the interviewer to prepare for the on-site interviews and pre-code the questionnaires.

During that period, pilot visits were made to establishments in the National Capital Region to test the questionnaire, the data collection process and the administrative procedures. The pilot visits demonstrated the value of pre-coding the data to be confirmed with participating establishments, thus speeding up data collection and reducing the length of on-site interviews.

Interviewers and CARS staff spent eight weeks holding face-to-face interviews with the participating establishments in all parts of the country. During the interviews, the interviewers confirmed matches, validated pre‑coded information and completed questionnaires. Subsequently, they followed up with the establishments to obtain data or documents that were not available during the interviews. As a result, the data collection period was extended by six weeks.

It should be noted that respondents were asked to provide information on permanent full-time employees only (working at least 30 hours per week). Self‑employed persons and casual, part‑time and contract employees were not considered and neither were physicians or other employees paid on a fee‑for‑service basis.

Data Validation

Upon completion of the field work, questionnaires were keyed in and verified by CARS staff in order to ensure the accuracy of the information. Collective agreements and other supporting documentation were researched to confirm data and investigate anomalies and revisions were made as required. CARS staff members also performed a spot check verification of job matches in order to verify their accuracy and reliability. Preliminary tables were then run and the aggregate data was verified in order to identify discrepancies which might not have previously been identified. A thorough investigation of these discrepancies was done and the tables were adjusted accordingly prior to publication.

Sample Weights

Random sampling enables us to scientifically measure the response of a subset of a certain population. If properly selected, the readings from this subset i.e., sample, represent the behaviour of the survey universe within a defined margin of error.

However, a sample only provides a portion of the actual measure. To obtain the total measurement, it is necessary to multiply the sample reading by a factor which is the inverse of the proportion that the sample is composed of, relative to the universe. This factor is known as the sample weight. To summarize, weighting enables the survey responses to be expanded in order to represent the survey universe.

In the presence of non-responding units, a non-response adjustment factor must be calculated. This factor represents the ratio of the initial sample size and the number of responding units. This non-response adjustment factor is multiplied by the sampling weight to obtain the final weight.

In this study, health-sector establishments form the sampling unit. The establishment-level base weights are equal to the reciprocal of the probabilities of selection of the sampled establishments.

The sample was selected using a stratified design on the basis of geographic regions and size of the establishments. The probability of selection of an establishment is the proportion of establishments sampled from the stratum. The province of Quebec, however, provided data for all health establishments in the province and was therefore assigned unit weight.

The Statistics Canada methodology prescribed the application of a non-response adjustment factor to the stratum weights in order to enable the respondent data to make up for the portion of the weights that was originally assigned to the non-respondents.

The non-response adjustment factor is calculated for each stratum by dividing the frequency of the sample establishments by the number of the responding units. The product of this factor and the original sample stratum weight then yields the final adjusted sample weight. This new factor is then multiplied with the number of responding establishments in each stratum to derive the number of establishments shown in the tables in this report.

This approach assumes that the responding establishments follow the same normal distribution by size (in terms of the number of employees, as reported in the 2001 Census) as that of the sample. After consultations with Statistics Canada, in situations where a disproportionate representation of large-size respondents was observed, the non-response adjustment factor for incumbents was re-calculated as the ratio of the size of the establishments in the sample and that of the responding establishments.

Previous || Next