This course is only offered in English | Le cours est offert en anglais seulement

Spaces limited. Register early to avoid disappointment.

Date: Wednesday, December 4, 2013

Time: 1:00 PM – 5:00 PM EST

Course Format: Interactive instruction and discussion.


Presenter:   Ruth Croxford, MSc, PStat Senior Research Coordinator Institute for Clinical Evaluative Sciences (ICES) ruth.croxford@ices.on.ca


Introduction to the Use of Administrative and Observational Data in Health Services Research

Click here to download the course description. 

Registration Fees:

Registrant Type Cost
Non-member $300 +hst
CSEB member $150 +hst
Student non-member $75 +hst
Student  CSEB member $40 +hst
Groups of 10 participants or more Special rates. Please contact us at secretariat@cseb.ca

All profits from this webinar will go to the CSEB Student Bursary Fund. HOW TO REGISTER:Registration is now closed. Cancellation Policy:  Refunds will be given for requests received in writing postmarked no later than November 20th, less an administration fee of $20.00. After that date, we regret no refunds will be issued; however, we will accept substitute delegates. Certificate of Completion: Participants may be issued an electronic certificate of completion issued by the Canadian Society for Epidemiology and Biostatistics upon completion of the course. (Upon request) Required Materials:

  • A computer/laptop
  • Speakers/headphones/headset (microphone and headphone combo)
  • A reliable internet connection

Optional Materials:

  • Microsoft Excel
  • A headset or microphone is required should the participant wish to ask questions or participate orally to the session

Webinar recording: The webinar will be recorded and made available to those attendees who have paid but are unable to participate in the live presentation. Ruth Croxford Bio:Ruth Croxford is a senior research coordinator at the Institute for Clinical Evaluative Sciences (ICES).  ICES receives administrative data sets from Ontario’s Ministry of Health, which are de-identified and linked in order to conduct research on a broad range of health care issues. As a research coordinator, Ruth helps design research studies, creates the analytical datasets and conducts the statistical analyses.  Her recent work has focused on research in arthritis, chronic obstructive pulmonary disease (COPD), measuring system performance, and health economics.  Prior to coming to ICES, she coordinated the Statistical Consulting Service at the University of Toronto. Ruth has a Masters in Computer Science from Queen’s University and a Masters in Statistics from the University of Toronto. Course Outline: Topic 1:  Using Administrative Data for Health Services Research: Small Area Variation Analysis and Indicators Learning objectives:

  • To obtain an overview of the sorts of data that are available for health services research
  • To understand the possibilities and limitations of research based on observational data
  • To explore one important component of health services research – quantifying variations in rates

Topic Outline:

  • What is health services research
    • A brief definition
    • An introduction to the data sources: observational data and administrative databases (vs. data obtained using planned experiments/clinical trials)
    • Examples of research using observational data
    • Pros and cons of observational data
    • Small Area Variation (investigating regional variation in rates of disease, health care use, surgical procedures, hospital error rates, etc.)
      • A brief history
      • Why rates vary
      • Standardizing rates: why standardize?, and how to standardize
      • Quantifying the variation in rates.
      • Indicators: a specific example of rates

Required prior knowledge: none

Recommended reading: Here are some of the earliest publications on unwarranted variations in health care:

  • Wennberg J, Gittelsohn A. Small area variations in health care delivery. Science 1973 Dec 14;182(117):1102-8.
  • Wennberg J, Gittelsohn A. Variations in medical care among small areas. Sci Am 1982 Apr;246(4):120-34.
  • Wennberg JE, Freeman JL, Culp WJ. Are hospital services rationed in New Haven or over-utilised in Boston? Lancet 1987 May 23;1(8543):1185-9.

To get an idea of the wealth of possibilities for research in rate variation, browse through the Atlases produced by a number of Canadian health services research institutions.  Here is a very abbreviated list:

To get an idea of the range of indicators of quality health care measured using observational data, look at the POWER (Project for an Ontario Women’s Health Evidence-Based Report) Study website: www.power.ca. Pre-Course Exercises: You are encouraged to think about a (simple) research question that you’re interested in and that you think might be answered using observational data.  At the end of the lecture, we can discuss your ideas – and whether they can be answered using available data.  So as not to completely stymie the lecturer, consider submitting your idea ahead of time! Some participants may have experience using information on rate variations, either across provincial health regions or among hospitals.  If you have questions about how they are calculated or how the variations in rates can be interpreted, you’re welcome to ask.  Again, you may want to submit the questions ahead of time, in the hopes of getting a more useful answer.

Topic 2: An Introduction to Using Administrative Data for Health Services Research:  Analysis of Observational Data – adjusting the analyses to control for bias

Learning objectives: Observational data is a valuable research tool, and can be used to compare health care treatments.  However, a major drawback is that the researcher has no control over which subjects are assigned to each treatment group. This webinar will discuss the propensity score – a method that is commonly used to adjust for bias.  It will also touch briefly on a second method, instrumental variable analysis. The objectives are:

  • To understand how observational data differs from experimental data
  • To understand how the propensity score is calculated and how it is used to answer research questions

Topic Outline:

  • Observational data versus randomized clinical trials
  • Statistical adjustment in observational studies
    • Multivariable regression adjustment
    • Calculating and using the propensity score
    • Instrumental variable analysis (very briefly)

Required prior knowledge:   This webinar will build on some of the material presented during the first session.   The first session is not a prerequisite, but some familiarity with administrative databases will be assumed.  I will also assume that participants have some familiarity with the concept of regression, and in particular, logistic regression.

Recommended reading:

  • The propensity score was introduced by Donald Rubin and Paul Rosenbaum.
  • An example of Rubin’s writing is: Rubin, DB.
  • Using Propensity Scores to Help Design Observational Studies:  Application to the Tobacco Litigation.
  • Health Services & Outcomes Research Methodology 2001; 2:169-188.
  • A comparison of 4 different methods to remove selection bias that results from using observational data: Stukel TA, Fisher ES, Wennberg DE, Alter DA, Gottlieb DJ, Vermeulen MJ.
  • Analysis of observational studies in the presence of treatment selection bias: Effects of invasive cardiac management on AMI survival.
  • Propensity score and instrumental variable methods.  JAMA. 2007; 297 (3): 278-85.
  • One of the early examples of the use of instrumental variable analysis in health services research is McClellan M, McNeil BJ, Newhouse JP.
  • Does more intensive treatment of acute myocardial infarction in the elderly reduce mortality?  Analysis using instrumental variables.  JAMA.  1994; 272(11): 859-66.

For numerous articles discussing how to perform propensity score analysis and/or providing examples of  research based on propensity score analyses, look for articles co-authored by Peter C. Austin.  Here are a few:

  • Assessing Balance in Measured Baseline Covariates when using Many-to-One Matching on the Propensity-Score.  Pharmacoepidemiol Drug Saf. 2008; 17(12): 1218-25 Austin PC, Mamdani MM.
  • A Comparison of Propensity Score Methods: A Case-Study Estimating the Effectiveness of Post-AMI Statin Use.
  • Statistics in Medicine. 2006; 25(12): 2084-106. Austin PC, Mamdani MM, Stukel TA, Anderson GM, Tu JV.
  • The use of the propensity score for estimating treatment effects: administrative versus clinical data. Statistics in Medicine 2005; 24:1563–1578 Clinicians tend not to be convinced by treatment comparisons conducted using observational data, preferring to see randomized clinical trials.

For an idea of some of the sensitivity analyses conducted in order to augment a propensity score analysis, see the on-line appendix for:

  • Gershon A, Croxford R, To T, Stanbrook MB, Upshur R, Sanchez-Romeu P, Stukel T.
  • Comparison of inhaled long-acting B-agonist and anticholinergic effectiveness in older patients with chronic obstructive pulmonary disease: a cohort study. Ann Intern Med. 2011; 154(9): 583-92

Pre-Course Exercises: Chances are that if you are registered to this course, you are considering a research project that will use observational data.

If you already have questions that are likely to be of general interest, consider submitting them ahead of time.