Ovarian cancer support

Abstract

Objectives To evaluate the potential effect of computer support on general practitioners' management of familial breast and ovarian cancer, and to compare the effectiveness of two different types of computer program.

Design Crossover experiment with balanced block design.

Participants Of a random sample of 100 general practitioners from Buckinghamshire who were invited, 41 agreed to participate. From these, 36 were selected for a fully balanced study.

Interventions Doctors managed 18 simulated cases: 6 with computerised decision support system Risk Assessment in Genetics (RAGs), 6 with Cyrillic (an established pedigree drawing program designed for clinical geneticists), and 6 with pen and paper.

Main outcome measures Number of appropriate management decisions made (maximum 6), mean time taken to reach a decision, number of pedigrees accurately drawn (maximum 6). Secondary measures were method of support preferred for particular aspects of managing family histories of cancer; importance of specific information on cancer genetics that might be provided by an "ideal computer program."

Results RAGs resulted in significantly more appropriate management decisions (median 6) than either Cyrillic (median 3) or pen and paper (median 3); median difference between RAGs and Cyrillic 2.5 (95% confidence interval 2.0 to 3.0; P [is less than] 0.0001). RAGs also resulted in significantly more accurate pedigrees (median 5) than both Cyrillic (median 3.5) and pen and paper (median 2); median difference between RAGs and Cyrillic 1.5 (1.0 to 2.0; P [is less than] 0.0001). The time taken to use RAGs (median 178 seconds) was 51 seconds longer per case (95% confidence interval 36 to 65; P [is less than] 0.0001) than pen and paper (median 124 seconds) but was less than Cyrillic (median 903 seconds; difference 23. (5 to 43; P = 0.02)). 33 doctors (92% (78% to 98%)) preferred using RAGs overall. The most important elements of an "ideal computer program" for genetic advice in primary care were referral advice, the capacity to create pedigrees, and provision of evidence and explanations to support advice.

Conclusions RAGs could enable general practitioners to be more effective gatekeepers to genetics services, empowering them to reassure the majority of patients with a family history of breast and ovarian cancer who are not at increased genetic risk.

Introduction

Continuing advances in the molecular genetics of common diseases mean that primary care will play an increasing role in providing genetic advice.[1] The recent increase in referrals of people at low risk of inherited cancer, particularly breast cancer, to genetics clinics suggests that general practitioners need a range of new skills to be effective gatekeepers.[2] The ability to obtain and interpret family history information accurately is fundamental to these new skills.[3] Computers could support primary care in this new task by simplifying pedigree drawing and implementing management guidelines.[4 5]

We previously reported a qualitative evaluation of Risk Assessment in Genetics (RAGs), a computer program to support the assessment of familial breast and ovarian cancer in primary care.[6] The results of this study informed the development of the software used in the current study so that it was more appropriate for primary care. The aim of the current study was to compare two different types of computer support--RAGs and Cyrillic (an established pedigree drawing program designed for clinical geneticists)--with traditional pen and paper methods in the recording and interpretation of family histories of cancer.

Participants and methods

Participants

We wrote to a random sample of 100 Buckinghamshire general practitioners inviting them to participate in the study. After one mailing, 41 doctors agreed to join the study, of whom the first 36 respondents were chosen. The participants were paid 80 [pounds sterling] for the two hours required to perform the study.

Computer support

RAGs was developed in a collaboration between JE and the ICRF Advanced Computation Laboratory. Pedigrees are generated by first entering information about the proband and then adding data about relatives by clicking on individual icons in the family tree. The program uses PROforma technology[7] to categorise risk of breast and ovarian cancer. The program implements detailed referral guidelines that are based on the Claus model[8] and then suggests appropriate management. The Claus model is a mathematical model that predicts risk of breast cancer and is based on data from a case-control study of 4730 breast cancer cases. Cyrillic draws pedigrees and assesses risk of breast cancer, also using the Claus model, and calculates the numerical risk of carrying a mutation that predisposes to breast cancer and the cumulative risks of developing breast cancer.[9] Cyrillic was originally designed for use by clinical geneticists. This study used a modified version of Cyrillic that takes the user through a series of question and answer boxes to construct the pedigree.

Simulated cases and recommended management

We developed 18 hypothetical cases, designed to cover a range of risk levels, based on the types of referral received by the Oxford genetics clinic in the previous year. An expert panel comprising a general practitioner and a health services researcher with knowledge of cancer genetics and a clinical cancer geneticist agreed by consensus the appropriate management for each case. Management decisions were based on the strategy proposed at a UK national consensus meeting: low risk women are managed in primary care, moderate risk women at a breast unit, and high risk women at a genetics clinic.[10] The panel decided that there were six high risk, five moderate risk, and seven low risk cases. The cases were randomly allocated into three sets of six.

Study design

Each doctor was asked to manage all 18 cases, six with each method of support (RAGs, Cyrillic, and pen and paper). We used a balanced block design. To avoid any learning effect, the order in which the methods and case sets were presented was completely balanced among the 36 doctors. We also ensured that each method was used equally often with each case set (see extra table on the BMJ website). For each case, the doctor was asked to create a pedigree and decide on management using the principle of triaging the patient as low, moderate, or high risk. The two computer programs were set up on a laptop computer in the doctor's consulting room. The participants were familiarised with each program with one or two test cases before conducting the study. When the doctors used pen and paper to manage cases they were allowed to use any paper referral guidelines that were available to them. Although all Buckinghamshire general practitioners had been mailed management guidelines in 1997, only three of the doctors in our study had access to these in their consulting room.

Outcome measures

The principal outcome measures were the number of appropriate management decisions made for each set of six cases, the mean time taken to reach a decision, and the number of pedigrees accurately drawn. A pedigree was considered correct only if it used standard symbols and lines and contained information about the age of the proband, type of cancer, and age of onset for each affected relative. After managing all 18 cases, the participants completed a questionnaire asking them to rate, on a five point Likert scale, the three methods for particular aspects of managing family histories of cancer, and the importance of specific functions or information on cancer genetics that might be provided by an "ideal computer program."

Sample size and statistical analysis

From the results of a pilot study, we calculated that we required 25 doctors to detect a mean difference of 1.5 in management scores (SD 1.6) between RAGs and Cyrillic with 90% power and two sided [Alpha] = 0.05. The sample size was increased to 36 to make a completely balanced study design. We used Friedman's two way analysis of variance to compare effects overall for each outcome. To compare different pairs of support for each outcome, we used Wilcoxon's matched pairs signed rank test. We used SPSS for Windows (version 8) for the Friedman's and Wilcoxon's tests, and the confidence interval analysis program (CIA)[11] to calculate differences in medians and associated confidence intervals.

Results

Characteristics of participants

The characteristics of the participants were similar to those who chose not to enter the study. Of the 36 doctors selected to participate, 69% were men, 61% held the MRCGP, and their median time since qualification was 21 (range 7-36) years. Of the 59 non-participants, 61% were men, 56% held the MRCGP, and their median time since qualification was 21 (range 8-39) years.

Outcomes