In Australia, as in many other parts of the world, patients cannot make appointments with a medical specialist without first consulting a GP. Patients are largely dependent on GPs for advice about the significance of symptoms. The GP must decide whether and when to seek a specialist opinion. Referral to a specialist requires a referral letter from the GP, which may be preceded or followed up by a telephone conversation with the specialist. Grounds for concern should be stated explicitly in the referral letter, especially if an urgent appointment is being sought.

In most cases, the referral letter is the main source of information used by the specialist to prioritise cases. A recent audit of UK GPs suggests that the referral letter, more so than the medical record, is used to determine relevant clinical details at the time of referral 1. However, it has been demonstrated that GPs generally record few clinical details in their correspondence with specialists, and that referral letters are often perceived as a 'ticket of entry' 2. This is of concern, as lack of clear information in the referral letter may be a contributing factor to delayed diagnosis.

Conversely, following specialist treatment, the GP is often the primary contact for patients seeking advice about follow-up care and ongoing support. This is particularly the case for patients from regional and rural areas whose cancer treatment is not delivered locally. Timely communication from the specialist to the GP about treatment plans and likely side effects of treatment is essential in order to ensure that care is adequately coordinated at the local level.

1. Jiwa M, Gordon M, Colwell B, Skinner P, Freeman J. Where is the relevant history and examination recorded? A review of documents in general practice. Quality in Primary Care. 2006;14:235-238.
2. Jiwa M, Coleman M, McKinley RK. Measuring the quality of referral letters about patients with upper gastrointestinal symptoms. Postgraduate Medical Journal. 2005;81:467-469.

The toolkit was derived from a pilot project conducted in Western Australia by Professor Moyez Jiwa and his team at the Curtin University of Technology. The pilot involved recruitment of GPs and specialists in Albany and Perth. The project used a process of review and feedback to rate the importance of a range of information in letters regarding colorectal, breast, urological, gynaecological, upper gastroenterology and respiratory cancers. The result is a checklist of information that the group of GPs and specialists consider to be of importance in providing treatment and care.

The project team has published a paper about the project:

Jiwa M, Deas K, Ross J, Shaw T, Wilcox H, Spilsbury K. An inclusive approach to raising standards in general practice: Working with a 'community of practice' in Western Australia. BMC Medical Research Methodology. 2009;9:13.

Toolkit developers

The team developing this toolkit included:

• Professor Moyez Jiwa and Ms Kathleen Deas, from the Western Australian Centre for Cancer and Palliative Care at Curtin University of Technology
• Associate Professor Tim Shaw and Ms Jackie Ross, from the Centre for Innovation in Professional Health Education and Research at the University of Sydney

The project was funded by Cancer Australia.

The project team gratefully acknowledges all the general practitioners and hospital specialists who participated in the pilot project in Western Australia on which this toolkit is based. Thanks also to Dr. Ruth McConigley for assistance with the early stages of this project, including coordinating ethics approval and recruitment of some practitioners.

Likely timeframe for this phase: 6 weeks

This is a challenging project that will require careful planning and clear identification of roles and responsibilities from the outset.

Suggested approach

• project coordinator to oversee the project: the project coordinator may be a nurse or other clinician or someone in an administrative role.
• champion to encourage participation and assist with troubleshooting: the champion should be a respected opinion leader from within or outside the health service area who has an understanding of the aims of the project and is supportive of the need for change; it may be useful to identify a GP champion and a specialist champion.
• reviewers who can score referral letters and/or discharge summaries: reviewers should have clinical experience or access to a GP and/or specialist for advice.

Ideally the project champion will also participate in the project and therefore should not be involved in scoring the letters.

• decide which specialties to focus on: the toolkit can be used for any or all of the following: colorectal, breast, urological, gynaecological, upper gastroenterology and respiratory specialists.
• decide what to measure: you may choose to include GP referral letters and specialist discharge summaries or to restrict the project to referral letters only or discharge summaries only.
• agree a project timeframe: set a timeframe for the project, with clearly defined milestones.
• consider risk management strategies: it can be helpful to pre-empt issues by thinking about potential barriers or risks to the project and what strategies you can use to overcome these issues; this toolkit provides some troubleshooting advice for each stage of the project based on the pilot study.
• Consider gaining ethics approval, only necessary if you wish to publish the findings of your project.

Likely timeframe for this phase: 4 weeks

Successful recruitment will depend on participants understanding the aims and benefits of participation and feeling confident that the information they provide will be treated confidentially. It is important that GPs and specialists are comfortable with providing information and comparing their practice with colleagues.

Depending on the size of the service you should aim for up to 15 participants. If your project includes specialists, ideally it would be helpful to recruit a representative group of specialist practitioners, either leaders in the local specialist network or respected senior figures.

It is likely that there will be fewer specialists than GPs, for example if you are going to focus on referrals to a particular service you may have 10-12 GPs but only 2-3 specialists etc. The specialists involved should ideally be the more senior or widely respected members of their discipline.

Suggested approach

• develop personal contact with potential participants – this will be critical to the success of the project; it can be helpful to enlist the support of practice managers when recruiting GPs and PAs, hospital service managers or other members of the specialist team when recruiting specialists.
• reassure participants that their data will be handled confidentially. Refer to letter of introduction for an example.
• clearly identify the rationale for the project and the importance of comprehensive referral letters.
• emphasise the benefits of reciprocal information about the quality of discharge summaries for the GP and communication from specialists; this may help to keep GPs engaged.

Troubleshooting

Below are some of the barriers you may encounter in recruiting general practitioners, together with some suggested solutions based on the experience of the toolkit developers.

"Letter writing is not a high priority for me"
Many practitioners view letter writing as relatively trivial and unimportant activity. Few will consider their letters wanting. In many cases this attitude may be justified with reference to referrals with obvious pathology, eg surgical correction of an inguinal hernia. It may be helpful to identify cases in which thorough and systematic screening for signs and symptoms is imperative to assist the specialist in determining the approach to investigation and diagnosis e.g. unexplained iron deficiency anaemia.

"My referrals are appropriate"
Some practitioners may feel that they assess patients adequately before referral but few will be able to find the evidence in their letters or notes. The letter is the only information available at the clinic to schedule an appointment and arrive at a timely diagnosis. Phone conversations with specialists are not routine for most referrals. GPs may not regard the pressure on hospital clinics to prioritise appointments as their problem. It is important to anticipate this argument and to respond with a clear rationale for promoting the project. It may be helpful but not sufficient to cite some relevant literature.

"What's in it for me?"
GPs may view the project as seeking only to highlight their poor practice. A focus on the quality of discharge summaries is a useful inclusion that acknowledges the importance of communication from specialists as a measure of quality of care.

Incomplete relay of information after the patient has been seen in hospital is a recognised and oft cited problem in primary care. The consequences may be serious, compromise patient safety and waste a great deal of general practitioner time. In our preliminary study specialists were keen to participate.

"Will you reimburse me for my time?"
Practitioners may seek payment for spending time on this aspect of the audit and local negotiations will need to address this issue.

Likely timeframe for this phase: 6–8 weeks – the toolkit developers note that a relatively short duration for collection of letters may be helpful in maintaining momentum.

Collection of referral letters from participating GPs and discharge summaries from specialists is integral to the audit process.

Suggested approach

• collect letters prospectively for 12 weeks; or
• collect at least 2 referral letters per specialty, per GP participant; and/or
• collect 6 - 12 letters per specialist (6–8 weeks for collection of letters).

Referral letters and discharge summaries should be de-identified, ie, submitted with all possible patient identifying details thoroughly removed. It is important to preserve the privacy of patients and this aspect will need close attention if breaches of confidentiality and complaints are to be avoided.

Troubleshooting

Below are some of the issues you may encounter in collecting letters and discharge summaries, together with some suggested solutions based on the experience of the toolkit developers.

"Can I provide letters retrospectively?"
In most cases it will not be possible for GPs to easily identify patients they have referred in the past.

"I'm only part time so won't generate enough letters"
It may be possible to accept referral letters penned on the basis of paper cases or vignettes. click here for some suggested 'paper cases'. Letters based on 'vignettes' or 'paper cases' are less than ideal but better than excluding enthusiastic and motivated practitioners who may wish to participate but are precluded by virtue of not referring many patients.

Likely timeframe for this phase: 6 weeks (can be happening at the same time as letter/discharge summary collection).

This is the ‘standard setting’ or benchmarking phase of the project and involves canvassing views from GPs and specialists about the information they view as being most and least important within a referral letter. This information will be used to score the letters collected from the GPs.

Suggested approach

1. inclusion of clinically relevant signs and symptoms in GP referral letters. Click here for a GP referral survey template.
2. inclusion of clinically relevant feedback from specialists to GPs in specialist discharge summaries. Click here for a discharge summary survey template.

The questionnaire could be administered electronically however a paper-based approach will work perfectly well. Make sure to include a reply paid envelope and/or a fax number for return. In addition, it is recommended that you collect practitioner demographics to help establish the characteristics of those practitioners most likely to respond to this exercise and change their practice. Click here for a sample demographic information questionnaire.

Troubleshooting

Below are some of the issues you may encounter in collecting letters and discharge summaries, together with some suggested solutions based on the experience of the toolkit developers.

Delayed responses
Responses may take longer than 6 weeks to collect. Practice managers and personal secretaries or other specialist team members may be helpful in encouraging a timely response.

Two reminder letters and regular contact with respondents may be necessary for a good response. In practice, with personal recruitment, it should be possible to get a 100% response, although even committed practitioners are notoriously slow to reply to surveys.

Realistic expectations
In our experience it is likely that participants will set very high standards, which may be difficult to achieve in practice. It is important to ensure that practitioners understand that each item of history and examination that is accorded importance will need to be documented in every referral letter regardless of whether that information is germane to the diagnosis. For example, if practitioners suggest that the patient’s occupation should be included in every referral letter, they may later be disappointed if their letters score poorly when in practice they seldom record this information or consider it superfluous in most referrals.

Likely timeframe for this phase: 6 weeks

Responses from GPs and specialists should be collated in order to guide the scoring of referral letters and discharge summaries.

Suggested approach

• Enter questionnaire responses into the Excel spreadsheet(s) provided: there is one spreadsheet for GP responses and one for Specialist responses.
• Use the formula in the spreadsheets to calculate median scores for up to 15 participants and derive the requisite summary measure.
• Ideally it is important to share this final scoring schedule with the participants at this point to ensure that all are comfortable with the benchmarks that have been set during this exercise.

Troubleshooting

Below are some of the issues you may encounter in collecting letters and discharge summaries, together with some suggested solutions based on the experience of the toolkit developers.

Why collect the median scores?
The responses to the questionnaires will be ‘skewed’. In other words most practitioners will regard each item of history and examination to be very important or very unimportant. Few items will evoke ambivalent responses. Therefore the data will need to be summarised as ‘median’ scores to reflect the majority view.

The scores do not represent a hierarchy of importance for clinical information. Many items of history and examination will be judged of equal importance.

Do I need statistical advice?
Specialist statistical advice is unnecessary for the purposes of this project.

Managing differing opinions
You may find that GPs and specialists have differing views on what is required information in a referral letter or discharge summary. Feedback to GPs and specialists can reflect these alternative views or be dealt with later after the practitioners have had a chance to reflect on their scores and made comment.

'Ownership' of information
Some GP participants may be more interested in the benchmarks set by specialists on the basis that as recipients of the letters they have an important stake in what information is relayed. However others may wish to set their own standards. As ownership is an important element of the audit process, it is recommended that in the first instance, feedback about survey responses reflects standards set by the GPs themselves.

Likely timeframe for this phase: 16 weeks - 12 weeks to collect the letters and 4 weeks to complete the scoring.

Each referral letter and/or discharge summary should be scored based on the benchmarking scores set by the participants themselves (in Step 3).

Suggested approach

Scoring

• be done by someone with clinical experience or with reference to a GP and/or specialist.
• be done by the same person for the first and second set of letters (i.e. before and after feedback).

Excel spreadsheets with the relevant statistical measures are provided: GP referral letter scoring spreadsheet; and Specialist discharge summary scoring spreadsheet.

Simple descriptive statistics will suffice, using the summary measures: mean scores, maximum scores, minimum scores and range.

Troubleshooting

Below are some of the issues you may encounter in collecting letters and discharge summaries, together with some suggested solutions based on the experience of the toolkit developers.

Interpreting shorthand
Some GPs use short-hand when describing patients. Several clinical details are summarised in one or two key phrases. Therefore ‘prostatism’ may be short hand for ‘hesitancy and frequency’.

A list of short-hand terms is included in the letter of introduction although we suspect there may be many others. It is possible that there is already local agreement about the terms used in letters or common local abbreviation. These short-hand items may not be highlighted until specific examples of letters are presented in this audit.

How to manage ambiguity
The audit exercise aims to encourage clear documentation. Therefore where there is ambiguity we recommend scoring the practitioner lower rather than making generous assumptions about the meaning of information or phrases recorded. On the downside this may increase any sense of injustice in the process and objections expressed will need to be handled carefully and immediately at the feedback stage.

Likely timeframe for this phase: 6 weeks

Feedback to practitioners is an important element of the audit process. It is important that this stage is handled sensitively and professionally.

Suggested approach

Giving feedback

• Provide feedback to each participant about their scores for each letter or discharge summary. The use of colour and symbols may help to make the material digestible. Click here for sample feedback letters for GP participants; and Specialist participants.
• For GPs, provide details of their overall performance for each specialty area compared with other participants in the project.

The style of feedback presented in this toolkit has been produced over several iterations and in close consultation with volunteers in the pilot study. It includes a brief guide on ‘how to’ make sense of the feedback.

The final style and content of the feedback will need to be endorsed by the reviewer. Therefore the format offered here is intended only as a guide to what was considered acceptable by volunteers when the process was road tested in Western Australia.

It is recommended that letters and discharge summaries are sent back to their authors when giving feedback as participants are unlikely to remember the document to which the score applies. Part of the letter sent at the time of recruitment may also need to be included.

It is important that the feedback is unambiguous, invites comment and is able to be refined. It also needs to be endorsed by the letter reviewer. Sensitivity is needed as participants may be disappointed and even upset by their scores.

Seeking and responding to comments

• Every comment received from participants should be acknowledged and a response drafted to explain how these comments helped to work with the participants in the project.

It is important to maintain an open and inclusive approach by inviting comment from participants on the format and contents of the feedback. This is important in creating and maintaining a sense of partnership and local ownership over the audit process, helping to ensure its success.

It may also be worth revisiting the survey (Step 3) so that practitioners are satisfied with their benchmarking scores in the preliminary standard setting exercise.

Troubleshooting

Below are some of the issues you may encounter in collecting letters and discharge summaries, together with some suggested solutions based on the experience of the toolkit developers.

Managing disappointment
The approach to giving feedback warrants particular attention. In the experience of the toolkit developers, many practitioners will be disappointed and even upset by their scores. This reaction may be an important element of the subsequent motivation to change; however it needs to be handled sensitively if participants are not to become disillusioned or disheartened.

Addressing issues of subjectivity
It may be difficult to be sure if feedback has captured some of the nuances relating to the subjectivity of the scoring process. It is recommended that a very clear statement is made when providing feedback to indicate that there is an element of subjective bias when scoring letters and that the scores will need to be interpreted with this in mind.

Building on lessons learned
In preparation for the final phase of the project it will be important to take on board lessons learned throughout the process. These lessons relate to local issues, in particular accepted shorthand terms used locally.

Revisiting the scoring process
It may be worth revisiting the questionnaire so that practitioners are satisfied with their scores in the preliminary standard setting exercise and do not place undue emphasis on items of history and examination that on closer scrutiny are clearly not as important in practice as they are considered at first glance. Volunteers in the preliminary audit in Western Australia were not asked to revisit the survey, although some participants were keen to revise their preliminary scores.

Likely timeframe for this phase: 16 weeks

The aim of this step is to demonstrate any improvement in the quality of the letters.

Suggested approach

• Follow the same approach used in the first round of letter/summary collection and review.
• Apply the templates prepared previously to score letters and present feedback for the final phase of the project.
• If you have refined the scoring process based on feedback about short-hand terms or changes in the priority given by practitioners to different elements of the patient’s history and examination, you should change the values in the scoring schedule accordingly.

It would be valuable to collect information about how the feedback was received and any information about what element of practice changed as a result of participating in the project. This information will help to refine the process described here. The team will have clear information about improvement in the quality of the letters.

It is likely that the quality of referral letters/discharge summaries will change substantially after feedback in some cases. It is possible to conduct standard statistical tests on these data in a ‘before and after’ study. For those who would prefer the data presented in statistical terms with ‘p’ values we recommend you consult a biostatistician if necessary.

Troubleshooting

Below are some of the issues you may encounter in collecting letters and discharge summaries, together with some suggested solutions based on the experience of the toolkit developers.

Lack of interest/engagement
It is possible and even probable that some practitioners will not participate in a subsequent round of this project. In the preliminary project in WA only 5 /15 practitioners submitted follow up letters in the second part of the audit cycle and within the 6 weeks following initial feedback. Because the project takes several months to complete some practitioners will have become disengaged, others will have forgotten to collect the letters and some may have had long periods of leave in the interim and may not have referred any patients.

It is important to ensure that practitioners and practice managers receive regular reminders to collect the letters for the follow up audit. It is recommended that the project coordinator send weekly reminders to keep the project on the practitioners’ agenda.

In the preliminary study practitioner scores changed as follows:

The clinical value of these changes cannot be inferred from this project. However there is some evidence in the literature cited in our bibliography that improving the amount of information relayed at referral may result in improved outcomes. Research is underway at Curtin University of Technology to demonstrate that specialists are much more likely to prioritise appointments confidently when offered more comprehensive referral letters.

Click here to see an example of how letters changed in the pilot project.

1. Jiwa M, Mathers N, Walters S. The quality of information on referrals to colorectal surgeons: towards consensus. Current Medical Research and Opinion. 2002;18,2:1-5.
2. Jiwa M, Burr J. GP Letter writing in colorectal cancer – a qualitative study. Current Medical Research and Opinion. 2002;18,6:342-346.
3. Jiwa M, Deas K, Ross J, Shaw T, Wilcox H, Spilsbury K. An inclusive approach to raising standards in general practice: Working with a 'community of practice' in Western Australia. BMC Medical Research Methodology. 2009;9:13.
4. Jiwa M, Walters S, Mathers N. Referral letters to colorectal surgeons - the impact of peer mediated feedback. British Journal of General Practice. 2004;54:123-126.
5. Jiwa M, Saunders C Fast track referral for cancer. British Medical Journal. 2007 Aug 11;335(7614):267-8.
6. Jiwa M, Mathers N. Quality of referral letters. British Journal of General Practice. 2003;406.
7. Jiwa M, Drury D, Hunt L. Improving the outcome in colorectal cases. British Journal of General Practice. 2002;52:329-330.
8. Jiwa M, Gordon M, Skinner P, Coker AO, Shaw L, Campbell MJ, Kenny R, Colwell B. Which symptomatic patients merit urgent referral for colonoscopy? A UK general practice perspective. Quality in Primary Care. 2007;15:21-5.
9. Jiwa M, Skinner P, Coker AO, Shaw L, Campbell MJ, Thompson J Implementing referral guidelines: Lessons from a negative cluster randomised factorial trial in general practice. BMC Family Practice. 2006 Nov 2;7:65.
10. Jiwa M, Freeman J, Tanner S. Correspondence between health care professionals: An evaluation of a facilitated medical student workshop. Education for Primary Care. 2006(17):155-161.
11. Keely E, Myers K, Dojeiji S, Campbell C. Peer assessment of outpatient consultation letters- feasibility and satisfaction. BMC Medical Research Methodology. 2007 May 22;7:13.
12. Ricketts DN, Scott BJ, Ali A, Chadwick RG, Murray CA, Radford JR, Saunders WP. Peer review amongst restorative specialists on the quality of their communication with referring dental practitioners. British Dental Journal. 2003 Oct 11;195(7):389-93; discussion 383.
13. Garåsen H, Johnsen R . The quality of communication about older patients between hospital physicians and general practitioners: a panel study assessment. BMC Health Services Research. 2007 Aug 24;7:133.
14. Fox AT, Palmer RD, Crossley JG, Sekaran D, Trewavas ES, Davies HA. Improving the quality of outpatient clinic letters using the Sheffield Assessment Instrument for Letters(SAIL). Medical Education. 2004 Aug;38(8):852-8.
15. Gandhi TK, Sittig DF, Franklin M, Sussman AJ, Fairchild DG, Bates DW. Communication breakdown in the outpatient referral process. Journal of General Internal Medicine. 2000 Sep;15(9):626-31.
16. Crossley GM, Howe A, Newble D, Jolly B, Davies HA. Sheffield Assessment Instrument for Letters (SAIL): performance assessment using outpatient letters. Medical Education. 2001 Dec;35(12):1115-24.
17. Gliardi A. Use of referral reply letters for continuing medical education: a review. Journal of Continuing Education in the Health Professions. 2002 Fall;22(4):222-9. Review.
18. Keely E, Myers K, Dojeiji S. Can written communication skills be tested in an objective structured clinical examination format? Academic Medicine. 2002 Jan;77(1):82-6.
19. Newton J, Hutchinson A, Hayes V, McColl E, Mackee I, Holland C. Do clinicians tell each other enough? An analysis of referral communications in two specialties. Family Practice. 1994 Mar;11(1):15-20.
20. Forrest CB, Glade GB, Baker AE, Bocian A, von Schrader S, Starfield B. Coordination of specialty referrals and physician satisfaction with referral care. Archives of Pediatrics and Adolescent Medicine . 2000 May;154(5):499-506.
21. Babington S, Wynne C, Atkinson CH, Hickey BE, Abdelaal AS Oncology service correspondence: do we communicate? Australasian Radiology. 2003 Mar;47(1):50-4.
22. Jacobs LG, Pringle MA. Referral letters and replies from orthopaedic departments: opportunities missed. British Medical Journal. 1990 Sep 8;301(6750):470-3.
23. Jenkins RM Quality of general practitioner referrals to outpatient departments: assessment by specialists and a general practitioner. British Journal of General Practice. 1993 Mar;43(368):111-3.
24. Idiculla JM, Perros P, Frier BM Do diabetes guidelines influence the content of referral letters by general practitioners to a diabetes specialist clinic? Health Bulletin (Edinburgh). 2000 Jul;58(4):322-7.
25. Pothier DD, Repanos C Referral letters: are we prioritizing consistently? Journal of Laryngology and Otology . 2005 May;119(5):377-80