Needs Assessment and Key Messages Evaluation

Introduction

The creation of an effective academic detailing intervention is a complex and iterative process. Each academic detailing organization must develop its own mechanism for navigating this process. The natural first step in the development of an intervention is identification of the barriers to appropriate, or contributing factors to inappropriate, prescribing practice that requires modification. This identification of a problem is often referred to as a "needs assessment". The determination of what the "need" is impacts all aspects of intervention development including what information will be required to create the intervention, what materials should be produced to facilitate the intervention and whom the intervention should target. The needs assessment represents a very broad concept encompassing everything from overall topic selection to determining a prescribing area within a topic that requires intervention.

Once the needs assessment has been completed and a topic has been chosen for a detailing intervention, a set of "key messages" is often developed. These key messages represent a distillation of the research conducted to support the academic detailing initiative. The key messages are concise statements that you wish your audience to remember and around which the academic detailing session will be focused. Their creation is an important and challenging component in the development of an academic detailing initiative.

Methods

The consultative process was initiated by surveys that were completed by all of the target academic detailing organizations. Once the surveys were returned, a follow-up interview was conducted to clarify or expand on the information provided. The content of the discussions was written down alongside the answers provided on the survey. Finally, the completed "transcript" was returned to the interviewee for final review and approval. This process was undertaken with five Canadian academic detailing organizations and three international organizations. The transcripts of the surveys and follow-ups are available in Appendix 1 to 8. The content of these surveys and interviews was reviewed to evaluate the needs assessment and key message development processes used by these academic detailing organizations. Since there is no existing tool available to evaluate these components of an academic detailing intervention, a pragmatic evaluation was developed. This evaluation involved the consideration of seven elements.

One researcher administered the survey, conducted the follow-up interviews and performed the evaluation. A second researcher then reviewed the transcripts and evaluations and provided additional comments.

The rationale for each element is discussed below:

Element #1: Consultation with primary care prescribers

The target audience of most academic detailing interventions is primary care prescribers. An important consideration within a needs assessment and key message development is the view of the target audience. As such, this evaluation will consider if and how primary care prescribers are incorporated into these processes.

Element #2: Consultation of primary literature

Inappropriate prescribing of agents or classes of agents is often documented in the medical literature. This information is often derived from population-based research and provides an indication of global drug utilization issues. Beyond this, a thorough understanding of primary literature on any given topic is required to allow for topic development. This evaluation will consider if and how evaluation of primary literature is incorporated into the needs assessment and key message development of the surveyed organizations.

Element#3: Objective evidence of prescribing practice within the jurisdiction

Determining prescribing patterns within a jurisdiction through analysis of drug claims or linked (drug claims and health records) databases can provide evidence of prescribing practice issues. Conducting this type of analysis before proceeding with an intervention also facilitates evaluation of the intervention’s effectiveness through follow-up analysis. This evaluation will consider if objective analysis of prescribing practice is considered in the needs assessment and key message development of the surveyed organizations.

Element #4: Consultation with a stakeholder/steering committee

The development of an academic detailing intervention is an extensive undertaking that takes into account many perspectives (prescribers, government payors, specialists, those conducting the intervention, etc). Academic detailing organizations may utilize a stakeholder or steering committee that brings all of the various perspectives together for consultation. This evaluation will consider if the academic detailing organizations consult a stakeholder or steering committee and what perspectives are represented.

Element #5: Involvement of provincial drug programs

This particular element was only considered with the Canadian academic detailing organizations. All of the Canadian organizations surveyed are provided with funding either directly by provincial ministries or indirectly through a regional health authority. The amount of influence the provincial drug program has over these components of the detailing intervention is important. Provincial drug programs tend to have a cost-containment focus to their activities. While this certainly should be an important consideration in the prescribing process, it is one of many considerations. It is also important for academic detailing groups to ensure the messages they are providing are unbiased (both of drug company and provincial drug plan) and evidence-based. The external validity and trust of physician depends on the detailers providing an unbiased message. This evaluation will consider the role that provincial drug programs have in needs assessment and key message development of the surveyed organizations.

Element #6: Local factors considered in needs assessment and key message development.

Each academic detailing organization may take into account other specific local factors during their needs assessment and key message development. This evaluation will ascertain how those local factors have been incorporated into the academic detailing intervention.

Element #7: Number of key messages

Academic detailing interventions often deal with complex disease management and prescribing behaviors. For practical purposes, the academic detailing organizations often distill a specific number of key messages that are delivered through the intervention. This evaluation will determine the limitations that organizations place on the number of key messages used in their detailing materials.

Results

Consultation occurred with eight academic detailing organizations (five from Canada and three international) that discussed their processes for needs assessment and development of key messages.

The detailed accounts of each program’s process in regards to the seven elements are documented as follows:

Organization: BC Community Drug Utilization Program (BC CDUP)

Base of Operation: North Vancouver, British Columbia, Canada

Contact Person: Anne Nguyen, Program Coordinator

Element #1: Consultation with primary care prescribers

Primary care physicians associated with the Lions Gate Health Region are surveyed every two years on the program. This survey is to assess prescribers’ satisfaction with the program, assess what changes physicians would like to see in the program (ex. provision of palm-based resources) and elicit direct recommendations on topics that would benefit from intervention. An example of the 2004 survey is included in Appendix 9. BC CDUP does not have any steering or advisory committee with physician representation that oversees or provides input into needs assessment or key messaging.

Element #2: Consultation of primary literature

Consideration is given to new published guidelines and current topics of controversy within the medical literature in the needs assessment process. Also, research conducted by the Center for Health Services and Policy Research (CHSPR) is considered during the needs assessment process. This organization researches a broad range of health-related topics, which includes research on drug utilization in B.C. The primary literature research is an important aspect of the key message development. The program coordinator conducts this review and is solely responsible for the distillation of key messages.

Element #3: Objective evidence of prescribing practice within the jurisdiction

In the past (>10 years ago) decisions on needs assessment were highly dependant on amount spent on particular drug categories and potential savings. This information was provided through a close relationship between BC CDUP and B.C. Pharmacare. Pharmacare would provide summary retrospective claims information to assist in decision making around topic selection. This process is no longer in place for the following reasons:

  • When B.C. Pharmacare moved to a reference-based pricing system, many high-use categories of medications came under strict pricing control that eliminated the measurable cost impacts of academic detailing.
  • The impression that BC CDUP was a cost-saving agency of the government hampered effectiveness with primary care practitioners.
  • BC CDUP has shifted its focus of its interventions to be more prescriber-needs focused.

Element #4: Consultation with a stakeholder/steering committee

BC CDUP does not have a board of stakeholders or steering committee.

Element #5: Involvement of provincial drug programs

Pharmacare funds the BC CDUP program, but BC CDUP is managed by the pharmacy department at the regional health authority where it is based. B.C. Pharmacare provided BC CDUP with timely drug utilization information to drive detailing interventions that would have high cost impact. However due to staffing changes, this is no longer available in a timely manner. Pharmacare does not dictate the content of the materials which BC CDUP produces, but expects an annual report on BC CDUP activities.

Element #6: Local factors considered in needs assessment and key message development

Other key local factors considered:

  • Key messages focus on matters relevant to primary care physicians.

  • Not all key messages would have measurable outcomes.

  • New drugs released on the market are considered in the needs assessment process for the potential need for academic detailing intervention. There are opportunities for both inappropriate overuse and misuse of newly marketed products.
  • New guidelines and controversial topics are also addressed if relevant to primary care physicians.

Element #7: Number of key messages

BC CDUP does not restrict the number of key messages provided for a given intervention.

Organization: Alberta Drug Utilization Program (ADUP)

Base of Operation: Calgary, Alberta, Canada

Contact Person: Paula Cunningham, Academic Detailer*

*Note: current contact is Diane Duncan, Academic Detailer, Chronic Disease Management Program, Calgary Health Region

 

Element #1: Consultation with primary care prescribers

Prescribing physicians were present on the ADUP Management Committee; as well, there was physician representation on the ADUP Steering Committees in both Calgary and Edmonton. These representatives provided insight on needs assessment from the prescriber’s perspective. The other avenue for physician consultation was via feedback from detailer encounters with prescribers during visits. This represents an avenue for informal consultation. Detailers have a form they fill out after the intervention encounter that documents this informal consultation. A copy of this form is available on pages 28 and 29 of the ADUP Internal Evaluation – Academic Detailing Initiative – Osteoporosis (Appendix 10). Element #2: Consultation of primary literature

 

Element #3: Objective evidence of prescribing practice within the jurisdiction

The ADUP conduct a "gap analysis" in the form of a Drug Utilization Review (DUR) before academic detailing on certain topics. An example of a comprehensive DUR conducted by the ADUP is included in Appendix 10. This DUR represents an exceptionally thorough evaluation of the prescribing behaviors around management of respiratory tract infections. The key findings (page 117) are an example of results obtained by this type of needs assessment. Because of the size and complexity of this analysis, the ADUP does not conduct this analysis for every topic they undertake. 

Element #4: Consultation with a stakeholder/steering committee

The ADUP had an extensive Management Committee that included representation from the following areas:

  • Alberta Blue Cross
  • Alberta College of Pharmacists
  • Alberta Health and Wellness
  • Alberta Medical Association
  • College of Family Physicians
  • College of Physicians and Surgeons
  • Pharmacists Association of Alberta
  • Regional Health Authorities
  • University of Alberta (Medicine and Pharmacy)
  • University of Calgary (Medicine)

Element #5: Involvement of provincial drug programs

As indicated above, a representative from Alberta Health and Wellness sat on the ADUP Management Committee. Therefore, they had a relatively small influence in operational aspects of the ADUP program and are consulted on needs assessment to the same degree as the overall Management Committee. Element #6: Local factors considered in needs assessment and key message development

Other key local factors considered in needs assessment included:

  • The perceived relevance of a topic to front-line practitioners.
  • The availability of up-to-date guidelines to guide content development.
  • Topics that have medications as a mainstay of treatment.
  • Educational deficiencies are the likely cause of inappropriate prescribing.
  • The effects of the intervention could be measured. The opportunity to measure the impact of key messages is considered important. The most common mechanism for evaluation is a measure in prescribing quantity of a target agent.

Other key local factors considered in key message development included:

  • The opportunity to measure the impact of key messages is considered important. The most common mechanism for evaluation is a measure in prescribing quantity of a target agent.

Element #7: Number of key messages

The ADUP produce a maximum of three key messages for any given academic detailing intervention.

Organization: RxFiles

Base of Operation: Saskatoon, Saskatchewan, Canada

Contact Person: Loren Regier, Program Coordinator

Element #1: Consultation with primary care prescribers

The main mechanism of soliciting physician input into the needs assessment process is through an annual survey. This survey is designed to assess prescribers satisfaction with the program but also to elicit direct recommendations on topics or areas of prescribing controversy that would benefit from intervention. A copy of the results of the 2004 physician survey is included in Appendix 11 (survey information on pages 9 to 33). Prescribers are also represented on the Advisory Committee (with other stakeholders), who are involved in the needs assessment process. The Advisory Committee is contacted three to four times per year to provide both a "progress report" and to consult on topics development. As well, the input of both physicians and pharmacists on the Advisory Committee are an important aspect of key message development. Finally, there is ongoing informal consultation with prescribers during the academic detailing visits.

Element #2: Consultation of primary literature

Primary literature is not routinely consulted during the needs assessment process, with RxFiles preferring to rely heavily on survey results from primary care prescribers and consultation with the Advisory Committee to guide topic selection. Key message development is dependant on review of primary literature. The review of this literature is conducted by the staff pharmacists with input from topic specialists. Key messages developed through this primary literature consultation are then vetted though the Advisory Committee.

Element #3: Objective evidence of prescribing practice within the jurisdiction

RxFiles have partnered with the Saskatchewan Health Quality Council (HQC) on two large interventions that have revolved around objective evidence of prescribing practice issues within Saskatchewan. The HQC is an independent agency that measures and reports on quality of care in Saskatchewan. When they identify these gaps or issues in prescribing practice using observational-epidemiological health data, they notify RxFiles and partner in initiatives that rectify these issues.

Element #4: Consultation with a stakeholder/steering committee

The RxFiles has an Advisory Committee made up of representatives from the general pharmacy and physician community as well as delegates from the following organizations:

  • Faculty of Family Medicine, University of Saskatchewan
  • Saskatchewan College of Physicians and Surgeons
  • Saskatchewan Medical Association

  • College of Pharmacy & Nutrition, University of Saskatchewan

  • Nurse Practitioners of Saskatchewan

  • Saskatoon Health Region

  • RxFiles Staff

  • Saskatchewan Health

The Advisory Committee is consulted three to four times per year to provide both a "progress report" and to consulate on topics/key message development.

Element #5: Involvement of provincial drug programs

The provincial drug plan, through its membership on the Advisory Committee has the opportunity to both forward potential topics to cover as well as provide input on other topics proposed during the needs assessment. It is estimated that the provincial drug program has directly proposed three topics in the nine years that RxFiles has operated and has given general support for all 27 topics for which detailing has occurred.

Element #6: Local factors considered in needs assessment and key message development

Other key local factors considered:

  • Although not represented on the Advisory Committee, input from the Continuing Medical Education department of the University of Saskatchewan is considered in needs assessment.
  • When choosing key messages for a given topic, the ability to measure the outcome of the message is not a pre-requisite. It is estimated that less than 1/3 of the key messages would result in measurable outcomes. This is in part due to the complex nature of drug utilization and prescribing.

Element #7: Number of key messages

The RxFiles limits the number of key messages to between three and five.

Organization: Prescription Information Services of Manitoba (PrISM)

Base of Operation: Winnipeg, Manitoba, Canada

Contact Person: Shawn Bugden, Executive Director

Element #1: Consultation with primary care prescribers

 

PrISM’s consultation with primary care prescribers is predominantly through their involvement with the Continuing Medical Education (CME) department of the University of Manitoba. Through this relationship, PrISM has built academic detailing interventions that expand upon CME lectures. The relationship has also provided opportunity to consult with individuals in the department that have an understanding of the educational needs of physicians. In addition, informal feedback on prescribing behaviors and deficiencies are derived from detailer contact with and questions received from prescribers.

 

 

Element #2: Consultation of primary literature

 

PrISM has incorporated regular review of primary (and secondary) literature on adverse drug events into its needs assessment process. The concept is that prescribing practices that result in negative patient outcomes should be considered for detailing intervention. As well, primary literature review is a predominant factor in the development of key messages. Once a topic is chosen, one of the PrISM staff is charged with the review and drafting of the key messages that will be associated with the topic.

 

 

Element #3: Objective evidence of prescribing practice within the jurisdiction

 

PrISM has not collected any objective evidence of prescribing deficiencies to facilitate its needs assessment. However, one of PrISM’s projects when initiating its academic detailing program was the Rural Evaluation of Academic Detailing. This project included a questionnaire component that preceded the intervention. This questionnaire sought to determine the root cause of prescribing practice deficiencies in both benzodiazepine utilization and beta-blocker utilization. A copy of the surveys is included in Appendix 12. The purpose of conducting this survey during key message development was to determine what type of message and resource tools would be most effective in inciting practice change. Conducting this survey was considered helpful in the formulation of key messages, but the work associated with producing and administering the survey makes it impractical for regular use in key message development.

 

 

 

Element #4: Consultation with a stakeholder/steering committee

 

PrISM does not have and does not consult with a formal stakeholder/steering committee. Informal discussion occurs regularly with a variety of individuals and organizations that influence PrISM’s approach to academic detailing topic selection and development.

 

 

Element #5: Involvement of provincial drug programs

 

The initial funding that created PrISM as a pilot project was from the provincial drug program. However, there is no formal connection between PrISM and the drug programs; only periodic informal suggestions have been received from the provincial drug program.

 

 

Element #6: Local factors considered in needs assessment and key message development

 

Other key local factors considered:

  • Because of the close working proximity between PrISM and the pharmacy regulatory authority (they share the same office-space), the patient-safety perspective has become prominent in detailing activities. As well, interventions have often targeted both prescribers and pharmacists.

Element #7: Number of key messages

 

PrISM generally has five or less key messages associated with any given academic detailing intervention.

 

 

Organization: Academic Detailing Service (ADS), Continuing Medical Education

Base of Operation: Halifax, Nova Scotia, Canada 

Contact Person: Michael Allen, Director

 

 

Element #1: Consultation with primary care prescribers

 

Being directly linked to the Continuing Medical Education Department at Dalhousie University, the Academic Detailing Service has extensive ties to primary care prescribers. The service has an Academic Detailing Advisory Committee made up of family physicians who are involved in topic selection and choosing key messages. In addition, the physicians that are “recipients” of detailing interventions are asked for suggestions on topics that should be covered in the evaluation forms that follow each detailing visit. Finally, broad surveys of all primary care physicians are conducted to gauge what physicians perceive to be their educational needs. An example of the results of the most recent survey on diabetes is included in Appendix 13. The three numbers following each point are the mean score out of five, the confidence interval and the number of respondents who rated the option.

 

 

Element #2: Consultation of primary literature

 

The specific portion of primary literature that the ADS incorporates into their needs assessment is new evidence. In other words, a topic may be developed around bringing new evidence forward to practitioners in a timely manner. The key message development is highly dependant on primary literature. One of the detailing pharmacists conducts the literature search on a topic. The pertinent information found is provided to the Working Group (program director, drug evaluation pharmacist, senior academic detailers) who then propose several key messages. It is the task of the Working Group and Physician Advisory Group to decide on the final key messages.

 

 

Element #3: Objective evidence of prescribing practice within the jurisdiction

 

No formal assessment of prescribing practice is undertaken before an academic detailing topic is developed. Through the intervention process, physicians are given the opportunity to request their own prescribing data from the provincial drug claims network. This allows target physicians to elicit their own objective evidence of prescribing practice.

 

 

Element #4: Consultation with a stakeholder/steering committee

 

Drug Evaluation Alliance of Nova Scotia (DEANS) is involved in the needs assessment process by providing direction on the topics addressed, in addition it monitors the execution and evaluation of each ADS topic and encourages and oversees complementary interventions. As of April 2006, the DEANS management committee is made up of representatives from the following organizations or specialty practice areas:

 

 

  • Continuing Medical Education, Dalhousie University
  • Department of Family Medicine, Dalhousie University
  • College of Pharmacy
  • Continuing Pharmacy Education, Dalhousie University
  • Population Health Research, Dalhousie University
  • Pharmacy Association of Nova Scotia
  • External Health Outcomes Consultant
  • Nova Scotia Prescription Monitoring Program
  • Drug Evaluation Unit, QEII Health Science Centre

 

The DEANS committee is accountable directly to the Minister of Health in Nova Scotia.

 

Element #5: Involvement of provincial drug programs

 

The DEANS management committee reports directly to the Minister of Health in Nova Scotia Health. Beyond this reporting relationship, the provincial drug program has no direct influence on topic selection or key message development.

 

Element #6: Local factors considered in needs assessment and key message development

 

Other key local factors considered:

  • Clinical content experts are involved early in the process of key message development to provide insight into the strengths and weaknesses of primary care prescribers.
  • Topics and key messages are not chosen based on their ability to generate a measurable outcome, but rather what will help physician incorporate the evidence into decision making in clinical practice.

Element #7: Number of key messages

The Academic Detailing Service limits the number of key messages to between three and four.

Organization: Drug & Therapeutic Information Service (DATIS)

Base of Operation: Lexington, Kentucky, United States*

Contact Person: Frank May, Service Director

*Note: DATIS is primarily an Australia-based organization with close ties to the National Prescribing Service (NPS). They are currently involved on a project in Kentucky where they are setting up an academic detailing program. Where applicable, information from DATIS activity in Australia is provided.

Element #1: Consultation with primary care prescribers

The approach that DATIS has to academic detailing is to cultivate very close relations between the detailers and their target prescriber audience. Through this relationship, information is gleaned on the prescribing practices of physicians. The assessment of the individual detailers on topics that are of contemporary interest to prescribers and how actual prescribing practices greatly differ from ideal is a key aspect of their needs assessment. When in Australia, the detailers working for DATIS covered 80-90% of the general practitioners within Southern Australia, thus detailers had a very good sense of the needs of the prescribing community. The DATIS operation in Kentucky does not have that extensive a network, but are hoping to build into this model over time.

An additional avenue for physician consultation is through the clinical support service that the DATIS program is providing. Information is collected on the types of clinical support requests they receive and these requests are considered in the needs assessment process.

A final avenue for prescriber consultation is through the Advisory Board, which include representatives from primary care setting and specialty care setting. The Advisory Board is presented with the potential topics derived from the needs assessment process and has final “say” on what topics are going to be developed into a detailing initiative.

Element #2: Consultation of primary literature

Consultation with the primary literature is not a key aspect of the needs assessment process, with DATIS being heavily focused on the assessed needs of prescribers. Primary literature review is however, a very large component of the key message development. Once a topic is chosen through the needs assessment process, usually one detailer is removed from the “field” to conduct a comprehensive systematic review of the literature on the topic. From this search, a bibliography of 500 to 1500 articles is abstracted and reviewed. After this comprehensive set of data is acquired, there is a “scoping” process that sets the boundaries for the given topic (i.e. what will be covered in the detail document). Once the topic is better refined, the review document is written using the pertinent primary literature. The goal of this process is to create a document that is comprehensive so that the academic detailers in the field have an unparalleled understanding of the literature in order to best meet the prescriber’s needs. Through development of the review document and consultative review by DATIS academic detailers, DATIS managers, external medical specialists in the particular field and selected interested general practitioners, a number of key messages are crystallized.

Element #3: Objective evidence of prescribing practice within the jurisdiction

The Australian program, prior to 1999, focused heavily on analysis of drug claims data to both guide topic development and assess effectiveness of the program. Needs assessment considered both the potential savings that could be generated through a given intervention and how readily measurable the outcome of an intervention would be. However, in the late 90’s the program changed its focus away from this process as it was felt that it did not accurately reflect the impact academic detailing was making. It was at this point the focus shifted to physician input for their needs assessment process.

The DATIS program in Kentucky does collect claims data from the State’s drug program database. However, the analysis of the drug claims information is minimally important in both their needs assessment and impact evaluation processes; it is of more interest to the State funders of the DATIS program.

Element #4: Consultation with a stakeholder/steering committee

The DATIS program has an Advisory Board that includes representation from the following areas:

  • Practicing primary care prescribers
  • Practicing physician specialists
  • Representative from the Faculty of Pharmacy, University of Kentucky
  • Representative from the Faculty of Medicine, University of Kentucky

The Advisory board is involved in the needs assessment as the final veto on topic selection.

Element #5: Involvement of provincial drug programs

N/A

Element #6: Local factors considered in needs assessment and key message development

Other key factors considered:

  • The selection and refinement of key message is an extensive process that takes into account the “marketability” of the message. A number of key messages are drafted through literature review and expert consultation. The final key messages are chosen using a rigorous process of defining features, benefits, barriers and enablers for each key message. Features of a key message are the qualities of the message that would make the organization want to support it (ex. A large amount of primary literature backing the message). Benefits of the key message are the positive results that would result of the dissemination of the message (ex. A message that promotes the safer use of a drug). Barriers of a key message would be anything that would stop or inhibit the adoption of the key message (ex. A barrier to withholding an antibiotic prescription for a child’s cold is the parent’s desire to receive a drug). Enablers are aspects of an academic detailing initiative that could facilitate the key message (ex. An information pad for parents about the dangers of over-prescribing antibiotics). The purpose of this process is to find the key messages that are the most marketable. This process also assists in the eventual development of broader written material that will be used by detailers to maximize the impact of their intervention on prescribers.
  • A consideration within the needs assessment process is identifying situations where there is uncertainty amongst prescribers about “correct” or “ideal” practice. It has been found that interventions around clarifying these practices are very well received.

Element #7: Number of key messages

The DATIS program does not pre-specify restrictions on the number of key messages within the interventions they develop. They divide their key messages into two broad categorizations, knowledge-imparting and behavioral. Knowledge-imparting key messages are developed when some particular prior knowledge of a point is a fundamental pre-requisite before a clinical practice change from other behavioral key messages could occur.

Organization: Independent Drug Information Service (iDiS)

Base of Operation: Boston, Massachusetts, United States*

Contact Person: Michelle Spetman, Project Manager

*Note: a team from the Harvard Medical School provide the clinical and administrative services for the operation of iDiS which is an initiative supported by the Pharmaceutical Assistance Contract for the Elderly (PACE) program in Pennsylvania, United States.

Element #1: Consultation with primary care prescribers

Primary care physicians are not involved in the needs assessment or key message development process. Among the researchers in the Harvard group that prepare the information for the detailing intervention are practicing physicians, but no prescribers from the target area of Pennsylvania are involved.

Element #2: Consultation of primary literature

Availability of primary literature is a key consideration in the needs assessment process for iDiS. Before a topic is chosen, the availability or lack of availability is assessed to determine if a topic can be adequately developed. The iDiS materials are designed to provide summaries of the relevant data and practice-relevant guidelines for prescribers. This requires that the selected topic has a solid base of evidence available for the Harvard researchers. Once a topic is chosen, literature analysis is the cornerstone to key message development. This research is conducted primarily by the clinicians and advisors at Harvard and the key messages are also defined by these groups.

Element #3: Objective evidence of prescribing practice within the jurisdiction

iDiS operates very closely with the PACE program in the key message development process. A group from Harvard, in association with a group from the PACE program, analyze PACE enrollment, claims, and claims expenditures to identify high volume claims, and assess therapeutic classes. This analysis determines areas where educational opportunities may exist to help improve appropriateness of prescribing. An example of this data extraction is included in an iDiS annual report included in Appendix 14 (section 7, pages 83-94).

Element #4: Consultation with a stakeholder/steering committee

There is no formal stakeholder or steering committee for iDiS; however, as the Pennsylvania PACE program is the sole funding source for the program, it has ultimate responsibility for the direction of the program. It is recognized that the topics chosen through the needs assessment process must be relevant to PACE Program objectives, which are to optimize medication therapy for the beneficiaries of Pennsylvania's state-supported drug programs, improve the quality of prescribing, and contain costs by providing the most appropriate medications to beneficiaries.

Element #5: Involvement of provincial drug programs

N/A

Element #6: Local factors considered in needs assessment and key message development

Other key local factors considered:

  • In the needs assessment process, potential cost savings aren’t the primary consideration, but rather quality of care for the PACE program enrollees.
  • When key messages are identified, they must:
  1. be evidence-based
  2. imply a change in behaviour/knowledge
  3. address a gap between ideal and actual practice
  4. allow for some measurement of impact (if possible)
  • iDiS has been contemplating the incorporation of key message refinement utilized by the DATIS program in Kentucky. Specifically, they are looking at concepts of barriers and enablers to key messages to further develop academic detailing materials that meet the needs of practitioners and hence have more impact.

Element #7: Number of key messages

The iDiS program does not pre-specify restrictions on the number of key messages within the interventions they develop.

Organization: National Prescribing Service (NPS)

Base of Operation: Surry Hills, New South Wales, Australia 

Contact Person: Judith Mackson, Manager, Education and Quality Assurance Program

Element #1: Consultation with primary care prescribers

NPS consults with primary care prescribers on the needs assessment in two ways. The academic detailers that deliver the NPS material are all employed by different Divisions of General Practice. Divisions of General Practice are geographic grouping of primary care healthcare providers, somewhat analogous to our regional health authorities (excluding the hospitals). The detailers are charged with surveying their individual Division for topic recommendations or educational needs. This information is collated centrally to identify commonality in needs. The second method of consultation occurs once the potential topic is chosen. A sampling of family physicians and community pharmacists are interviewed by telephone to test the acceptability of the topic for intervention. This final process demonstrates a rigorous needs assessment process to external parties.

Element #2: Consultation of primary literature

Consultation with primary literature is not a large component of the needs assessment process for NPS. The availability of primary literature to create an intervention is considered as part of the criteria of topic selection. Largely NPS’s focus is on consultation with primary care prescribers as well as the input from an expert advisory council. There is extensive consultation of the primary literature in key message development. Members of the expert advisory committee conduct systematic literature evaluation in order to develop the detail document that supports the intervention. The key messages are developed and refined by this group of experts. The process of creating the detail document and the developing/refining of key messages can take as long as six months.

Element #3: Objective evidence of prescribing practice within the jurisdiction

A combination of resources is used to collect background information on prescribing practice. The Medicare database information is combined with collated data from the community pharmacies to provide a complete picture of all medication prescribing. This information is used to provide baseline information and an assessment of the gap between optimal and current practice. This baseline information is then also incorporated into the program evaluation.

Element #4: Consultation with a stakeholder/steering committee

The NPS has a board of directors and an extensive group of membership organizations (36 medical or other Australian healthcare associations); however, these groups are not involved in the needs assessment or key message development. This work is left to the expert advisory committee.

Element #5: Involvement of provincial drug programs

N/A

Element #6: Local factors considered in needs assessment and key message development

Other key local factors considered:

  • The NPS chooses five intervention topics per year. Only two of those intervention topics will include an academic detailing component. The topics that receive academic detailing are those where the key barriers are knowledge deficits.
  • The criteria that are considered during topic selection in the needs assessment process are:
  1. Priority for GPs or other target groups
  2. The information will assist in providing best patient care – (Ability to make an impact; ability to affect patient outcomes)
  3. New information is available
  4. Systems issues which will impact on patient care
  5. Evidence of therapeutic problem, variation in prescribing, adverse outcomes
  6. Potential impact on drug program expenditure
  7. Data are available to support delivery of the message
  8. Evidence exists to guide better practices
  9. New drug
  10. Ability to link with parallel programs
  11. Presence of therapeutic uncertainty or controversy

Element #7: Number of key messages

The NPS program usually adheres to a maximum of five key messages per intervention with most interventions having four to five.

Discussion

Importance

The needs assessment and key message development are not components of the toolkit that COMPUS will be creating; however, the results of their development (ex. a list of key messages) are integral components of the toolkit. They are processes that COMPUS should be considering at the forefront of the academic detailing material development to be delivered through the toolkit. Appropriate needs assessment and key messages are vitally important to a successful academic detailing intervention. Developing an intervention for which minimal inappropriate prescribing is occurring or developing an intervention with tools that do not address the root cause of the inappropriate prescribing will lead an ineffective intervention. The results of the surveys of domestic and international academic detailing organizations illustrates the variety of ways that needs assessments can be conducted and key messages derived. None of these methods will necessarily fit COMPUS directly, but key practice elements from each can be considered as COMPUS devises its own processes.

Content/Format

The content and format generally refer to physical materials produced for the toolkits. As the information on needs assessment and key message development are largely process-based, there is little that can be commented on in these regards. Examples of the content and format of various items that may be involved in the needs assessment are described in the “Results” section. As these documents are for internal reference purposes, style and formatting considerations are far less important then the content itself.

Process

Ideally, needs assessment should occur as a first-step in the development of an intervention. We recognize that some of the information on needs assessment is coming well after COMPUS has established many of these processes with its Proton Pump Inhibitor project. It is our hope that the information and template recommendations will provide COMPUS with new ideas or help refine existing processes. There is a gap analysis currently underway for the Proton Pump Inhibitor project involving interviews with primary care prescribers and consumers. This analysis should also help to meet some of the requirements for needs assessment.

The process of developing key messages is another area where COMPUS may already have progressed with their extensive review work on the Proton Pump Inhibitors. As is evident from the results of our surveys, the practice of key message development by academic detailing organizations is highly variable. In some organizations like PrISM and the B.C. CDUP, key message are created solely by employees working on the literature evaluation for a topic. In others such as RxFiles, there is more consultation with a stakeholder body after key messages are drafted by internal staff. It became apparent that one organization, DATIS, had established a much more rigorous process around key message development and refinement. As described in the Results Section, after potential key messages are drafted from an extensive literature review and consultation with experts, they enter into a process of defining the features, benefits, barriers and enablers for each key message. The DATIS methodology is being considered for implementation by the iDiS program and has generated a great deal of interest from the Canadian detailing organizations. COMPUS may wish to explore this process in more detail for future projects.

The needs assessment and key message development processes that COMPUS develops are not to be adopted by academic detailing and non-academic detailing organizations, although it is worthwhile that they are informed of COMPUS’s processes.

The time associated with the needs assessment and key message development is of interest. The needs assessment process for four of the domestic programs (BC CDUP, ADUP, RxFiles and ADS) and one of the international programs (DATIS) involved either direct prescriber surveys or intervention feedback response. Some needs assessment processes that involved extensive consultation with stakeholder groups who are consulted on a scheduled basis throughout the year (RxFiles) take more time then programs where no outside consultation is required (BC CDUP and PrISM). The variety of structures and lack of tracking of related time make it impossible to define a specific time association. In short, we could determine no average time to conduct a needs assessment process. More consultation with physician and stakeholder groups may extend the time required but may produce a more relevant product.

The key message development process also varied between programs and within programs depending on the topic being considered. Often key message production occurs simultaneously with the development of background materials and printed resources associated with the intervention. The estimated time for this development work varied from a few weeks to several months depending on the depth of topic being considered. Organizations that undertake extensive systematic literature evaluation in topic development such as NPS and DATIS reported it taking approximately six months for one employee to take the process from topic to completion. Organizations that conducted more targeted interventions where there was less data to consider (ex. specific medication incidents) were able to develop key messages and materials in three to four weeks.

Part of the ADUP needs assessment process is looking for topics wherein significant variation or deficiencies are known to exist in usual care. This process involves a search of primary literature as well as consultation with specialists that are well versed in the literature on a given topic. During the development of key messages, one detailer will be tasked with drafting of key messages. A literature search (including grey literature) is included in the development of these draft key messages.