Develop a Comprehensive Learning Needs Assessment for Your Research Staff

Tracy Graham, RN, BSN, CCRC

Clinical Researcher—November 2018 (Volume 32, Issue 9)

PEER REVIEWED

Tracy Graham, RN, BSN, CCRC

 

Targeted continuing education for research staff is an important component of any research program. Continuing education that is meaningful to research staff promotes professional development and improves the quality of research. The foundation of a successful continuing education plan is a comprehensive learning needs assessment. This article will summarize key strategies for conducting a learning needs assessment that will help you efficiently identify the specific needs of your staff.

Setting the Foundation

The development and implementation of relevant continuing education for research personnel is, and should be, a vital, beneficial component of any research program. Continuing education can equip research staff with the tools necessary to handle increasingly complex and rigorous research protocols by fostering effective protocol selection, implementation, study management, and recruitment and retention strategies.

Continuing education can also assist busy research staff to quickly identify and integrate updates, changes, and revisions to federal, state, institutional, and sponsor regulations, requirements, and policies. In an increasingly competitive climate, where only the most compliant and effective sites are offered study participation, effective continuing education that addresses quality and compliance issues, identified through internal and external audits, inspections, and monitoring, can heighten the desirability of a research site to sponsors.

Finally, continuing education plays a crucial role in the professional development of research staff. However, the creation and conduct of relevant continuing education takes an extraordinary amount of time and resources, regardless of whether you are from a large healthcare center with hundreds of researchers at multiple satellite sites, an academic institution, or a small clinic or physician office. A comprehensive learning needs assessment helps focus time and effort on pertinent topics, which ultimately is a time-saver for both administrators and research staff.

Figuring out how and where to identify topics for continuing education does not have to be a daunting task. With a little investigation and planning, one can isolate the most meaningful and relevant topics.

The foundation of any education plan should be a comprehensive assessment of learning needs. The key to developing such an assessment is to identify, select, and then evaluate the sources that house information on the strengths and weaknesses of staff and the research program as a whole. Tapping into these key sources will assist in identifying reoccurring themes and topics in need of being addressed through education and training.

Sources for Identifying Learning Needs

Internal Audits

One source to examine when looking for potential learning needs is the output of your site’s Quality Assurance (QA) and/or Research Compliance program; most large research centers have one or both. Even small clinics or an independent physician office may have current or past QA projects to examine. Ideally, tracking findings from these internal audits will help you to gain insight into the strengths and weaknesses of your research program. Review any audit findings to identify problematic areas, such as drug accountability or informed consent, to help pinpoint specific learning and skill development needs.

Lessons Learned

Looking at the mistakes made by other sites and study staff can be an excellent source of relevant educational topics for your staff. One valuable resource to review is publicly available U.S. Food and Drug Administration (FDA) Warning Letters. By looking at letters from the past 12 to18 months, valuable insights can be formed about areas that the FDA has focused on and the problems found at other sites.

During your review, identify the top five most common citations issued by the FDA. These citations can serve as external sources in the learning needs assessment, and can later be compared to the problem areas identified from internal sources. If there is overlap, you have strong evidence to include these areas in the continuing education plan for the upcoming year.

Direct Observation

Direct observation of staff is another effective source for identifying learning needs. Select a random sample of research coordinators and schedule dates to observe them conducting informed consent discussions and study visits. Observe other study-delegated tasks such as drug or device accountability, including receipt, dispensation, and reconciliation. Although this method takes time and, in the case of larger healthcare centers or academic institutions, may involve travel, observation of actual practice can provide great insight on problem areas to target with education and training.

Job Descriptions/Competencies

Review the job descriptions and competencies for your research staff. Identify competencies that have not been the focus of QA activity either recently or in the past. Look for new responsibilities or competencies that have been added. Staff may be unfamiliar with these added responsibilities or the processes involved in achieving them. Finally, when evaluating job descriptions as a potential source for learning needs, always validate with leadership and staff that the listed competencies reflect current expectations and practice.

Monitoring Letters

A random sample of interim monitoring visit letters from sponsor monitoring visits can offer a window on the conduct of research at your site, and serve as an excellent source to identify learning needs. Be sure your sample includes monitoring of industry trials, federally funded trials, and investigator-initiated research. In addition, ensure the random sample includes research from all service lines or disease categories covered at your site; collect data on common errors, problems that required a corrective and preventive action (CAPA) plan, and items requiring follow-up.

Examine the data collected from external monitoring letters and identify the top three to five issues found. Again, if the common issues identified from monitoring letters overlap with learning needs identified from other internal and external sources, these areas should be included in the continuing education plan for your staff.

Survey Stakeholders

Research takes the effort of numerous members of the research team, including managers, coordinators, regulatory specialists, research associates, investigators, institutional review board (IRB) staff, and sponsors. A comprehensive learning needs assessment should take into consideration the needs of all your research staff. These research team members are key stakeholders in the education plan created from the learning needs assessment, and will be much more receptive to future education they have personally identified as important to them. Surveying these stakeholders can enhance your understanding of their learning needs from a variety of perspectives.

Those closest to the trenches, so to speak, are often familiar with the most confusing topics, policies, and forms, as well as the most common mistakes and frequently asked questions. Managers, for example, have first-hand knowledge of staff strengths, weakness, and goals, and can provide valuable insight on topics that present the most difficulty for staff. Similarly, the local IRB office staff or IRB members can provide a lens on the trends or problem areas they have observed from recent projects. If your current training program uses preceptors or mentors to train new staff, these team members are another great source to assist in identifying areas that require ongoing education.

When surveying your stakeholders, it is important to ask which policies, procedures, or forms represent frequent sources of questions from research staff. A confusing policy, procedure, or form can be addressed and incorporated into your continuing education plan. Learning how and why these documents are used and needed will enhance compliance with them and improve the conduct of research at your site.

There are many effective ways to survey stakeholders, including:

  • focus group meetings
  • web-based surveys
  • web-based quizzes

If time permits, schedule a focused meeting with each stakeholder group, as such meetings allow the various stakeholders to speak freely. For example, a meeting with both management and research coordinators may not be as effective as a meeting with research coordinators alone, since staff may feel more hesitant to share weakness in front of management. Similarly, management will be able to speak more freely about their observations without the research staff that reports to them present.

The benefit of in-person meetings is the ability to clarify suggestions and feedback in real time. However, since these meetings take time on the part of all attendees, be sure to provide the agenda for your meeting well in advance. Not only is this respectful to the time of each attendee, it allows all in attendance time to think about research topics or processes that are most problematic, topics they want to learn more about, or education they feel is needed or desired.

If the number of key stakeholders in one focus group is large, the logistics of scheduling a meeting is more difficult. As an alternative, web-based surveys may be used to target large and widespread groups, as they allow stakeholders to provide suggestions and feedback at their convenience. Web-based surveys also allow results to be quickly analyzed and converted to tables, graphs, and charts suitable for presentations.

Meanwhile, although receiving them may seem intimidating to some, surveys targeting staff on a variety of research topics with multiple choice and true/false questions can gather valuable information. Questions about staff experiences with consenting non-English speaking subjects, adverse event/serious adverse event (AE/SAE) reporting, and investigational drug or device accountability are examples of topics to consider. However, developing good questions for such surveys takes time and practice; be sure to take this time and reference books and articles that will help you design well-constructed questions to ensure the questions measure the outcome you are trying to assess.

Ideally, the survey should be administered via a web-based application, so its results can be easily analyzed. There are many web-based tools (e.g., Survey Monkey) that are free for the creation of smaller surveys or require a minimal fee for larger surveys. A little research on the front end should help you find a tool that is just right for your needs.

To minimize staff discomfort, assure staff that results from these surveys are collected anonymously and that data are analyzed collectively, not individually. Be sure to discuss this plan with leadership, in advance, so that they can endorse the surveys. Emphasize to leaders and staff that, if participation in (and completion of) the surveys is not 100%, the value of the data to the organization will be compromised.

Learning Needs Analysis—Putting it All Together

Take all of the learning needs you have gathered from your internal and external sources and compare them to identify trends and patterns. Note any learning needs or topics that were identified from several sources. For example, if the accurate and prompt identification and reporting of AEs/SAEs was identified as an area of need in your staff survey, in the most recent quality audit, and by mentors and managers, then you have found a definite area of need. Training on this topic would be appropriate during an upcoming continuing education event.

After careful comparison and analysis, the top three to five topics in need of continuing education and training should emerge. Table 1 is an example of sources used for an annual learning needs assessment and topics that were identified for continuing education. Based on this assessment, mandatory training should be held on AEs/SAEs, unanticipated problems, and violation identification/reporting.

 

Table 1: Sources Used for and Topics Determined by Learning Needs Assessment

Topics Identified for Training in 2017 Source(s) Evaluated in Fall 2016
  • AE/SAE identification and reporting
  • Manager focus group recommends training on AEs/SAEs
  • Top 5 FDA citation – failure to conduct the investigation according to the investigational plan (AE/SAE reporting)
  • Top 3 topic identified from staff quiz
  • Top 3 topic identified from staff survey/focus group
  • Unanticipated problem identification and reporting
  • Previous finding on external audit of site
  • Top 3 topic from staff quiz
  • Top 3 topic identified from staff survey/focus group
  • Violation identification and reporting
  • Top 5 most common FDA citation
  • #1 internal compliance audit finding
  • Top 3 from staff quiz

 

Conclusion

Meaningful continuing education for research staff promotes the professional and compliant conduct of research. The key to planning relevant continuing education is the completion of a comprehensive learning needs assessment. Using the strategies and sources for conducting a learning needs assessment outlined in this article will maximize the time devoted to this activity.

Beyond the suggestions given here, be sure to incorporate additional strategies and sources of your own. Varying the sources of the learning needs assessment from year to year can expand and validate the organization’s introspection, ensuring it continues to capture and address the changing needs of staff.

Time spent on the front end identifying learning needs and evaluating the trends found is the first, and perhaps most important, step in conducting a comprehensive learning needs assessment for developing valuable continuing education for research personnel.

Resources

Haladyna TM, Downing SM, Rodriguez MC. 2002. Applied Measurement in Education. Mahwah, N.J.: Lawrence Erlbaum Associates, Inc.

Carriveau RS. 2016. Connecting the Dots. Sterling, Va.: Stylus Publishing, LLC.

Morgan DL. 1998. The Focus Group Guidebook. Thousand Oaks, Calif.: Sage Publications, Inc.

U.S. Food and Drug Administration. Inspections, Compliance, Enforcement, and Criminal Investigations—Compliance Actions and Activities—Warning Letters. https://www.fda.gov/ICECI/EnforcementActions/WarningLetters/default.htm

Tracy Graham, RN, BSN, CCRC, (tracy.graham@aurora.org) is a research educator and clinical research coordinator at the Aurora Research Institute in Milwaukee, Wis.