Evidence-Based Performance Management: Applying Behavioral Science to Support Practitioners
Matthew D. Novak & Florence D. DiGennaro Reed & Tyler G. Erath & Abigail L. Blackman & Sandra A. Ruby & Azure J. Pellegrino
© Association for Behavior Analysis International 2019
Abstract
The science of behavior has effectively addressed many areas of social importance, including the performance management of staff working in human-service settings. Evidence-based performance management entails initial preservice training and ongoing staff support. Initial training reflects a critical first training component and is necessary for staff to work independently within an organization. However, investment in staff must not end once preservice training is complete. Ongoing staff support should follow preservice training and involves continued coaching and feedback. The purpose of this article is to bridge the research-to-practice gap by outlining research-supported initial training and ongoing staff support procedures within human-serving settings, presenting practice guidelines, and sharing information about easy-to-implement ways practitioners may stay abreast of current research.
Keywords Performance management . Staff training . Supervision
Portions of this manuscript were presented as an invited talk by Florence D. DiGennaro Reed at the OBM in Health and Human Services Conference, Denver, CO.
Florence D. DiGennaro Reed
fdreed@ku.edu
4001 Dole Human Development Center, Department of Applied Behavioral Science, University of Kansas, 1000 Sunnyside Avenue, Lawrence, KS 66045, USA
Published online: 26 November 2019
Train people well enough so they can leave, treat them well enough so they don’t
want to.
—Richard Branson
The science of behavior has effectively addressed many areas of social importance including education (e.g., Sulzer-Azaroff & Gillat, 1990), traffic safety (e.g., van Houten, Nau, & Marini, 1980; Yeaton & Bailey, 1978), substance use (e.g., Higgins, Silverman, & Heil, 2007), parent training (e.g., Lindgren et al., 2016; Phaneuf & McIntyre, 2007), behavioral medicine(e.g., Piazza, Milnes, & Shalev, 2015),and others. Perhaps the most widely known application is the behavioral treatment of autism (Freedman, 2016). Due to the research supporting its effectiveness (e.g., National Autism Center, 2009, 2015), autism treatment based on the principles of behavior analysis is endorsed by the U.S. Surgeon General (U.S. Department of Health & Human Services, 1999), National Institutes of Health (Strock, 2007), and the National Research Council (Lord & McGee, 2001). The model that has evolved for the provision of services to individuals with autism and other disabilities presents a challenge that is unique to those in health and human services. Whereas psychological therapy is delivered by licensed psychologists and medical procedures are performed by doctors, behavior analysts with the most amount of training and education (i.e., Board Certified Behavior Analysts®) often do not deliver services directly. Rather, they typically over see the provision of services delivered by others. Thus, the challenge for those providing behavioral services to vulnerable populations is ensuring that individuals with relatively less training and education (i.e., Registered Behavior Technicians® or Board Certified Assistant Behavior Analysts®) provide direct services in an effective and responsible manner.
To be successful, this unique service model necessitates effective performance management—the careful training and supervision of staff who implement behavioral treatment. Unfortunately, recent research suggests the field could benefit from improvements in this area. DiGennaro Reed and Henley (2015) administered a survey to certified and aspiring behavior analysts to determine the extent to which evidence-based staff training practices were implemented by their employers. The results were worrisome and launched efforts by the authors to disseminate recommended practices to practitioners and organizational leaders (e.g., DiGennaroReed , 2016, 2017, 2018, 2019; Henley, 2018, 2019). Only 55% of respondents indicated they received initial training after being hired; when training was provided, organizations relied on tactics with little empirical support, such as instruction alone. Fortunately, 71% of respondents reported they received ongoing training and support, but primarily in the forms of a lecture or monthly feedback, which are typically less effective than other forms of performance management. Respondents who worked as supervisors were also queried about the training they received to prepare them for the important responsibility of leading and managing staff. Only 33% indicated they received any formal training on how to effectively implement research-supported supervision practices. These results suggest training and performance management in settings that employ behavior analysts are not consistent with recommended practices. Many respondents began working without any formal training, received less-than-ideal ongoing support, and were expected to supervise staff without any guidance regarding best practices.
Many practicing behavior analysts lack sufficient training and ongoing support, which is worrisome because the lack of evidence-based staff training and performance management may lead to negative outcomes, including staff turnover, poor service quality, and charges of unethical conduct. Kazemi, Shapiro, and Kavner (2015) surveyed 100 behavior technicians from several companies and found that satisfaction with initial training, ongoing staff support, and supervisor behavior were key factors influencing respondents’ intent to quit their jobs. These results suggest that providing evidence-based staff training may help to mitigate staff turnover, which is reported to be as high as 75% within human-service organizations (Kazemi et al., 2015). Moreover, staff turnover places substantial financial strain on an organization, with average costs ranging from $5,000 for a single behavior technician to $10,000 for a certified behavior analyst (Sundberg, 2016).
In addition to affecting workforce stability, insufficient training and performance management may further disrupt the quality of behavioral services by influencing treatment integrity—the extent to which prescribed treatment procedures are implemented correctly (Peterson, Homer, & Wonderlich, 1982). In a review of 19 parametric analyses of treatment integrity, Brand, Henley, DiGennaro Reed, Gray, and Crabbs (2019) reported that treatment integrity errors produce unpredictable client outcomes. Although, in some instances, errors in behavioral treatment did not negatively influence client skill acquisition or problem behavior (e.g., Leon, Wilder, Majdalany, Myers, & Saini, 2014), in general researchers showed that integrity errors resulted in slower learner progress and less effective treatment. These findings underscore the importance of ensuring that staff implement treatment protocols as designed, which is fostered by the quality of training and performance management staff experience.
Insufficient performance management practices may have consequences that extend beyond affecting direct services; they may carry severe consequences for behavior analysts individually as well as for the profession more broadly. To ensure quality provision of services, the Behavior Analyst Certification Board® (BACB®) Professional and Ethical Compliance Code for Behavior Analysts (BACB, 2014) outlines a set of guidelines that all behavior analysts must follow. For example, section 5.03 of the Code states, “If the supervisee does not have the skills necessary to perform competently, ethically, and safely, behavior analysts provide conditions for the acquisition of those skills” (p. 14). Thus, supervisors are obligated to provide staff with effective training. On the topic of ongoing staff support, section 5.06 states, “Behavior analysts design feedback and reinforcement systems in a way that improves supervisee performance. Behavior analysts provide documented, timely feedback regarding the performance of a supervisee on an ongoing basis” (p. 14). Together these statements indicate that behavior analysts must rely on evidence-based staff training and performance management procedures to ensure compliance with this code. Failure to do so may result in ethical charges against the credentialed behavior analyst, which could lead to loss of certification or state licensure.
Evidence-based performance management involves a program that includes initial preservice training and ongoing staff support. Initial training, often referred to as orientation or onboarding, reflects a critical first training component and is necessary for staff to work independently within an organization. However, investment in staff must not end once preservice training is complete. Ongoing staff support should follow preservice training and it involves supervisors providing continued coaching and feedback regarding the staff’s performance in one or more areas. The purpose of this article is to bridge the research-to-practice gap by outlining research-supported initial training and ongoing staff support procedures within human-service settings, presenting practice guidelines based on our experience and current research, and sharing information about ways to stay up to date with research.
Preservice Training
The natural first step to ensuring high-quality staff training is the delivery of preservice training, which allows new employees to learn relevant job skills in a controlled, distraction-free environment. Although orientation and onboarding procedures are ubiquitous across all forms of employment, the presence of training alone does not guarantee proficiency with job skills. To be most effective, preservice training should use empirically supported training techniques. One empirically supported technique is behavioral skills training (BST), which is a procedure used to train new skills through a package including instruction, modeling, rehearsal, and feedback (Parsons, Rollyson, & Reid, 2012). Behavioral skills training can be implemented individually or in groups and has been used to train several tasks in the human service industry, including discrete trial instruction (Sarokoff & Sturmey, 2004) and mand training (NigroBruzzi & Sturmey, 2010) among others. Each component of BST is outlined below.
Instruction
The first component of BST is instruction, which involves describing a target skill or behavior one expects the trainee to perform (Miltenberger, 2016). Trainers can deliver instruction by describing procedures vocally (e.g., lecture, discussion) or textually (e.g., written protocols). Written protocols can include text alone or text with supplemental components, such as diagrams and images (i.e., enhanced written instructions; Berkman, Roscoe, & Bourret, 2019; Graff & Karsten, 2012). All too often, preservice training ends after instruction alone (DiGennaro Reed & Henley, 2015). It is critical to know that instruction alone is insufficient in training new skills (Ducharme & Feldman, 1992). Instead, training should begin with instruction followed by the other components that comprise BST.
This is not to say that instruction does not influence performance. Indeed, there are several ways in which instruction can be delivered to influence trainees’ performance. For example, Henley, Hirst, DiGennaro Reed, Becirevic, and Reed (2017) compared the use of directive instructions (e.g., “you must”) to nondirective instructions (e.g., “you might consider”) on trainees’ responding under changing reinforcement schedules in a laboratory task. When presented with directive instructions, participants responded in accordance with the instructions, even when the instructions were inaccurate (i.e., instructions did not match the reinforcement schedules). Participants in the nondirective condition, however, responded in accordance with the reinforcement schedules, independent of the instructions. Studies on this topic suggest that, when creating instructions for staff, it is important to consider how instructions are delivered, as minor variations may affect performance.
There are isolated instances in the literature of participants correctly performing skills after detailed instruction alone. For example, Graff and Karsten (2012) taught staff to effectively conduct preference assessments using enhanced written instructions, which included a detailed data sheet, diagrams, and step-by-step instructions written without technical jargon. However, Shapiro, Kazemi, Pogosjana, Rios, and Mendoza (2016) only partially replicated Graff and Karsten’s findings and had to include additional training procedures to reach criterion performance. Instances of performing to criterion after instruction alone are rare and may depend on various factors, such as the skill being trained, the instruction format, and the trainee’s history. Because these several factors must align for instruction alone to be effective, it is prudent to implement all components of BST when conducting preservice training.
Practice Guidelines We recommend that trainers present clear and succinct instructions about procedures, as Jarmolowicz et al. (2008) demonstrated that treatment adherence is better when instructions are written in a conversational form than in technical language. Moreover, it may be helpful to provide jargon-free, written protocols with diagrams (Graff & Karsten, 2012) that staff can use during training and refer to later, if needed. In our consultation experience, we have observed that trainees often appreciate the rationale for certain procedures, and providing that information may help guard against performance drift, though research is still needed in this area. Finally, we recommend trainers and supervisors be transparent about specific behaviors that will be observed and measured. It is presumed that the behaviors that will be observed and measured are of the utmost importance and, thus, should not be kept a secret from trainees. Also, new staff members generally want to perform well and may become nervous about their responsibilities and being observed. In our experience, transparency about what is expected and how performance will be measured generally reduces trainees’ nervousness.
Modeling
The second component of BST is modeling, which involves an experienced staff person demonstrating perfect performance of a target skill or behavior one expects the trainee to imitate (Miltenberger, 2016). Modeling can take place in person or shown via video. Video modeling includes the advantages of conserving resources if used over time and being transportable, allowing trainees the opportunity to view the video model outside of the training session. Further, a live model may contain slight errors or inconsistencies across trainings that affect training outcomes; video modeling provides the added benefit of standardizing training procedures and ensuring that only desired models are shown (DiGennaro Reed, Blackman, Erath, Brand, & Novak, 2018; Shapiro & Kazemi, 2017).
Recent research has shown video modeling, in particular video modeling with voiceover instruction, to be more effective than written instructions alone—though feedback was required for some participants—when training new staff to implement discrete trial instruction and behavior reduction interventions (e.g., DiGennaro Reed, Codding, Catania, & Maguire, 2010; Giannakakos, Vladescu, Kisamore, & Reeve, 2016). Furthermore, staff effectively learned how to conduct preference assessments after receiving training comprised of video modeling with voiceover instruction (e.g., Delli Bovi, Vladescu, DeBar, Carroll, & Sarokoff, 2017).
Practice Guidelines When incorporating modeling in training, it is important to model the entire procedure for the trainee. In addition, we recommend standardizing the models across training episodes to ensure the relevant skills are being modeled consistently. The latter recommendation may be accomplished by relying on video models or ensuring live models receive written protocols to follow during training. Finally, to aid in generalization, modeling should entail multiple exemplars of the target skill (Moore & Fisher, 2007). DiGennaro Reed, Erath, Brand, and Novak (2019) provide resources for creating video models, which readers may find beneficial.
Rehearsal
The third component of BST, rehearsal—also referred to as practice or role play— involves having trainees practice a target skill (Miltenberger, 2016). Rehearsal can be arranged in multiple ways, such as using a trained researcher or confederate (e.g., Iwata et al., 2000; McGimsey, Greene, & Lutzker, 1995; Phillips & Mudford, 2008), another trainee who is acquiring the skill being rehearsed (e.g., Palmen, Didden, & Korzilius, 2010; Wallace, Doney, Mintz-Resudek, & Tarbox, 2004), a service recipient (e.g., Erbas, Tekin-Iftar, & Yucesoy, 2006), or by varying the number of rehearsal opportunities (e.g., Jenkins & DiGennaro Reed, 2016). An important distinction is that rehearsal alone does not guarantee high levels of performance; in fact, it may allow trainees to practice errors, which may impede acquisition (Ward-Horner & Sturmey, 2012). Thus, rehearsal is typically accompanied by performance feedback and continues until trainees achieve mastery (Reid, Parsons, & Green, 2011).
Practice Guidelines In suggesting guidelines for trainers, we offer that practice does not make perfect; rather, practice with feedback makes perfect. That is, feedback should be delivered immediately following each rehearsal opportunity; thus, rehearsal and feedback should occur in tandem. Feedback is described in more detail in the next section. To help trainees acquire all relevant skills, the trainer must engineer opportunities for the trainee to practice the entire procedure, respond to a range of client responses, receive feedback, and meet a mastery criterion. These opportunities include using confederates in-vivo or in an analog setting, and having the confederate behave in scripted or predetermined ways to ensure the trainee practices all the components of a procedure and responds to a range of behaviors. Allowing trainees to respond to many situations is important, especially when dealing with vulnerable populations and procedures that need to be implemented with high integrity.
Feedback
The fourth component of BST is feedback, which refers to information about past performance that specifies how the trainee can improve performance in the future (Miltenberger, 2016). At this stage of training, feedback should be delivered immediately after each rehearsal opportunity and specify steps performed correctly and steps requiring correction (DiGennaro Reed et al., 2018). Several reviews of the literature reveal there are various dimensions of feedback, such as its source (supervisor, peer), medium or mode (written, verbal), frequency (daily, weekly), recipients (individuals, groups), privacy (private, public), and content (type of information provided; Alvero, Bucklin, & Austin, 2001; Balcazar, Hopkins, & Suarez, 1985; Prue & Fairbank, 1981). In addition, the way in which corrective and positive feedback are sequenced can influence its efficacy (Henley & DiGennaro Reed, 2015; Slowiak & Lakowske, 2017). Although supervisors and trainers have flexibility regarding how to implement the various dimensions of feedback, it is likely most efficient and effective to deliver both corrective and positive verbal feedback immediately upon completion of rehearsal.
Practice Guidelines The numerous variations and dimensions of feedback present a challenge to researchers seeking to examine the most effective combinations. Thus, no practice recommendations with an exact combination of methods can be made, which may be impractical as most supervisors are subject to various organizational and logistical constraints that limit their control over certain factors. However, research on the dimensions of feedback and its delivery has produced results with enough consistency to support general practice guidelines.
We recommend trainers base their feedback on direct observations rather than verbal reports or indirect measures. The use of observational data provides important information for the trainer and may also be used as a data source for delivering feedback. Next, trainee performance may improve faster if corrective feedback is presented before positive feedback (Henley & DiGennaro Reed, 2015); however, we often ask trainees to specify their preference for the order in which feedback is delivered. Third, supervisors should deliver feedback immediately after performance, as research suggests immediate feedback is more effective than delayed feedback (Goodman, Brady, Duffy, Scott, & Pollard, 2008) and staff prefer immediate feedback compared to delayed feedback (Reid & Parsons, 1996). We also recommend that feedback is used in conjunction with the components of BST described previously. Finally, it is important that feedback is delivered in a respectful and professional manner to maintain a positive working relationship, in particular when corrective feedback must be shared.
Ongoing Staff Support
It would be a mistake to assume that, once preservice training is complete, staff will perform all skills with high integrity while on the job and in a complex real-world environment. Thus, high-quality preservice training should be viewed as the initial investment in the professional development of staff. Based on decades of research, it is reasonable to assume that staff may not generalize skills learned in a contrived training environment to the actual work setting or maintain high levels of performance over extended periods of time. To address these challenges and support the maintenance and growth of staff skills, ongoing coaching is necessary. In the following sections, we outline how supervisors can provide ongoing support to staff following preservice training.
Observations and Feedback
The next step in the training process is the provision of ongoing supervision and support. This usually involves supervisory observations of the staff implementing varied procedures. The purpose of direct observations is twofold: to provide the supervisor with opportunities to monitor treatment integrity in the natural environment and to use data to provide performance feedback to staff. Depending on contextual variables operating in the organization (e.g., staff schedules, supervisor schedules, the setting), these observations can vary widely in when, how, and how often they are employed. For example, observations can be scheduled in advance or unscheduled drop-ins (when); viewed live (i.e., in-vivo) or via video recordings (how); and conducted as often or as little as needed, depending on staff performance, feasibility, and other related variables (how often).
Although variability exists regarding how supervisory observations are implemented, one commonality across all variations is to ensure recommended practices are being used. Table 1 depicts an on-the-job training protocol (adapted from Ricciardi, 2005) for using recommended practices to conduct ongoing observations. This protocol uses a competency-based approach to training (e.g., Reid & Parsons, 2002), and shares many of the same components used in initial training procedures (e.g., instruction, rehearsal, performance feedback). Thus, divergence from initial training procedures is more in relation to the training process, not training content, wherein the goal of ongoing supervision observations is to provide practice opportunities for staff to demonstrate high levels of treatment integrity in their actual workplace setting. The provision of supervision and support in this capacity also affords other benefits, such as the opportunity for supervisors and staff to troubleshoot issues regarding implementation in practice—an issue which may not have been present in the controlled training settings where initial training typically occurs. Related to this, supervisors can also embed coaching procedures (e.g., prompting, prompt fading, reinforcement) into their observations to facilitate staff implementation at mastery levels.
Table 1 On-the-job training protocol (adapted from Ricciardi, 2005)
| Step | Implementation guidelines |
| Verbal instruction | □ Meet with the trainee.
□ Verbally review training checklist item-by-item, answer questions, and provide clarification if needed. |
| Written instruction | □ Provide the trainee with a copy of the training checklist. □ Verbally describe how the checklist is a resource tool. |
| Trainee observations | □ On-the-job demonstrations are preferred over role-play scenarios.
□ Minimize reactivity during observation. □ Record data on trainee performance for each skill. |
| Deliver feedback | □ Review performance on each skill providing positive and corrective feedback, as needed.
□ Provide praise for steps and skills implemented correctly. □ Provide corrective feedback for steps and skills implemented incorrectly, along with information on how to implement correctly during future performance. □ Practice skills performed incorrectly via role play, model correct performance when needed. □ Ask the trainee if they have any questions and answer. |
| Repeat steps until mastery achieved | □ Observations should occur over time (e.g., across days, shifts).
□ Conduct observation probes intermittently to assess performance. |
Pay for Performance
An important consideration for maintaining desirable performance is the method by which staff are compensated for their work. Most compensation systems in the United States involve pay-for-time systems, in which compensation is primarily based on the amount of time spent at work, not performance of job duties. The prevalence of payfor-time systems presents an interesting challenge for behavior-analyst supervisors, as the primary contingency (i.e., pay) is delivered largely independent of performance of job duties. Although pay-for-performance systems likely maintain higher quality and quantity work than pay-for-time, several factors impede implementation of the pay-for performance systems.
The first barrier to performance-contingent pay is that many supervisors may not be in a position in their organization where they are able to dramatically change existing pay structures. Those supervisors who have control over organizational pay structures would also be likely to experience resistance from staff due to the ubiquity of traditional pay structures. A second barrier to pay-for-performance systems is a concern that these systems may place undue stress on employees—because their income is tied directly to their performance (Ganster, Kiersch, Marsh, & Bowen, 2011). This issue is made worse when incentive systems are dependent on factors beyond staff control.
Notwithstanding these barriers, supervisors must identify methods for reinforcing desirable performance within a traditional pay-for-performance system. A large body of research on the use of incentives in organizational settings suggests that desirable staff performance can be maintained by monetary incentives that account for a relatively small proportion of total compensation for staff (e.g., Dickinson & Gillette, 1994). That is, desired performance is maintained by the presence of an incentive contingency, not the percentage of incentive pay (Poling, Dickinson, Austin, & Normand, 2000). Thus, one solution may be for supervisors to use monetary incentives in conjunction with traditional pay systems—readers in organizations capable of and interested in providing performance-based pay are directed to Abernathy (1996, 2014) for additional resources. Note that this type of system is not without limitations. First, monetary incentives must be arranged so they are sustainable over an extended duration, and many human-service settings simply do not have the funds to maintain these efforts. Second, monetary incentives may be subject to certain federal and state labor regulations (e.g., Fair Labor Standards Act of 1938), which may create unwanted logistical difficulties for an organization. Thus, despite the advantages of using a generalized conditioned reinforcer in the form of monetary incentives, supervisors may seek alternative forms of incentive delivery to maintain desirable staff performance.
Identifying Potential Incentives Although supervisors may feel confident in their ability to select effective nonmonetary incentives for their staff, we urge caution in doing so. Wilder, Rost, and McMahon (2007) asked supervisors to rank-order potential incentives based on how effective they thought the incentive would be for each of their employees. When comparing supervisor predictions with employee rankings, Wilder et al. observed high agreement between supervisors’ indication of their employees’ most preferred items or activities; however, few supervisors accurately predicted moderate- and lower-preferred items or activities. These findings were replicated by Wilder, Harris, Casella, Wine, and Postma (2011) and suggest that supervisors generally do a poor job at predicting effective incentives.
Daniels and Bailey (2014) offer a systematic procedure to assist with selecting incentives. First, supervisors should consider what types of incentives they can reasonably provide. At this stage it is important to consider that incentives may be a mix of items, activities, or privileges. Whereas monetary and most tangible incentives have an inherent cost to an organization, many activities or privileges may come at little or no cost. For example, Iwata, Bailey, Brown, Foshee, and Alpern (1976) maintained high levels of daily care and training activities from staff working in a residential facility using an incentive where staff could rearrange their days off for the following week. Another important consideration at this stage is that supervisors should select items or activities that are sustainable for use for the indefinite future. Although reinforcement schedules may be thinned over time, the incentive program is only effective for as long as it is being implemented. Supervisors may need to have conversations with their organization’s management to identify incentives that the organization allows and can support—so the supervisor does not have to pay for it out-of-pocket.
Second, supervisors may wish to discuss these options for potential incentives with staff to identify a small number of items, activities, or privileges. This step should be conducted after first identifying potential items and activities because, otherwise, employees do not know what is available or may ask for things that cannot be delivered (see Daniels & Bailey, 2014).
Finally, after potential incentives have been identified, supervisors should conduct a systematic preference assessmentwith staff.Although thereis a richliteratureevaluating methods for assessing preference in clinical populations with limited verbal repertoires (e.g., Fisher et al., 1992), these methods are likely not appropriate for staff with strong verbal repertoires (see Waldvogel & Dixon, 2008). Two commonly used methods for assessing employee preference for incentives are a reinforcer survey (Daniels & Bailey, 2014) and a ranking procedure (Waldvogel & Dixon, 2008; Wine, Reis, & Hantula, 2014). The reinforcer survey asks employees to rate how much work they would be willing to do for each item or activity on a Likert-type scale from 0 (none at all) to 4 (very much). The ranking procedure is similar but provides a relative value for each potential incentive by asking employees to rank the items or activities from least to most preferred. Both the survey and ranking procedure provide relatively accurate indications of effective incentives—although the survey method may be better at identifying a greater range of effective incentives (see Wine et al., 2014 for a comparison).
Incentive Delivery A final consideration for ensuring effective use of incentives is the way they are delivered. Some variables that warrant additional consideration for use with staff include incentive quality, probability of delivery, and delay to delivery. Incentive quality is largely determined by preference, although incentive magnitude and schedule also affect quality. Incentives may be delivered on a dense schedule initially and thinned once the employee has consistently demonstrated desired performance. Two considerations with respect to quality are that staff preferences may shift over time (Wine, Gilroy, & Hantula, 2012) and the use of incentives of varied preference may maintain high levels of performance (Wine & Wilder, 2009). Thus, supervisors should consider assessing preference on a regular basis and using a variety of moderate- to high-preferred incentives.
Although rules can help bridge the gap of delays to reinforcement, supervisors should seek to minimize delays as much as possible. In addition, with respect to the probability of incentive delivery, reinforcers may not need to be delivered each time. Though there is little research assessing probability of incentive delivery in OBM settings, several studies have demonstrated the effectiveness of lottery systems at maintaining performance (e.g., Cook & Dixon, 2006; Iwata et al., 1976). Taken together, there has been little OBM research assessing the relative impact of reinforcer dimensions. However, recent research incorporating the area of behavioral economics has provided a promising methodology through which a thorough understanding may be developed (e.g., Henley, DiGennaro Reed, Reed, & Kaplan, 2016; Wine et al., 2012).
Assessing Performance Problems
Despite best efforts to train and provide follow-up support to staff, supervisors will likely encounter instances of less-than-desirable performance. As is true with service delivery in clinical populations, performance management interventions typically provide benefit when they are informed by preintervention functional assessment. Though several informant assessments exist and have been implemented with success (e.g., Daniels & Lattal, 2017; Mager & Pipe, 1984), the Performance Diagnostic Checklist– Human Services (PDC–HS; Carr, Wilder, Majdalany, Mathisen, & Strain, 2013) has emerging support for use in human-service settings. The PDC–HS may be especially advantageous for BCBAs in supervisory positions who may have limited OBM experience as it is less time consuming and does not require the same level of expertise as behavioral systems analysis (for more information about behavioral systems analysis, see Johnson, Casella, McGee, & Lee, 2014; McGee & Diener, 2010; Sigurdsson & McGee, 2015). The PDC–HS is an informant assessment that may be used to identify variables causing or maintaining substandard performance. The assessment consists of 20 questions across four domains: (a) training; (b) task clarification and prompting; (c) resources, materials, and processes; and (d) performance consequences, effort, and competition. As its name suggests, the PDC–HS was developed specifically for use in human service settings, and the questions and domains were developed to target environmental variables that commonly affect performance in this particular setting.
The first section, training, contains questions about the type of training received and whether the employee has demonstrated desired levels of performance in the past. The second section, task clarification and prompting, includes questions about the employee’s knowledge of job requirements and the environment where the behavior is expected to occur. Third, the resources, materials, and processes section contains questions about the organization’s systems and resources that may be beyond the employee’s control. Finally, the performance consequences, effort, and competition section includes questions about overall supervisor support and supervision, feedback, and potential competing activities.
Carr et al. (2013) indicated that seven of the items on the PDC–HS can be answered through direct observation and recommended administering the remaining 13 items through discussion with the employee’s supervisor. Practitioners administering the PDC–HS might consider also administering it with multiple supervisors or across different levels of the organization (e.g., management, supervisors, employees) as there may not be 100% agreement across all relevant parties (e.g., Merritt, DiGennaro Reed, & Martinez, 2019).
When complete, results of the PDC–HS are scored by counting the number of items for which an area of concern was flagged in each domain (see Carr & Wilder, 2016; Carr et al., 2013); the domain with the most flagged items typically indicates the “function” of the performance problem, or the area to be targeted for intervention. Where the PDC–HS may be particularly useful is the accompanying intervention planning guide, which provides a list of recommended interventions (with supporting literature) for each domain. For example, Bowe and Sellers (2018) conducted a PDC– HS to assess inaccurate implementation of error correction procedures during teaching sessions, and results indicated that insufficient training was likely the greatest contributing factor for performance problems. The experimenters then compared an indicated intervention of BST with an intervention recommended in the task clarification and prompting domain (posting reminders; i.e., a nonindicated intervention) and found that performance improved only following the indicated intervention. In sum, the PDC–HS is particularly beneficial for supervisors in human service settings as it provides a systematic assessment of performance problems, is tailored specifically for use in human service settings, and helps supervisors select a function-indicated intervention.
Continuing Education for Staff
As employees continue their professional growth at an organization, their interests may begin to extend beyond the scope of what a supervisor is able to provide. These interests may be viewed as an opportunity to develop skills by facilitating additional learning opportunities. Many of the approaches described in the section below on staying up to date with research can also be adapted for staff development. For example, a supervisor may encourage staff to attend regional conferences or workshops by providing time off work or assisting with registration costs. Those who oversee a large number of staff may consider inviting speakers to deliver workshops on topics related to the services the organization provides. By facilitating continuing education opportunities, supervisors create varied learning opportunities for staff while also creating natural rewards for staff who stay with an organization.
Staying Up to Date in Performance Management Research
Staying current with the research literature is an important skill for practitioners as it may foster the provision of high-quality supervision, training, and services across all levels of the organization. Section 1.01 of the Professional and Ethical Compliance Code includes a provision that states “behavior analysts rely on professionally derived knowledge based on science and behavior analysis when making scientific or professional judgements in human service provision, or when engaging in scholarly or professional endeavors” (BACB, 2014, p. 4). To meet this standard, credentialed behavior analysts must remain abreast of current scientific findings. At the broadest level, this often involves various classes of behavior that can be categorized into two areas: (a) attending professional development opportunities and (b) reading peerreviewed publications in staff training and performance management.
Attending professional development opportunities often means attending local, regional, and/or international behavior-analytic conferences and workshops to learn about recent developments in a given area. With current technology, webinars, onlinebased trainings, and podcasts may be options. For example, the OBM Network (http://obmnetwork.com) provides numerous webinars from excellent researchers in the field of OBM on topics such as feedback, remote supervision, pay-for performance, leadership, and staff turnover, among others.
Carr and Briggs (2010) offer helpful recommendations for overcoming barriers to staying abreast of the scholarly literature. In addition, a means of remaining up to date on latest publications is to use software- and internet-based tools for alerts on recently published literature in a journal or content area. For example, many journal providers offer table-of-content alerts, which present the user with a brief snapshot of the articles published in the newest issue of a journal. Pubcrawler (http://pubcrawler.gen.tcd.ie/) is an internet-based alert service that allows users to create personal queries for recently published articles, which are then compiled into a list and sent via email. A notable feature is that users can set their search criteria for the query using an extensive list of settings, including but not limited to commonly used components such as keywords and articles from particular journals. For example, if a user were interested in receiving updates for recent publications on staff training, the user could set up a search query by setting the search criteria for “staff training” as the keyword and a list of behavioranalytic journals as the places in which the query should search.
Given the expansive and ever-growing body of literature, it can be difficult to discern which literature is most relevant, in particular when many articles are accessible only through fees or university subscriptions. However, there are numerous methods to find articles that contain staff training or performance management content. One simple and straightforward way is to purchase access to behavior-analytic journals through the Association for Behavior Analysis International©, which can often be done at a discounted rate for members. Credentialed behavior analysts also have access to select journals through the online portal of the BACB. Another method is to use Google Scholar (https://scholar.google.com), where articles may be freely available. An additional benefit to using Google Scholar is that searches expand beyond the scope of just peer-reviewed articles, as their database also indexes books, conferences, and other nonpeer-reviewed materials. ResearchGate© (https://www.researchgate.net) is another tool for individuals to use to either download an article or reach out to the researcher directly. Finally, although many articles can be accessed using the methods described above, authors will typically be happy to send articles to those who reach out to them directly via email (often found on their university web page) or ResearchGate.
Conclusion
Recent research suggests that recommended staff training and performance management practices are not being regularly implemented in organizations that hire behavior analysts, which could affect the quality of services being provided. This article attempted to bridge the research-to-practice gap by outlining research-supported initial training and ongoing staff support procedures within human service settings, presenting practice guidelines, and sharing information about easy-to-implement ways practitioners may stay abreast of current research. Assessment and intervention procedures based on the science of behavior analysis have empirical support, but we must ensure they are being implemented with high integrity by a well-trained and supported workforce.
References
Abernathy, W. B. (1996). The sin of wages. Atlanta, GA: Performance Management.
Abernathy, W. B. (2014). Beyond the Skinner box: The design and management of organization-wide performance systems. Journal of Organizational Behavior Management, 34, 235–254. https://doi.org/10.1080/01608061.2014.973631
Alvero, A. M., Bucklin, B. R., & Austin, J. (2001). An objective review of the effectiveness and essential characteristics of performance feedback in organizational settings. Journal of Organizational Behavior Management, 21(1), 3–29. https://doi.org/10.1300/J075v21n01_02
Balcazar, F. E., Hopkins, B. L., & Suarez, Y. (1985). A critical, objective review of performance feedback. Journal of Organizational Behavior Management, 7(3–4), 65–89. https://doi.org/10.1300/J075v07n03_05
Behavior Analyst Certification Board. (2014). Professional and ethical compliance code for behavior analysts. Littleton, CO: Author. Retrieved from https://www.bacb.com/ethics/ethics-code/ (Accessed August 19, 2016).
Berkman, S. J., Roscoe, E. M., & Bourret, J. C. (2019). Comparing self-directed methods for training staff to create graphs using GraphPad Prism. Journal of Applied Behavior Analysis, 52, 188–204. https://doi.org/10.1002/jaba.522
Bowe, M., & Sellers, T. P. (2018). Evaluating the Performance Diagnostic Checklist–Human Services to assess incorrect error-correction procedures by preschool paraprofessionals. Journal of Applied Behavior Analysis, 51, 166–176. https://doi.org/10.1002/jaba.428
Brand, D., Henley, A. J., DiGennaro Reed, F. D., Gray, E., & Crabbs, B. (2019). A review of published studies involving parametric manipulations of treatment integrity. Journal of Behavioral Education, 28, 1–26. https://doi.org/10.1007/s10864-018-09311-8
Carr, J. E., & Briggs, A. M. (2010). Strategies for making regular contact with the scholarly literature. Behavior Analysis in Practice, 3, 13–18. https://doi.org/10.1007/BF03391760
Carr, J. E., & Wilder, D. A. (2016). The Performance Diagnostic Checklist–Human Services: A correction. Behavior Analysis in Practice, 9, 63. https://doi.org/10.1007/s40617-015-0099-3
Carr, J. E., Wilder, D. A., Majdalany, L., Mathisen, D., & Strain, L. A. (2013). An assessment-based solution to a human-service employee performance problem. Behavior Analysis in Practice, 6, 16–32. https://doi.org/10.1007/BF03391789
Cook, T., & Dixon, M. R. (2006). Performance feedback and probabilistic bonus contingencies among employees in a human service organization. Journal of Organizational Behavior Management, 25(3), 45–63. https://doi.org/10.1300/J075v25n03_04
Daniels, A. C., & Bailey, J. S. (2014). Performance management: Changing behavior that drives organizational performance (5th ed.). Atlanta, GA: Performance Management Publications.
Daniels, A. C., & Lattal, A. D. (2017). Life’s a PIC/NIC… when you understand behavior. Cornwall on Hudson, NY: Sloan Publishing.
Delli Bovi, G. M., Vladescu, J. C., DeBar, R. M., Carroll, R. A., & Sarokoff, R. A. (2017). Using video modeling with voice-over instruction to train public school staff to implement a preference assessment. Behavior Analysis in Practice, 10, 72–76. https://doi.org/10.1007/s40617-016-0135-y
Dickinson, A. M., & Gillette, K. L. (1994). A comparison of the effects of two individual monetary incentive systems on productivity: Piece rate pay versus base pay plus incentives. Journal of Organizational Behavior Management, 14(1), 3–82. https://doi.org/10.1300/J075v14n01_02
DiGennaro Reed, F. D. (2016, November). ABCs of staff support: Evidence-based performance management. Presentation for the Iowa Association for Behavior Analysis Annual Conference, Des Moines, IA.
DiGennaro Reed, F. D. (2017, May). Evidence-based performance management: Applying behavioral science to support practitioners. Presentation for the OBM in Health and Human Services Conference, Denver, CO.
DiGennaro Reed, F. D. (2018, May). Using behavioral science to support educators during consultation. Presentation for the annual meeting of the Association for Behavior Analysis International, San Diego, CA.
DiGennaro Reed, F. D. (2019, April). Performance management in ABA service settings. Presentation for the annual meeting of the Association for Professional Behavior Analysts, Atlanta, GA.
DiGennaro Reed, F. D., Blackman, A. L., Erath, T. G., Brand, D., & Novak, M. D. (2018). Guidelines for using behavioral skills training to provide teacher support. Teaching Exceptional Children, 50, 373–380. https://doi.org/10.1177/0040059918777241
DiGennaro Reed, F. D., Codding, R., Catania, C. N., & Maguire, H. (2010). Effects of video modeling on treatment integrity of behavioral interventions. Journal of Applied Behavior Analysis, 43, 291–295. https://doi.org/10.1901/jaba.2010.43-291
DiGennaro Reed, F. D., Erath, T. G., Brand, D., & Novak, M. D. (2019). Video modeling during coaching and performance feedback. In A. Fischer, E. Dart, T. Collins, & K. Radley (Eds.), Technology applications in school consultation, supervision, and school psychology training. New York, NY: Routledge.
DiGennaro Reed, F. D., & Henley, A. J. (2015). A survey of staff training and performance management practices: The good, the bad, and the ugly. Behavior Analysis in Practice, 8, 16–26. https://doi.org/10.1007/s40617-015-0044-5
Ducharme, J. M., & Feldman, M. A. (1992). Comparison of staff training strategies to promote generalized teaching skills. Journal of Applied Behavior Analysis, 25, 165–179. https://doi.org/10.1901/jaba.1992.25165
Erbas, D., Tekin-Iftar, E., & Yucesoy, S. (2006). Teaching special education teachers how to conduct functional analysis in natural settings. Education and Training in Developmental Disabilities, 41, 28–36. Fair Labor Standards Act of 1938. (n.d.). 29 U.S.C. §201 et seq.
Fisher, W., Piazza, C. C., Bowman, L. G., Hagopian, L. P., Owens, J. C., & Slevin, I. (1992). A comparison of two approaches for identifying reinforcers for persons with severe and profound disabilities. Journal of Applied Behavior Analysis, 25, 491–498. https://doi.org/10.1901/jaba.1992.25-491
Freedman, D. H. (2016). Improving public perception of behavior analysis. The Behavior Analyst, 39, 89–95. https://doi.org/10.1007/s40614-015-0045-2
Ganster, D. C., Kiersch, C. E., Marsh, R. E., & Bowen, A. (2011). Performance-based rewards and work stress. Journal of Organizational Behavior Management, 31, 221–235. https://doi.org/10.1080/01608061.2011.619388
Giannakakos, A. R., Vladescu, J. C., Kisamore, A. N., & Reeve, S. A. (2016). Using video modeling with voiceover instruction plus feedback to train staff to implement direct teaching procedures. Behavior Analysis in Practice, 9, 126–134. https://doi.org/10.1007/s40617-015-0097-5
Goodman, J. I., Brady, M. P., Duffy, M. L., Scott, J., & Pollard, N. E. (2008). The effects of “bug-in-ear” supervision on special education teachers’ delivery of learn units. Focus on Autism and Other Developmental Disabilities, 23, 207–216. https://doi.org/10.1177/1088357608324713
Graff, R. B., & Karsten, A. M. (2012). Evaluation of a self-instruction package for conducting stimulus preference assessments. Journal of Applied Behavior Analysis, 45, 69–82. https://doi.org/10.1901/jaba.2012.45-69
Henley, A. J. (2018). Empirical foundations and practical applications of behavioral skills training. Invited presentation and workshop for Behavior Services of Western Massachusetts, Springfield, MA.
Henley, A. J. (2019). Supervising like a boss: Healthy behavioral practices for promoting effective supervision. Invited presentation for the Massachusetts Association of 766 Approved Private Schools, Marlborough, MA.
Henley, A. J., & DiGennaro Reed, F. D. (2015). Should you order the feedback sandwich? Efficacy of feedback sequence and timing. Journal of Organizational Behavior Management, 35, 321–335. https://doi.org/10.1080/01608061.2015.1093057
Henley, A. J., DiGennaro Reed, F. D., Reed, D. D., & Kaplan, B. A. (2016). A crowdsourced nickel-and-dime approach to analog OBM research: A behavioral economic framework for understanding workforce attrition. Journal of the Experimental Analysis of Behavior, 106, 134–144. https://doi.org/10.1002/jeab.220
Henley, A. J., Hirst, J. M., DiGennaro Reed, F. D., Becirevic, A., & Reed, D. D. (2017). Function-altering effects of rule phrasing in the modulation of instructional control. The Analysis of Verbal Behavior, 33, 24–40. https://doi.org/10.1007/s40616-016-0063-5
Higgins, S. T., Silverman, K., & Heil, S. H. (Eds.). (2007). Contingency management in substance abuse treatment. New York, NY: Guilford Press.
Iwata, B. A., Bailey, J. S., Brown, K. M., Foshee, T. J., & Alpern, M. (1976). A performance-based lottery to improve residential care and training by institutional staff. Journal of Applied Behavior Analysis, 9, 417– 431. https://doi.org/10.1901/jaba.1976.9-417
Iwata, B. A., Wallace, M. D., Kahng, S., Lindberg, J. S., Roscoe, E. M., Conners, J., et al. (2000). Skill acquisition in the implementation of functional analysis methodology. Journal of Applied Behavior Analysis, 33, 181–194. https://doi.org/10.1901/jaba.2000.33-181
Jarmolowicz, D. P., Kahng, S., Ingvarsson, E. T., Goysovich, R., Heggemeyer, R., & Gregory, M. K. (2008). Effects of conversational versus technical language on treatment preference and integrity. Intellectual & Developmental Disabilities, 46, 190–199. https://doi.org/10.1352/2008.46:190-199
Jenkins, S. R., & DiGennaro Reed, F. D. (2016). A parametric analysis of rehearsal opportunities on procedural integrity. Journal of Organizational Behavior Management, 36, 255–281. https://doi.org/10.1080/01608061.2016.1236057
Johnson, D. A., Casella, S. E., McGee, H., & Lee, S. C. (2014). The use and validation of pre-intervention diagnostic tools in Organizational Behavior Management. Journal of Organizational Behavior Management, 34, 104–121. https://doi.org/10.1080/01608061.2014.914009
Kazemi, E., Shapiro, M., & Kavner, A. (2015). Predictors of intention to turnover in behavior technicians working with individuals with autism spectrum disorder. Research in Autism Spectrum Disorders, 17, 106–115. https://doi.org/10.1016/j.rasd.2015.06.012
Leon, Y., Wilder, D. A., Majdalany, L., Myers, K., & Saini, V. (2014). Errors of omission and commission during alternative reinforcement of compliance: The effects of varying levels of treatment integrity. Journal of Behavioral Education, 23, 19–33. https://doi.org/10.1007/s10864-013-9181-5
Lindgren, S., Wacker, D., Suess, A., Schieltz, K., Pelzel, K., Kopelman, T., et al. (2016). Telehealth and autism: Treating challenging behavior at lower cost. Pediatrics, 137, S167–S175. https://doi.org/10.1542/peds.2015-2851O
Lord, C., & McGee, J. P. (Eds.). (2001). Educating children with autism. Washington, DC: National Academy Press, Committee on Educational Interventions for Children with Autism, Division of Behavioral and Social Sciences and Education, National Research Council.
Mager, R., & Pipe, P. (1984). Analyzing performance problems (2nd ed.). Belmont, CA: Lake.
McGee, H. M., & Diener, L. H. (2010). Behavioral systems analysis in health and human services. Behavior Modification, 34, 415–442. https://doi.org/10.1177/0145445510383527
McGimsey, J. F., Greene, B. F., & Lutzker, J. R. (1995). Competence in aspects of behavioral treatment and consultation: Implications for service delivery and graduate training. Journal of Applied Behavior Analysis, 28, 301–315. https://doi.org/10.1901/jaba.1995.28-301
Merritt, T. A., DiGennaro Reed, F. D., & Martinez, C. E. (in press). Using the Performance Diagnostic Checklist–Human Services to identify an indicated intervention to decrease employee tardiness. Journal of Applied Behavior Analysis, 52, 1034–1048. https://doi.org/10.1002/jaba.643
Miltenberger, R. G. (2016). Behavior modification: Principles and procedures (6th ed.). Boston, MA: Cengage Learning.
Moore, J. W., & Fisher, W. W. (2007). The effects of videotape modeling on staff acquisition of functional analysis methodology. Journal of Applied Behavior Analysis, 40, 197–202. https://doi.org/10.1901/jaba.2007.24-06
National Autism Center. (2009). National Standards Project: Findings and conclusions. Randolph, MA: Author.
National Autism Center. (2015). National Standards Project, Phase 2: Findings and conclusions. Addressing the need for evidence-based practice guidelines for autism spectrum disorder. Randolph, MA: Author.
Nigro-Bruzzi, D., & Sturmey, P. (2010). The effects of behavioral skills training on mand training by staff and unprompted vocal mands by children. Journal of Applied Behavior Analysis, 43, 757–761. https://doi.org/10.1901/jaba.2010.43-757
Palmen, A., Didden, R., & Korzilius, H. (2010). Effectiveness of behavioral skills training on staff performance in a job training setting for high-functioning adolescence with autism spectrum disorder. Research in Autism Spectrum Disorders, 4, 731–740. https://doi.org/10.1016/j.rasd.2010.01.012
Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-based staff training: A guide for practitioners. Behavior Analysis in Practice, 5, 2–11. https://doi.org/10.1007/BF03391819
Peterson, L., Homer, A. L., & Wonderlich, S. A. (1982). The integrity of independent variables in behavior analysis. Journal of Applied Behavior Analysis, 15, 477–492. https://doi.org/10.1901/jaba.1982.15-477
Phaneuf, L., & McIntyre, L. L. (2007). Effects of individualized video feedback combined with group parent training on inappropriate maternal behavior. Journal of Applied Behavior Analysis, 40, 737–741. https://doi.org/10.1901/jaba.2007.737-741
Phillips, K. J., & Mudford, O. C. (2008). Functional analysis training for residential caregivers. Behavioral Interventions, 23, 1–12. https://doi.org/10.1002/bin.252
Piazza, C. C., Milnes, S. M., & Shalev, R. A. (2015). A behavior-analytic approach to the assessment and treatment of pediatric feeding disorders. In H. S. Roane, J. E. Ringdahl, & T. S. Falcomata (Eds.), Clinical and organizational applications of applied behavior analysis (pp. 69–94). Cambridge, MA: Academic Press. https://doi.org/10.1016/B978-0-12-420249-8.00004-6
Poling, A., Dickinson, A. M., Austin, J., & Normand, M. P. (2000). Basic behavioral research and organizational behavior management. In J. Austin & J. E. Carr (Eds.), Handbook of applied behavior analysis (pp. 295–320). Reno, CA: Context Press.
Prue, D. M., & Fairbank, J. A. (1981). Performance feedback in organizational behavior management: A review. Journal of Organizational Behavior Management, 3(1), 1–16. https://doi.org/10.1300/J075v03n01_01
Reid, D. H., & Parsons, M. B. (1996). A comparison of staff acceptability of immediate versus delayed verbal feedback in staff training. Journal of Organizational Behavior Management, 16(2), 35–47. https://doi.org/10.1300/J075v16n02_03
Reid, D. H., & Parsons, M. B. (2002). Working with staff to overcome challenging behavior among people who have severe disabilities: A guide for getting support plans carried out. Morganton, NC: Habilitative Management Consultants.
Reid, D. H., Parsons, M. B., & Green, C. W. (2011). Evidence-based ways to promote work quality and enjoyment among support staff: Trainee guide. Washington, DC: American Association of on Intellectual and Developmental Disabilities.
Ricciardi, J. N. (2005). Achieving human service outcomes through competency-based training: A guide for managers. Behavior Modification, 29, 488–507. https://doi.org/10.1177/0145445504273281
Sarokoff, R. A., & Sturmey, P. (2004). The effects of behavioral skills training on staff implementation of discrete-trial teaching. Journal of Applied Behavior Analysis, 37, 535–538. https://doi.org/10.1901/jaba.2004.37-535
Shapiro, M., & Kazemi, E. (2017). A review of training strategies to teach individuals implementation of behavioral interventions. Journal of Organizational Behavior Management, 37, 32–62. https://doi.org/10.1080/01608061.2016.1267066
Shapiro, M., Kazemi, E., Pogosjana, M., Rios, D., & Mendoza, M. (2016). Preference assessment training via self-instruction: A replication and extension. Journal of Applied Behavior Analysis, 49, 794–808. https://doi.org/10.1002/jaba.339
Sigurdsson, S. O., & McGee, H. M. (2015). Organizational behavior management: Systems analysis. In H. S. Roane, J. E. Ringdahl, & T. S. Falcomata (Eds.), Clinical and organizational applications of applied behavior analysis (pp. 627–647). Cambridge, MA: Academic Press. https://doi.org/10.1016/B978-0-12420249-8.00025-3
Slowiak, J. M., & Lakowske, A. M. (2017). The influence of feedback statement sequence and goals on task performance. Behavior Analysis: Research & Practice, 17, 357–380. https://doi.org/10.1037/bar0000084
Strock, M. (2007). Autism spectrum disorders (pervasive developmental disorders). Bethesda, MD: National Institute of Mental Health. Retrieved from https://eric.ed.gov/?id=ED495219 Accessed 19 Aug 2019.
Sulzer-Azaroff, B., & Gillat, A. (1990). Trends in behavior analysis in education. Journal of Applied Behavior Analysis, 23, 491–495. https://doi.org/10.1901/jaba.1990.23-491
Sundberg, D. B. (2016) Why people quit your company and how behavior analysis can slow the revolving door at ABA service providers. Retrieved from https://bsci21.org/why-people-quit-your-company-andhow-behavior-analysis-can-slow-the-revolving-door-at-aba-service-providers/. Accessed 2 Nov 2016.
U.S. Department of Health & Human Services. (1999). Mental health: A report of the Surgeon General. Rockville, MD: U.S. Department of Health & Human Services, Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, National Institutes of Health, National Institute of Mental Health.
Van Houten, R., Nau, P., & Marini, Z. (1980). An analysis of public posting in reducing speeding behavior on an urban highway. Journal of Applied Behavior Analysis, 13, 383–395. https://doi.org/10.1901/jaba.1980.13-383
Waldvogel, J. M., & Dixon, M. R. (2008). Exploring the utility of preference assessments in organizational behavior management. Journal of Organizational Behavior Management, 28, 76–87. https://doi.org/10.1080/01608060802006831
Wallace, M. D., Doney, J. K., Mintz-Resudek, C. M., & Tarbox, R. S. (2004). Training educators to implement functional analyses. Journal of Applied Behavior Analysis, 37, 89–92. https://doi.org/10.1901/jaba.2004.37-89
Ward-Horner, J., & Sturmey, P. (2012). Component analysis of behavior skills training in functional analysis. Behavioral Interventions, 27, 75–92. https://doi.org/10.1002/bin.1339
Wilder, D. A., Harris, C., Casella, S., Wine, B., & Postma, N. (2011). Further evaluation of the accuracy of managerial prediction of employee preference. Journal of Organizational Behavior Management, 31, 130–139. https://doi.org/10.1080/01608061.2011.569202
Wilder, D. A., Rost, K., & McMahon, M. (2007). The accuracy of managerial prediction of employee preference: A brief report. Journal of Organizational Behavior Management, 27(2), 1–14. https://doi.org/10.1300/J075v27n02_01
Wine, B., Gilroy, S., & Hantula, D. A. (2012). Temporal (in)stability of employee preferences for rewards. Journal of Organizational Behavior Management, 32, 58–64. https://doi.org/10.1080/01608061.2012.646854
Wine, B., Reis, M., & Hantula, D. A. (2014). An evaluation of stimulus preference assessment methodology in organizational behavior management. Journal of Organizational Behavior Management, 34, 7–15. https://doi.org/10.1080/01608061.2013.873379
Wine, B., & Wilder, D. A. (2009). The effects of varied versus constant high-, medium-, and low-preference stimuli on performance. Journal of Applied Behavior Analysis, 42, 321–326. https://doi.org/10.1901/jaba.2009.42-321
Yeaton, W. H., & Bailey, J. S. (1978). Teaching pedestrian safety skills to young children: An analysis and one-year follow-up. Journal of Applied Behavior Analysis, 11, 315–329. https://doi.org/10.1901/jaba.1978.11-315
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
To our knowledge, the extant research has focused on effects of models of exemplar performance only; more research is needed on the effects of including models of nonexemplar performance on training outcomes.
