by Robin Foster, PhD, CAAB, CHBC A clicker-trained OTTB is torn between approaching and running away when he sees his owner walking into the pasture, but, as soon as he hears the sound of the clicker, he trots toward her for a treat. A Grand-Prix warmblood hears the barn door open in the morning and starts to pace, call, and paw at the ground until her grain is delivered. A two-year old wild mustang was fearful of most humans at first, but after he learned to target people with his muzzle for a hand-fed treat, the young horse now boldly walks up to every stranger and bumps the person with his nose. These examples are familiar to horse owners. They confirm that food is a powerful motivator of both desirable and unwanted behavior, and more importantly, they also illustrate that cues associated with food are quickly learned and also have the power to influence behavior. In each example, the horses responded to a cue that predicted food. When food is used as a reward in training, contrived cues—such as a clicker, special word, or gesture—may be specifically introduced to signal that food is forthcoming. A number of accidental cues also predict food that may not be part of the training plan, but are meaningful to the animal. Examples of accidental cues include: objects always present during training (a particular person, a target, a treat bag, a special halter); the location where food is delivered (from the hand, a dish, or a hay net); and the training context (facility, arena, paddock, or stall). An abundance of research confirms that cues associated with rewards have a powerful influence on motivation and behavior. Most research on reward learning has involved rodents in controlled laboratory studies, but the reward centers and neural pathways in the brains of all vertebrates are remarkably similar, and many of the findings in rodents apply to humans, horses, and other mammals, as well. What are Sign Tracking and Goal Tracking? "Sign-tracking” [1] (ST) refers to the finding that, for some animals, the cues associated with food and other rewards become powerful “motivational magnets”, charged with value (referred to in the scientific literature as incentive salience.) Sign-trackers approach and manipulate the cue first, before approaching the food. In animals that develop ST responses, simply being in contact with the cue is rewarding, and the cue functions as an effective conditioned reinforcer.[2] [3] Not all animals develop ST behavior. Certain individuals are predisposed to develop ST, but for others the cue does not become charged with value. These animals respond by “goal-tracking” (GT) and move directly to the location where the food will be delivered.[4] The cue predicts reward for both sign-trackers and goal-trackers the cue predicts reward, but the animals’ responses are markedly different. ST has been extensively studied in laboratory animals. It is essentially unknown in applied animal behavior, but almost certainly occurs. Using clicker training as an illustrative example, a horse predisposed to ST may persistently watch, approach, touch, and even mouth the clicker, before shifting its attention and moving to the location where the food will be delivered, such as an outstretched hand or food dish. In some cases the ST animal may even become unusually fixated on the clicker. In contrast, a GT horse will move immediately to the location of food delivery when it hears the clicker sound. Some individuals are more likely to engage in ST and others in GT. These distinct responses to cues are stable and may be governed by distinct learning systems.[5] Researchers studying individual differences are interested in how environment, neurobiology and genetic make-up interact to influence the incentive value of cues associated with rewards, including food, drugs, and sex. Why is Sign Tracking a Concern in Training? ST is an expected and common response to cues that are associated with rewards. Even though it is “normal”, ST has been linked to other behavior and learning patterns that can interfere with training and raise welfare concerns, and a goal in training should be to minimize the incidence of ST behavior. A few of these issues are described in the following sections. Sign-tracking has been linked to an increased risk for compulsive behavior and addiction[6] Sign-trackers may be especially prone to compulsive behaviors and addiction, because they place excessive incentive value on reward cues. In contrast, GT individuals are resistant to compulsive behavior and addiction.[7] [8] ST has been used as model of eating disorders and drug addiction, and is believed to reflect an “addiction-prone phenotype”.[9] It is widely recognized that food and drugs can take control over an individual’s thoughts and actions, depending on individual’s vulnerability to addiction as well as the qualities of the food or drug. Animals prone to ST are unable to shift their thoughts and actions away from both the food and cues associated with food (or cues associated with drugs.) Similarly, in humans, binge eaters are attracted by food-cues (e.g., food smells), and drug addicts are strongly attracted to drug-cues (e.g., syringes). Unusually strong attraction to the cue seen in sign-trackers can lead to excessive indulgence in the food or drug. The value of the cue can be enhanced in sign-trackers—but not in goal-trackers—by exposing the animal to drugs like cocaine and amphetamines that sensitize the reward-learning neural pathways.[10] Whether certain foods can similarly sensitize the reward-learning neural pathways isn’t known, but is possible. Behavior change protocols often recommend using high quality treats, and trainers often complain that high value treats seem to take control over a horse’s thoughts and actions, and can even interfere with training. “Lusting while loathing”[11] can be triggered by cues associated with rewards Desire and pleasure areas of the brain are activated by food, drugs, sex, and other pleasurable stimuli, as well as by cues linked to these stimuli. An important scientific discovery is that wanting (desire) and liking (pleasure) appear to be distinct experiences with different neural pathways. Wanting is the motivation to seek out and consume, and liking is the warm and fuzzy feeling of satisfaction. The urgent, intense, cravings of extreme wanting characteristic of addictive and compulsive behaviors may hold no pleasure whatsoever, and may even be distressing. Desire is easy to recognize through behaviors such as approach and consumption; pleasure is much more challenging to directly observe and objectively measure in animals, but even in rats the body language associated with liking and wanting are distinct if one observes closely enough.[12] “When rats taste something sweet, they do what human babies do – poke their tongues out and lick their lips. The more they enjoy a taste, the more their tiny tongues waggle with delight…. Tongue-waggling and lip-licking, believe it or not, are Berridge’s pleasure barometers.”[13] Learning to recognize the body language of liking, and to distinguish it from wanting, has welfare implications when using food-reward training. Wanting may motivate behavior, but it doesn’t follow that the experience is pleasurable. Sign-trackers are susceptible to and unable to inhibit desire, even when the pleasure of consuming the food or drug diminishes. For sign-trackers both desire and pleasure will be high at first, but the wanting will persist—triggered by the sight, smell, and proximity of the reward—even when the pleasure is gone. For goal-trackers, desire and pleasure are more closely linked and the animal will lose interest as it becomes satiated; desire will decrease as pleasure diminishes because the cues that predict treats hold no incentive value. Sign-trackers may be vulnerable to fear conditioning Sign-trackers attribute excessive value to a learned cue, regardless of its emotional valence. Most studies on ST and GT have focused on cues that predict food and drugs, but sign-trackers also appear to be vulnerable to PTSD[14] and may have more severe responses to fear-conditioned cues. PTSD and addiction are very different, but there is a strong link between PTSD and substance abuse. Both share an increased “hyperreactivity to trauma- or drug-related cues.” [15] A recent study in rats found that sign-trackers were vulnerable to fear incubation (i.e., the animal’s fear worsened in the absence of conditioning experiences), whereas goal-trackers had normal fear responses that decreased over time. In addition, ST mice have higher corticosterone levels, which is a marker of vulnerability to drug abuse and is associated with stress and fear.[16] Fear and avoidance contribute to many behavior problems in horses, and an animal’s vulnerability to fear conditioning is an important consideration when exposing it to potential sources of discomfort or stress (e.g., during training or veterinary procedures). For sign-trackers but not goal-trackers, the cues not only predict the aversive event, but may also acquire aversive properties, elicit a strong fear and avoidance responses, and be resistant to extinction. Not much is known about the relationship between ST and fear conditioning, but the same principles that operate in reward learning may apply. How can Sign-tracking be Influenced by the Environment and Training Conditions? Individuals differ in their predisposition to ST behavior, but the specific learning conditions also play a significant role. Even in adult animals, whether ST or GT develops can be modified by changing the environment and training protocol. Decades of laboratory research on ST offers practical information about how to decrease ST behavior in applied animal training. Increasing the delay and distance between the cue and reward reduces sign-tracking Sign-tracking response are more likely to emerge if there is short delay between the cue and reward, and when the cue and reward are physically close or in the same location. Increasing the spatial and temporal separation between the cue and reward leads to a reduction in ST and an increase in GT.[17] Mugging is a common complaint about horses trained with food rewards. Sign-trackers will approach and manipulate any cue associated with food reward, and mugging occurs when a horse approaches and “manipulates” the trainer or, more often, her coat pockets. Mugging may be evidence of ST, which is likely to develop during training if the cue and food are in the same location and if the time delay between them is short. It may be particularly pronounced if the trainer embodies both the cue (holds a clicker in one hand) and food (feeds the horse from the other hand, with treats in a pouch on the person’s waist). In this example the person, the pouch, and the marker combine into a “super-cue” that predicts food. It’s important to recognize that ST is persistent and resistant to extinction (see “Extinction” section below). This means that ST horses will continue to approach and bump the person, even when the she doesn’t have food, because the person has become a motivational magnet. Sign-tracking response are more likely to develop to some cues than others In rats, ST is very common when a moving lever predicts food; animals will approach and manipulate the lever before approaching the food-delivery location. In contrast, when a sound predicts food, rats nearly always show GT and move directly to the food-delivery location. The reasons for this difference are unclear; the different cues are equally effective predictors of food, but only some (the lever) acquire incentive value and can function as a conditioned reinforcer.[18] Using a sound for a cue seems to prevent the development of ST behavior, whereas objects that can be manipulated appear to encourage ST. In training, exactly where clickers fit in isn’t clear; they produce a sound, but are also tangible objects with visual, tactile, and dynamic properties. Most trainers have experienced horses that become focused on and touch (or even lick) the clicker, which suggests that it has acquired incentive value. These ST horses may also be vulnerable to compulsive behavior around the food and clicker, and continue to show a strong motivation for training even when it is no longer pleasurable. At present there is no systematic published research on the development of ST and GT responses with the use of a clicker as the cue. Sign-tracking is resistant to Pavlovian extinction Classically (Pavlovian) conditioned responses to the cue are slower to extinguish in sign-trackers than in goal-trackers.[19] During extinction the cue is repeatedly presented alone without the reward, and the conditioned response to the cue is expected to decrease or disappear. For example, extinction of a clicker-reward association would involve presenting the clicker sound repeatedly, without food, until the animal stops responding to the clicker and no longer expects a treat. Goal-trackers would quickly stop responding to the clicker sound, whereas sign-trackers would continue to respond to the sound. Biased attention to the cue may result in this “persistent and inflexible responding” by sign-trackers during extinction.[20] Resistance to extinction does not seem to occur for cues in instrumental conditioning. [21] An example of a cue (discriminative stimulus) in positive reinforcement training is the verbal command “back”; when the horse steps backward in response to the command, it gets a treat reward. In positive reinforcement training, extinction of the verbal command would be similar for sign-trackers and goal-trackers. Studies show that sign-trackers are also slower to discriminate between reward and no-reward situations, and seem to be attracted to the cue regardless of whether or not it predicts a reward in that situation. For example, a food dish (cue) may predict that the horse will get grain (reward) only when it is in the stall, but not when it is in the barn aisle. Sign-trackers will continue to approach the food dish regardless of its location and whether or not food is available, and are slower than goal-trackers to learn that the dish only predicts food in certain contexts. Pre-exposure to the cue reduces sign-tracking If the animal is familiar with the cue, for example by repeatedly presenting it alone, before it has ever been associated with a reward, ST is inhibited (this is known as latent inhibition). Prior exposure to the cue does not affect the emergence of GT responses directed at the food-delivery location.[22] In clicker training, presenting the clicker sound repeatedly before linking it to a food reward should reduce the incidence of ST and increase GT. If the clicker is introduced and only used during training with food rewards, then ST is more likely to develop. Uncertainty between the cue and reward increases sign-tracking The emergence of ST response is high if the reward follows the cue only some of the time, but only when training begins with partial reinforcement. When training begins with continuous reinforcement and then later adjusts to a partial reinforcement schedule, the incidence ST is low and does not increase. Uncertainty usually refers to training with partial reinforcement, and in most research studies the reward follows the cue half of the time. Certainty refers to continuous reinforcement, and the reward always follows the cue. The effect of uncertainty on the development of ST responses can be dramatic; in one study, when rats were trained with partial reinforcement the proportion of sign-trackers was 90% and goal-trackers was 5%. When rats were trained with continuous reinforcement the proportion of sign-trackers was only 50% and goal-trackers increased to 35%.[23] Uncertainty also refers to variability in the value of the reward, for example if the cue is sometimes followed by a small, low value reward, and at other times by a large, high value reward. ST responses increase with uncertainty, especially when it combines uncertainty in both partial reinforcement and reward value.[24] Environmental enrichment reduces sign-tracking The overwhelming evidence is that environmental enrichment results in a lower incidence of ST.[25] How enrichment interacts with other factors is more complex. For example, in one study, adolescent rats showed less ST behavior than adults, but were more sensitive to food restriction and social isolation. When adolescent rats were placed on a food restricted schedule, ST was higher in socially isolated rats, and GT was higher in pair-housed rats.[26] More research is needed to clarify the complex interactions between enrichment, age, and ST. Take home message Food is commonly used in positive reinforcement training and in desensitization and counter-conditioning behavior modification. An understanding of ST is valuable for animal trainers, particularly those who use clicker-training, because the clicker is a cue specifically introduced to predict a food reward. The goals of this article are to raise awareness of ST, the potential problems linked to ST—including compulsive behavior and resistance to extinction—and suggest ways to reduce ST in order to improve training outcomes and animal welfare. Another goal of this article is to encourage applied research on ST in horses. As a general rule, it’s prudent to be cautious about extrapolating findings from laboratory studies to applied animal training. With respect to ST, many laboratory experiments are set-up to encourage a high incidence of ST and may not represent real-life situations that concern animal trainers and behaviorists. What is needed in horses is a systematic description of ST behavior, identification of contexts that affect the emergence of ST, and recognition of individuals that are at risk for ST responses. References [1] Hearst, E., & Jenkins, H. M. (1974). Sign-tracking: The stimulus–reinforcer relation and directed action. Austin, TX: The Psychonomic Society. As cited in Burns, M., and Domjan, M. (1996) Sign tracking versus goal tracking in the sexual conditioning of male Japanese quail (Coturnix japonica ). Journal of Experimental Psychology: Animal Behavior Processes, Vol 22(3), 297-306). 2 Robinson, T.E., 2009. Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences. Biological Psychiatry 65(10) 869-873. [3] Beckmann, J.S., and Chow, J.J. (2014) Isolating the incentive salience of reward-associated stimuli: value, choice, and persistence. Learning & Memory, 116-127. 4 Boakes, R. A. (1977). Performance on learning to associate a stimulus with positive reinforcement. In H.Davis & H. M. B.Hurwitz (Eds.), Operant-Pavlovian interactions (pp. 67–97). Hillsdale, NJ: Erlbaum. As cited in Burns, M., and Domjan, M. (1996) Sign tracking versus goal tracking in the sexual conditioning of male Japanese quail (Coturnix japonica ). Journal of Experimental Psychology: Animal Behavior Processes, Vol 22(3), 297-306. [5] Nilsson, J., Kristiansen, T.S., Fosseidengen, J.E., Ferno, A, and van den Bos, R. (2008) Sign- and goal-tracking in Atlantic cod (Gadus morhua). Animal Cognition 11(4), 651-659. [6] Anderson, R.I., Bush, P.C., and Spear, L.P. (2013) Environmental manipulations alter age differences in attribution of incentive salience to reward-paired cues. Behavioural Brain Research 257, 83-89 [7] Flagel, S.B., Akil, H., and Robinson, T.E. (2009). Individual differences in the attribution of incentive salience to reward-related cues: Implications for addiction. Neuropharmacology 56 (S1), 139-148. [8] Flagel, S.B., Watson, S.J., Akil, H., and Robinson, T.E. (2008) Individual differences in the attribution of incentive salience to a reward-related cue: Influence on cocaine sensitization. Behavioural Brain Research 186(1) 48-56. [9]Tunstall, B.J. and Kearns, D.N. (2015) Sign-tracking predicts increased choice of cocaine over food in rats. Behavioural Brain Research 281, 222-228. [10] Flagel, S.B., Watson, S.J., Akil, H., and Robinson, T.E. (2008) Individual differences in the attribution of incentive salience to a reward-related cue: Influence on cocaine sensitization. Behavioural Brain Research 186(1) 48-56. [11] Litt, A., Khan, U., and Shiv, B. (2010) Lusting while loathing: Parallel counterdriving of wanting and liking. Psychological Science 21(1), 118-125. [12] Fleming, A. (2015) The science of craving. The Economist: The Intelligent Life (May/June) http://www.intelligentlifemagazine.com/content/features/wanting-versus-liking?https%3A%2F%2Fwww.socialflow.com%2Fpublish#?fsrc=scn/fb/te/bl/ed/100socialflow=science+of+craving [13] Fleming, A. (2015) The science of craving. The Economist: The Intelligent Life (May/June) http://www.intelligentlifemagazine.com/content/features/wanting-versus-liking?https%3A%2F%2Fwww.socialflow.com%2Fpublish#?fsrc=scn/fb/te/bl/ed/100socialflow=science+of+craving [14] Morrow, JD, Saunders, BT, Maren, S, and Robinson, TE. 2015. Sign tracking to an appetitive cue predicts incubation of conditioned fear in rats. Behavioural Brain Research 276, 59-66. [15] Morrow, JD, Saunders, BT, Maren, S, and Robinson, TE. 2015. Sign tracking to an appetitive cue predicts incubation of conditioned fear in rats. Behavioural Brain Research 276, 59-66. [16] Tomie, A., Lincks, M., Nadarajah, S.D., and Pohorecky, L.A., (2012). Pairings of lever and food induce Pavlovian conditioned approach of sign-tracking and goal-tracking in C57BL/6 mice. Behavioural Brain Research 226(2), 571-578. [17] Silva, F.J., Silva, K.M., and Pear, J.J. (1992) Sign- versus goal-tracking: Effects of conditioned-stimulus-to-unconditioned-stimulus distance. Journal of the Experimental Analysis of Behavior 57(1), 17-31. [18] Beckmann, J.S., and Chow, J.J. (2014) Isolating the incentive salience of reward-associated stimuli: value, choice, and persistence. Learning & Memory, 116-127. [19] Ahrens, A.M., Singer, B.F, Fitzpatrick, C.J., Morow, J.D., and Robinson, T.E., (2015) Rats that sign-track are resistant to Pavlovian but not instrumental extinction. Behavioural Brain Research July 30, 2015 [20] Ahrens, A.M., Singer, B.F., Fitzpatrick, C.J., Morrow, J.D, and Robinson, T.E. (2015) Rats that sign-track are resistant to Pavlovian but not to instrumental extinction. Behavioural Brain Research, July 30 (update ref) [21] Ahrens, A.M., Singer, B.F., Fitzpatrick, C.J., Morrow, J.D, and Robinson, T.E. (2015) Rats that sign-track are resistant to Pavlovian but not to instrumental extinction. Behavioural Brain Research, July 30 (update ref) [22] Boughner, R.L., and Papini, M.R., (2003) Appetitive latent inhibition in rats: Now you see it (sign tracking), now you don't (goal tracking). Learning & Behavior 31(4), 387-392. [23] Robinson, M.J.F., Anselme, P., Suchomel, K., and Berridge, K.C. (2015) Amphetamine-induced sensitization and reward uncertainty similarly enhance incentive salience for conditioned cues. Behavioral Neuroscience 129(4), 502-511 [24] Anselme, P., Robinson, M.J.F., and Berridge, K.C., (2013) Reward uncertainty enhances incentive salience attribution as sign-tracking. Behavioural Brain Research 238, 53-261 [25] Beckmann, J.S. and Bardo, M.T. (2012) Environmental enrichment reduces attribution of incentive salience to a food-associated stimulus. Behavioural Brain Research 226, 331—334. [26] Anderson, R.I., Bush, P.C., and Spear, L.P. (2013) Environmental manipulations alter age differences in attribution of incentive salience to reward-paired cues. Behavioural Brain Research 257, 83-89. Comments are closed.
|
