9.26 Behavior - Reward, Motivation, Addiction

Physician's Notebooks 9 - http://physiciansnotebook.blogspot.com - See Homepage


26. Behavior  -  Reward, Motivation, Addiction -  Update 15 Septr 2021

Important headings in order of appearance are in the below column. Use search & find  or scroll down to them.

Motivation

William James

Ivan Pavlov's discovery of the conditioned reflex

classical conditioning

Thomas Watson in the 1920's discovered operant conditioning

 key experiments in animal behavior

Direct and indirect electrical self stimulation of the brain

Motivation is determined by the state of an animal's need for something, e.g., the state of an animal's hunger determines his seeking out of food and eating it; or the availability of a sexy woman produces a state that strengthens or weakens the man's behavior toward obtaining the sex. Motivation based on an internal body state makes much basic behavior. As Yogi Berra was wont to say: One strongly seeks sleep when one is sleepy. But for humans many goal-seeking behaviors are complex. For example, one may not be hungry at all - may have just eaten - but still be motivated to seek an ice cream cone to lick because one saw it in an advertisement and recalls now how good it tasted. 

The great, late 19th Century American psychologist William James first got the idea that we are strongly affected by the feedback from our behavior. The idea got reinforced by Russian scientist Ivan Pavlov's discovery of the conditioned reflex. When Pavlov's hungry dogs were presented with food, the initial sight or smell of the food started them salivating and, as soon as the food was at their mouths, it got them eating. Pavlov called the food the unconditioned stimulus and he used salivation as the response to the food stimulus. Behavioral scientists today call the examples of the salivation at the sight of food, appetitive behavior, and the actual eating, consummatory behavior.  Pavlov wondered: If I associate a neutral effect like a bell ring, with the presentation of food, could it take the place of the food in causing the dog to salivate? And after several pairings of the food with a bell ring in front of the hungry dog, the bell ring alone could start the dog to salivate just as though it were the food. The effecting bell (or any other paired, originally neutral stimulus) was called the conditioned stimulus. (Cf. the food's being the unconditioned stimulus) This experiment developed into the theory of incentive-motivation, which notes that an object (food) becomes an incentive to an animal whose drive (hunger, food deprivation or imaged desire for a tasty type of food) is strong. The incentive and the animal's drive each cwork together to produce a motivational state in the animal. The motivation then produces appetitive behavior (Salivating, searching for the food) in the animal and the at the mouth-nearness of the incentive object (the food) leads to consummatory behavior (eating). One can generalize from an eating food experiment example to all kinds of motivation (A man's sexual desire for an object, etc.)

Note that Pavlov's so-called classical conditioning is feedback. In the dog's case, the hungry animal sees food being presented and without even thinking (involuntarily) starts salivating. The neural input of the sight and smell of food quickly gets associated with a simultaneous come-and-get-it bell sound, and the fact that the bell can replace the food so immediately shows that the salivation response behavior is not a thought-out decision but rather the result of the almost immediate neural impulse of the sound (or the smell and sight of the food) signal running from the ear (or nose or eyes) into the brain and ticking off the remembered motor effect that leads to salivation. It is essentially a knee jerk-type reflex, also referred to as zombie behavior when it involves the brain.

       Now let us get to some very practical observations that I recently made during my hospitalization for fractured pelvis and hip replacement surgery. I noted that it was very easy to become conditioned by the pain and discomfort of trying to walk as my pelvic fracture was healing and as I was recovering from the hip surgery. Initially, I had lots of pain when I tried to walk so I correctly did not attempt to walk. That was protective. However, as my fractures healed further and as my surgery had its good effect, I also noted that I had become so conditioned to the expected pain of trying to walk that I was afraid to even attempt to walk. So what had started as a protective conditioning was ending as a response that blocked my recovery. And as I became aware of this through my own self psychoanalysis (cognitive consideration), I purposely, worked on removing the negative conditioning. Of course, I checked with my surgeon that I was not harming myself by walking too soon. But once he assured me, I discovered that this new change in attitude greatly assisted my recovery. 
  Another example I discovered of practical observation of the behaviorism had to do with my pain medication. I was taking a pain medication that had a bitter taste under the tongue. In the beginning this medication took about 15 minutes to have its good effect, but, after a couple of days of use I discovered that it was starting to work almost at once, i.e., immediately I detected the bitter taste my pain left me. This seems to me, clearly, a conditioned response, i.e., I had become conditioned to the bitter taste always being followed by relief of pain, which normally occurred 15 or so minutes after taking the pill, but, due to the conditioning, soon began to be immediately experienced. 
You may go from these two examples and figure out how useful an understanding of behavior and conditioning can be in a practical sense for you. These are examples of feedback, or bottom-up conditioning. In the opposite direction feed-forward, or top-bottom, is seen in my observations of crazy-appetite eating behavior which sometimes afflicts me in bed at night even after I’ve had a big supper, when I start thinking about apple pie a la mode that I love to top off a big meal with, so much so, that it forces me out of bed into a cold night walk to an all-night Lawson’s to buy the pie and devour it (Top-down because the thought from my cerebral cortex continues down to my spinal cord activating muscles that get me up, buying).

Thomas Watson in the 1920s discovered operant conditioning. In it, a preceding behavior is either immediately reinforced by a reward (positive reinforcement) or inhibited by withholding an expected reward  (negative reinforcement). For example, a Pavlovian trained dog that salivates on the ringing of a bell goes into a training mode where the conditioned stimulus (the bell ringing) is always followed by the withholding of the unconditioned stimulus (the food) and, oppositely, the timed absence of the bell results in the reward of the food. In this case, the Pavlovian conditioned stimulus (the bell ringing), becomes, opposite to the original conditioning, associated with a high probability (in the animal's mind based on the manipulation of the experiment) of a negative outcome (No food), so the conditioned stimulus loses its power and the dog gets unconditioned and stops salivating when the bell rings and avoids places where the bell is ringing because he has learned it means a high probability of no-food. Note that operant conditioning involves memory that extends over a time period, it is not a knee-jerk reaction. We call this type of conditioning "top-down" because it's based on the mind interfering with the direct experience. It contrasts with the "bottom-up" Pavlovian conditioning which is purely  knee-jerk and does not involve mind. This gets to the question of how the brain produces (also enhances or inhibits) motivation or even more elementary, what stimulates us towards actions during a life. The Pavlovian reward is stimulated from a biological need like food, which satisfies a basic urge (chemically mediated by low blood sugar, or hypoglycemia; and is a much stronger factor in motivating lower animals than in humans, who are more motivated by operant conditioning.
  In key experiments in animal behavior, a rat has a catheter attached into its spinal cord to deliver a cocaine-high jolt (the reinforcing reward) and the rat controls each dose of delivery of the cocaine by pressing the lever, which it learns to do quickly by the experimenter demonstrating to the rat that a press of the lever delivers a milligram dose of the cocaine into the rat and gives a cocaine high. In the series of experiments, the dose of the cocaine delivered by a lever-press is varied by increasing or decreasing the dose of cocaine, and the rate of the rat's lever-pressing to get his dose is recorded. It was discovered, first, that the rat does not become tolerant to the cocaine, as drug addicts do to opioids, i.e., a rat's responses do not decrease over the period of the drug taking. Second, it was found that as the dose of the delivered drug is reduced, a lower threshold is reached, shown by the rat's stopping pressing the lever to get a drug-high below a certain dose, which is interpreted as a sign that the rat is no longer getting the high. Above this threshold, the rat will exactly control his lever-pressing so that regardless of the amount of the delivered dose, the rate of delivery of the cocaine over a unit time (24 hours) equalizes, i.e., if the dose is doubled, the rate of lever-pushing is halved. Thus, under natural conditions of cocaine self administration the drug does not tend to be abused in rats. Finally, and most important, an experiment where an inhibitor of the neurotransmitter, dopamine, was mixed with the cocaine upped the threshold for lever-pushing and at specific supra-threshold doses increased the rate of lever-pushing. This shows that dopamine is the neurotransmitter that is important in getting the pleasure from cocaine (and presumably many drugs of addiction).

The effects shown are the result of release of the neurotransmitter dopamine (Because, in a related experiment, the dopamine antagonist blunted the effect). This suggests that neurotransmitters like dopamine are important in animal (human too) habit formation behavior.
    Direct and indirect electrical self stimulation of the brain (Deep Brain Stimulation or DBS) to give experimentally produced pleasure or to treat intractable incessant pain or other mental conditions has given insight into the molecular chemical sources of motivation. Especially the discovery that a wire implanted in the brain's subcortical basal ganglia involved in reward motivation produces a delightfully pleasurable feeling very like a person that is not tolerant to an opioid feels on receiving an opium injection. Experimenting with a rat, using the brain wired , got the same results as with cocaine. Furthermore, pre-treatment with dexedrine or cocaine (both stimulants that are known to work by internal dopamine nerve ending release) markedly lowered the threshold for stimulation (Less electric current for the lower threshold). This confirms that an animal's (human's too) internal neural reward system (that makes one behave positively or prevents from behaving that way when blocked) is based on neurotransmitter release or inhibition.  Dopamine is a main neurotransmitter but also serotonin has been shown to give positive reinforcement by internal pleasure. This explains the current use of serotonin-enhancing anti depression drugs’ good effect against pain.
   I hope this gives a look at the emerging neuroscience of behavior which is showing that we behave in ways derived from calculating probabilities of reward or non reward outcomes based on our memories feeding forward to strengthen our decisions. It is a complex area that those interested in should continue to track from here starting by reading Fundamental Neuroscience, 4th ed. and going from there. 
  Comment on my behavioral observations. “Cognitive considerations” are what you are thinking as you decide on behaviors; behavior is what you end up doing. Too often in pursuing our decisions we end up like the knee-jerk preparation, or a zombie, mindlessly going through rituals of conditioned bad behavior that we could have prevented by a little self-psychoanalysis.
   Especially once we have come to a preliminary decision like “I am going out to buy curry rice for suppers, we should say, “Au contraire!” and keep open to change, up till the last moment, keep flexible, rethinking the bases of our decisions (good health, economy, convenience). This can be generalized from the most trivial to the most important. The slang is: hang loose. Don’t go into lockdown mode based on previous decisions. It is a secret of my happy survival to age 88.
End of Chapter. To read next Sociobiology----How a billion years of evolution has shaped your deepest desires and  9.27 Manic Depressive Bipolar Disorder - Good Adv...