I once cared for a patient for 10 years, which is a pleasingly long time in oncology. Alas, the years didn’t bond us. I found her, in turns, combative and annoying, and I confess she probably found me the same. Before each encounter, I would take a deep breath and talk myself into greeting her with an ease I never felt.

She was my “heart-sink” patient. When she didn’t show up, I worried, but when she did, my stomach tightened. My “surface feeling” was impatience, but inside, I felt terrible that any patient should arouse such antipathy in a member of the “caring profession”. When she was finally discharged in good health, we were both relieved for different reasons.

I found myself thinking about this when reading an illuminating study about what makes “difficult” patients difficult. (I do love that doctors choose to study such things.)

The authors interrogated studies from different countries and different non-psychiatric settings including primary care, emergency, inpatient and specialty clinics. Twenty-eight of the 45 studies used a reliable instrument called the Difficult Doctor-Patient Relationship Questionnaire, a self-administered tool completed by the doctor to identify patients whose care was experienced as difficult.

Questions include:

How time consuming is caring for this patient?

To what extent are you frustrated by this patient’s vague complaints?

Do you find yourself secretly hoping this patient will not return?

How difficult is it to communicate with this patient?

How manipulative is this patient?

How much are you looking forward to this patient’s next visit after seeing this patient today?

The authors found strong evidence that physicians rate one in six patients as “difficult”. Mostly, these patients have personality disorders, depression, anxiety and chronic pain. Patients considered difficult reported more symptom burden and worse function. Finally, as in every walk of life so in medicine: women were somewhat more likely to be perceived as difficult patients although this perception was not supported by the evidence.

Not surprisingly, patients experienced as difficult reported more unmet expectations and were less satisfied with their healthcare, although there was insufficient evidence they in fact received inferior healthcare.

When it comes to physicians, the characteristic with the strongest evidence for perceived difficulty was the level of experience. Doctors in training were more likely to rate patients as difficult than veteran physicians. Those with higher measured empathy were also less likely to rate patients as difficult. Burnout went both ways: doctors with burnout were more likely to rate their patients as difficult and doctors who reported a high prevalence of difficult patients were more likely to report burnout. Tellingly, those with higher job satisfaction reported fewer challenging patients, but the lower the job satisfaction the higher the prevalence of such patients.

It is difficult to tease out the many contributors to what makes a patient difficult, but the study is commendable for providing food for thought.

It is not the number of medical conditions but the type of problem that tests doctors. Most doctors are not trained to manage mental health and personality disorders in an ongoing, meaningful way. Short appointments, competing demands and increasing administrative tasks don’t help.

“Difficult” patients force us to depart from the familiar, intellectually satisfying script of matching symptom to diagnosis and treatment to cure. Hence, they can make us uncomfortable.

For instance, from a strictly medical angle, my patient was “fine”. Despite her high-risk disease, here she was 10 years later free of cancer. In this time, her children had grown up and given her grandchildren, something many of my patients will never get to see.

Then, what made her difficult? Her chronic complaints about the hospital and her bad mood didn’t help. I had no solution for her unemployment and unstable housing. I had no way of managing her very real dissatisfactions apart from writing copious petitions on her behalf which were in vain.

I would have appreciated just the occasional acknowledgment that modern medicine had cured her and a battalion of doctors and nurses continued to invest substantial time and effort into her advocacy. This small gesture could have replenished my diminishing sympathies, although I appreciate the irony of expecting a vulnerable patient to make a doctor feel better.

But to be fair to dissatisfied patients, there are times where doctors fail them in ways they ought not. Chronic pain is a good example of how patients are labelled difficult when the tests are normal, but the person doesn’t feel right. When doctors can provide neither symptom relief nor neat explanation and either say or imply that “it must be in your head”, patients rightly feel perplexed and disappointed.

To prevent difficult medical problems from being redefined as difficult patients, doctors need help. We need better access to long-term mental health, allied health and social work providers. Communication skills training for doctors (and everyone else in healthcare) remains an optional extra – when patient welfare is at stake, it shouldn’t be.

Finally, the study raises a rather poignant question. The timeless answer to why we become doctors is “to help people”. If this is true, we can’t be content with helping the five in six “easy” patients who are so because they comply with our advice and don’t complain.

We must engage with the one in six people who are dissatisfied and demanding, finding ways to understand them and navigate their difficulties. In this era of automation, the doctors who will stand out are those willing to bear witness to human suffering despite the challenges.

• Ranjana Srivastava is an Australian oncologist, award-winning author and Fulbright scholar. Her latest book is called Every Word Matters: Writing to Engage the Public