The Signal & The Noise · Vol. 10
A doctor can only say no to an AI system's proposal if they could also judge the case themselves. A study from Poland suggests that ability can slip once a system is in use. The job of overseeing AI is now written into frameworks in Abu Dhabi and in Europe, without a profession behind it.
This research has been sitting in my drawer since August. I knew it would become a piece one day, just not when. This week it found its place, through something that came out of Abu Dhabi in October.
On 21 October the Department of Health in Abu Dhabi published a framework for what it calls AI clinician systems: systems that could in time carry out tasks that today require a licensed doctor. It is meant to guide research into them. According to the reports, it requires safeguards at every level, including oversight and accountability by the physician and referral to a physician when needed.
I think that is a sensible order: first settle who oversees, and only then give the system more freedom. There is one question in it that you only see once you look a little longer. A doctor can only say no to a proposal from the system if they could also assess the case themselves, and whether they still can in a year depends on what they do in the meantime. That is what this piece is about. It applies to any country that uses human oversight as its safety net, and so not only to the Emirates.
At Samture we design the foundation beneath AI: what a system knows, where that comes from and who signs for what. In the Excellence Centre we train the people who build it, and they build for themselves from the first week. My answer to this question is already in there, a little.
At four endoscopy centres in Poland an AI system was introduced that watches along during a colonoscopy and points out polyps. Before it arrived, the doctors found adenomas, polyps that can grow into cancer, in 28.4% of their examinations without the system. Once the system had been in use for a while, the figure for examinations they again carried out without it was 22.4%. It covered 795 examinations before and 648 after. The researchers published this in 2025 in The Lancet Gastroenterology & Hepatology.
What strikes me is that the fall sits in the examinations where the system was not involved.
This needs careful reading, and the researchers say so themselves. It is a retrospective study and not a trial, they write that continuous use of AI might lower detection in ordinary examinations, and other things may have changed in that period. It says something about these centres in this period and not about every doctor everywhere. What it does show, in a real hospital, is something we know from other trades. A skill is something you maintain, and it wears down when you do not use it. That is true of flying, of surgery and of a second language.
What we measure, we mostly measure on the machine: how accurate it is, how it scores on benchmarks, how often it makes something up. That is important work. This study looked at something else, namely what happens to the person who works with the machine, and far less attention goes there.
A framework like Abu Dhabi's sets down that there must be oversight and that the doctor remains responsible. Europe does something similar for high-risk AI: human oversight is part of the requirements, and in July the start dates moved to 2 December 2027 for stand-alone systems and 2 August 2028 for AI in products that are already regulated.
Both set down that a person oversees. What that person needs to be able to do, and how they show they still can, is a different question. For a doctor there is a profession around it, with training, registration and continuing education, and the study from Poland suggests that even then judgement can shift. For the colleague who has to assess a system's proposals at a bank, in a call centre or in a factory, that profession does not exist.
So a function has been invented, the person who oversees, without a profession behind it. You do not fill a function like that by hiring someone. A course on the first day helps only for a while too, because the skill has to be kept up.
In the Emirates the doctor's oversight is being set down before an AI clinician system may do anything on its own, and that is a good basis. What I did not find in the reports is how the doctors themselves stay in shape once the system is in place. Perhaps it is in the framework itself, which I know only through the reports, so I leave it here as a question.
At a bank it becomes concrete. Whoever signs as second reader on a decision by the system has to be able to see something the first reader misses, and that is only possible if they could also assess the case themselves.
Look this week at one number your organisation probably already has: how often the people in your process reject a proposal from the system, compared with a year ago. If that percentage falls, the system may have improved. It may also be that the reviewer is reading less carefully. The number itself does not tell you which of the two it is. If you want to know, have someone who normally works with the system assess one case a month without it, and put the results side by side.
Curious what this could look like in your own practice? A first impression is at samture.com/solutions, for banks at samture.com/banking, and if you would like to talk it over I am happy to set aside half an hour.
Abu Dhabi is setting down that the doctor remains responsible for what a system may one day do. I hope that someone is also writing down, in the meantime, how we keep in shape the people who sit beside it.
A foundation where what a system knows, where that comes from and who signs for what are written down, so that the people who oversee it have something to check against. See how it works, for banks and for other regulated organisations.
See the solutionsThe Signal & The Noise
A fortnightly read on what is actually changing in AI across the UAE and the Gulf, written personally. No content calendar, no generated filler.