The article below is an (automatic) translation of the original Dutch article that can be read here.

Automatic speech recognition (ASR) is attracting increasing interest from the medical field and may also mean something for pharmacists. A survey has shown that pharmacists are particularly interested in its use in medication review meetings to reduce the administrative workload. However, most are still taking a wait-and-see attitude.

PW36 - 09-09-2022 | by John Davelaar, Berrie van der Molen, and Toine Pieters

Pharmacists have a wide and diverse range of tasks, which can differ enormously from those of other pharmacists, depending on the sector in which they work. However, one element is often the same: time pressure. New systems are regularly introduced to make work faster and more efficient, such as shortcut keys and user-friendly interfaces. Automatic speech recognition (ASR) may eventually be added to this.

We talked to ten pharmacists from different sectors (pharmaceutical industry, community pharmacy, hospital pharmacy and the research world) to see how they think ASR could be implemented in daily practice, what advantages and disadvantages are involved and what possible objections are foreseen.

ASR enables the conversion of spoken text into written, searchable text. The technology must be able to recognise the language spoken and be able to cope with different acoustic conditions [1]. An everyday example of this is the virtual smartphone assistant that you can ask questions to. The digital assistant converts spoken text, recognises your question and then searches for an answer. Even in some pharmacies, basic applications of speech recognition are already in place, for example when patients call the pharmacy, they first hear a menu with options and are connected to the right employee based on what they say. For application in specialist areas, such as healthcare, there are additional challenges: recognition of complex medical terms and specific acoustic settings such as consulting rooms and counter discussions.
Although the application of ASR in healthcare is not without its challenges, some doctors are already using it, for example during patient record-keeping, in order to reduce the administrative workload. In addition, the privacy sensitivity of medical calls must be taken into account. The HoMed research project is working on ASR of doctor-patient and pharmacist-patient conversations. We do this by using archived sound recordings of the Netherlands Institute for Health Research (Nivel).

Upon completion of the project, HoMed's ASR infrastructure will be made freely available for further applications in the Dutch healthcare domain. Reason enough to anticipate what ASR could mean for pharmaceutical care. In the questionnaire and interviews, we asked pharmacists how ASR could support pharmaceutical care and what the advantages and disadvantages might be. Our interviews revealed that pharmacists can mainly envision using ASR when conducting medication reviews. The advantage of this is that pharmacists can then concentrate fully on what the patient is saying and not on the typing, which means that fewer things are overlooked. Because the system eliminates the need for typing, the administrative workload may be reduced. "You are now talking and typing, but that does not always run smoothly," says a pharmacist. A disadvantage mentioned is that often only a small part of the medication review is relevant, while the programme records everything.
A pharmacist remarked: "I don't want to have to read through chunks of text to see what is important and what is not; too much is too much. This shows that there is not only a need for recognition of the spoken text, but also for more advanced applications such as recognition of terms, names or specific information in a conversation. In addition, ASR can be used to make many hours of conversation recordings searchable using keywords. As a result, important medical issues, such as the recognition of medication abuse and therapy non-adherence, can be investigated [2]. This can lead to pharmacists recognising such behaviour earlier and thus being able to respond in a timely and appropriate manner.

Possible objections

There is some reticence among pharmacists regarding the positive predictive ability of automatic speech recognition. As indicated earlier, they foresee problems with the conversion of spoken words to text, as a result of which pharmacists expect to have to check the noted text for errors, which would take time. They expect that especially complex drug names will lead to problems with speech recognition.

In the HoMed research project, this is one of the challenges we are currently working on: how can you ensure that the automatic speech recognition system is able to recognise as many pharmaceutical and medical terms as possible, especially as these are often pronounced in different ways? For this we make grateful use of lists of terms that have been made available to HoMed by the Medicines Evaluation Board (CBG), Health Base Foundation, the International Council for Harmonisation of Technical Requirements Registration Pharmaceuticals Human Use (ICH), Institute for the Dutch Language (INT), Institute for Responsible Medicine Use (IVM), KNMP, NHG, Nictiz and the Netherlands Care Institute (Zorginstituut Nederland).

There are also concerns that patients will be less free to speak out about embarrassing events and side effects if conversations are recorded for ASR purposes. The privacy challenges are part of HoMed, as this also concerns the practice material used to test the infrastructure, a necessary stage in the development of ASR for specific domains. The speech recogniser must not only be able to recognise medical text, but also be delivered with a method for privacy assurance.

The fact that several pharmacists are often in one room is mentioned as a potential problem for the effective deployment of ASR; are those acoustic conditions optimal? One pharmacist is concerned: "Are we all going to have to wear headphones so that the sound is well-captured and we are less affected by background noise? It's like we're going to become a call centre!"

The research within HoMed, in which we also investigate pharmacy conversations, will have to reveal this. The costs of the materials and the software itself are mentioned: "What kind of price tag will this carry?" Will these costs yield a profit below the line, pharmacists wonder. Many of the objections mentioned already have HoMed's attention, so solutions may come more quickly.

Food for thought

The tour of the pharmaceutical field for this article is an initial exploration of the possibilities and challenges of automatic speech recognition in pharmaceutical care. Pharmacists see the added value, but there are also reservations about its effectiveness. The aforementioned objections are currently being investigated in the development of ASR for the medical domain within HoMed, which is encouraging.

We expect to make the tool available through the Open Speech Technology Foundation in late 2023. Pharmacists who have their own ideas about specific applications in pharmaceutical practice can contact us.

John Davelaar is an honours student in pharmacy. Berrie van der Molen is a researcher at the Freudenthal Institute. Toine Pieters is professor at the Discipline Group Pharmacoepidemiology and Clinical Pharmacology, Department of Pharmaceutical Sciences at Utrecht University.

Literature

1. Malik M, Malik MK, Mehmood K, Makhdoom Automatic speech recognition: a survey. Multimed Tools Appl. 2021;80(6):9411–57.

2. Tejedor-García C, van der Molen B, van den Heuvel H, van Hessen A, Pieters Towards an Open-Source Dutch Speech Recognition System for the Healthcare Domain. In: Proceedings of the 13th Conference on Language Resources and Evaluation. Marseille: ELRA; 2022. p. 1032–9.