New Operations in Speech

Miami Children's Hospital, a world leader in pediatric health care,
has a medical staff of more than 650 physicians and over 2,000
employees. The hospital specializes in all aspects of pediatric
medical care from birth through adolescence.  Although the hospital
draws children with very specific needs from all over the world, it is
also the only licensed specialty hospital exclusively for children in
South Florida.

Originally opened in 1950 as Variety Children's Hospital, its name was
changed to Miami Children's Hospital in 1986. MCH now treats more than
185,000 patients each year. It is a non-profit, freestanding hospital.
Today, MCH is in the midst of a renovation project that will enlarge
the hospital by 78,000 square feet, making it one of the most
attractive health care facilities in South Florida.

With all that is going on at the hospital, MCH is also in the process
of incorporating speech recognition technologies into their existing
i-Rounds clinical documentation application from Teges Corporation.
Teges Corporation visited IBM in November of 2004 and saw a
demonstration of the X+V programming language. The demonstration used
IBM's WebSphere speech technology to speech-enable Web pages.  Teges
realized that IBM's speech technology could be used in their i-Rounds
application.  In January of 2005, Teges teamed with IBM to create a
speech-enabled prototype of the i-Rounds application.  The entire
i-Rounds application was speech-enabled for navigation and other
functional components, such as an Operating Room (OR) timer and a
Cardiac Intensive Care Unit (CICU) voice recording application, within
30 days. The resulting application can be used with traditional
keyboard, mouse and display as well as with speech input and output.

It was MCH's need for hands-free and eyes-free access to computing
that would help clinicians access or record critical information
without having direct contact with equipment that led to their
implementation of the multimodal i-Rounds. There are many areas within
the hospital where clinicians do not have access to computers or
hardware because the environment is sterile or the clinicians' hands
are busy with surgery or other tasks.  These are the areas that MCH
wanted to focus on with speech technologies. Their goals with this
project included hands-free access to all critical patient information
from anywhere with any device, increased efficiency and speed with
which clinicians access and create patient information (including
voice recordings) within the hospital, an interactive environment for
users to communicate with the computer, text-to-speech (TTS)
capability of the voice application to warn users of dangerous
trends/events and TTS capability to remind users not to forget key
elements of the medical treatment.

The speech application's ability to meet these goals would depend
heavily on its ability to adapt to dynamic and noisy environments.
MCH found ways to minimize ambient noise and increase recognition
accuracy by finding, positioning and outfitting MCH staff and rooms
with the appropriate microphone technology.  The application had to be
easy to use so clinical staff would quickly adopt the new technology.
In the end, IBM's WebSphere Everyplace Multimodal Environment allowed
the Hospital to design new features as well as enhance the existing
i-Rounds application to become multimodal. The underlying technology
for doing this was a Web-based programming language called XHTML+Voice
(X+V). Based on Web standards, X+V sped the addition of speech to
i-Rounds and it used application development skills to which Teges
developers were accustomed. Also, since the speech technology is user
independent, the medical staff was not required to do any speaker
training to use the application.

The speech-enabled application underwent a litany of tests both for
endurance and accuracy before it was used in a production environment.
This included IBM's testing procedures such as usability, automation,
lifecycle testing, and audio analysis. The browser with which the
voice application is delivered has a series of debugging tools,  such
as logging, tracing and audio capturing, that the IBM team used to
reproduce the exact environment and interaction the user had with the
computer. The OR, though an uncommon environment, shares many similar
characteristics of other environments that IBM has speech-enabled,
such as automobiles.  For example, heart monitors provide constant
noise to the "always listening" speech system.  This situation is
similar to constant noises in a car. Adjusting variables in the
multimodal system such as microphone type, placement and speech engine
settings enabled the system to be tuned to the operating room
environment. The debugging functionality available in the system
proved critical to the success of the project.

The speech features of i-Rounds have been enabled in the Cardiac
Intensive Care Unit (CICU) and in the OR for use during pediatric
cardiac surgery. They provide a hands-free mechanism for physicians to
enter information, retrieve information and record their voice
directly into the patient's medical record making the doctor's
assessment/diagnosis available immediately without waiting for
transcription services.  In the OR, the computer speaks to the surgeon
through four speakers that are embedded in the ceiling.  The surgeon
interacts with the computer using a cardioid wireless microphone,
which activates the speech system when the surgeon utters the keyword,
"Computer."  In the CICU, clinicians access the system by using a
wireless tablet PC.  They use the tablet's built-in microphone and
activate the speech system using a push-to-talk mechanism (depressing
a button on the tablet).

When MCH started testing the speech recognition in different areas of
the facility, they encountered three major hurdles.  First, ambient
noises, such as beeping oxygen saturation monitors, drills, and saws
in the OR and bedside monitors, alarms, families, patients and staff
making noise in the CICU, created a challenge for the speech
recognition system. MCH researched microphones, both wireless and hard
wired, to improve recognition. Once they determined the position and
type of microphones to use, they were able to eliminate much of the
ambient noise and significantly increase the speech recognition
capability.

Second, MCH had to overcome resistance in some environments to pushing
a button to indicate to the computer that they were going to talk. The
users wanted to talk to the computer spontaneously and have it respond
to their commands. In order to fulfill this request, IBM made changes
in the WebSphere Everyplace Multimodal Environment that extended
existing features built into speech technology giving the system the
ability to first listen for a keyword (e.g "Computer") and then listen
until the user stopped speaking.  Certain design changes were made to
the i-Rounds application that resulted in users having more
flexibility with the system; for instance, during surgery, the surgeon
tells the system when they pass key phases of the operation such as
going "on bypass" or "cross clamp off." The surgeon needs to discuss
passing these phases with the team and saying the phrase "on bypass"
could accidentally trigger the computer to process that event.  With
the keyword activation feature, the surgeon says "Computer,
on-bypass," which solved the problem in a way that was natural and
intuitive for the clinicians.

The third and final significant challenge was developing a common
dialogue or nomenclature for navigating and delivering speech commands
within the application. Early users were, literally, getting lost in
the system.  More specifically, some users of the speech application
never look at a computer monitor when using the system and are only
talking to the voice-activated room. To handle this, navigating to all
the major sections of the application were made common by always
saying "Go to . . . ."  Then on each page, a pop-up window shows the
speech commands that are active for that particular part of the
application.  Having a consistent way to navigate around the i-Rounds
application and then having the application show which commands were
valid at the time relieved users of the burden of remembering
commands. This made the system easy as well as efficient for users who
knew the commands.

Once all the challenges were overcome and the tuning of the
application was complete, the system was implemented in the OR and
CICU.  In the OR, the technology serves as a safety net.  It provides
hands-free verification of the patient, diagnosis and planned
procedure.  It also provides timed reminders for the delivery of
medication and other procedures that must be administered at specific
intervals. The safety net of the speech application reduces the
occurrence of human error by offering an alternative reference for the
administering clinician to use before performing any life-threatening
tasks on the patient.  This may eventually work to improve the
hospital's mortality rate, which is already among the best in the
world at 98 percent.

Also, in the CCU, it works as a new way to record transcription by
recording the user's voice directly into the Web browser which then
becomes part of the patient's electronic medical record.  Others can
use the browser to access the information from anywhere and at any
time.  By capturing the doctor's audio transcription into the
patient's database record, this information becomes immediately
available to other doctors and medical staff.  Speech-to-text occurs
subsequently offline and the results are updated in the patient's
database record.   Health care practitioners then have real-time
access to the doctor's notes, which previously was not the case.
Also, by having the transcribed text in the same electronic document,
the data can be mined in the future.  This system is an improvement
over handheld recorders because the data is in the record under
application control/protection and not the doctor's pocket or courier
bag traveling to a transcription service.

MCH has only just started to use the multimodal i-Rounds solution and
has put it to use in two departments.  At this time, it's difficult to
predict exactly what impact the system will have on further reducing
mortality rates or preventable medical errors throughout the hospital.
Research is available suggesting the relationship between improved
outcomes and increased availability of patient information.  The
multimodal i-Rounds application provides speech input and output as
yet another vehicle for medical staff to access patient information
faster and more efficiently than ever before.

Another potential benefit is that speech-enabled systems use built-in
libraries of words and phrases.  This has the positive side effect of
enforcing common spelling and phraseology with the staff.  It improves
data accuracy and avoids handwriting and typing errors.  Another
potential, but as yet unproven benefit, is more efficient collection
of patient data from caregivers because entry is quick, easy and can
be done on the spot.

Making a huge paradigm shift, MCH's users had to adapt from both a
clinical documentation and computer interaction stand point.
Therefore, MCH has received mixed reactions from these users. One of
the main concerns expressed by physicians is that they don't have the
time to learn new technology at the expense of losing time with their
patients.  Since IBM's speech engine is user independent, it does not
require voice training, which solves one of MCH's major obstacles for
implementing speech technology.

In an effort to continue to tear down barriers to implementing speech
recognition hospital-wide, MCH is updating its IT Vision and Strategy
and aligning it with the clinical focus.  Once complete, the
multimodal capability will be an option that can be rolled out to
physicians wishing to use it for their documentation needs, and
potentially to other caregivers including nurses, technicians, etc.
Also, because the multimodal i-Rounds application is a fully working
traditional application for keyboard, display and mouse, users can
experiment with using speech at their own pace.  That is, they can
keep using the system the way they're used to doing it and use speech
when they want to.

MCH will continue with its testing to determine if this technology
will be widely accepted by the clinical staff.  Physicians are often
conservative and MCH anticipates some resistance, only because it is
new technology and a new process compared to the way care is
historically delivered.  Once practicing physicians see the benefits
gained through the system's efficiency and accuracy, MCH predicts that
their resistance will be replaced with excitement and adoption of the
speech system. 

By Stephanie Owens
Speech Technology Magazine

Contributed by:

 

Mohamad Saleem

Research Scholar

INDIA