Ronald Schilling, PhD

Diasonics MRI was an amazing success story – it was No. 2 in the race during the old days. With the development being UCSF-based, Diasonics had access to doctors on a routine basis. This was a wonderful thing for a start-up that is grounded on the Clinical/Technical Tie – having clinical experts and technical experts working side-by-side.

William G. Bradley, Jr, MD, PhD, was in his last year of medical school when he learned about Diasonics. He was staking his career around MRI as Diasonics was staking its company around MRI. We were in the same boat from the start, paddling together.

The early adopters were often private docs who were not as expert as the university players, and therefore relied on additional guidance. Bradley came up with the idea of the “over-read” concept. The docs would provide images to Bradley using teleradiology and receive clarification of the clinical findings from Bradley. This represented a major breakthrough in “customer engagement.”

It was a pleasure to talk with Bradley about business aspects of Diasonics MRI, for example, the sales strategy. Bradley, based on other consulting activities, had a feeling that the private market was going to play a key role in the expansion of MRI. After several strategy sessions, the approach that Diasonics followed came to fruition: Focus on the private market; selective in the hospital market; and opportunistic in the university market.[[{"type":"media","view_mode":"media_crop","fid":"64715","attributes":{"alt":"William G. Bradley, Jr, MD, PhD","class":"media-image media-image-right","id":"media_crop_1313330930810","media_crop_h":"0","media_crop_image_style":"-1","media_crop_instance":"8287","media_crop_rotate":"0","media_crop_scale_h":"0","media_crop_scale_w":"0","media_crop_w":"0","media_crop_x":"0","media_crop_y":"0","style":"height: 207px; width: 150px; float: right;","title":"William G. Bradley, Jr, MD, PhD","typeof":"foaf:Image"}}]]

• Private Market: Organizations were forming to build MRI centers. This included Texas Medical Equipment, Health Images, Medical Imaging Centers of America, and others. Being run by business people, they were in sync with the Diasonics’ “Total Business Solution” model. Bradley was an advisor to several of these groups.

• Hospital Market: Usually a very conservative market segment. Imagine a member of the decision committee asking radiology the question, “do GE and other large companies provide MRI?” Despite Bradley’s presentations, papers, etc., based on his work at UCSF and his Diasonics’ site at Memorial Research Center in Pasadena, it was an uphill battle not worth pursuing.

• University Market: Bradley’s efforts provided the confidence for UCSF, Memorial, University of Michigan, Orlando Regional Medical Center, and the University of Texas to make the big leap to buy from a start-up. These centers, all “triggered” by Bradley’s activity became the family that succeeded.

Bradley’s work with David Stark, MD, created a foundation publication – everyone knew about Bradley and, hence, about Diasonics. As the mobile market started to develop, Bradley once again played a powerhouse position in convincing Diasonics of the merits of mobile. Mobile was a key to finally getting the hospitals into the MRI business. Having the workstation inside the hospital kept the workflow at a high level. The MRI van was most often travelling between a number of sites or stationary on the hospital campus.

Throughout the above activities, it was wonderful to have Bradley on the team. He was, for Diasonics, Dr. MRI. Alex Margulis taught us that, “if you can’t see it on MRI, it’s only because you don’t have the appropriate driving algorithm.” Bradley found them for Diasonics.

We will all miss such an intelligent, caring, hard-driving team member.

Articles

Paying tribute to Bill Bradley, “Dr. MRI.” 

Bill Bradley – a Powerhouse

Being an advisor to several medical imaging publications, requires one to provide a prognosis of what the key dynamics in medical imaging might be in the following year.

At this point in time, I am personally involved in several areas: big data (BD) and interactive virtual reality (IVR). Both areas are beginning to take hold in terms of impacting patient outcomes (PO). This discussion is focused on the synergy between BD and IVR, and the resulting potential for significant gains in PO – an important dynamic for medical imaging.

 about BD is increasing as IBM’s efforts with Watson are coming to fruition. It is most noteworthy that Virginia Rometty, IBM CEO, characterizes Watson as a cognitive-based system that represents supervised learning using BD techniques. Cognition is indeed a critical element in that it impacts knowledge, which is dependent on both cognition and intuition. Knowledge, in turn, represents a fundamental factor in driving PO.

The other critical element in the knowledge equation, intuition, has been shown to be driven by 

. Hence, the power of the BD/IVR intersection.

Consider IVR as the ability to view an object in open 3D space and directly interact with the object (eg, removing a tumor electronically).

Radiologists have noted, that IVR provides a way to communicate more effectively with surgeons. Surgeons have noted that IVR allows them to finally achieve surgical visualization, the ability to view anatomy in a manner that corresponds to their 3D world view – they never open up patients and see 2D views.

Essentially, IVR represents a new paradigm, or “language,” with which different specialists discover an enhanced ability for communicating more effectively with one another.

Why might 2017 be the key year to realize the potential of the BD/IVR intersection? Based on several years of clinical trials, significant improvement in the core areas of PO, clinical efficacy (CE), and workflow (WF), have been demonstrated in a number of applications, due to IVR driven increases in knowledge.

We might think of IVR as focused on “doing the right things” and BD as focused on “doing things right.” A case in point would be a Protocol for optimizing both CE and WF for virtual colonoscopy. Based on the intuitive properties of IVR, a concept of dividing the colon into linear segments emerged with each segment separately analyzed for polyps. This breakthrough led to significant gains in both clinical efficacy and workflow.

BD has already been effectively applied to 

. BD could also be applied to the analysis of all users of the above protocol to establish best practices from data representing all user motions (head, stylus, etc.). In this manner, BD is a key to “doing things right.” This direction was intimated by 

The intersection of BD and IVR therefore provides a significant opportunity for achieving improvements in PO. We could think of this 

 as knowledge modeling. Today, leading organizations in health care are establishing knowledge modeling as the core of effective data management. What could be better than to apply these principles to increasing PO?

Big data meets interactive virtual reality. 

Intersection of Big Data and Interactive Virtual Reality

There is much excitement about big data and its potential applications in medical imaging. The purpose of this blog is to consider related approaches that are providing results today and can therefore serve as a guide for big data driven applications.

Virtual colonoscopy will be used as a clinical example to clarify this subject. Big data in the form of machine learning has been applied to CADx and utilized for virtual 

The CADx approach of analyzing large numbers of 2D images, creating representative algorithms, and applying them to diagnosis is an established approach. Medical Sieve, from 

, is one of the latest versions of diagnostic interpretation of 2D medical imaging datasets guided by clinical knowledge.

“Medical Sieve is an ambitious, long-term exploratory grand challenge project to build a next generation cognitive assistant with advanced multimodal analytics, clinical knowledge, and reasoning capabilities that is qualified to assist in clinical decision making in both radiology and cardiology.”

In all of the above, the fundamental source of data remains in the form of 2D views. When the 2D views are created as a projection of 3D volume data sets, we might refer to them as 2.5D 

. The use of 2D/2.5D as the application of big data in medical imaging, is dated and therefore can be considered as thinking “inside the box.”

, “Deriving value out of insightful analytics that could impact care processes and outcomes is an important, and often the most critical, goal of efforts related to big data.”

By working with “objects,” available with 

 (IVR), as opposed to being limited by images alone, the goals noted by Shrestha are already being realized in virtual colonoscopy as follows:

“With IVR, I can lift the colon out of the display, open up the colon and rotate it in innumerable planes,” Judy Yee, MD, of UCSF, 

. “Image manipulation is more intuitive and I can use the spatial resolution of lesions relative to folds and the background wall more easily, providing an ideal situation to identify polyps including flat lesions.”. This approach can be thought of as thinking, “outside the box,” with intuition driven knowledge, resulting in increasing clinical efficacy and workflow/patient outcomes.

The ultimate goal of big data is to improve patient outcomes. With IVR, the initial results are encouraging. They provide a ground truth and guide for raising the bar further with big data techniques applied to objects. At that point, big data will think outside the box.

Technology can help big data think outside the box in radiology. 

Can Big Data Think Outside The Box?

I just returned from an excellent health care conference, CARS 2016, held this year in the beautiful city of Heidelberg. Now in its 30th year, CARS (computer-assisted radiology and surgery) has been focused on closing the radiology-surgery gap.

Significant progress has recently been made, in closing the gap, with the advent of technologies like interactive virtual reality (IVR), an effective visualization communication tool. By viewing objects in open 3D space, in addition to images, the radiologist can, “speak more intelligently with the surgeon." In addition, the surgeon, who "never opens a patient and sees a 2D view," can have a workstation that relates effectively with their practice.

There’s nothing like excellent communication to bring diverse professionals together. In the case of radiology and surgery, the use of a visualization-based communication tool is the perfect solution.

At CARS, during which several related surgery organizations meet simultaneously, one often finds radiologists and surgeons, side by side, discussing the challenges and future of their joint trade.

There was discussion of radiology creating a surgical plan and streaming the results to the surgeon in the form of a visual report. Consider the area of colonoscopy. The radiologist, using virtual colonoscopy, can find the polyps, create scenes (3D data sets delineating the specific situation) for each polyp, and stream the "visual report" to the surgeon for polyp removal.[[{"type":"media","view_mode":"media_crop","fid":"49977","attributes":{"alt":"PACS","class":"media-image media-image-right","id":"media_crop_4331099277536","media_crop_h":"0","media_crop_image_style":"-1","media_crop_instance":"6076","media_crop_rotate":"0","media_crop_scale_h":"0","media_crop_scale_w":"0","media_crop_w":"0","media_crop_x":"0","media_crop_y":"0","style":"height: 113px; width: 169px; border-width: 0px; border-style: solid; margin: 1px; float: right;","title":"Â©sfam photo/Shutterstock.com","typeof":"foaf:Image"}}]]

Since approximately 10% of patients actually have polyps at the time of an examination, the cost savings are significantly increased and the difficulty for patients is significantly reduced. The radiologist plays a significant role in using IVR to not only characterize standard polyps, but also to find the challenging flat lesions that are twice as likely to be cancerous – followed by creating the scenes for the surgeon. The surgeon, will be provided with only cases containing polyps so that their efforts are more productive.

During discussions at CARS, dealing with the “operating room of the future,” it was evident that by working together, patient outcomes can be significantly improved – and that of course, is the name of the game.

Radiology and surgery speak the same language at CARS 2016.

Radiology and Surgery: Progress in Coming Together

Many surgeons and imaging specialists have recognized the incredible potential of 3D printing. Using 3D printers, we can turn real patient data into actionable, physical objects. The applications are numerous - from custom size prosthetics to surgical models to sizing implants.

Companies and medical professionals are looking at a range of opportunities. Stratasys and Vital Imaging announced a partnership last year to offer a “print on demand” service. And at RSNA last year, you could hardly go 30 minutes without overhearing an excited conversation about 3D printing.

There have been incredible successes. Nicklaus Children’s Hospital in Miami has been one of the strongest early adopters in 3D printing. In October, they made headlines for a surgical operation on a rare congenital heart defect in a five-year-old girl. The surgeons printed a 3D model of her heart in order to fully understand its unusual anatomical structures and prepare for a successful operation.

Despite success stories like this, many questions remain about the road to mass adoption of 3D printing. Printers are expensive to buy and maintain, and the technology is always evolving. By buying now, hospitals run the risk of purchasing technology that will soon become outdated. And I don’t think the industry has determined exactly how 3D printing fits into everyday medical workflows. Nevertheless, hospitals are moving quickly to spend large amounts of money on printers.

Lessons From the 90s[[{"type":"media","view_mode":"media_crop","fid":"48570","attributes":{"alt":"3D printing","class":"media-image media-image-right","id":"media_crop_7299693794629","media_crop_h":"0","media_crop_image_style":"-1","media_crop_instance":"5804","media_crop_rotate":"0","media_crop_scale_h":"0","media_crop_scale_w":"0","media_crop_w":"0","media_crop_x":"0","media_crop_y":"0","style":"height: 177px; width: 200px; border-width: 0px; border-style: solid; margin: 1px; float: right;","title":"Â©Mopic/Shutterstock.com","typeof":"foaf:Image"}}]]


In a way, this whole situation reminds me of the 1990s, when hospitals were beginning to adopt PACS. Here was an exciting new technology, promising huge return on investment. People couldn’t stop talking about PACS, and the money followed the hype.

Hospitals spent millions on these systems, but they largely failed to address the workflow question. Following the investment in infrastructure, hospitals failed to invest in training and onboarding. Many CIOs refused to deal with PACS, and the systems frequently sat unused.

As a result, the whole market crashed, and it took years to recover. It wasn’t a problem with the technology, but with the timing and method of its adoption.

I worry that the same could happen with 3D printing. If you could print a model directly in the office, in a matter of minutes - that would be a tremendous asset. But 3D printing is notoriously slow. Creating a model makes sense for exceptional cases - like the heart surgery at Nicklaus - but perhaps not for everyday operations. There’s often too much effort for the output.

This begs the question: if hospitals spend millions on 3D printers, will doctors actually use them? Are there other ways to deliver some of the benefits of 3D printing, at faster speeds and without spending so much money on a piece of equipment that could soon be outdated?

Another Option: Advanced Imaging Technologies


There’s another option that offers many of the same advantages to medical professionals, at much less cost. New imaging systems are emerging that create true volumetric images of patient anatomy. We call these virtual reality (VR) systems. Unlike today’s “3D” imaging, this is not a 3D view on a flat screen. Rather, it is a real-size interactive image (similar to a hologram) of a patient’s body part projected into open space.

Like a 3D printer, an interactive VR system can create a workable model from actual patient data. These help medical professionals understand anatomy, and even practice their operations. It also allows radiologists to provide a better product to surgeons - to communicate more intelligently, in a language (3D anatomy) that they both understand.

There are advantages to both technologies. Printed models have a physical consistency that more closely resembles body parts - you can hold them, and they provide resistance to incisions. But unlike prints, 3D images can be generated instantly, and with no cost for materials. In many cases, they can provide the same quality of information as 3D models. For example, doctors have seen huge success correctly sizing medical devices using VR systems.

Imaging systems also offer data analytics and feedback that you can’t obtain with a static model. A surgeon could practice an operation over and over again with virtual reality, with detailed feedback, and without needing to generate additional models. They can help radiologists communicate more effectively with surgeons, by letting them label patient anatomy and structures from multiple angles, at each step of the procedure.

They can also be dynamic - allowing doctors to visualize blood flow, for example, or to view anatomy from multiple angles. And they allow professionals to view and study many different models quickly and easily.


Ultimately, doctors will use both technologies; it’s a matter of choosing the right tool for the right situation. If adopted correctly, 3D printers and VR systems will bring tremendous benefit to the busy workflows of medical professionals. But let’s make sure 3D printing doesn’t become the latest technology to flounder due to poor timing and adoption strategy. As always, hospitals should carefully consider workflow, training, and all available options before rushing to buy.

3D printing has a promising future in radiology, if we take our time and manage expectations.

3D Printing: Proceed with Caution

Interoperability was a buzz word on steroids at HIMSS. It was found in big letters within many of the prominent booths all over the show.

Interoperability demands that communications regarding all aspects of patient care be connected, from entry into the system to discharge. This is required to realize the best results for increasing patient outcomes. However, this can't properly happen if departments, such as radiology and surgery, do not effectively talk with each other – something that often occurs due to the lack of a common visualization language.

This was confirmed at HIMSS by discussing this matter with many C- and IT-level individuals. They did confirm, after participating in a demo, that interactive virtual reality (IVR) has the potential to resolve the communication issue.

Why might this be true? For radiology, the "object" views (vs current limitation of using only image views) of IVR will allow radiology to "speak intelligently with surgeons." For surgeons, the “object” views of IVR is, by definition, the representation of the anatomical world of the surgeon. As one surgeon stated, “I've never opened up a patient and seen a 2D view.”

I won't be so bold as to say that IVR is the "key" ingredient to realizing interoperability. However, I will be so bold as to say that the proven, significant gains of both clinical efficacy and workflow (using IVR) are "key" ingredients to increasing patient outcomes. The enhancement of patient engagement due to IVR has been shown to further increase patient outcomes.[[{"type":"media","view_mode":"media_crop","fid":"46856","attributes":{"alt":"Ronald Schilling, PhD","class":"media-image media-image-right","id":"media_crop_6255425767733","media_crop_h":"0","media_crop_image_style":"-1","media_crop_instance":"5467","media_crop_rotate":"0","media_crop_scale_h":"0","media_crop_scale_w":"0","media_crop_w":"0","media_crop_x":"0","media_crop_y":"0","style":"height: 144px; width: 180px; border-width: 0px; border-style: solid; margin: 1px; float: right;","title":"Ronald Schilling, PhD","typeof":"foaf:Image"}}]]

An editorial report summarizing HIMSS said:

“At HIMSS16 buzzwords are dead (including big data, interoperability, personalized medicine, population health, etc.) It’s about innovating at all costs,” Dr. John D. Halamka, CIO for Harvard Medical School, wrote for 

. “We will no longer be driven by compliance imperatives (Meaningful Use, HIPAA, Affordable Care Act, and ICD10), but instead will need to improve outcomes in order to survive financially. No one is completely sure how to do that, but there are enablers.”

Interoperability is not dead. Based on meaningful use of IVR, interoperability can finally realize its potential in improving patient outcomes. That’s where innovation will come into play.

Interactive virtual reality can help radiologists, surgeons work better together.

Interactive Virtual Reality Key to Interoperability

Today, there is serious intent to lower dose in medical imaging studies. At RSNA 2014, there were many examples of meaningful results. Similarly, at CARS 2015, surgeons indicated an increasing focus on lowering dose.

The questions that follow from this are, “Can performance be improved by other means than increasing dose?” And, “Can performance be improved despite the fact that dose is being reduced?”

There is some indication that this may indeed be possible – the key being a new paradigm for visualization. Today, docs are mainly viewing images using 2D views of 3D volume data sets (referred to as 2.5D). This requires the docs to use their cognitive ability to create 3D views in their brain. This process often results in cognitive strain and loss of time. How might a new paradigm impact this situation?

Knowledge, a combination of cognition and intuition, is the driver behind increasing visualization performance. Hegarty

 points out that, “complementary actions have the potential to relieve the user of the need to perform an internal computation.”

Let’s begin our search for complementary actions by recalling the movie Avatar. Armed with stereo glasses, we can view the imagery with depth perception. Suppose we now add virtual reality to the equation – the ability to look around and interact with objects. We now have interactive virtual reality (IVR), a way of providing for complementary actions.

What is the doctor’s initial reaction to using such a system?[[{"type":"media","view_mode":"media_crop","fid":"41934","attributes":{"alt":"Ronald Schilling, PhD","class":"media-image media-image-right","id":"media_crop_3422326607196","media_crop_h":"0","media_crop_image_style":"-1","media_crop_instance":"4493","media_crop_rotate":"0","media_crop_scale_h":"0","media_crop_scale_w":"0","media_crop_w":"0","media_crop_x":"0","media_crop_y":"0","style":"height: 128px; width: 160px; margin: 1px; border-width: 0px; border-style: solid; float: right;","title":"Ronald Schilling, PhD","typeof":"foaf:Image"}}]]

A radiology student at UCSF, upon seeing IVR, stated, “I’ve just spent two years learning how to construct 3D from 2D views. With IVR, the 3D is present from the start and I can spend all of the required time to focus on solving the clinical problem.”

Another opportunity from IVR is to bridge the radiology-surgery gap. When seeing IVR at RSNA, a radiologist from UCLA noted, “now I can speak intelligently to the surgeon.” A surgeon from El Camino Hospital in Los Altos, CA, noted, “I’ve never opened up a patient and seen a 2D view.”

In future blogs, specific examples will be presented that provide evidence of the points made above. The results are based upon a “clinical/technical tie” process of understanding the clinical “needs” and providing complementary actions to optimize both clinical efficacy and workflow.

1. Hegarty M. “The Cognitive Science of Visual – Spatial Displays, Topics in Cognitive Science,” 2011 – Wiley Online Library.

Can new technology in radiology maximize efficiency without sacrificing quality?

Lowering Dose While Increasing Performance

It is worthwhile to begin with some key points from a foundation article by Hegarty et al on the subject of spatial cognition in medicine.

“All medical professionals depend on a detailed understanding of anatomy, which involves such spatial concepts as the shape of anatomical structures, where they are located relative to one another and how they are connected.

Performance of surgical tasks requires a broad range of spatial processes in order to plan, navigate, and reason using complex representations of space. In MIS the surgeon must rely on imagery showing anatomical structures from unusual angles never seen in anatomy textbooks.”

The challenges described above, known as spatial cognition, are compounded by the limitations associated with 2D or 2.5D (viewing 3D data sets in 2D). Having to construct 3D recreations in one’s mind while viewing a series of 2D images is a cognitively intensive process.

This is the present practice in medical imaging and continues to give rise to the 

 gap. As a surgeon recently told me, “I have never opened up a patient and seen a 2D view.” At RSNA, a radiologist told me, when viewing holographic-generated images: “with this capability, I will be able to communicate more effectively with surgeons.”

Hegarty suggested that working with holographic 3D images could make spatial cognitive abilities less of a concern when it comes to reading medical images. It’s as if the holographic images are doing the work for the reader so they don’t have to rely only on their internal visualization skills.

The concept of using holography has been presented by Hackett.

In this paper, it is demonstrated that medical holography for basic anatomy training provides statistically significant improved performance over traditional textbook materials in understanding anatomy, especially in regards to spatial relationships. The study also points out that cognitive load measures were lower using the holograms, lending credence to the notion that it is easier and more intuitive to process complex medical information from holograms.[[{"type":"media","view_mode":"media_crop","fid":"29593","attributes":{"alt":"Clinical efficacy and workflow","class":"media-image media-image-right","id":"media_crop_6062893802128","media_crop_h":"0","media_crop_image_style":"-1","media_crop_instance":"3070","media_crop_rotate":"0","media_crop_scale_h":"0","media_crop_scale_w":"0","media_crop_w":"0","media_crop_x":"0","media_crop_y":"0","style":"height: 263px; width: 350px; border-width: 0px; border-style: solid; margin: 1px; float: right;","title":"Figure 1. Clinical efficacy and workflow potential with expertly-derived protocols to IHS.","typeof":"foaf:Image"}}]]

In order for this type of advanced visualization to be fully accepted by surgeons, the ability to grasp and interact with objects in open 3D space is essential. This capability, or interactive holographic system (HIS), is now being developed by a number of companies in the field.

Applying expert derived protocols to IHS provides potential for optimizing clinical efficacy and workflow (see Figure 1). IHS alone may be of low benefit, similar to the situation with the early days of PACS - which brought the trend of buying new displays without changing behavior and still printing lots of film. Being protocol driven, with present technology, has a moderate benefit. The key to success is believed to be the combination of IHS and a protocol-driven workflow.

Can we close the radiology-surgery gap? The tools are available. The opportunity is there. It is up to us.

1. Hegarty M, et al. The Role of Spatial Cognition in Medicine: Applications for Selecting and Training Professionals, Chapter 11; Applied Spatial Cognition, G.L.Allen, Ed., Erlbaum, Mahwah, NJ.

2. Hackett M. Medical Holography for Basic Anatomy Training, I/ITSEC, 2013.

The technology exists to help radiologists and surgeons work better together.