Virtual Abdomen
ALTAIR's Virtual Abdomen is a toolbox that allows to segment, reconstruct and simulate patient specific abdomens.
There are several surgical simulators actually on the market. These allows a reconstruction of the abdomen on a computer (thus the name "virtual abdomen"). Organs are drawn in a three-dimensional accurate reconstruction the user can navigate into. This makes it possible to clearly see the abdomen from different points of view, without the heavy constraints present while operating on a real patient. Furthermore, it is possible to interact with organs. Our simulator, like many others, uses the haptic rendering as well as the graphical one. This means that using an appropriate controller users can "feel" the contact, in a realistic way. A physical model analyzes the "input" the user gives by moving the controller in the environment, and provide a graphical deformation and a force feedback.
The great novelty of our approach is its ability to render not only generic organs like a medical atlas but also patient specific abdomens. This result is made possible by the many innovations we introduce in the reconstruction phases. With innovative segmentation techniques, specifically studied to be as automatic as possible and specific for each tissue, we extract and classify tissues in CT medical images. From the segmented data we build custom organs, getting a 3D model of the real organ of the specific patient, which could differ from a standard one for several reasons (e.g. shape, disease location, density and stiffness). The interaction with deformable organs is global, i.e. we do not restrict physical deformations to a sub part of the model, but we compute the whole body physics thus allowing more realistic interactions.
ALTAIR's Virtual Abdomen is accurate, can render a 3D reconstruction of patient specific abdomens and allow navigation and interaction with visual and haptic feedback. Probing and Grabbing allow the touch of the deformable organs and the whole intervention can be simulated before the surgery occurs. The "reconstruction pipeline" from patient specific data involves several research topics, the three major phases involved in the simulation are: medical data segmentation, virtual organ initialization and interactive simulation.
Medical Data Segmentation
We develop semi automatic algorithms for medical image segmentation: since different tissues requires different methods special attention should be paid in the selection of the correct algorithm and parameters for each tissue: we are developing tissue and geometry specific methods that greatly reduce the time and interaction required to the radiologist.
The outputs of the segmentation are surfaces and point clouds that describe reconstructed organs in the same frame of reference of the segmented data. This allows us to use the CT scans to initialize organ physical properties and greatly helps during the registration of virtual organs with real ones.
Organ Initialization
In the simulation phase we use mass spring models that have to be carefully initialized starting from the medical data and the segmentation results. Along with a volumetric, tetrahedral, model, there is the need to build a voxelization of the data obtained from the segmentation. All the processing needs to keep care of several constraints induced by the presence of deformable organs. In fact the realism of the simulation greatly depends on the stiffness and the volumetric reconstruction of models, so special attention should be paid in the selection of the correct tetrahedrization algorithm and parameters for each tissue: we are developing ad hoc methods for volumetric reconstruction and for physical properties initialization.
Voxelization of the reconstructed abdomen
We are investigating various methods for model tetrahedrization and we developed a method that is based on CT data and reference tissue properties to calibrate mass spring models.
Based on the assumption that a segmented tissue is homogeneous in its composition the tissue density can be computed from its Hounsfield value and in the same way tissue elastic properties can be identified considering that where the tissue is denser it is stiffer. We are working on more powerful calibration techniques based on the fusion of data coming from different sources i.e. MR, elastography and ultrasound.
Density distribution of a reconstructed liver (data in Kg/m3)
Interactive Simulation
Pushing gesture
The simulation of the deformation in interactive environments implies a tradeoff between realism and computational complexity. In fact, for a realistic rendering of interactions, the evolution of the environment should be computed at a framerate of at least 1 kHz. We are thus investigating several different models, in particular the mass spring models, the particle systems and the finite element methods and evaluating their realism, ease of use and computational requirements.
Grabbing gesture
The Simulator is a 3D environment, based on the multiplatform OpenGL library, that is used for several tasks: navigation, rendering of optimal trajectories to reach specific points on the organs and surgery simulation. It mixes I/O bound and Processing-bound tasks. Complicate physics calculations runs realtime reflecting the consequences of user-driven interaction. Also, some data comes from peripherals and sensors, so there is a network transmission of data which needs to be taken into account. This being the case, the simulator needs to be very modular, its parts wisely distributed through all the available resources. One of the novelty we introduce is the capability of the simulator to carry out calculation on GPUs (Graphics Processing Unit): parallel processors present on video-cards commonly available on the market. Video-cards are highly parallel computation devices, which can make use of their GPUs to execute computation at an amazingly high speed. The use of GPUs for scientifical purpose has started with the birth of GPGPU as scientist used the video cards for general processing tasks. Through an intensive use of GLSL, all the physics runs in parallel on the video-card, as well as the graphic rendering through shaders, allowing the CPU to care about I/O, Network and other computations.
Currently the simulator can update the state of models composed by more than 38000 tetrahedra at more than 5 kHz on a Intel Core Duo 2 @ 2.16 GHz equipped with an Nvidia GeForce 8800 GTX, allowing the user to perform various kind of interaction with the deformable environment (pushing and pulling tissues) with the two tools. Communication with master devices takes place at 1 kHz.
A screenshot of the simulated virtual abdomen
The interaction with the user has a main role in the simulation. On the graphical side, we uses advanced lighting, shadowing and textures to achieve a good level of realism, also using the advanced shader programs. The camera can be used to navigate through the environment, a detailed surgery room, with the patient body rendered with transparency to give a clear idea of position. The laparoscopic area is rendered with details, showing bones, organs and vessels. Two (or more) tools are rendered and synchronized with input peripherals. They offer basic animations to show the current action, so for instance, a grasper tool closes the mobile parts when pinching the liver surface. Furthermore, a clear, unobtrusive UI (User Interface) constantly show all the data the surgeon might need. The force calculation keeps being updated independently from graphical render, though it directly control it. Moving a tool in our Simulator offers then a good simulation of real surgery phases, with optimal haptic rendering and realtime deformations. Many constraints which come from the real surgeries can be removed, allowing a better analysis of the single phase. Everything is rendered on the screen, but it is possible to use the simulator with proper devices, such as shutter glasses or specific displays, in order to have a stereo visualization.
A planned trajectory is suggested to the trainee
To render forces we use both commercial and non standard haptic devices. This requires the ability to handle hardware with different features (frame of reference, rendered force/torque magnitude, controller...) in a standardized way. Moreover an important aspect of force rendering in this project is the reduction of delay in the communication over LAN. Currently the simulator is interfaced with commercial haptic devices, like Sensable PHANToM Omni, MPB Freedom, and with less standard devices, like NASA JPL FRHC force feedback joystick.
The communication is carried out by an architecture we developed for generic teleoperation, called Penelope. The Simulator can run in a standalone version or within the Penelope architecture due to its high modularity. The advantages of the Penelope Architecture are many, as multiple sensors can acquire data which are transparently read by the simulator, no matter where they are placed. Also controllers can be on different machine within the network. This pushes even forward the modularity concept, allowing data flow and management to be split through the network. This allows to put simulator and master devices on different computers and to get optimal performances.
Thanks to the libraries its used and its design the simulator is cross-platform: we used it on Windows(tm) Systems, Linux/Unix, and theoretically on MacOSX. Both 32 and 64 bit architecture are supported. nVidia Cards are required, though AMD/ATi and other cards might be supported in the future.
Different versions of the simulator interacting with NASA frhc and Sensable PHANToM Omni
Future and Ongoing Development
The Simulator has shown to be enough flexible to apply to several contexts, for example dentistry. Its development keeps introducing optimization, upgrades and new features. We are investigating a port of the physics calculations from GLSL to the newer CUDA. A careful analysis is undergoing and some tests gave promising results. Particle based models and FEM are under close study and an implementation on this new branch. The aim is to offer optimal solutions in all possible contexts, and combined use of both the techniques is potentially possible. Surgical operations such as cut and suture is undergoing.
