Surgical Data Science

Research areas

Surgical data science and team mission

Surgical Data Science (SDS) is about improving outcomes of surgery with AI-based software systems driven by clinical data. We aim to improve the training and capabilities of surgical teams with SDS, to reduce complications, make surgery easier and safer, and achieve better patient outcomes.

Disrumpere

Ultrasound (US) is a key technology to detect abdominal cancer early, and treat it with minimal intervention. The disrumpere project aims to combine low-cost ultrasound devices with innovative AI and robotics technologies, to make US easier, faster more widely used.

Laparoscopic surgical guidance systems with Augmented Reality

We research computer systems to improve laparoscopic surgery with Augmented Reality (AR) technologies. 3D medical image data such as CT or MR is automatically combined with the laparoscopic video, to show hidden critical structures such as tumours and major vessels.

Percutaneous surgical guidance systems

We research computer systems to improve percutaneous surgery with Virtual Reality (VR) and 3D tracking technologies.

Ultrasound and flexible endoscopy educational systems

Objective skill assessment is becoming an increasingly important component of surgery education and high-stakes skill assessment for accreditation. Our goal is to combine low-cost mechanical simulators with AI to make these tools broadly accessible.

Software

Sight

Sight, the Surgical Image Guidance and Healthcare Toolkit facilitates the creation of software based on medical imaging.

It includes various features such as 2D and 3D medical image processing (CT/MRI/US), video processing, visualization, augmented reality, and connectivity with tracking systems. It can be used to write navigation systems, simulators, planning software, or even simple video filtering applications.

Sight is written in C++ and built on top of the best open-source libraries in the field such as OpenCV, ITK, VTK, PCL, and Qt and makes their usage easier by providing data common formats and wrappers. It is based on a modular object/service architecture, making building software application as simple as connecting together data, algorithms and user interface. It runs on Windows and Linux and is freely available under the LGPL.

Visualisation

Multi-Planar Reconstruction
Direct volume rendering
Mixed Rendering (Volume + Surfaces)
OpenGL 4.x programmable pipeline
Fast rendering pipeline

Formats and protocols

DICOM CT/MRI images
SCP/SCU
OpenIGTLink
Video files: MP4/AVI/MKV/…
Cameras: Intel Realsense, most USB webcams.
RTP/RTSP streaming

Fast prototyping

Interface and application design with XML or QML files
Reusable algorithms and widgets as services
Modular code, dynamic library loading
Easy application packaging
Prebuilt binaries for 3rd part libraries

Augmented reality

Camera calibration
Lens distortion and undistortion compensation
Virtual 2D/3D scene superimposition onto video
Precise synchronisation of the video and the virtual layer
Optical tracking with Aruco tags

Software applications

SightViewer

Medical image and segmentation viewer. It supports many popular formats including DICOM and VTK.

SightCalibrator

User-friendly application to calibrate mono and stereo cameras. Very handy since camera calibration is a prerequisite in any AR application.

Clinical and technical publications

View all >

2017

On-patient see-through augmented reality based on visual SLAM

Mahmoud N., Grasa O.G., Nicolau S.A., Doignon C., Soler L., Marescaux J., Montier J.M.M.

1-11

2017

Augmented reality in minimally invasive digestive surgery

Soler L., Nicolau S., Pessaux P., Mutter D., Marescaux J.

421-432

2016

Augmented reality guidance for the resection of missing colorectal liver metastases

Ntourakis D., Memeo R., Soler L., Marescaux J.

419-26

2016

Automatic localization of endoscope in intraoperative CT image: A simple approach to augmented reality guidance in laparoscopic surgery

Bernhardt S., Nicolau S.A., Agnus V., Soler L., Doignon C., Marescaux J.

130-143

8 / 43 First Previous 5 6 789 10 11 Next Last

Datasets

Liver segmentation

3D-IRCADb-01

This dataset is composed of the CT-scans of 10 women and 10 men with hepatic tumors in 75% of cases.

Where appropriate, the Couinaud segment number corresponding to the location of tumors is also provided.

Respiratory cycle

3D-IRCADb-02

This dataset is composed of 2 anonymized CT-scans.

The first one has been realized during the arterial phase in inhaled position, whereas the second one has been realized during the portal phase in exhaled position.

The patient has a hepatic focal nodular hyperplasia in segment VII according to Couinaud’s description.

The DEPOLL dataset for evaluating registration accuracy in AR-guided liver surgery

DePoLL (the Deformable Porcine Laparoscopic Liver) dataset was created to quantitatively evaluate registration accuracy for AR-guided liver surgery using a pre-operative CT model.