Back to Research

Poster · 2024

A Simple Method for Synchronising Rich Kinematic Video Data with Local Field Potentials from DBS Systems

G. Sarapata, M. Beudel, D. Hubers and J. O’Keeffe

Introduction

Several deep brain stimulation (DBS) systems now allow for simultaneous stimulation and local field potential (LFP) recording.1 While conventional DBS is an effective therapy for neurological disorders like Parkinson’s disease (PD), synchronising neuronal activity signals with behavioural measures could reveal mechanisms underlying motor control in health and disease.2 Computer vision assessment of PD3 can probe neural population codes in a data-driven manner when LFP signals are synchronised with kinematic video data. To achieve this, we define a method for synchronising LFP signals from a stimulation device with external visual data with sub-second precision.

Method

Data were captured during a standard MDS-UPDRS part III assessment in four medication–stimulation states (ON-ON, ON-OFF, OFF-ON, OFF-OFF). The tools used included:

  • Percept™ PC (Medtronic PLC, USA); a neurostimulation device implanted in PD patients capable of measuring LFP signals.
  • Video-enabled smart device; such as a consumer-grade smartphone or tablet running iOS or Android.
  • Kelvin Clinic™ App (Machine Medicine Technologies, UK); which is available in the iOS and Android app stores and allows for video recording of the MDS-UPDRS motor assessment and export of the device’s accelerometer data.

The data capture procedure involved the following steps:

  1. Updating the DBS device’s internal clock through the tablet controller, synchronising the stimulator’s clock with the control tablet.
  2. Starting the video assessment on the smart device and beginning the LFP capture session.
  3. While recording, tapping four times on the patient’s implanted pulse generator with the smart device to induce artefacts in the LFP signal. The tapping pattern is aperiodic, created by counting slowly (≈ 1 Hz) and steadily to 8, tapping on the 1st, 2nd, 4th and 8th beats.
  4. Recording the rest of the motor assessment, stabilising the camera device (e.g. using a tripod) and ensuring the patient is entirely in the shot.

The first step achieves coarse synchronisation in the order of seconds. Sub-second synchronisation of the LFP and video data is obtained by aligning the induced LFP artefacts with acceleration spikes from the smart device’s built-in accelerometer. This alignment is performed by finding a time lag that maximises the cross-correlation between the signals, where f(t) and g(t) are values of the two signals at time t:

Equation: the argmax over x in the interval from minus w to w of the sum, over t from minus infinity to infinity, of f(t minus x) times g(t).

Here w denotes a time window around the coarse alignment (x = 0) for which the cross-correlation is computed. Since the accelerometer and the camera share the smart device’s internal clock, aligning video frames with the DBS signal follows naturally.

Figure 1: a four-step schematic of the data capture procedure, showing the DBS device clock being updated from the control tablet, video and LFP capture starting together, the patient's implanted pulse generator being tapped on beats 1, 2, 4 and 8 of a slow count, and the resulting LFP trace aligned with the smart device's acceleration trace, where three sharp acceleration spikes coincide with artefacts in the LFP signal during the highlighted synchronisation window that precedes the motor assessment.
Figure 1. Data capture procedure. After updating the internal clock of the DBS device through its control tablet, the video and LFP capture is initiated. LFP signal artefacts get induced through a series of taps made with the video-recording mobile device. The same device is then used to capture the subsequent motor assessment. The LFP signal gets aligned with the mobile device’s accelerometer reading which results in LFP–video synchronisation.

Results

Across 5 subjects and 20 assessments, human-identifiable artefacts were produced by tapping in 14 cases, with successful automatic synchronisation in 10. A dependence between tapping force (as indicated by recorded acceleration) and the likelihood of LFP artefact occurrence was observed. Wider adoption of this procedure will enable a detailed quantitative investigation of this relationship.

Discussion

This method is a crucial step in unravelling the neural basis of various movement disorders and could enhance DBS therapy by providing real-time feedback on neuronal activity and motor functions. Future research can build on this foundation to explore other neurological conditions and improve synchronisation processes. Notably, Medtronic’s Percept™ PC neurostimulation device has an internal accelerometer, although not enabled by default. Activating this capability and accessing the resulting inertial data offers the potential to synchronise LFP and kinematic video data without tapping the IPG.

While this work focuses on synchronising LFP and kinematic video data, the Medtronic Percept™ PC system outputs JSON files that include current stimulation parameters. Combined with the Kelvin™ platform, which stores clinical ratings and medication status, this allows for the creation of rich, multimodal datasets. These encompass both inputs (stimulation and medication) and outputs (kinematic data and clinical ratings) across multiple patients and sites, making them ideal for developing machine learning-based programming systems. Collaborative filtering engines, like those used by Netflix and Amazon for recommendations, demonstrate the potential of these systems to efficiently solve complex problems by identifying desirable solutions from vast possibilities.

Disclosure

Grzegorz Sarapata and Jonathan O’Keeffe are employees of Machine Medicine Technologies. Deborah Hubers received funding from the Amsterdam UMC TKI-PPP grant (2023 call). Martijn Beudel received research funding from the Amsterdam UMC TKI-PPP grant (2021 & 2023 call), the EU Joint Programme – Neurodegenerative Disease Research (JPND) project (2021 call) and Stichting ParkinsonFonds (2023) and Medtronic (2023, 2024).

References

  1. Neumann, W.-J., Gilron, R., Little, S. & Tinkhauser, G. Adaptive deep brain stimulation: from experimental evidence toward practical implementation. Movement Disorders 38 (2023).↑
  2. van Wijk, B., Bie, R. & Beudel, M. A systematic review of local field potential physiomarkers in Parkinson's disease: from clinical correlations to adaptive deep brain stimulation algorithms. Journal of Neurology 270 (2022).↑
  3. Morinan, G., Dushin, Y., Sarapata, G. et al. Computer vision quantification of whole-body Parkinsonian bradykinesia using a large multi-site population. npj Parkinson's Disease 9, 10 (2023). https://doi.org/10.1038/s41531-023-00454-8↑

About this poster

This research was originally presented as a poster by Machine Medicine Technologies. This HTML version has been reformatted for accessibility and web presentation, while preserving the content of the original poster.