Sign in Subscribe
By petr zhegin in astrocyte* bci newsletter —

Neurotech monthly. August 2023

High-performance speech neuroprosthesis, speech, image, & music reconstruction from ECoG, EEG, & fMRI, in situ organic bioelectronics assembly, Neuralink's fresh funding, a story of a visual implants pioneer

Ahmed et al., 2023

Hello!

August Astrocyte* nesletter is here. This issue is delayed due to moving to the ghost platform from the Substack and merging two domains. Neurotech takes a more significant part of my life, so moving it to my personal domain made sense. It also trims costs, as there is no need to keep a separate domain/website for the newsletter.

This issue covers, among other things:

Neurotech reading list

  • High-performance speech neuroprosthesis, three papers that made me renew my Nature subscription;
  • Non-invasive stimulation - tES, TMS, nano-magnets;
  • Assembling bioelectrodes in the brain;
  • Speech reconstruction from ECoG, image from EEG, music from fMRI.

Startup/Corporate news

  • Nauralink, Universal Brain rasied funding;
  • A story of Second Sight's adventures, how visual implants are getting a second chance.

Below are a few things I'm working on. Subscribe to get these notes and a monthly newsletter into your mailbox.

Soon to be published:

  1. A summary/reflection note on the Royal Society Neural Interfaces Summit 2023 - November;
  2. A note on Synchron, an endovascular brain-computer interface - November;
  3. An updated version of the BCI startup landscape - December.
Subscribe

I. Reading List 📚

Techstack/Hardware/Wetware 🧠💿

Two high-performance speech neuroprosthesis papers no one should miss A high-performance neuroprosthesis for speech decoding and avatar control (Metzger et al.) and A high-performance speech neuroprosthesis (Willett et al.) were published in August. The results were then reviewed in Brain implants that enable speech pass performance milestones.

Via two invasive systems, brain signals were translated into sentences at speeds close to everyday speech, 62/78 words per minute (the speed of the natural conversation is 150-160 words per minute). These systems represented a significant improvement over previous systems, up to a factor of four. Both high-performance BCIs are not yet ready for clinical translation, however. And they are not for consumerisation for sure. Among the challenges are:

  • Custom sophisticated hardware - Willett et al. used microelectrode arrays (MEAs) (initially four, eventually two was enough) with 64 electrodes per array; Metzger et al. employed a silicon sheet embedded with 253 electrocorticography (ECoG) electrodes. Systems also included external components (see below). The longevity and reliability of devices will have to be examined.
  • Invasive delivery methods - electrodes were implanted (placed on the cortical surface or penetrated the cortex) by teams of highly qualified professionals. Also, external units attached to patients' skulls were required to process/transmit signals from electrodes. For example, a separate surgery took place to install a detachable digital headstage that collects data from ECoG, then '... minimally processes and digitizes the acquired cortical signals and then transmits the data to a computer for further signal processing'.
  • Recurrent neural networks (RNNs) used to translate brain signals into text were trained on unique patient data, so the systems could not be pre-trained on other peoples' data and operate out of the box. For instance, in Willett et al., the data collection took 140 minutes every day for eight days. 
  • The RNN decoder also needs to be updated, and the device has to be paused and recalibrated, as changes in the user's brain neural activity occur over several days.

Before doubling down on solving these challenges, one must determine the dominant approach to speech decoding. Less vs more electrodes, where they should be placed (Willett et al. observed. for example, that neural activity recorded from Broca’s area could not be decoded), penetrative vs. on-surface. Technological/engineering challenges could be faster iterated and solved when more clarity on these questions emerges.

A system fit for clinical translation, according to Thomas Oxley (@tomoxl), a founder of Synchron, an endovascular BCI should be:

... fully implanted, low training, low troubleshooting, generalisable, platform-agnostic solution that can be implanted in a highly accessible clinical setting.

By @WillettNeuro, @FanChaofei, @shenoystanford, @JaimieHenderson, @neuroleigh:

By @SeanMetzger5, @KayloLittlejohn, @jessierliu:

A clinical trial of using focused ultrasound to help with the delivery of chemotherapy into the brain to treat progressive diffuse intrinsic pontine gliomas

It's a great example of using multiple modalities together. I can't stop comparing various hardware and pharmaceutical modalities with periphery devices connected to the PC. Probably, we'll end up with multiple modalities working under the single 'umbrella' of the 'brain-computer interface'.

Focused ultrasound can be used to activate microbubbles inside the brain’s blood vessels. When activated, the microbubbles oscillate and temporarily open the tight junctions between the cells, enabling higher concentrations of therapies to enter brain tissue. 

Closing the loop between brain and electrical stimulation: towards precision neuromodulation treatments

A detailed look at transcranial electrical stimulation (tES) and real-time functional magnetic resonance imaging (rtfMRI) used together to establish closed-loop tES-fMRI for individually optimized neuromodulation. By @EkhtiariHamed

TMS provokes target-dependent intracranial rhythms across human cortical and subcortical sites

Here, transcranial magnetic stimulation (TMS) was used in combination with intracranial EEG (iEEG). An experiment demonstrated that TMS, a non-invasive modality, could indirectly stimulate the brain's deeper areas.

... it is also clear that non-invasive, cortically-targeted stimulation can modulate electrical activity in deep brain structures that are not directly accessed by stimulation itself... Within the limits of the moderately-sized samples in this study (up to 17 subjects depending on the stimulation site and recording location), our data suggest that TMS directed at the DLPFC – at least following single pulses – suppresses high-frequency activity in the hippocampus for several hundred milliseconds. There is also weak but intriguing evidence that parietal stimulation can instigate theta rhythms in the hippocampus – a finding that has profound implications for how we might use stimulation in modulating core cognitive functions of the hippocampus itself.

By @boeslab, @esolomon, @DrCoreyKeller

BRAINSTORM project will develop nano-magnets for neuromodulation

An alternative view on non-invasive stimulation - 'nanoinvasive neuromodulation technology'. A team of researchers from multiple universities collaborate on stimulation via controlling the magnetic nanomaterials after they have been delivered to the brain area.  To achieve this, they developed an end-to-end system with two core components: 1) an ultrasound-based delivery system allowing tiny magnets to cross the blood-brain barrier, avoiding cranial injection; 2) a helmet-type device to precisely control the magnetic nanomaterials.

Led by @dgregurec

Reusable, flexible, and lightweight chronic implants for Neuropixels probes

A cheaper Neuropixel-based open-sourced implant that aims to help with research on animals. Twitter: @MatteoCarandini

“Apollo implant” - for the reversible chronic implantation of Neuropixels probes [1.0 and 2.0]. It comprises two modules: the recoverable payload module accommodates up to two Neuropixels probes and is reused across animals, and the docking module is permanently cemented to the skull during implantation. The design is open source and can be readily adjusted to change the distance between probes, implantation depth, or angle of insertion.

Two papers on nature-inspired electrodes blew me away. One represents an advancement in flexible probes - biohybrid transition microelectrode array (TMEA) - based on substituting exogenous electrode materials with endogenous ones. Another one proposes to assemble bioelectronics in the zebrafish's brain.

The pace of innovations is relentless. While we are still perfecting existing hardware modalities like ECoG, MEA, and others, the work is already happening on merging hardware and wetware even closer.

Implementation and testing of a biohybrid transition microelectrode array for neural recording and modulation

The polymer microneedles have channels containing a hydrogel material to facilitate neural growth. This microneedle array is attached to a silicon-based chip which carries liv- ing neurons in gold-coated microwells (μWells) on the surface. The immobilized neurons are intended to project their neurites through the hydrogel- filled shaft channel and, after implantation, form synaptic connections with the brain’s local neu- rons. They thus function as a living interlink between the base chip’s electrical recording/s- timulating site and the region of interest in the human brain’s cortex.

In situ assembly of bioresorbable organic bioelectronics in the brain

Here we place bioresorbable electrodes with a brain-matched shear modulus—made from water-dispersed nanoparticles in the brain—in the targeted area using a capillary thinner than a human hair. Thereafter, we show that an optional auxiliary module grows dendrites from the installed conductive structure to seamlessly embed neurons and modify the electrode’s volume properties

Data Science 💻🧮

Direct speech reconstruction from sensorimotor brain activity with optimized deep learning models

Data from five patients with high-density ECoG was used for the optimization and evaluation of deep-learning models for speech reconstruction during a speech production task. 

We found that end-to-end deep learning approaches... overall benefited from model optimization, and that the choice of output parameters of these models (target spectrograms) had the largest effect on the reconstruction quality. ... Word recognition in reconstructed audio was markedly more accurate, stable across subjects and robust compared to raw brain input. ... we quantified the relationship between the reconstruction accuracy and location of HD ECoG electrodes, revealing that the largest contribution to model performance was made by small clusters of electrodes throughout the ventral and dorsal premotor and motor cortices.

Twitter: @ju_berezutskaya

How does artificial intelligence contribute to iEEG research?

This is a fantastic dive into applying machine learning with intracranial EEG. It covers:

... key machine learning concepts, specifics of processing and modeling iEEG data and de- tails of state-of-the-art iEEG-based neurotechnology and brain-computer interfaces.

Twitter: @karimjerbineuro, @ju_berezutskaya, @AnneLiseSaive, @marcelge

Brain2music: reconstructing music from human brain activity

We introduce a method for reconstructing music from brain activity, captured using functional magnetic resonance imaging (fMRI). Our approach uses either music retrieval or the MusicLM music generation model conditioned on embeddings derived from fMRI data.

Twitter: @TimoDenk

Seeing through the brain: image reconstruction of visual perception from human brain signals

Image reconstruction from signals. It's exciting to explore to what extent generative models can make sense out of noisy EEG signals. I can hardly imagine this approach working in healthcare, but it looks relevant for consumer applications.

... comprehensive pipeline for Neural Image generation, namely NEUROIMAGEN... [it ]incorporate[s] a multi-level seman-tics extraction module which decodes different semantic in-formation from the input signal with various granularity. Specifically, the extracted information contains sample-level semantics which is easy to decode, and pixel-level seman-tics such as the saliency map of silhouette information that tends to more decoding difficulties. The multi-level outputs are further fed into the pretrained diffusion models with the control of the generation semantics.

Misc neurotech reading 👨‍💻

Biomarker-guided neuromodulation aids memory in traumatic brain injury

Neurotech startups do not address memory frequently. I came across only a few of them. For example, on the memory uploading side of things, there is Nectome, and for memory improvement, there is Nia Therapeutics. It's great to see researchers using existing technologies like ECoG to work with it. Memory boosting still seems far away from mass adoption, as it requires invasive electrodes. Here's what has been done by researchers, including Nia's co-founder @KahanaMichael:

The data from indwelling electrodes was collected as patients studied and recalled lists of words, then personalized machine-learning classifiers were trained to predict momentary fluctuations in mnemonic function in each patient. Classifiers then triggered high-frequency stimulation of the lateral temporal cortex (LTC) at moments when memory was predicted to fail. This strategy yielded a 19% boost in recall performance on stimulated as compared with non-stimulated lists (P = 0.012).

Thread by @DanRizzutoPhD of Nia:

The future of brain circuit-targeted therapeutics

New stimulation and imaging modalities are emerging. It's crucial to systematically translate them into practice. This paper offers a framework for identifying and modulating treatment targets.

By @Khosravan2Sanaz, @shansiddiqi, @RolstonJohn, @foxmdphd

Business 📈

Startup/Corporate News 📰💰

🇺🇸 Musk's Neuralink raises $280 mln in funding led by Thiel's Founders Fund - the BCI heavyweight raised capital soon after reported secondary sale, which I've mentioned in June's issue.

🇺🇸 Universal Brain, which I've mentioned earlier raised a $1.7M seed round. The company builds a brain-based personalized treatment for mental health; the system includes an EEG headset, neurofeedback, and an analytical layer for brain input/output data(EEG, fMRI).

🇺🇸 Second sight’s implant technology gets a second chance. A great story of how a neurotech startup's trajectory could look like through initial development and trials, merger, and repurposing of the tech. Here is a summary.

In 2020, California-based biotech company Second Sight faced financial struggles, putting at risk hundreds of users worldwide who depended on its Argus II retinal implant and the Orion implant for vision. As the company ceased most operations and merged with biotech startup Nano Precision Medical, Second Sight's tech was spun out, creating a new entity named Cortigent. This new business ensures a continued supply of replacement parts for Argus II users, though no further development of retinal-device upgrades. Instead, the $15 million from an upcoming IPO will finance the Orion brain implant's development, aiming to restore vision in some blind individuals and repurpose the technology for a device designed to aid post-stroke mobility recovery.

Cortigent's technology, stemming from stimulating the brain's surface to help regain motor skills after a stroke, offers a more sophisticated approach than previous attempts in this field. Unlike, for example, Northstar Neuroscience's single-driver device from the early 2000s, Cortigent's version boasts 60 independent channels, allowing for more precise motor cortex stimulation.

Twitter @meharris

📬📬📬 Subscribe for a monthly update on neurotech and neurotech investment/commercialisation activity.

Subscribe

Subscribe to peter zhegin | future platform

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.
jamie@example.com
Subscribe