We have located links that may give you full text access.
Text and image generation from intracranial electroencephalography using an embedding space for text and images.
Journal of Neural Engineering 2024 April 23

Invasive brain-computer interfaces (BCIs) are promising communication devices for severely paralyzed patients. Recent advances in intracranial electroencephalography (iEEG) coupled with natural language processing have enhanced communication speed and accuracy. It should be noted that such a speech BCI uses signals from the motor cortex. However, BCIs based on motor cortical activities may experience signal deterioration in users with motor cortical degenerative diseases such as amyotrophic lateral sclerosis (ALS). An alternative approach to using iEEG of the motor cortex is necessary to support patients with such conditions.
Approach:
In this study, a multimodal embedding of text and images was used to decode visual semantic information from iEEG signals of the visual cortex to generate text and images. We used contrastive language-image pretraining (CLIP) embedding to represent images presented to 17 patients implanted with electrodes in the occipital and temporal cortexes. A CLIP image vector was inferred from the high-γ power of the iEEG signals recorded while viewing the images.
Main results:
Text was generated by CLIPCAP from the inferred CLIP vector with better-than-chance accuracy. Then, an image was created from the generated text using StableDiffusion with significant accuracy.
Significance:
The text and images generated from iEEG through the CLIP embedding vector can be used for improved communication.
.
Full text links
Related Resources
Trending Papers
Obesity pharmacotherapy in older adults: a narrative review of evidence.International Journal of Obesity 2024 May 7
SGLT2 Inhibitors in Kidney Diseases-A Narrative Review.International Journal of Molecular Sciences 2024 May 2
Use of Intravenous Albumin: A Guideline from the International Collaboration for Transfusion Medicine Guidelines.Chest 2024 March 5
Get seemless 1-tap access through your institution/university
For the best experience, use the Read mobile app
All material on this website is protected by copyright, Copyright © 1994-2024 by WebMD LLC.
This website also contains material copyrighted by 3rd parties.
By using this service, you agree to our terms of use and privacy policy.
Your Privacy Choices
You can now claim free CME credits for this literature searchClaim now
Get seemless 1-tap access through your institution/university
For the best experience, use the Read mobile app