Spelled words by users
1. Introduction
There exist different methods to analyze and study biomedical signals. Some of these methods are based on medical imaging, involving Magnetic Resonance Imaging (MRI), Computed Tomography (CT), Nuclear Scintigraphy, etc. This techniques show the image of a specific part of the human body. But there exits other methods which study the different signals based in 1-dimensional analysis, such as Electroencephalography (EEG), Electrocardiography (ECG), etc. These do not show an image, but they also show relevant information.
These techniques are not only used to detect any anomaly, but in the case of the EEG, it is also possible to develop communication systems by Brain Computer Interface (BCI) [1]. The BCI technology has numerous applications which can improve the quality of life of those people who need external help at the time of communication or controlling their movements.
EEG records the electrical activity along the scalp produced by the neurons. This activity happens since the brain cells communicate each other giving place to tiny electrical impulses.
The impulse begins with a chemical discharge which origins a current in the membrane of the emitting cell. Once the impulse gets the extreme of the connection between cells, the neuron secretes a protein that inhibits or excites another neuron.
After receiving the signal, the neuron releases ions to the outside of the cell. When lots of ions are expelled at the same time they can stimulate other neurons. At the time this wave of ions gets to the electrodes, the ions can attract or push the metal of the electrodes. This difference of pressure can be measured by a voltmeter and the record of this activity along time is the EEG signal.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image1.jpeg)
Figure 1.
EEG operation
Different studies show that, with the correct training, it is possible to control different external devices with the BCI technology. In [2], Galán et al. present a study where two users have to control a wheelchair using a system based in BCI. They have concluded that the success has been higher after learning the interaction with the system.
With respect to the communication, the most used technique is the detection of the P300 wave as result of a photostimulus [3]. In these cases, the user has to watch one keyboard where the columns and rows are illuminated randomly. If the letter the person is watching is illuminated, the brain sends a signal as a response to that photostimulus. This answer is called P300.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image2.jpeg)
Figure 2.
Keyboard used for the detection of P300. Source: http://www.bbci.de/competition/ii/albany_desc/albany_desc_ii.html
In [4] the authors use a system based on evoked potential to control one robotic arm. The system illuminates different actions which can be made by the arm. To know which action the user is focused, they detect the P300 and the N2pc.
There are also different applications of BCI technology in different games, such as Mindball [5], which consist on moving a ball depending on the brain activity: the more relaxed and focused the user is, the more the ball will move.
But electrodes are not only used to measure brain signals, but they also register the electrical activity from other parts, such as the heart (ECG) or the skeletal muscles (EMG). It is also possible to pick up the muscular activity, as it could be the measurement of the eye movements (EOG).
We have decided to use an EEG as electrocardiograph (ECG) and as electrooculography (EOG). The used device is the same for both applications: a portable EEG with 4 different channels.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image3.jpeg)
Figure 3.
PET 4.0: EEG of 4 channels
This device was initially acquired to work in one application in which it was necessary to carry a portable EEG. Due to its portability, it can collect data during long periods of time and outdoors.
The electrooculography (EOG) is based on the electrical activity of the eyes movement and it has been used for the user’s communication. There are other applications based on the eye movements to control devices. They detect the user’s iris, so it can know the position the user is looking through the coordinates of the iris. In [6] the study presents a method to control a wheelchair with one camera establishing three points as reference: the eyes and the nose. This three points form a triangle.
The ECG is a register of the electrical activity of the heart in order to know if there is any anomaly depending on the heart behavior.
We have used two different placements of the electrodes:
Bipolar placement: each channel represents the difference between two adjacent electrodes. This montage has been used for the EOG.
Referential or monopolar placement: each channel represents the difference between one electrode and another one established as reference. The ECG has been acquired with this montage.
The chosen electrodes are adhesive because of their facility to be placed both in the face as well as in the breast. The diameter of each electrode is 24 millimeters.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image4.jpeg)
Figure 4.
Kendall Arbo electrodes. Source: http://www.brainquiry.com/KendallArbo.html
The electrodes are responsible for registering the voltage inside a single cell. An electric impulse, known as the action potential, is an electric discharge which travels along the cellular membrane. The action potential is used to transport the information between the tissues. Although they can be generated for different kind of corporal cells, the most active are the cells of the nervous system [7].
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image5.jpeg)
Figure 5.
Action potential
For the monitoring and the posterior analysis we have developed two user’s interface in Matlab: one for the ECG and another one for the EOG.
2. Relation between EEG and EOG
During the register of an EEG, there can appear different signals called artifacts. These signals can be originated by multiple causes: muscular movements, skin impedance, technical problems, etc. One of the causes of these artifacts can be the ocular movement, such as eye blinking. This electrical activity is registered in the frontal region of the brain, being FP1, FP2 the channels where can best be appreciable.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image6.jpeg)
Figure 6.
EEG with two eye blinking
But to register the eye activity it is not necessary to place the electrodes on the head, it can be registered placing the electrodes in the face, which is called EOG.
An EOG is a method to register the eyes movement by placing little electrodes near the eyes muscles. Under normal conditions there exists a potential difference (10µV to 5 mV approximately) between the cornea and the Bruch´s membrane, located on the back of the eye. This is known as cornea-retinal potential. The cornea corresponds to the positive extreme and the retina corresponds to the negative of the dipole.
Figure 7 represents the differential of potential between the cornea and the retina:
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image7.jpeg)
Figure 7.
Cornea - retinal potential
Figure 8 show the behavior of the eye as a dipole:
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image8.jpeg)
Figure 8.
Eyes movement potential
In [8] it is well explained the different kinds of eye tracking systems and the evolution of them. For studying the eyes movement the criteria for divide the different techniques is different. There are three different kinds of eye tracking:
Invasive technique: this technique uses a special contact lens with an incorporated mirror or a magnetic field senor. The data acquired are more detailed because it is in contact with the eyeball, but is more uncomfortable because the user has to wear the contact lens.
Electrical potential: it uses electrodes placed around the eyes to detect the movement [9]. It is a robust method to measure the ocular movements when there are eyes blinking or change of the gaze. The main disadvantage is that it may become uncomfortable and the register can be affected by the electrodes movement. One of the main advantages is that it registers the eye movement with closed eyes so they can be used in the analysis of sleeping problems [10] and [11]. The EOG is within this kind of eye tracking.
Non - invasive technique: there is no contact with the eyes. The eyes movement is located through a camera or an optical sensor. The infrared light generates different reflections of the cornea of the user’s eyes known as Purkinje images [12] and [13].
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image9.jpeg)
Figure 9.
Purkinge images
The montage of the electrodes is bipolar: we register the difference between two electrodes. As the device is a 4 channel EEG, we have used them to perform 2 different positions: one for register if the user is looking on the left or on the right; and a second one to check if the user is looking up or down. The reference electrodes are placed in the ears.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image10.jpeg)
Figure 10.
Electrodes position of EOG
The difference between channels 2 and 3 shows whether the user is looking up (positive) or down (negative). The difference between channels 4 and 5 determines if the user has looked to the left (positive) or to the right (negative).
The final application will have one user interface divided into two tasks:
Communicate between the device and the main application via Bluetooth.
Interpret what the user wants to express.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image11.jpeg)
Figure 11.
User's interface of EOG
For the first task, we have used the API provided by the fabricant. To allow the user’s communication, the application will show the same keyboard as Figure 12:
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image12.jpeg)
Figure 12.
Keyboard
The user moves one circle through the keyboard by moving his eyes to the left, to the right, up or down. Once the user selects “OK”, the application will use the T9 dictionary to interpret the correct word. The dictionary has been programmed taking in count the words which meaning is related with expressing feelings and emotional states.
Figure 13 show the difference between looking up and looking down. The subtraction of CH2 and CH3 accentuates the difference between the eye movements.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image13.jpeg)
Figure 13.
Difference between looking up and down
In Figure 14, which shows the difference between looking to the left or to the right, it can be seen that the situation is similar than looking up and down: looking to the left produces an increment of the signal value, while looking to the right decrements it.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image14.jpeg)
Figure 14.
Difference between looking to the left and to the right
3. Relation between EEG and ECG
An ECG is the graphical representation of the electrical activity precedent from the heart. It is a non - invasive method to study and to analyse the condition of the heart with the aim of detecting possible anomalies or diseases.
In order to synchronize the cyclical contraction of the heart, the fibers of the cardiac muscle transmit electrical impulses.
The electrical activity is the potential difference generated by the cardiac cells (each cell acts as a voltage generator) which are appreciable on the skin surface, where the electrodes pick up this activity in order to get a graphical representation.
This representation shows 4 significant parts:
“P Wave”: the cardiac cycle begins with the depolarization, which leads on the contraction of the atriums in order to deposit the blood in the ventricles. The sinus node (SA node) indicates the atrium muscles that they have to contract to begin the sequence. The potential action is propagated through the specialized cells of the cardiac muscle to the AV nodule.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image15.jpeg)
Figure 15.
Atrial systole
“QRS complex”: the wave of depolarization arrives to the ventricles through the Bundle of His and the Purkinje fibers, producing the contraction of the ventricles.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image16.jpeg)
Figure 16.
QRS complex
“ST segment”: indicates the time between the end of the contraction of the ventricles and the beginning of the resting period.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image17.jpeg)
Figure 17.
ST segment
“T wave”: indicates the repolarization of the ventricles.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image18.jpeg)
Figure 18.
T wave
Figure 19 is the representation of a normal ECG. There can be appreciated the different waves explained before.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image19.png)
Figure 19.
Heart signals
When the electrical impulse arrives to a cardiac cell it provokes the interchange of ions inside the cell, giving place to a change of polarity. The action potential is the base for the depolarization and polarization of the myocardium: it is what generates the electrical impulse.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image20.jpeg)
Figure 20.
Cardiac action potential
The electrodes are places along the trunk of the users, taking the shoulders as reference. Figure 21 shows the position of the electrodes for the correct register of the ECG.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image21.jpeg)
Figure 21.
Electrodes placement
The user’s interface has the aspect of Figure 22. The user can select the channels in both graphs.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image22.jpeg)
Figure 22.
User's interface for ECG
4. Practice examples
4.1. EOG
In this section, it will be described how the EOG has been used to move a mark along a keyboard on the screen in order to help the people who are not able to communicate with the rest due a disease.
First of all, the application calibrates the position of the electrodes and establishes the correspondent thresholds to know the position of the eyes. After that, it will show the keyboard the user will use for the communication.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image23.jpeg)
Figure 23.
General diagram
Once the application shows the keyboard, it will do the operations described in Figure 24 and Figure 25 to know if the user has made any action with the eyes.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image24.jpeg)
Figure 24.
Up and down criterion
The signal is obtained during 200ms and then the application calculates the average during this time interval. After that, it compares if there has been an enough increment or decrement to detect if the user has moved the eyes intentionally.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image25.jpeg)
Figure 25.
Left and right criterion
Figure 26 and Figure 27 show the calibration state of the EOG before showing the keyboard. The first two images correspond to looing up (yellow rectangle) and down (green rectangle).
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image26.jpeg)
Figure 26.
Looking up and down
Below, the pictures show the user looking to the left (dark blue) or to the right (cyan).
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image27.jpeg)
Figure 27.
Looking to the left and to the right
The EOG has been tested with 5 different users (3 female, 2 male) with ages between 23 and 30. They have been asked to spell 5 different words: WATER, FISH, SUN, DOG and MIND.
4.2. ECG
This section describes the ECG used to monitor the heart activity and to detect possible anomalies. In this case, the montage is monopolar, so we can obtain information from 4 different points (4 channels), establishing as a reference the user’s shoulders. The signals the ECG records are smoothed before being monitored.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image28.jpeg)
Figure 28.
General diagram of ECG monitoring
After acquiring the signal (Figure 29), it is filtered with a Butterworth bandpass filter between 15Hz and 50Hz to detect the QRS complex (Figure 30).
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image29.jpeg)
Figure 29.
Original ECG
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image30.jpeg)
Figure 30.
Butterworth filter
Once the signal has been filtered, we have established a threshold to facilitate the detection of the R wave (Figure 31):
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image31.jpeg)
Figure 31.
Signal after establishing the threshold
5. Results
5.1. EOG
We have asked the users to move a circle along a keyboard to spell five different words. We have divided the results into areas: the spelling of the whole word and the spelling of the single letters.
It has been established that if the user expends more than three attempts to move the mark, the objective letter is considered as an error.
Table 1 shows the obtained results for the different users:
|
|
|
|
|
|
WATER | o | o | o | o | o |
FISH | x | o | x | x | o |
SUN | o | o | o | o | o |
DOG | o | o | o | o | o |
MIND | o | o | x | o | o |
Average letter | 78.95% | 100% | 84.21% | 89.47 | 100% |
Average word | 80% | 100% | 60% | 80% | 100% |
Table 1.
Table 2 contains the average of each word by letter and by users. The number inside the parenthesis indicates the total number of letters of each word involving the five sessions (five different users).
|
|
|
WATER (25) | 100% | 100% |
FISH (20) | 65% | 40% |
SUN (15) | 100% | 100% |
DOG (15) | 100% | 100% |
MIND (15) | 90% | 80% |
Table 2.
Average of each word
After establishing different thresholds, it is possible to select the wished letter controlling the movement with the eyes with an average of the 90.53%.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image32.jpeg)
Figure 32.
Eye movements for the word DOG
Figure 32 and Figure 33 present the different movements for the spelling of the words DOG and WATER respectively.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image33.jpeg)
Figure 33.
Eye movements for the word WATER
The following two graphs are presented where the word FISH is correctly spelled (Figure 34) and wrongly spelled (Figure 35).
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image34.jpeg)
Figure 34.
Word FISH well spelled
It can be seen that an exception has happened with the EOG at the time the user was spelling fish.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image35.jpeg)
Figure 35.
Word FISH bad spelled
5.2. ECG
The ECG allows the detection of the principal waves for the posterior analysis of anomalies.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image36.jpeg)
Figure 36.
Normal ECG
The ECG shown in Figure 37 presents a premature ventricular contraction after the T wave.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image37.jpeg)
Figure 37.
ECG for different channels with anomaly
Figure 38 contains an ECG with arrhythmias: the cardiac frequency presents an alteration in the second 83 and the second 86.
![](http://cdnintech.com/media/chapter/43035/1512345123/media/image38.jpeg)
Figure 38.
ECG for different channels with arrhythmias
The three graphs explained above correspond to three different users, showing the channels acquired for the 4 electrodes. The first one presents a normal ECG while the other two present two kinds of anomalies: premature ventricular contraction and arrhythmia.
Comparing the normal ECG with the rest, it can be seen that there are differences between the form of the wave and the RR intervals. In the second graph, there is a beat which is advanced in respect of a normal frequency. Because of that, the second graph presents few numbers of RR intervals (6 beats in 10 seconds).
Regarding to the third graph, the form of the waves is similar than the normal ECG, but the heart frequency is not constant: it can be appreciated that the heart frequency is not constant due to a little pause between the seconds 83 and 86.
6. Conclusions
As it has been demonstrated, the sensors used in BCI have an extra potential for applications which need high sensitivity.
The signals recorded by an EEG, an EOG and an ECG are bioelectrical signals; that is why the techniques to obtain the electrical activity of the brain, the eye movements and the heart activity are very similar. In the three cases, it is necessary to place electrodes in order to measure the biologic potentials.
The electrical activity of the eyes and the heart movement can be better appreciated than the activity from the brain. Because of that, it has been possible to use one EEG device as ECG and EOG as the resolution to distinguish the different signals is less than in an EEG. On the contrary, it could be not possible to use as EEG a device which has not been designed for that.
The election of the type of electrodes that are needed to be used is very important depending on the signal to measure and the needed resolution. The chosen electrodes are good for the recording of an EOG and an ECG, but their resolution may not be sufficient in some EEG applications.
Regarding to the EOG, the users have communicated the different letters with an average of 90.53%. With the appropriate training, the times of spelling could be reduced and it would be possible to increase the hit rate.
As future work there could be three different lines:
Use the EEG for analyzing more bioelectrical signals, for example EMG.
Analyze the signals in real time in order to detect any possible anomaly.
Improve the results of the EOG adding new functions to the interface and check the most adequate trainings to increase the percentage of success.
Acknowledgments
Finally, we would like to thank to those people who has helped to develop the application, especially to Eneko Lopetegui, because his advices and knowledge.
References
- 1.
Graiman B Allison B. Z Pfurtscheller G 2011 - 2.
Galán F Nuttin M Lew E Ferrez P. W Vanacker G Philips J et al 2008 Sep;119 9 2159 69 - 3.
Guger C Edlinger G 2010 462 94 8 - 4.
Sirvent J. L Azorín J. M Iáñez E Úbeda A Fernández E 2011 Apr;8 2 103 11 - 5.
Lin T. A John L. R 2006 - 6.
Purwanto D Mardiyanto R Arai K 2009 Dec 15;14 3 397 400 - 7.
Bear M. F Connors B. W Paradiso M. A 2007 - 8.
Duchowski A. T 2007 - 9.
Muthmainnah N Noor M Ahmad S 2012 3 5 - 10.
Liang S-F Chen Y-H Kuo C-E Chen J-Y Hsu S-C 2011 2104 2107 - 11.
Tagluk M. E Sezgin N Akin M 2010 Aug;34 4 717 25 - 12.
Talukder A Morookian J Monacos S Lam R Lebaw C Bond A 2004 - 13.
Weigle C Banks D. C 2008