ISSN: 2375-379X
Health Sciences Research  
Manuscript Information
Integration of Emotional Assessment into E-Health Care Delivery: For Autism Spectrum Disorder: A Review
Health Sciences Research
Vol.4 , No. 6, Publication Date: Sep. 26, 2017, Page: 46-56
470 Views Since September 26, 2017, 160 Downloads Since Sep. 26, 2017

Olumide Obe, Department of Computer Science, Federal University of Technology, Akure, Nigeria.


Folasade Oluwayemisi Akinloye, Department of Computer Science, Federal University of Technology, Akure, Nigeria.


The deployment of emotional assessment into electronic healthcare is gradually finding its way into healthcare domain especially for people with autism spectrum disorder because of their inability to relate their feelings or emotions with people around them or their environment. Therefore the integration of emotional assessment for treatment of Autism Spectrum Disorder (ASD) patient will help improve the understanding of their emotion when feedback to them. This paper is to provide an overview of the state-of-the-art on the subject of emotional assessment in affective computing (AC) incorporated into electronic healthcare technology. The overview includes the basic terminologies and definitions, a survey of existing technologies and recent advancement in the field of Affective computing i.e. affective/emotional technology, the modalities used in gathering data to detect the emotional state of an individual suffering from ASD, e-healthcare delivery and finally, inclusion of emotional assessment in e-healthcare system. This paper also offers a comprehensive number of references for each concept, methods, and applications in the emotional detection in e-healthcare delivery.


ASD, AC, Ambient Technology (AmT), Pervasive Computing, Emotion, E-Health


American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4 ed., Washington, D. C.


A. M. Wetherby, J. Woods, L. Allen, J. Cleary, H. Dickson, & C. Lord. Early indicators of autism spectrum disorders in the second year of life. Journal of Autism and Developmental Disorders, 34, 473-493, 2004.


Centres for Disease Control and Prevention (CDC). 2014. Available Online:


GTbank’s Orange Ribbon Initiative offers Therapy to Autism in Nigeria. 2015 See Available Online: (


M. L. Richins. Measuring Emotions in the Consumption Experience. Journal of Consumer Research. 24: 127–146, 1997.


R. W. Picard. Affective Computing. MIT Press; 2000.


J. M. Jenkins, K. Oatley, N. L. Stein. Human emotions: A Reader. Black-well Publishers; 1998


R. A. Calvo, and S. D’Mello, S. Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing, 1 (1), 18–37, 2010.


P. Ekman, W. Friesen. Unmasking the Face. Prentice-Hall, 1975.


D. Filko and G. Martinovic, “Emotion Recognition System by a Neural Network Based Facial Expression Analysis,” in Automatika - Journal for Control, Measurement, Electronics, Computing and Communications. ATKAFF. vol. 54, no. 2, pp. 263-272, 2013.


T. Hu, L. C. De Silva and K. Sengupta “A hybrid approach of NN and HMM for facial emotion classification,” in Pattern Recognition Letters, vol. 23, no. 11, pp. 1303-1310, 2002.


G. R. Javier, S. David, R. Rahim, M. C. Antonio “Automatic emotion recognition through facial expression analysis in merged images based on Artificial Neural Network”. 12th Mexican International Conference on Artificial Intelligence 978-1-4799-2605-3/13, 2013 IEEE


B. Shruti and N. Pravin, “Emotion Recognition from Facial Expression based on Bezier Curve” International Journal of Advanced Information Technology (IJAIT) Vol. 5, No. 3/4/5/6, 2015.


B. V. Gaurav, R. S. Senjaliya, J. T. Alpesh, V. K. Prajesh, H. J. Hardik “Human Emotional State Recognition Using Facial Expression Detection” Research Inventy: International Journal Of Engineering And Science Issn: 2278-4721, Vol. 2 Issue 2, Pp 42-44, 2013.


J. M. Gorman, J. Martinez, J. D. Coplan, J. Kent, M. Kleber. The effect of successful treatment on the emotional and physiological response to carbon dioxide inhalation in patients with panic disorder”. Biol Psychiatry. Pp 862–867, 2004.[PubMed]


G. Iovane, S. Salerno, P. Giordano, G. Ingenito, & G. R. Mangione, G. R. A computational model for managing emotions and affections in emotional learning platforms and learning experience in emotional computing context. Sixth International Conference on Complex, Intelligent, and Software Intensive Systems (CICIS’12; pp. 873–880), 2012.


A. Luneski, E. Konstantinidis, & P. D. Bamidis. Affective medicine: A review of affective computing efforts in medical informatics. Methods on Information in Medicine, 49 (3), 207–218, 2010.


R. I. García-Betances, G. Fico, D. Salvi, M. Ottviano, and Arredondo M. T. On the Convergence of Affective and Persuasive Technologies in Computer-Mediated Health-Care System. An Interdisciplinary Journal on Humans in ICT Environments Volume 11 (1), 71–93, 2015.


E. Hill, S. Berthoz and U. Frith, Brief report: Cognitive processing of emotions in individuals with autistic spectrum disorder and their relatives, Journal of Autism and Developmental Disorders, vol. 34, no. 2, pp. 229–235, 2004.


G. R. Javier, S. David, R. Rahim, L. Aron, M. C. Antonio, B. Isis “Speech emotion recognition in emotional feedback for Human-Robot Interaction” International Journal of Advanced Research in Artifical Intelligence, Vol. 4, No 2., 2015.


O. Pierre-Yves “The production and recognition of emotions in speech: feature and algorithm” International Journal Human-Computer Studies pp 157-183, 2003.


A. Luneski, P. Bamidis, & M. Hitoglou-Antoniadou. Affective computing and medical informatics: State of the art in emotion aware medical applications, 2008.


H. Hamdi, A. Suteau, & Allain, P. Emotion assessment for affective computing based on physiological responses. IEEE World Congress on Computational Intelligence (WCCI 2012; pp. 1–8), 2012.


M. Z. Poh, T. Loddenkemper, C. Reinsberger, N. C. Swenson, S. Goyal, M. C. Sabtala, and R. W. Picard. Convulsive seizure detection using a wrist-worn electrodermal activity and accelerometry biosensor Epilepsia, 53 (5), e93–e97, 2012.


M. Z. Poh, N. C. Swenson, R. W. Picard. A wearable sensor for unobtrusive, long-term assessment of Electrodermal activity. IEEE Transactions Biomedical Engineering, 57 (5), 1243–1252, 2010.


J. K. Baker, R. M. Fenning, M. Howland, and C. Murakami, I second that emotion: Concordance and synchrony in physiological arousal between children with ASD and theirparents. In A. Esbensen (Chair), Expanding research on family environment: How, who, and when to measure. Symposium presented at the 47th Annual Gatlinburg Conference on Intellectual and Developmental Disabilities. Chicago, IL, 2014.


C. D. Katsis, N. E. Ntouvas, C. G. Bafas, D. I. Fotiadis. Assessment of Muscle Fatigue During Driving Using Surface EMG. Proceedings of the 2nd IASTED International Conference on Biomedical Engineering, BioMED 2004, February 16–18, Innsbruck, Austria.


T. Takahashi, T. Murata, T. Hamada, M. Omori, H. Kosaka, M. Kikuchi, H. Yoshida, Y. L. Wada Changes in EEG and autonomic nervous activity during meditation and their association with personality traits. Int J Psychophysiol. 55: 199–207, 2005 [PubMed]


Picard, R. W., and Healey J. (1997). Affective wearables. Personal Technologies 1 (4), 231–240


D. Moore, P. Mcgrath, and J. Thorpe. Computer-aided learning for people with autism—a framework for research and development. Innovations in Education and Training International 37 (3), 218–228, 2000


J. Klein, Y. Moon and R. W. Picard, This computer responds to user frustration: theory, design and results, Interacting with Computers, vol. 14 (2), pp. 119-140, 2002.


F. T. Nicolás, Tecnologías de Ayuda en Personas con Trastornos del Espectro Autista: Guía Para Docentes. 2004. Available online:


T. Takeo, N. Toshitaka, K. Daisuke, Development application softwares on PDA for autistic disorder children. IPSJ SIG Tech. Rep. 12, 31–38, 2007.


B. Ingersoll, A. Wainer, Initial efficacy of project ImPACT: A parent-mediated social communication intervention for young children with ASD. J. Autism Dev. Disord. 43, 2943–2952, 2013.


N. Aresti-Bartolome and B. Garcia-Zapirain Technologies as Support Tools for Persons with Autistic Spectrum Disorder: A Systematic Review, International Journal Environ. Res. Public Health 11, 7767-7802, 2014.


M. Wang, D. Reid, Virtual reality in pediatric neurorehabilitation: Attention deficit hyperactivity disorder, autism and cerebral palsy. Neuroepidemiology 36, 2–18, 2011.


B. A. Brigadoon An Innovative Online Community for People Dealing with Asperger’s Syndrome and Autism. 2006. Available online:


S. Parsons, A. Leonard, P. Mitchell, Virtual environments for social skills training: Comments from two adolescents with autistic spectrum disorder. Comput. Educ. 47, 186–206, 2006.


N. Josman, H. M. Ben-Chaim, S. Friedrich, P. L. Weiss, Effectiveness of virtual reality for teaching street-crossing skills to children and adolescents with autism. Int. J. Disabil. Hum. Dev. 7, 49–56, 2011.


Marc Fabri SYAEDM. Emotionally Expressive Avatars for Chatting, Learning and Therapeutic Intervention. Available online:


G. Herrera, X. Casas, J. Sevilla, L Rosa, C. Pardo, J. Plaza, Pictogram room : Aplicación detecnologías de interacción natural para el desarrollo del niño con autismo. Anu. Psicol. clínica yla salud 41–46, 2012.


Orang, F. Fundación Orange—E-Mintza. Available online:


M. E. Delano, Improving written language performance of adolescents with asperger syndrome. J. Appl. Behav. Anal. 40, 345–351, 2007


J. A. DeQuinzio, D. B. Townsend, P. Sturmey, C. L. Poulson, Generalized imitation of facial models by children with autism. J. Appl. Behav. Anal. 40, 755–759, 2007.


I. Torii, K. Ohtani, N. Shirahama, T. Niwa, N. Ishii, N., Voice Output Communication Aid Application for Personal Digital Assistant for autistic Children. In Proceedings of the 2012 IEEE/ACIS 11th International Conference on Computer and Information Science, Shanghai, China, 2012


I. Torii, K. Ohtani, T. Niwa, N. Ishii, Development and Study of Support Applications for Autistic Children. In Proceedings of the 2013 14th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, Honolulu, HI, USA 2013.


J. W. Tanaka, J. M. Wolf, C. Klaiman, K. Koenig, J. Cockburn L. Herlihy, C. Brown, S Stahl, M. South, J. C. McPartland, The perception and identification of facial emotions innindividuals with autism spectrum disorders using the Let’s Face It! Emotion skills battery J. Child Psychol. Psychiat. 53, 1259–1267, 2012.


B. O. Ploog, S. Banerjee, P. J. Brooks, Attention to prosody (intonation) and content in childrenwith autism and in typical children using spoken sentences in a computer game. Res. Autism Spectr. Disord 3, 743–758, 2009.


J. Diehl, J., Schmitt, L. M., Villano, M., & Crowell, C. R. The clinical use of robots for individuals with autism spectrum disorders: A critical review. Research in Autism Spectrum Disorders, 6 (1), 249–262, 2012


M. A. Goodrich, M. Colton, B. Brinton, M. Fujiki, J. A. Atherton, L. Robinson, D. Ricks, M. H. Maxfield, A Acerson, A. Incorporating a robot into an autism therapy team. IEEE Intell. Syst, 27, 52–59, 2012.


J. Lee, H. Takehashi, C. Nagai, G. Obinata, D. Stefanov, Which robot features can stimulate better responses from children with autism in robot-assisted therapy? Int. J. Adv. Robot. Syst., 9, 2012.


B. Huskens, R. Verschuur, J. Gillesen, R. Didden, E. Barakova Promoting question-asking in school-aged children with autism spectrum disorders: Effectiveness of a robot intervention compared to a human-trainer intervention. Dev. Neurorehabil, 16, 345–356, 2013.


E. T. Bekele, U. Lahiri, A. R. Swanson, J. A. Crittendon, Z. E. Warren, N. Sarkar, A step towards developing adaptive robot-mediated intervention architecture (ARIA) for children with autism. IEEE Trans. Neural Syst. Rehabil. Eng. 21, 289–299. 2/51128, 2013.


P. Howlin “Children with Autism and Asperger Syndrome: A Guide for Practitioners and Careers”, John Wiley & Sons, 1998.


T. Westeyn “Recognizing Mimicked Autistic Self-Stimulatory Behaviours Using HMMs,”Proc. 9th IEEE Int’l Symp. Wearable Computers (ISWC 05), IEEE CS Press, pp. 164-167, 2005.


A. J. Kientz, G. R. Hayes, T. L. Westeyn, T. Starner, and G. D. Abowd Pervasive Computing and Autism: Assisting Caregivers of Children with Special Needs IEEE Computer Society ¦ 1536-1268/07, 2007


H. Raffler. “Other perspectives on ambient intelligence”, 2006.


J. C. Augusto and P. McCullagh. “Ambient Intelligence: Concepts and Application” ComSIS Vol. 4, No. 1, 2007


A. Allen., (1996) “Home health visits using a cable television network: user satisfaction”, Journal of Telemedicine and Telecare, vol. 2, pp. 92–94, 1996.


C. Lisetti, F. Nasoz, C. Lerouge, O. Ozyer, & K. Alvarez, Developing multimodal intelligent affective interfaces for tele-home health care. International Journal of Human–Computer Studies, 59, 245–255, 2003.


WHO 2005


S. Lindgren, W. David., S. Alyssa, S. Kelly, P. Kelly, K. Todd, L. John, R. Patrick, and W. Debra (2015) “Telehealth and Autism: Treating Challenging Behaviour at Lower Cost” University of Iowa Children’s Hospital, 100 Hawkins Dr, 341A CDD, Iowa City, IA 52242.


S. L. Hepburn, A. Blakeley-Smith, B. Wolff, and J. A. Reaven (2016) “Telehealth delivery of cognitive-behavioural intervention to youth with autism spectrum disorder and anxiety: A pilot study” Autism 2016, Vol. 20 (2) pp 207-218


B. Parmanto, I. W. Pulantara, J. L. Schutte, A. Saptono, M. P. McCue, An integrated telehealth system for remote administration of an adult autism assessment. Telemed. J. E-Health 19, 88, 2013.


E. Hedman, L. Miller, S. Schoen, D. Nielsen, M. Goodwin, R. Picard (2012), “Measuring Autonomic Arousal During Therapy”, Proceedings of 8th International Design and Emotion Conference London


D. K. Christos, G. George and I. F. Dimitrios “An integrated telemedicine platform for the assessment of affective physiological states”. 2006. Avalaible from: Diagnostic pathology


C. Lisetti and C. LeRouge Affective Computing in Tele-home Health: Design Science Possibilities in Recognition of Adoption and Diffusion Issues. Proceedings HICSS, 37th IEEE Hawaii International Conference on System Sciences, Hawaii, USA, 2004.


N. Dahlback, A. Jonsson, and L. Ahrenbert, (eds.) Wizard of Oz Studies - Why and How. Morgan Kaudfan, San Francisco, CA, 1998.


H. Predinger and M. Ishizuka “What Affective Computing and Life-Like Character Technology can do for Tele-Home Health Care” In Workshop on HCI and Homecare: Connecting Families and Clinicians (Online Proceedings) in conj. with CHI, 2003.

  Join Us
  Join as Reviewer
  Join Editorial Board