Assessment by telematic means and artificial agents: A new challenge for psychometrics?

Daniela Conti, Santo Di Nuovo

Accepted April 28, 2023

First published April 28, 2023

https://doi.org/10.26387/bpa.2023.00006

Abstract

During the COVID-19 restrictions, the administration of psychodiagnostic tools not through direct
interaction with the psychologist, but administered electronically and/or with artificial agents, opened a new challenge. A change of setting and interactive conditions are introduced which can alter the reliability and validity of tools consolidated for diagnostic use in face-to-face assessments. A sample of 122 licensed psychologists participated in the study. They were either attending or teaching post-graduate specialization courses of different theoretical-methodological focus. The participants were given an online survey via Google forms that included a questionnaire composed of 21 items on a 5-point scale, built according to the UTAUT model and adapted for the acceptability and willingness to use online questionnaires for mental well-being, and a semantic differential for evaluating the attitude towards technology in general. The overall attitude of the psychologists toward the innovative modalities of assessment is positive, and the intention to use online testing is very high. The predictors of the positive attitude and intention of use are analyzed. A multidimensional analysis suggested that the attitude towards online testing, and the intention to use it, are located in the crossing dimensions of technical (psychometric) aspects and concrete usability. The administration of tests electronically or through artificial agents requires adaptation studies and in many cases a reformulation of the tools that are offered using these modalities.
The need to raise awareness amongst psychologists about the new forms of assessment, and to train those who intend
to use them, has be underlined.

References

BAIG, M.Z. & KAVAKLI, M. (2019). A survey on psycho-physiological analysis & measurement methods in multimodalsystems. Multimodal Technologies and Interaction, 3 (2), 37.
BREARLY, T.W., SHURA, R.D., MARTINDALE, S.L., LAZOWSKI,R.A., LUXTON, D.D., SHENAL, B.V. & ROWLAND, J.A. (2017).Neuropsychological test administration by videoconference: Asystematic review and meta-analysis. Neuropsychology Review,27 (2), 174-186.
BUCHANAN, T. (2002). Online assessment: Desirable or dangerous?Professional Psychology: Research and Practice, 33 (2), 148.
BUKIE, O.F. (2014). Understanding technologies for e-assessment:A systematic review approach. Journal of Emerging Trends inComputing and Information Sciences, 5 (12), 936-947.
CONTI, D., COMMODARI, E. & BUONO, S. (2017). Personalityfactors and acceptability of socially assistive robotics in teacherswith and without specialized training for children with disability.Life Span and Disability, 20 (2), 251-272.
CONTI, D., DI NUOVO, S., BUONO, S. & DI NUOVO, A. (2017).Robots in education and care of children with developmentaldisabilities: A study on acceptance by experienced and futureprofessionals. International Journal of Social Robotics, 9, 51-62.
CORTELLESSA, G., FRACASSO, F., SORRENTINO, A.,ORLANDINI, A., BERNARDI, G., CORACI, L., DEBENEDICTIS, R. & CESTA, A. (2018). ROBIN, a telepresencerobot to support older users monitoring and social inclusion:Development and evaluation. Telemedicine and E-Health, 24 (2),145-154.
CULLUM, C.M., HYNAN, L.S., GROSCH, M., PARIKH, M. &WEINER, M.F. (2014). Teleneuropsychology: Evidence for videoteleconference-based neuropsychological assessment. Journal ofthe International Neuropsychological Society: JINS, 20 (10), 1028.
DE GRAAF, M.M.A., BEN ALLOUCH, S. & VAN DIJK, J.A.G.M.(2019). Why would I use this in my home? A model of domesticsocial robot acceptance. Human-Computer Interaction, 34 (2),115-173.
DINUOVO,A.,VARRASI,S.,LUCAS,A.,CONTI,D.,McNAMARA,J. & SORANZO, A. (2019). Assessment of cognitive skills viahuman-robot interaction and cloud computing. Journal of BionicEngineering, 16, 526-539.
DINUOVO,S.F.&NARZISI,V.(2021).Telepsychology:Acceptabilityof online assessment and treatment in users and professionalpsychologists. Mediterranean Journal of Clinical Psychology, 9(3).
FISKE,A.,HENNINGSEN,P.&BUYX,A.(2019).Yourrobottherapistwill see you now: ethical implications of embodied artificialintelligence in psychiatry, psychology, and psychotherapy.Journal of Medical Internet Research, 21 (5), e13216.
IVANOVIC, M. & JAIN, L.C. (2013). E-learning paradigms andapplications: Agent-based Approach (Vol. 528). Springer.
OSGOOD, C.E., SUCI, G.J. & TANNENBAUM, P.H. (1957). Themeasurement of meaning (Issue 47). University of Illinois Press.
PRESTON, J.A. & SHACKELFORD, R. (1999). Improving on-lineassessment: An investigation of existing marking methodologies.Proceedings of the 4th Annual SIGCSE/SIGCUE ITiCSEConference on Innovation and Technology in Computer ScienceEducation, 29-32.
RICHES, S., AZEVEDO, L., VORA, A., KALEVA, I., TAYLOR, L.,GUAN, P., JEYARAJAGURU, P., McINTOSH, H., PETROU, C.& PISANI, S. (2022). Therapeutic engagement in robot‐assistedpsychological interventions: A systematic review. ClinicalPsychology & Psychotherapy, 29 (3), 857-873.
ROSSI, S., CONTI, D., GARRAMONE, F., SANTANGELO, G.,STAFFA, M., VARRASI, S. & DI NUOVO, A. (2020). The roleof personality factors and empathy in the acceptance andperformance of a social robot for psychometric evaluations.Robotics, 9 (2), 39.
ROSSI, S., SANTANGELO, G., STAFFA, M., VARRASI, S., CONTI,D. & DI NUOVO, A. (2018). Psychometric evaluation supportedby a social robot: Personality factors and technology acceptance.2018 27th IEEE International Symposium on Robot and HumanInteractive Communication (RO-MAN), 802-807.
SCASSELLATI, B., ADMONI, H. & MATARIC, M. (2012). Robots foruse in autism research. Annual Review of Biomedical Engineering,14, 275-294.
doi.org/10.1146/annurev-bioeng-071811-150036
SORA, B., NIETO, R., DEL CAMPO, A.M. & ARMAYONES, M.(2021). Acceptance and use of telepsychology from the clients’́́perspective: Questionnaire study to document perceivedadvantages and barriers. JMIR Mental Health, 8 (10), e22199.
TAKAHASHI, H., BAN, M. & ASADA, M. (2016). Semanticdifferential scale method can reveal multi-dimensional aspects ofmind perception. Frontiers in Psychology, 7, 1717.
VARRASI, S., LUCAS, A., SORANZO, A., McNAMARA, J. & DINUOVO, A. (2019). IBM cloud services enhance automaticcognitive assessment via human-robot interaction. In G.Carbone, M. Ceccarelli & D. Pisla (Eds.), New trends in medicaland service robotics. Springer International Publishing.
WADSWORTH, H.E., DHIMA, K., WOMACK, K.B., HART JR.J.,WEINER, M.F., HYNAN, L.S. & CULLUM, C.M. (2018).Validity of teleneuropsychological assessment in older patientswith cognitive disorders. Archives of Clinical Neuropsychology, 33(8), 1040-1045.
WAINER, H., DORANS, N.J., FLAUGHER, R., GREEN, B.F. &MISLEVY, R.J. (2000). Computerized adaptive testing: A primer.Routledge.

SHOW ALL REFERENCES (24)HIDE REFERENCES

DOWNLOAD PDF

Issue:

Issue 296

Keywords:

Views:

1147

Downloads:

Cite the article:

Conti Daniela . Di Nuovo Santo . Assessment by telematic means and artificial agents: A new challenge for psychometrics?. BPA Applied Psychology Bulletin. 2023;296(1):40-52. doi:10.26387/bpa.296.1.

CITE THE ARTICLE

7685

Assessment by telematic means and artificial agents: A new challenge for psychometrics?

Abstract

References

Article info

Issue:

Keywords:

Views:

Downloads:

Cite the article:

BPA - BULLETIN OF APPLIED PSYHOLOGY

MANAGEMENT AND EDITORIAL OFFICE

ABOUT US

Citation tool

How to cite this article