Nicolás Vizioli,
Alejandro Pagano
Universidad de Buenos Aires, Buenos Aires, Argentina



The present study takes a tour of the concept of reliability as one of the fundamental psychometric properties in Classical Tests Theory. The concept and its different practical applications are developed to investigate the degree of reliability of a measuring instrument. Focusing on the calculation of internal consistency from alpha and omega as the most used coefficients and the importance of calculating these coefficients by using polychoric correlation matrices (PCM). The main objective is to present a guide in Spanish for the calculation of ordinal reliability coefficients using the R/Rstudio program. Providing an example at the empirical level that shows the relevance of calculating this type of coefficients for calculating the reliability of an instrument. Using a sample of 266 adults between the ages of 18 and 63 years (M = 31.91 SD = 11.50), the version adapted to Argentina of the Beck Anxiety Inventory and the Emotional Regulation Questionnaire were administered. In this way, coefficients are exposed to estimate the reliability of the instruments that account for their advantages and disadvantages, performing the calculation using MCP, Pearson’s correlation matrix and Pearson’s covariance matrix. From the results, it is evident that the calculation using MCP provided higher degrees of reliability compared to the calculation using the other two matrices. This document is expected to be of importance to researchers unfamiliar with R.

Keywords: Reliability, Single Administration, Alpha, Omega, Polychoric Correlations Matrix, R/Rstudio



Aguirre-Urreta, M. I., Rönkkö, M., & McIntosh, C. N. (2019). A cautionary note on the finite sample behavior of maximal reliability. Psychological Methods, 24(2), 236-252.

Armor, D. J. (1973). Theta reliability and factor scaling. Sociological Methodology, 5, 17-50.

Beck, A. T., Epstein, N., Brown, G., & Steer, R. A. (1988). An inventory for measuring clinical anxiety: Psychometric properties. Journal of Consulting and Clinical Psychology, 56(6), 893–897.

Bernaards, C. A., & Jennrich, R. I. (2005). Gradient projection algorithms and software for arbitrary rotation criteria in factor analysis. Educational and Psychological Measurement, 65, 676-696.

Browne, M. W., & Cudeck, R. (1993). Alternative ways ofassessing model fit. En K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162). Sage.

Catalán, H. E. N. (2019). Reliability, population classification and weighting in multidimensional poverty measurement: A Monte Carlo study. Social Indicators Research, 142(3), 887-910.

Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, (1), 98-104.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.

Deng, L., & Chan, W. (2017). Testing the difference between reliability coefficients alpha and omega. Educational and Psychological Measurement, 77(2), 185-203.

Domínguez-Lara, S. A. (2012). Propuesta para el cálculo del Alfa Ordinal y Theta de Armor. Revista de Investigación en Psicología, 15(1), 213-217.

Domínguez-Lara, S. A. (2016). Evaluación de la confiabilidad del constructo mediante el Coeficiente H: breve revisión conceptual y aplicaciones. Psychologia. Avances de la Disciplina, 10(2), 87-94.

Domínguez-Lara, S. (2018). Fiabilidad y alfa ordinal. Actas Urológicas Españolas, 42(2), 140-141.

Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problema of internal consistency estimation. British Journal of Psychology, 105(3), 399-412.

Elosua, P., & Zumbo, B. (2008). Coeficientes de fiabilidad para escalas de respuesta categórica ordenada. Psicothema, 20(4), 896-901.

Flora, D. B. (2020). Your coefficient alpha is probably wrong, but which coefficient omega is right? A tutorial on using R to obtain better reliability estimates. Advances in Methods and Practices in Psychological Science, 3(4), 484-501.

Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9(4), 466-491.

Fox, J., & Bouchet-Valat, M. (2019). Rcmdr: R Commander. R package version 2.5-2.

Gadermann, A. M., Guhn, M., & Zumbo, B. D. (2012). Estimating ordinal reliability for Likert-type and ordinal item response data: A conceptual, empirical, and practical guide. Practical Assessment, Research, and Evaluation, 17(1), 3.

Green, S. B., & Yang, Y. (2015). Evaluation of dimensionality in the assessment of internal consistency reliability: Coefficient alpha and omega coefficients. Educational Measurement: Issues and Practice, 34(4), 14-20.

Gross, J. J., & John, O. P. (2003). Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being. Journal of Personality and Social Psychology, 85(2), 348-362.

Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10(4), 255-282.

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2014). Multivariate data analysis (7th ed.). Pearson Education Limited Harlow.

Hancock, G. R., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. En R. Cudeck, S. du Toit, & D. Soerbom (Eds.), Structural equation modeling: Present and future—A festschrift in honor of Karl Jöreskog (pp. 195-216). Scientific Software International.

Jackson, P. H., & Agunwamba, C. C. (1977). Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: I: Algebraic lower bounds. Psychometrika, 42(4), 567-578.

Jöreskog, K. G. (1994). On the estimation of polychoric correlations and their asymptotic covariance matrix. Psychometrika, 59(3), 381-389.

Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y. (2020). semTools: Useful tools for structural equation modeling (R package Version 0.5-3).

Kelley, K., & Lai, K. (2012). MBESS: MBESS. R package version 3.3.2.

Kline, R. B. (2011). Principles and practice of structural equation modeling. Guilford Press.

Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Addison Wesley.

Lorenzo-Seva, U., & Ferrando, P. (2020). Manual of the program FACTOR v. 10.

Manterola, C., Quiroz, G., Salazar, P., & García, N. (2019). Metodología de los tipos y diseños de estudio más frecuentemente utilizados en investigación clínica. Revista Médica Clínica Las Condes, 30(1), 36-49.

McDonald, R. P. (1981). The dimensionality of tests and items. British Journal of mathematical and statistical Psychology, 34(1), 100-117.

McDonald, R.P. (1999). Test theory: A unified treatment. Erlbaum.

McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412-433.

Mikulic, I. M., (2007). Construcción y adaptación de pruebas psicológicas [Manuscrito inédito]. Facultad de Psicología, Universidad de Buenos Aires.

Moltner, A., & Revelle, W. (2015). Find the Greatest Lower Bound to Reliability.

Muñiz, J. (2010). Las teorías de los tests: teoría clásica y teoría de respuesta a los ítems. Papeles del Psicólogo, 31(1), 57-66.

Pagano, A. E., & Vizioli, N. A. (en prensa). Adaptación del Cuestionario de Regulación Emocional (ERQ) en población adulta de la Ciudad Autónoma de Buenos Aires y el Conurbano Bonaerense. Psicodebate. Psicología, Cultura y Sociedad.

R Development Core Team. (2020). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing.

Racine, J. S. (2011). RStudio: A Platform-Independent IDE for R and Sweave. Journal of Applied Econometrics, 27(1), 167–172.

Raykov, T. (1998). Coefficient alpha and composite reliability with interrelated nonhomogeneous items. Applied Psychological Measurement, 22(4), 375-385.

Raykov, T. (2012). Scale development using structural equation modeling. En Hoyle, R. (Ed.), Handbook of structural equation modeling (pp. 472-492). Guilford Press.

Raykov, T., & Marcoulides, G. A. (2019). Thanks coefficient alpha, we still need you! Educational and psychological measurement, 79(1), 200-210.

Revelle, W. (1979). Hierarchical cluster analysis and the internal structure of tests. Multivariate Behavioral Research, 14(1), 57-74.

Revelle, W. (2011). An overview of the psych package.

Revelle, W. (2020). Package ‘psych’.

Revelle, W. (2021). How To: Use the psych package for Factor Analysis and data reduction.

Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145-154.

Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354.

Rosseel, Y. (2012). lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software, 48(2), 1–36.

Rosseel, Y. (2020). The lavaan tutorial.

Savalei, V., Reise, S. P., Vazire, S., & Fried, E. (2019). Don’t Forget the Model in Your Model-based Reliability Coefficients: A Reply to McNeish (2018). Collabra: Psychology, 5(1).

Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107.

Sijtsma, K., & van der Ark, L. A. (2015). Conceptions of reliability revisited and practical recommendations. Nursing Research, 64(2), 128-136.

Thurstone, L. L. (1928). Attitudes can be measured. American Journal of Sociology, 33(4), 529-554.

Trizano-Hermosilla, I., & Alvarado, J. M. (2016). Best alternatives to Cronbach’s alpha reliability in realistic conditions: congeneric and asymmetrical measurements. Frontiers in Psychology, 7, Article 769.

Ventura-León, J. L. (2018). ¿Es el final del alfa de Cronbach? Adicciones, 31(1), 80-81.

Viladrich, C., Angulo-Brunet, A., & Doval, E. (2017). Un viaje alrededor de alfa y omega para estimar la fiabilidad de consistencia interna. Anales de Psicología, 33(3), 755-782.

Vizioli, N. A., & Pagano, A. E. (2020). Adaptación del Inventario de Ansiedad de Beck en población de Buenos Aires. Interacciones, e171-e171.

Zumbo, B. D., Gadermann, A. M., & Zeisser, C. (2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods, 6(1), 4.

Zumbo, B. D., & Kroc, E. (2019). A measurement is a choice and Stevens’ scales of measurement do not help make it: A response to Chalmers. Educational and Psychological Measurement, 79(6), 1184-1197.