Incomplete block design as an alternative for project evaluation in governmental and non-governmental organizations

Authors

DOI:

https://doi.org/10.21640/ns.v11i22.1867

Keywords:

incomplete block design, projects evaluation, projects for scientific research, experimental design statistical, calls for the allocation of financial resources

Abstract

Introduction: The transparent evaluation of projects proposals for scientific research with financial support through calls made by governmental institutions or private organizations becomes extremely important, along with increasing confidence in the researchers who participate in the calls for the allocation of financial resources. Therefore, this research presents an alternative to the process of evaluating projects with financial resources. The current method of evaluating funded projects is to obtain an average of the scores assigned to each project by a group of reviewers. However, this method does not discount the effect of the reviewer and produces a bias in the final evaluation of each project, since projects are likely to be favored when reviewers are relaxed and affected when reviewers are demanding and very strict.               

Method: This research proposes the use of two experimental design statistical methods for project evaluation: (a) incomplete balanced block design and (b) Alpha block design. In addition, we provide codes in the statistical software R, for both statistical designs, which allows us to implement project evaluation quickly and accurately.                  

Results: Based on the proposed methods, it was found that the traditional method presents a very important bias due to the heterogeneity among the reviewers.                     

Discussion and Conclusion: The use of experimental design statistical methods (balanced incomplete block design or Alpha design) can reduce this bias considerably, obtaining a higher accuracy for evaluations.

Downloads

Download data is not yet available.

Author Biographies

Osval Antonio Montesinos López, Universidad de Colima

Facultad de Telemática

Francisco Javier Luna Vázquez, Universidad de Colima

Facultad de Telemática

Emeterio Franco Pérez, Universidad de Colima

Escuela de Mercadotecnia

Alberto Pastrana Palma, Universidad Autónoma de Querétaro

Facultad de Contaduría y Administración, División de Estudios de Posgrado

Raymundo Buenrostro Mariscal, Universidad de Colima

Facultad de Telemática

References

Campos-Arceiz, A., Primack, R., & Koh, L. P. (2015). Reviewer recommendations and editors’ decisions for a conservation journal: Is it just a crapshoot? And do Chinese authors get a fair shot? Biological Conservation, 186, 22-27. doi:http://dx.doi.org/10.1016/j.biocon.2015.02.025

Bates, D., Máchler, M., Bolker, B., & Walker, S. (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67, 1--48. doi:10.18637/jss.v067.i01

CONACYT. (2015). Convocatoria 2016 - Proces de evaluación de las propuestas y selección para apoyo. Proceso de evaluación de las propuestas y selección para apoyo, Consejo Nacional de Ciencia y Tecnologia. Obtenido de https://www.conacyt.gob.mx/index.php/sni/convocatorias-conacyt/convocatorias-programa-de-estimulos-a-la-innovacion/convocatoria-2016/11036-proceso-de-evaluacion-2016/file

CONACYT. (2016). Evaluación 2016 y Preguntas frecuentes. Correo, Consejo Nacional de Ciencia y Tecnologia. Obtenido de https://www.conacyt.gob.mx/index.php/sni/convocatorias-conacyt/convocatorias-programa-de-estimulos-a-la-innovacion/convocatoria-2016/10586-evaluacion-2016-preguntas-frecuentes/file

De Mendiburu, F. (2017). agricolae: Statistical Procedures for Agricultural Research. Obtenido de https://CRAN.R-project.org/package=agricolae

ELSEVIER. (2009). Home. Recuperado el 08 de octubre de 2018, de https://www.elsevier.com/reviewers/what-is-peer-review

ELSEVIER. (2018). https://www.elsevier.com/reviewers/what-is-peer-review. Recuperado el 19 de octubre de 2018

González- Hernández, I. J. (2006). Diseños experimentales de bloques incompletos y aplicaciones en la industria. Pachuca de Soto, Hidalgo, México: Instituto de ciencias básicas e ingeniería. Recuperado el 11 de Octubre de 2017

Maj Gen A.K, .. (April de 2016). Peer review' for scientific manuscripts: Emerging issues, potential threats, and possible remedies. Medical Journal Armed Forces India, 72(2), 172-174. doi:https://doi.org/10.1016/j.mjafi.2016.02.014

Mead, R. (2010). The design of experiments. New York, USA: Cambridge University Press. doi:ISBN 9780521287623

Mendoza, H., & López, L. A. (2001). Estimación de datos faltantes en bloques incompletos balanceados conectados. Revista Colombiana de Estadística, 24(2), 73-89. Obtenido de http://www.redalyc.org/articulo.oa?id=89924201

Muhammad , A., & Wahida, S. (2018). The Application of Algebraic Methods in Balanced. Journal of Physics:, 1028(1), 012123. doi:10.1088/1742-6596/1028/1/012123

Publishing Research Consortium. (2016). Publishing Research Consortium Peer review survey 2015. 58. Obtenido de https://www.elsevier.com/__data/assets/pdf_file/0007/655756/PRC-peer-review-survey-report-Final-2016-05-19.pdf

Sailer, M. O. (2013). crossdes: Construction of Crossover Designs. Obtenido de https://CRAN.R-project.org/package=crossdes

SINECYT. (2012). Código de ética del sistema nacional de evaluación científica y tecnológica. Obtenido de CONACYT: 2006-2012.conacyt.gob.mx/registros/sinecyt/Documents/CODIGO_ETICA.pdf

Spier, R. (2002). The history of the peer-review process. TRENDS in Biotechnology, 20(8), 357-358. doi:doi.org/10.1016/S0167-7799(02)01985-6

Wickham, H., & Henry, L. (2017). tidyr: Easily Tidy Data with 'spread()' and 'gather()' Functions. Obtenido de https://CRAN.R-project.org/package=tidyr

Wickham, H., Francois, R., Henry, L., & Müller, K. (2017). dplyr: A Grammar of Data Manipulation. Obtenido de https://CRAN.R-project.org/package=dplyr

Zuckerman, H., & Merton, R. (1972). Patterns of Evaluation in Science:Institutionalisation, Structure and Functions of the Referee System. Minerva, 9, 66-100. doi:https://doi.org/10.1007/BF01553188]

Published

2019-05-29

How to Cite

Montesinos López, O. A., Luna Vázquez, F. J., Franco Pérez, E., Pastrana Palma, A., & Buenrostro Mariscal, R. (2019). Incomplete block design as an alternative for project evaluation in governmental and non-governmental organizations. Nova Scientia, 11(22), 433–477. https://doi.org/10.21640/ns.v11i22.1867

Issue

Section

Human and Social Sciences

Metrics

Similar Articles

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 > >> 

You may also start an advanced similarity search for this article.