Standardized educational evaluations from the actors’ experience

Authors

DOI:

https://doi.org/10.1590/S1678-4634202349248451es

Keywords:

Standardized assessment, Uses of assessment, Social experience

Abstract

This article presents a study that approaches, from the sociology of experience, the problematic of the relationship between the actors of the school context and the standardized evaluation devices. Despite the criticisms that these instruments have raised in Chilean society, there are s school principals and local administrators willing to implement them voluntarily. The objective of this study was to investigate the reasons and meanings that they give to the use of a low-stakes evaluation program developed by a prestigious university in Chile, oriented to measure the performance and progress of students in relation to the national curriculum. The design contemplated the execution of a qualitative research based on semi-structured interviews with school principals and administrators of public and private educational networks that use the evaluation program, selected according to purposive sampling criteria (N=33). The information was analyzed based on Grounded Theory guidelines for open coding and, subsequently, interpreted from the sociology of experience. The results revealed the presence of heterogeneous logics of action (integration, strategic and subjectivation) underlying the decisions and uses of the assessment, whose possibilities of articulation lie with the actors and depend to a large extent on the organizational conditions of the establishments, giving rise to different experiences in relation to standardized evaluations. This study identified three types: the academicist experience, relative adherence, and the underutilization of assessments. It concludes by pointing out the implications of this study for educational research, policy and practice.

Downloads

Download data is not yet available.

References

ABARZÚA, Andrea; CONTRERAS, Johana. Sistema de Evaluación de Progreso del Aprendizaje, SEPA: evidencia de su confiabilidad y validez. In: MANZI, Jorge; GARCÍA, María; TAUT, Sandy (org.). Validez de evaluaciones educacionales en Chile y Latinoamérica. Santiago de Chile: UC, 2019. p. 127-156.

AGENCIA DE CALIDAD DE LA EDUCACIÓN. Nuevo sistema nacional de evaluación de aprendizajes: la evaluación al servicio de los aprendizajes. Santiago de Chile: Agencia de Calidad de la Educación, 2018.

ALKIN, Marvin; KING, Jean. The historical development of evaluation use. American Journal of Evaluation, Los Angeles, v. 37, n. 4, p. 568-579, 2016. https://doi.org/10.1177/1098214016665164

» https://doi.org/10.1177/1098214016665164

ASESORIAS PARA EL DESARROLLO. Estudio sobre el funcionamiento de los servicios de Asistencia Técnica Educativa (ATE) chileno: informe final. Santiago de Chile: Ministerio de Educación: PNUD, 2014.

BALL, Stephen John. Subjectivity as a site of struggle: refusing neoliberalism? British Journal of Sociology of Education, Oxford, v. 37, n. 8, p. 1129–1146, 2016. https://doi.org/10.1080/01425692.2015.1044072

» https://doi.org/10.1080/01425692.2015.1044072

BALL, Stephen John. The teacher’s soul and the terrors of performativity. Journal of Education Policy, London, v. 18, n. 2, p. 215-228, 2003. https://doi.org/10.1080/0268093022000043065

» https://doi.org/10.1080/0268093022000043065

BEAVER, Jessica; WEINBAUM, Elliot. State test data and school improvement efforts. Educational Policy, Los Altos, v. 29, n. 3, p. 478-503, 2015. https://doi.org/10.1177/0895904813510774

» https://doi.org/10.1177/0895904813510774

BOURDIEU, Pierre; PASSERON, Jean-Claude. La reproducción: elementos para una teoría del sistema de enseñanza. Madrid: Popular, 2001.

BRAVO, Juan. Simce: Pasado, presente y futuro del sistema nacional de evaluación. Estudios Públicos, Santiago de Chile, n. 123, p. 189-211, 2011. https://doi.org/10.38178/cep.vi123.336

» https://doi.org/10.38178/cep.vi123.336

BRÖCKLING, Ulrich; KRASMANN, Susanne; LEMKE, Thomas. From Foucault’s lectures at the Collège de France to studies of governmentality: an introduction. In: BRÖCKLING, Ulrich; KRASMANN, Susanne; LEMKE, Thomas (org.). Governmentality. New York: Routledge, 2010. p. 9-41. https://doi.org/10.4324/9780203846476-4

» https://doi.org/10.4324/9780203846476-4

BROWN, Chris; ZHANG, Dell. How can school leaders establish evidence-informed schools: an analysis of the effectiveness of potential school policy levers. Educational Management Administration and Leadership, London, v. 45, n. 3, p. 382-401, 2017. https://doi.org/10.1177/1741143215617946

» https://doi.org/10.1177/1741143215617946

CHILE. Ministerio de Educación. Política para el fortalecimiento de la evaluación en el aula. Santiago de Chile: Mineduc, 2018.

CONTRERAS, Johana. Les modes de production des inegalités, les cultures scolaires et les expériences des élèves: une comparaison des systèmes éducatifs francais et chilien. 2014. 581 p. Tesis (Doctorado en Sociología) – École Doctorale: Sociétés, politique, santé publique, Université de Bordeaux, Bordeaux, 2014. Disponible en: https://tel.archives-ouvertes.fr/tel-01152627 Acceso en: 20 oct. 2020.

» https://tel.archives-ouvertes.fr/tel-01152627

DE LA VEGA, Luis; PICAZO, María Inés. La accountability de la calidad y equidad de una política educativa: el caso de la subvención escolar preferencial para la infancia vulnerable en Chile. Reforma y Democracia, Caracas, v. 2016, n. 65, p. 193-224, jun. 2016.

DEMAILLY, Lise. En Europe: l’évaluation contre la crise des systèmes scolaires, l’évaluation en crise. Education et Societes, Paris, v. 17, n. 1, p. 105-120, 2006. https://doi.org/10.3917/es.017.0105

» https://doi.org/10.3917/es.017.0105

DEMAILLY, Lise. L’évaluation de l’action éducative comme apprentissage et négociation. Revue Francaise de Pedagogie, Paris, n. 142, p. 115-130, 2003. https://doi.org/10.3406/rfp.2003.29

» https://doi.org/10.3406/rfp.2003.29

DUBET, François. Les instruments et l’institution: le cas de l’école. Sociologie du Travail, Paris, v. 58, n. 4, p. 381-389, 2016. https://doi.org/10.1016/j.soctra.2016.09.019

» https://doi.org/10.1016/j.soctra.2016.09.019

DUBET, François. Sociologie de l’expérience. Paris: Seuil, 1994.

DUBET, François. The return of society. European Journal of Social Theory, London, p. 1-19, 2020. https://doi.org/10.1177/1368431020950541

» https://doi.org/10.1177/1368431020950541

DUBET, François; MARTUCCELLI, Danilo. En la escuela: sociología de la experiencia escolar. Buenos Aires: Losada, 1998.

DURU-BELLAT, Marie. Le mérite contre la justice. Paris: Presses des Sciences Po, 2009.

EBBELER, Johanna et al. The effects of a data use intervention on educators’ satisfaction and data literacy. Educational Assessment, Evaluation and Accountability, Dordrecht, v. 29, n. 1, p. 83-105, 2017. http://dx.doi.org/10.1007/s11092-016-9251-z

» http://dx.doi.org/10.1007/s11092-016-9251-z

ELACQUA, Gregory et al. Short-run effects of accountability pressures on teacher policies and practices in the voucher system in Santiago, Chile. School Effectiveness and School Improvement, Abingdon, v. 27, n. 3, p. 385-405, 2016. https://doi.org/10.1080/09243453.2015.1086383

» https://doi.org/10.1080/09243453.2015.1086383

FALABELLA, Alejandra. ¿Qué aseguran las políticas de aseguramiento de la calidad? Un estudio de casos en distintos contextos escolares. Estudios Pedagógicos, Valdivia, v. 42, n. 1, p. 107-126, 2016. https://doi.org/10.4067/S0718-07052016000100007

» https://doi.org/10.4067/S0718-07052016000100007

FALABELLA, Alejandra. The ethics of competition: accountability policy enactment in Chilean schools’ everyday life. Journal of Education Policy, London, v. 35, n. 1, p. 23-45, 2020. https://doi.org/10.1080/02680939.2019.1635272

» https://doi.org/10.1080/02680939.2019.1635272

FALABELLA, Alejandra; DE LA VEGA, Luis. Políticas de responsabilización por desempeño escolar: un debate a partir de la literatura internacional y el caso chileno. Estudios Pedagógicos, Valdivia, v. 42, p. 395-413, 2016.

FIMYAR, Olena. Using governmentality as a conceptual tool in education policy research. Educate, London, v. 1, n. 1, p. 3-18, 2006.

FLICK, Uwe. Introducción a la investigación cualitativa. Madrid: Morata, 2004. https://doi.org/M-15.306-2007

» https://doi.org/M-15.306-2007

FLÓREZ, María Teresa; ROZAS, Tamara. Accountability from a social justice perspective: Criticism and proposals. Journal of Educational Change, Berkeley, v. 21, n. 1, p. 157-182, 2020. https://doi.org/10.1007/s10833-019-09361-3

» https://doi.org/10.1007/s10833-019-09361-3

FOUCAULT, Michel. Le sujet et le pouvoir. In: DEFERT, Daniel (org.). Dits et écrits II 1976-1988. Paris: Gallimard, 2001. p. 1041-1062.

GIBBS, Graham. El análisis de datos en investigación cualitativa. Madrid: Morata, 2012.

GOBO, Giampietro. The renaissance of qualitative methods. Forum Qualitative Sozialforschung, Berlin, v. 6, n. 3, p. 1-12, 2005. https://doi.org/10.17169/fqs-6.3.5

» https://doi.org/10.17169/fqs-6.3.5

HEIN, Andreas; TAUT, Sandy. El uso de información evaluativa externa con fines formativos: el caso de establecimientos educacionales chilenos participantes de SEPA. Revista Iberoamericana de Evaluación Educativa, [S. l.], v. 3, n. 2, 160-181, 2010.

KENNEDY, Brianna; THORNBERG, Robert. Deduction, induction, and abduction. In: FLICK, Uwe (org.). The SAGE handbook of qualitative data collection. London: SAGE, 2018. p. 49-64. https://doi.org/10.4135/9781526416070.n4

» https://doi.org/10.4135/9781526416070.n4

LOEB, Susanna; BYUN, Erika. Testing, accountability, and school improvement. Annals of the American Academy of Political and Social Science, Thousand Oaks, v. 683, n. 1, p. 94-109, 2019. https://doi.org/10.1177/0002716219839929

» https://doi.org/10.1177/0002716219839929

MALIN, Joel et al. World-wide barriers and enablers to achieving evidence-informed practice in education: what can be learnt from Spain, England, the United States, and Germany? Humanities and Social Sciences Communications, London, v. 7, n. 1, p. 1-14, 2020. https://doi.org/10.1057/s41599-020-00587-8

» https://doi.org/10.1057/s41599-020-00587-8

MARTUCCELLI, Danilo. Cambio de rumbo: la sociedad a escala del individuo. Santiago de Chile: LOM, 2007.

MARTUCCELLI, Danilo. Variantes del individualismo. Estudios Sociológicos, Ciudad de México, v. 37, n. 109, p. 7-37, 2019.

MARTUCCELLI, Danilo; DE SINGLY, F. L’individu et ses sociologies. Paris: Armand Colin, 2018.

MERLE, Pierre. Les pratiques d’évaluation scolaire. Paris: PUF, 2018.

MONARCA, Héctor. Evaluaciones externas: mecanismos para la configuración de representaciones y prácticas en educación. Buenos Aires: Miño y Dávila, 2015.

MONS, Nathalie. Effets théoriques et réels des politiques d’évaluation standardisée. Revue Francaise de Pedagogie, Paris, n. 169, p. 99-140, 2009. https://doi.org/10.4000/rfp.1531

» https://doi.org/10.4000/rfp.1531

OZGA, Jenny. The politics of accountability. Journal of Educational Change, Berkeley, v. 21, n. 1, p. 19-35, 2020. https://doi.org/10.1007/s10833-019-09354-2

» https://doi.org/10.1007/s10833-019-09354-2

PARCERISA, Lluís; FALABELLA, Alejandra. La consolidación del Estado evaluador a través de políticas de rendición de cuentas: trayectoria, producción y tensiones en el sistema educativo chileno. Education Policy Analysis Archives, Tempe, v. 25, p. 1-24, 2017. https://doi.org/10.14507/epaa.25.3177

» https://doi.org/10.14507/epaa.25.3177

PARRA, Victoria; MATUS, Gladys. Usos de datos y mejora escolar: una aproximación a los sentidos y prácticas educativas subyacentes a los procesos de toma de decisiones. Calidad en la Educación, Santiago de Chile, n. 45, p. 207-250, 2016. https://doi.org/10.4067/s0718-45652016000200007

» https://doi.org/10.4067/s0718-45652016000200007

PERRENOUD, Philippe. La evaluación de los alumnos. Buenos Aires: Colihue, 2008.

PINO, Mauricio; OYARZÚN, Gonzalo; SALINAS, Iván. Crítica a la rendición de cuentas: narrativa de resistencia al sistema de evaluación en Chile. Cadernos Cedes, São Paulo, v. 36, n. 100, p. 337-354, 2016. https://doi.org/10.1590/CC0101-32622016171362

» https://doi.org/10.1590/CC0101-32622016171362

RAMOS-ZINCKE, Claudio. Dispositivo de evaluación y gubernamentalidad del sistema educacional: entretejimiento de ciencia social y poder. Cinta de Moebio, Santiago de Chile, v. 61, p. 41-55, 2018. https://doi.org/10.4067/S0717-554X2017000100041

» https://doi.org/10.4067/S0717-554X2017000100041

RAMOS-ZINCKE, Claudio; FALABELLA, Alejandra. Dispositivo de evaluación educacional y gubernamentalidad en Chile: los orígenes (1844-1970). Cuhso, Temuco, v. 2, n. 30, p. 1-30, 2020. https://doi.org/10.7770/0719-2789.2019.cuhso.03.a07

» https://doi.org/10.7770/0719-2789.2019.cuhso.03.a07

SANTIAGO, José. Repensar la acción social tras la crisis de la idea de sociedad: las promesas de las nuevas sociologías del individuo. In: GATTI, Gabriel; TEJERINA, Benjamín (org.). Pensar la agencia en la crisis. Madrid: Centro de Investigaciones Sociológicas, 2016. p. 103-121.

SCHILDKAMP, Kim. Data-based decision-making for school improvement: research insights and gaps. Educational Research, Berkshire, v. 61, n. 3, p. 257-273, 2019. https://doi.org/10.1080/00131881.2019.1625716

» https://doi.org/10.1080/00131881.2019.1625716

SCHILDKAMP, Kim et al. Factors promoting and hindering data-based decision making in schools. School Effectiveness and School Improvement, Abingdon, v. 28, n. 2, p. 242-258, 2016. https://doi.org/10.1080/09243453.2016.1256901

» https://doi.org/10.1080/09243453.2016.1256901

SMITH, William. National testing policies and educator based testing for accountability. OECD Journal, Paris, v. 2017, n. 1, p. 131-149, 2017. https://doi.org/10.1787/eco_studies-2016-5jg1jxftj4r3

» https://doi.org/10.1787/eco_studies-2016-5jg1jxftj4r3

STOBART, Gordon. Tiempos de pruebas: los usos y abusos de la evaluación. Madrid: Morata, 2010.

STRAUSS, Anselm; CORBIN, Juliet. Bases de la investigación cualitativa: técnicas y procedimientos para desarrollar la teoría fundamentada. Medellín: Universidad de Antioquia, 2002.

SUSPITSYNA, Tatiana. Accountability in american education as a rhetoric and a technology of governmentality. Journal of Education Policy, London, v. 25, n. 5, p. 567-586, 2010. https://doi.org/10.1080/02680930903548411

» https://doi.org/10.1080/02680930903548411

SUSSMAN, Joshua; WILSON, Mark. The use and validity of standardized achievement tests for evaluating new curricular interventions in mathematics and science. American Journal of Evaluation, New York, v. 40, n. 2, p. 190-213, 2019. https://doi.org/10.1177/1098214018767313

» https://doi.org/10.1177/1098214018767313

TANNENBAUM, Richard J.; KANE, Michael T. Stakes in testing: tot a simple dichotomy but a profile of consequences that guides needed evidence of measurement quality. ETS Research Report Series, Princeton, v. 2019, n. 1, p. 1-16, 2019. https://doi.org/10.1002/ets2.12255

» https://doi.org/10.1002/ets2.12255

TOLO, Astrid et al. Intelligent accountability in schools: a study of how school leaders work with the implementation of assessment for learning. Journal of Educational Change, Berkeley, v. 21, n. 1, p. 59-82, 2020. https://doi.org/10.1007/s10833-019-09359-x

» https://doi.org/10.1007/s10833-019-09359-x

VANLOMMEL, Kristin; SCHILDKAMP, Kim. How do teachers make sense of data in the context of high-stakes decision making? American Educational Research Journal, Thousand Oaks, v. 56, n. 3, p. 792-821, 2019. https://doi.org/10.3102/0002831218803891

» https://doi.org/10.3102/0002831218803891

Published

2023-12-22

How to Cite

Standardized educational evaluations from the actors’ experience. (2023). Educação E Pesquisa, 49(contínuo), e248451. https://doi.org/10.1590/S1678-4634202349248451es