Monitoring and Evaluation as a Way to Complete and to Implement a Policy : The Case of Brazilian National Innovation Policy
(2022) The 14th European Evaluation Society (EES) Biennial Conference- Abstract
- The present National Innovation Policy (henceforth called PNI) was instituted in Brazil by Decree No. 10.537, of October 28, 2020, within the federal public administration. The policy's objective is to guide, coordinate and articulate strategies, programs, and actions across several public entities to foster innovation in the productive sector. Thus, it aims to increase the productivity and competitiveness of companies and other institutions throughout the innovation process. The policy is structured around six axes: funding innovation, innovation culture, technological background, market development, educational systems, and intellectual property. Objectives and guidelines are defined for each of these Axes. However, neither a previous... (More)
- The present National Innovation Policy (henceforth called PNI) was instituted in Brazil by Decree No. 10.537, of October 28, 2020, within the federal public administration. The policy's objective is to guide, coordinate and articulate strategies, programs, and actions across several public entities to foster innovation in the productive sector. Thus, it aims to increase the productivity and competitiveness of companies and other institutions throughout the innovation process. The policy is structured around six axes: funding innovation, innovation culture, technological background, market development, educational systems, and intellectual property. Objectives and guidelines are defined for each of these Axes. However, neither a previous Theory of Change, a Logical Framework, nor a monitoring and evaluation system was established to guide the implementation. Operationally speaking, the policy was implemented by gathering ongoing actions from eighteen different federal ministries and agencies and then assigning them to the axes above. It was an ad hoc measure to put the policy into practice. As can be inferred, the selected actions did not emerge as a breakdown from axes. In 2021 the design and implementation of a monitoring and evaluation model (M&E) were commissioned to an independent evaluation group. The objective was to develop an original methodology and indicators for monitoring and evaluating outputs, outcomes and impacts of the PNI. The challenge of the model is twofold: to adjust the matching amongst actions, axes, and the policy's objectives, and to implement a common framework for M&E able to coordinate the involved agencies. A Theory of Change and a Logical Framework were ex-post designed and validated, and based on these references, the model was conceived. It defined a classical template for monitoring the ongoing actions (using flags and lights according to execution), adding indicators of outcomes and impacts. It is an integrated M&E model that, once implemented, will require that new actions be justified not only in terms of their outputs but also on the expected outcomes and impacts. Standard indicators of science, technology and innovation outcomes and impacts from different sources as for OECD, UNESCO, Global Innovation Index and Sustainable Development Goals (SDG/UN) have been gathered and organized to be employed in the M&E process. Non-standard indicators are also foreseen so that the model can measure internationally comparable and specific indicators. The whole model is now under development and validation among stakeholders. We expect this process to go beyond the domain of M&E and influences the way priorities are defined, and governance is established and run. By involving different actors, incorporating the SDGs into the policy's core, the evaluation process facilitated the policy's implementation and coordination. Furthermore, the articulation between the different stakeholders, promoted by the M&E, minimized fragmentation and uncoordinated actions within the scope of the policy's formulation. It is in this sense that the evaluation process is a way to transform the policy itself (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/69c6f196-dde3-41bd-aa70-af8f898c8f62
- author
- Salles-Filho, Sergio Luiz Monteiro ; Carneiro, Ana Maria ; Spatti, Ana Carolina ; Muraro da Silva, Vinicius LU and Falcini dos Santos, Gabriel
- organization
- publishing date
- 2022-06-09
- type
- Contribution to conference
- publication status
- published
- subject
- keywords
- Innovation, Innovation Policy, Brazil, Theory of Change, Monitoring and Evaluation
- conference name
- The 14th European Evaluation Society (EES) Biennial Conference
- conference location
- Copenhagen, Denmark
- conference dates
- 2022-06-06 - 2022-06-10
- language
- English
- LU publication?
- yes
- id
- 69c6f196-dde3-41bd-aa70-af8f898c8f62
- date added to LUP
- 2022-10-27 17:25:15
- date last changed
- 2022-10-28 10:19:26
@misc{69c6f196-dde3-41bd-aa70-af8f898c8f62,
abstract = {{The present National Innovation Policy (henceforth called PNI) was instituted in Brazil by Decree No. 10.537, of October 28, 2020, within the federal public administration. The policy's objective is to guide, coordinate and articulate strategies, programs, and actions across several public entities to foster innovation in the productive sector. Thus, it aims to increase the productivity and competitiveness of companies and other institutions throughout the innovation process. The policy is structured around six axes: funding innovation, innovation culture, technological background, market development, educational systems, and intellectual property. Objectives and guidelines are defined for each of these Axes. However, neither a previous Theory of Change, a Logical Framework, nor a monitoring and evaluation system was established to guide the implementation. Operationally speaking, the policy was implemented by gathering ongoing actions from eighteen different federal ministries and agencies and then assigning them to the axes above. It was an ad hoc measure to put the policy into practice. As can be inferred, the selected actions did not emerge as a breakdown from axes. In 2021 the design and implementation of a monitoring and evaluation model (M&E) were commissioned to an independent evaluation group. The objective was to develop an original methodology and indicators for monitoring and evaluating outputs, outcomes and impacts of the PNI. The challenge of the model is twofold: to adjust the matching amongst actions, axes, and the policy's objectives, and to implement a common framework for M&E able to coordinate the involved agencies. A Theory of Change and a Logical Framework were ex-post designed and validated, and based on these references, the model was conceived. It defined a classical template for monitoring the ongoing actions (using flags and lights according to execution), adding indicators of outcomes and impacts. It is an integrated M&E model that, once implemented, will require that new actions be justified not only in terms of their outputs but also on the expected outcomes and impacts. Standard indicators of science, technology and innovation outcomes and impacts from different sources as for OECD, UNESCO, Global Innovation Index and Sustainable Development Goals (SDG/UN) have been gathered and organized to be employed in the M&E process. Non-standard indicators are also foreseen so that the model can measure internationally comparable and specific indicators. The whole model is now under development and validation among stakeholders. We expect this process to go beyond the domain of M&E and influences the way priorities are defined, and governance is established and run. By involving different actors, incorporating the SDGs into the policy's core, the evaluation process facilitated the policy's implementation and coordination. Furthermore, the articulation between the different stakeholders, promoted by the M&E, minimized fragmentation and uncoordinated actions within the scope of the policy's formulation. It is in this sense that the evaluation process is a way to transform the policy itself}},
author = {{Salles-Filho, Sergio Luiz Monteiro and Carneiro, Ana Maria and Spatti, Ana Carolina and Muraro da Silva, Vinicius and Falcini dos Santos, Gabriel}},
keywords = {{Innovation; Innovation Policy; Brazil; Theory of Change; Monitoring and Evaluation}},
language = {{eng}},
month = {{06}},
title = {{Monitoring and Evaluation as a Way to Complete and to Implement a Policy : The Case of Brazilian National Innovation Policy}},
year = {{2022}},
}