(Impact) evaluation and assessment framework - Revision history

Karlijn Schipper: /* Transformative potential */

2020-01-20T08:05:11Z

Transformative potential

← Older revision		Revision as of 10:05, 20 January 2020
Line 57:		Line 57:

	In general, the transformative potential of evaluations and assessments is dependent on their perceived importance by decision-makers and/or their citizens. These approaches are tools to reveal problems and suggest ways forward, but they do not have inherent transformative potential. Provided that they make it to the agendas of urban areas, most approaches in this overview have transformative potential. For example, Food-chain analysis may challenge the power relations between large and small, local producers, IBL reveals how small experiments disturb the status quo and spark sustainability transitions, predictive health systems and the FUG index problematize race or place based discrimination in cities.		In general, the transformative potential of evaluations and assessments is dependent on their perceived importance by decision-makers and/or their citizens. These approaches are tools to reveal problems and suggest ways forward, but they do not have inherent transformative potential. Provided that they make it to the agendas of urban areas, most approaches in this overview have transformative potential. For example, Food-chain analysis may challenge the power relations between large and small, local producers, IBL reveals how small experiments disturb the status quo and spark sustainability transitions, predictive health systems and the FUG index problematize race or place based discrimination in cities.

			Participants of arena#1 in Rotterdam identified the following important aspects and opportunities associated with (Impact) evaluation and assessment frameworks: their ability to inform policy and empower citizens with open data, their ability to create co-learning experiences between different actors, and the importance of having shared and consistent evaluation frameworks across cities/regions.

			The following concerning aspects and challenges of the approach were also discussed: ensuring that the data is as neutral as possible, difficulty measuring qualitative processes with quantitative data methods, limited comparability of findings across different contexts, the development of good and meaningful indicators and clear objectives of the framework, and the possible exclusion of marginalized voices while developing an assessment framework.

	==Illustration of approach==		==Illustration of approach==

Karlijn Schipper: /* Opportunities and Challenges */

2020-01-20T08:04:56Z

Opportunities and Challenges

← Older revision		Revision as of 10:04, 20 January 2020
Line 61:		Line 61:

	'''Initiative Based Learning: Case study of Brixton community energy project in London (From PATHWAYS project)''' <ref> https://www.pathways-project.eu/sites/default/files/Case%20study%20Brixton%20energy.pdf </ref> IBL was used to study the evolution of a cooperatively owned solar energy project in London, England, and the UK’s first inner-city renewable energy co-operative. PATHWAYS studied the gestation, development and implementation of the initial program, and analyzed its potential for replication and transfer across contexts and scales. After the smaller-scale project in Brixton proved successful, it became a not for profit <ref> https://www.repowering.org.uk/ </ref> which is currently working to help other communities develop renewable co-ops. The IBL assessment included great detail on specific actor networks, financing, policy influence, community involvement and so on. Initial replication efforts were mainly peer-to-peer on a smaller scale, while later on the initiative was transferred to a larger scale by pulling information from its smaller projects and feeding it ‘upstream’ into the Repowering Ltd. organization. This case study is a good example of an initiative that connects sustainability and justice in an urban setting, and demonstrates how assessment methods like IBL are used to better understand the role of local initiatives in energy transitions.		'''Initiative Based Learning: Case study of Brixton community energy project in London (From PATHWAYS project)''' <ref> https://www.pathways-project.eu/sites/default/files/Case%20study%20Brixton%20energy.pdf </ref> IBL was used to study the evolution of a cooperatively owned solar energy project in London, England, and the UK’s first inner-city renewable energy co-operative. PATHWAYS studied the gestation, development and implementation of the initial program, and analyzed its potential for replication and transfer across contexts and scales. After the smaller-scale project in Brixton proved successful, it became a not for profit <ref> https://www.repowering.org.uk/ </ref> which is currently working to help other communities develop renewable co-ops. The IBL assessment included great detail on specific actor networks, financing, policy influence, community involvement and so on. Initial replication efforts were mainly peer-to-peer on a smaller scale, while later on the initiative was transferred to a larger scale by pulling information from its smaller projects and feeding it ‘upstream’ into the Repowering Ltd. organization. This case study is a good example of an initiative that connects sustainability and justice in an urban setting, and demonstrates how assessment methods like IBL are used to better understand the role of local initiatives in energy transitions.

	~~==Opportunities and Challenges==~~

	Participants of arena#1 in Rotterdam identified the following important aspects and opportunities associated with (Impact) evaluation and assessment frameworks: their ability to inform policy and empower citizens with open data, their ability to create co-learning experiences between different actors, and the importance of having shared and consistent evaluation frameworks across cities/regions.

	The following concerning aspects and challenges of the approach were also discussed: ensuring that the data is as neutral as possible, difficulty measuring qualitative processes with quantitative data methods, limited comparability of findings across different contexts, the development of good and meaningful indicators and clear objectives of the framework, and the possible exclusion of marginalized voices while developing an assessment framework.


	==References==		==References==

Philipp Spaeth: transferred the feedback from Rotterdam, that Sophia had put in a dedicated secion into the flow of the description.

2019-12-16T10:59:54Z

transferred the feedback from Rotterdam, that Sophia had put in a dedicated secion into the flow of the description.

← Older revision		Revision as of 12:59, 16 December 2019
Line 36:		Line 36:

	The Common Good Matrix, a value-oriented benchmark system developed for the institutional sector (companies, municipalities or other organisations) allows the assessment of their impact on the common good. This assessment tool positions increasing the common good (e.g. quality of life, social justice, environmental sustainability) as a main goal of organizations, rather than purely financial gain. This Matrix has been developed by The Economy for the Common Good, which has been endorsed by the European Economic and Social Committee. While the Matrix appears to be supported by important actors, a foreseeable limitation is that its success hinges on uptake among organizations which are not required to use it.		The Common Good Matrix, a value-oriented benchmark system developed for the institutional sector (companies, municipalities or other organisations) allows the assessment of their impact on the common good. This assessment tool positions increasing the common good (e.g. quality of life, social justice, environmental sustainability) as a main goal of organizations, rather than purely financial gain. This Matrix has been developed by The Economy for the Common Good, which has been endorsed by the European Economic and Social Committee. While the Matrix appears to be supported by important actors, a foreseeable limitation is that its success hinges on uptake among organizations which are not required to use it.

			Further examples of how (Impact) evaluation and assessment frameworks can be used to create sustainable and just cities include: an in-progress framework (included in this [https://www.mdpi.com/2071-1050/11/5/1304/ paper] which is being built to evaluate transformative capacity according to diversity and inclusivity parameters among others, the [https://unalab.eu/home/ UnaLab project] which is developing a common evaluation framework for NBS (based on environmental, social, economic factors) within the Nature4Cities project, and the City of Augsburg's ongoing development of sustainability indicators that are to be mainstreamed into its various City departments.


	==Relation to UrbanA themes: Cities, sustainability, and justice==		==Relation to UrbanA themes: Cities, sustainability, and justice==
Line 59:		Line 62:
	'''Initiative Based Learning: Case study of Brixton community energy project in London (From PATHWAYS project)''' <ref> https://www.pathways-project.eu/sites/default/files/Case%20study%20Brixton%20energy.pdf </ref> IBL was used to study the evolution of a cooperatively owned solar energy project in London, England, and the UK’s first inner-city renewable energy co-operative. PATHWAYS studied the gestation, development and implementation of the initial program, and analyzed its potential for replication and transfer across contexts and scales. After the smaller-scale project in Brixton proved successful, it became a not for profit <ref> https://www.repowering.org.uk/ </ref> which is currently working to help other communities develop renewable co-ops. The IBL assessment included great detail on specific actor networks, financing, policy influence, community involvement and so on. Initial replication efforts were mainly peer-to-peer on a smaller scale, while later on the initiative was transferred to a larger scale by pulling information from its smaller projects and feeding it ‘upstream’ into the Repowering Ltd. organization. This case study is a good example of an initiative that connects sustainability and justice in an urban setting, and demonstrates how assessment methods like IBL are used to better understand the role of local initiatives in energy transitions.		'''Initiative Based Learning: Case study of Brixton community energy project in London (From PATHWAYS project)''' <ref> https://www.pathways-project.eu/sites/default/files/Case%20study%20Brixton%20energy.pdf </ref> IBL was used to study the evolution of a cooperatively owned solar energy project in London, England, and the UK’s first inner-city renewable energy co-operative. PATHWAYS studied the gestation, development and implementation of the initial program, and analyzed its potential for replication and transfer across contexts and scales. After the smaller-scale project in Brixton proved successful, it became a not for profit <ref> https://www.repowering.org.uk/ </ref> which is currently working to help other communities develop renewable co-ops. The IBL assessment included great detail on specific actor networks, financing, policy influence, community involvement and so on. Initial replication efforts were mainly peer-to-peer on a smaller scale, while later on the initiative was transferred to a larger scale by pulling information from its smaller projects and feeding it ‘upstream’ into the Repowering Ltd. organization. This case study is a good example of an initiative that connects sustainability and justice in an urban setting, and demonstrates how assessment methods like IBL are used to better understand the role of local initiatives in energy transitions.

	==~~Contributions from Arena One Participants~~==		==Opportunities and Challenges==

	~~'''Examples of application'''~~

	Examples of how (Impact) evaluation and assessment frameworks can be used to create sustainable and just cities include: an in-progress framework (included in this [https://www.mdpi.com/2071-1050/11/5/1304/ paper] which is being built to evaluate transformative capacity according to diversity and inclusivity parameters among others, the [https://unalab.eu/home/ UnaLab project] which is developing a common evaluation framework for NBS (based on environmental, social, economic factors) within the Nature4Cities project, and the City of Augsburg's ongoing development of sustainability indicators that are to be mainstreamed into its various City departments.

	~~'''Important elements and opportunities'''~~

	While the discussion focused on examples and challenges with the topic, participants identified the following important aspects and opportunities associated with (Impact) evaluation and assessment frameworks: their ability to inform policy and empower citizens with open data, their ability to create co-learning experiences between different actors, and the importance of having shared and consistent evaluation frameworks across cities/regions.

	~~'''Concerning elements~~ and ~~challenges'''~~		Participants of arena#1 in Rotterdam identified the following important aspects and opportunities associated with (Impact) evaluation and assessment frameworks: their ability to inform policy and empower citizens with open data, their ability to create co-learning experiences between different actors, and the importance of having shared and consistent evaluation frameworks across cities/regions.

	~~Participants identified the~~ following concerning aspects and challenges ~~associated with (Impact) evaluation and assessment frameworks~~: ensuring that the data is as neutral as possible, difficulty measuring qualitative processes with quantitative data methods, limited comparability of findings across different contexts, the development of good and meaningful indicators and clear objectives of the framework, and the possible exclusion of marginalized voices while developing an assessment framework.		The following concerning aspects and challenges of the approach were also discussed: ensuring that the data is as neutral as possible, difficulty measuring qualitative processes with quantitative data methods, limited comparability of findings across different contexts, the development of good and meaningful indicators and clear objectives of the framework, and the possible exclusion of marginalized voices while developing an assessment framework.

Sophia Silverton at 15:20, 3 December 2019

2019-12-03T15:20:13Z

← Older revision		Revision as of 17:20, 3 December 2019
Line 58:		Line 58:

	'''Initiative Based Learning: Case study of Brixton community energy project in London (From PATHWAYS project)''' <ref> https://www.pathways-project.eu/sites/default/files/Case%20study%20Brixton%20energy.pdf </ref> IBL was used to study the evolution of a cooperatively owned solar energy project in London, England, and the UK’s first inner-city renewable energy co-operative. PATHWAYS studied the gestation, development and implementation of the initial program, and analyzed its potential for replication and transfer across contexts and scales. After the smaller-scale project in Brixton proved successful, it became a not for profit <ref> https://www.repowering.org.uk/ </ref> which is currently working to help other communities develop renewable co-ops. The IBL assessment included great detail on specific actor networks, financing, policy influence, community involvement and so on. Initial replication efforts were mainly peer-to-peer on a smaller scale, while later on the initiative was transferred to a larger scale by pulling information from its smaller projects and feeding it ‘upstream’ into the Repowering Ltd. organization. This case study is a good example of an initiative that connects sustainability and justice in an urban setting, and demonstrates how assessment methods like IBL are used to better understand the role of local initiatives in energy transitions.		'''Initiative Based Learning: Case study of Brixton community energy project in London (From PATHWAYS project)''' <ref> https://www.pathways-project.eu/sites/default/files/Case%20study%20Brixton%20energy.pdf </ref> IBL was used to study the evolution of a cooperatively owned solar energy project in London, England, and the UK’s first inner-city renewable energy co-operative. PATHWAYS studied the gestation, development and implementation of the initial program, and analyzed its potential for replication and transfer across contexts and scales. After the smaller-scale project in Brixton proved successful, it became a not for profit <ref> https://www.repowering.org.uk/ </ref> which is currently working to help other communities develop renewable co-ops. The IBL assessment included great detail on specific actor networks, financing, policy influence, community involvement and so on. Initial replication efforts were mainly peer-to-peer on a smaller scale, while later on the initiative was transferred to a larger scale by pulling information from its smaller projects and feeding it ‘upstream’ into the Repowering Ltd. organization. This case study is a good example of an initiative that connects sustainability and justice in an urban setting, and demonstrates how assessment methods like IBL are used to better understand the role of local initiatives in energy transitions.

			==Contributions from Arena One Participants==

			'''Examples of application'''

			Examples of how (Impact) evaluation and assessment frameworks can be used to create sustainable and just cities include: an in-progress framework (included in this [https://www.mdpi.com/2071-1050/11/5/1304/ paper] which is being built to evaluate transformative capacity according to diversity and inclusivity parameters among others, the [https://unalab.eu/home/ UnaLab project] which is developing a common evaluation framework for NBS (based on environmental, social, economic factors) within the Nature4Cities project, and the City of Augsburg's ongoing development of sustainability indicators that are to be mainstreamed into its various City departments.

			'''Important elements and opportunities'''

			While the discussion focused on examples and challenges with the topic, participants identified the following important aspects and opportunities associated with (Impact) evaluation and assessment frameworks: their ability to inform policy and empower citizens with open data, their ability to create co-learning experiences between different actors, and the importance of having shared and consistent evaluation frameworks across cities/regions.

			'''Concerning elements and challenges'''

			Participants identified the following concerning aspects and challenges associated with (Impact) evaluation and assessment frameworks: ensuring that the data is as neutral as possible, difficulty measuring qualitative processes with quantitative data methods, limited comparability of findings across different contexts, the development of good and meaningful indicators and clear objectives of the framework, and the possible exclusion of marginalized voices while developing an assessment framework.


	==References==		==References==

Lin at 21:06, 21 November 2019

2019-11-21T21:06:37Z

← Older revision		Revision as of 23:06, 21 November 2019
Line 69:		Line 69:
	[[Category: DESAFIO]]		[[Category: DESAFIO]]
	[[Category: FOODMETRES]]		[[Category: FOODMETRES]]
			[[Category: PULSE]]

Lin at 21:06, 21 November 2019

2019-11-21T21:06:05Z

← Older revision		Revision as of 23:06, 21 November 2019
Line 63:		Line 63:

	[[Category: Approaches]]		[[Category: Approaches]]
			[[Category: (Impact) evaluation and assessment framework]]
	[[Category: Regeneration of disused urban space]]		[[Category: Regeneration of disused urban space]]
	~~[[Category: Alternative indicators]]~~
	[[Category: Policies and practices for inclusion of disadvantaged groups]]		[[Category: Policies and practices for inclusion of disadvantaged groups]]
	[[Category: Sustainable Food Supply Chains]]		[[Category: Sustainable Food Supply Chains]]
			[[Category: DESAFIO]]
			[[Category: FOODMETRES]]

Sophia Silverton at 08:25, 8 November 2019

2019-11-08T08:25:53Z

← Older revision		Revision as of 10:25, 8 November 2019
Line 7:		Line 7:
	==General introduction to approach==		==General introduction to approach==

	While each evaluation/assessment method discussed here is highly unique, they share several commonalities. Each one includes an in-depth research on their issue and its context. This could include research on governance arrangements and the impacts of certain events, system change, development patterns etc. The assessments are carried out using case studies, pilot projects, and in one case, big-data. Each assessment framework is mainly aimed at helping policymakers by: producing policy recommendations, decision-support tools, public-use data repositories, scenarios, enhancing policy learning, predicting and mitigating risks, and even changing norms in policy making. Each assessment method has a strong focus on environmental sustainability, and a varying degree of justice considerations.		While each evaluation/assessment method discussed here is highly unique, they share several commonalities. Each one includes in-depth research on their issue and its context. This could include research on governance arrangements and the impacts of certain events, system change, development patterns etc. The assessments are carried out using case studies, pilot projects, and in one case, big-data. Each assessment framework is mainly aimed at helping policymakers by: producing policy recommendations, decision-support tools, public-use data repositories, scenarios, enhancing policy learning, predicting and mitigating risks, and even changing norms in policy making. Each assessment method has a strong focus on environmental sustainability, and a varying degree of justice considerations.

	<u>Note:</u> Two approaches, Socio-technical strategy assessment (DESAFIO, 2013-2015)<ref> https://cordis.europa.eu/project/rcn/106722/reporting/en </ref> and Impacts quantification of globalization (GLOBAL-IQ, 2011-2014)<ref> https://cordis.europa.eu/project/rcn/99713/reporting/en </ref>, are worth noting but do not sufficiently cover UrbanA concepts of sustainability and justice in urban settings to be further elaborated upon here. See references for more information about the projects.		<u>Note:</u> Two approaches, Socio-technical strategy assessment (DESAFIO, 2013-2015)<ref> https://cordis.europa.eu/project/rcn/106722/reporting/en </ref> and Impacts quantification of globalization (GLOBAL-IQ, 2011-2014)<ref> https://cordis.europa.eu/project/rcn/99713/reporting/en </ref>, are worth noting but do not sufficiently cover UrbanA concepts of sustainability and justice in urban settings to be further elaborated upon here. See references for more information about the projects.

Karlijn Schipper at 13:29, 1 November 2019

2019-11-01T13:29:53Z

← Older revision		Revision as of 15:29, 1 November 2019
Line 9:		Line 9:
	While each evaluation/assessment method discussed here is highly unique, they share several commonalities. Each one includes an in-depth research on their issue and its context. This could include research on governance arrangements and the impacts of certain events, system change, development patterns etc. The assessments are carried out using case studies, pilot projects, and in one case, big-data. Each assessment framework is mainly aimed at helping policymakers by: producing policy recommendations, decision-support tools, public-use data repositories, scenarios, enhancing policy learning, predicting and mitigating risks, and even changing norms in policy making. Each assessment method has a strong focus on environmental sustainability, and a varying degree of justice considerations.		While each evaluation/assessment method discussed here is highly unique, they share several commonalities. Each one includes an in-depth research on their issue and its context. This could include research on governance arrangements and the impacts of certain events, system change, development patterns etc. The assessments are carried out using case studies, pilot projects, and in one case, big-data. Each assessment framework is mainly aimed at helping policymakers by: producing policy recommendations, decision-support tools, public-use data repositories, scenarios, enhancing policy learning, predicting and mitigating risks, and even changing norms in policy making. Each assessment method has a strong focus on environmental sustainability, and a varying degree of justice considerations.

	<u>Note:</u> Two approaches, Socio-technical strategy assessment (DESAFIO, 2013-2015)<ref> https://cordis.europa.eu/project/rcn/106722/reporting/en </ref> and Impacts quantification of globalization (GLOBAL-IQ, 2011-2014)<ref> https://cordis.europa.eu/project/rcn/99713/reporting/en </ref>, are worth noting ~~in this cluster~~ but do not sufficiently cover UrbanA concepts of sustainability and justice in urban settings to be further elaborated upon here. See references for more information about the projects.		<u>Note:</u> Two approaches, Socio-technical strategy assessment (DESAFIO, 2013-2015)<ref> https://cordis.europa.eu/project/rcn/106722/reporting/en </ref> and Impacts quantification of globalization (GLOBAL-IQ, 2011-2014)<ref> https://cordis.europa.eu/project/rcn/99713/reporting/en </ref>, are worth noting but do not sufficiently cover UrbanA concepts of sustainability and justice in urban settings to be further elaborated upon here. See references for more information about the projects.

	==Shapes, sizes and applications==		==Shapes, sizes and applications==

Sophia Silverton at 15:24, 30 October 2019

2019-10-30T15:24:07Z

← Older revision		Revision as of 17:24, 30 October 2019
Line 1:		Line 1:

			Research on sustainable and just urban areas has involved a variety of evaluation and assessment methods. This cluster summarizes a sample of these methods, which have been employed in the study of the following topics: environmental conflict in coastal urban areas, food supply chains, transitions to sustainable and low-carbon societies, environmental public health risks, distribution of green amenities, and common good contributions by companies and other organizations. What unites these approaches is not their substantive content, but rather the innovative, fruitful, and transferable methods that can be employed in many contexts.

	This page is part of an ongoing, open-ended online collaborative database, which collects relevant approaches that can be used by city-makers to tackle unsustainability and injustice in cities. It is based mainly on knowledge generated in EU-funded projects and touches on fast changing fields. As such, this page makes no claims of authoritative completeness and welcomes your suggestions.		This page is part of an ongoing, open-ended online collaborative database, which collects relevant approaches that can be used by city-makers to tackle unsustainability and injustice in cities. It is based mainly on knowledge generated in EU-funded projects and touches on fast changing fields. As such, this page makes no claims of authoritative completeness and welcomes your suggestions.

Sophia Silverton: /* General introduction to approach */

2019-10-30T15:23:25Z

General introduction to approach

← Older revision		Revision as of 17:23, 30 October 2019
Line 5:		Line 5:
	==General introduction to approach==		==General introduction to approach==

	Research on sustainable and just urban areas has involved a variety of evaluation and assessment methods. This cluster summarizes a sample of these methods, which have been employed in the study of the following topics: environmental conflict in coastal urban areas, food supply chains, transitions to sustainable and low-carbon societies, environmental public health risks, distribution of green amenities, and common good contributions by companies and other organizations. What unites these approaches is not their substantive content, but rather the innovative, fruitful, and transferable methods that can be employed in many contexts.		While each evaluation/assessment method discussed here is highly unique, they share several commonalities. Each one includes an in-depth research on their issue and its context. This could include research on governance arrangements and the impacts of certain events, system change, development patterns etc. The assessments are carried out using case studies, pilot projects, and in one case, big-data. Each assessment framework is mainly aimed at helping policymakers by: producing policy recommendations, decision-support tools, public-use data repositories, scenarios, enhancing policy learning, predicting and mitigating risks, and even changing norms in policy making. Each assessment method has a strong focus on environmental sustainability, and a varying degree of justice considerations.

	While each evaluation/assessment method is highly unique, they share several commonalities. Each one includes an in-depth research on their issue and its context. This could include research on governance arrangements and the impacts of certain events, system change, development patterns etc. The assessments are carried out using case studies, pilot projects, and in one case, big-data. Each assessment framework is mainly aimed at helping policymakers by: producing policy recommendations, decision-support tools, public-use data repositories, scenarios, enhancing policy learning, predicting and mitigating risks, and even changing norms in policy making. Each assessment method has a strong focus on environmental sustainability, and a varying degree of justice considerations.

	<u>Note:</u> Two approaches, Socio-technical strategy assessment (DESAFIO, 2013-2015)<ref> https://cordis.europa.eu/project/rcn/106722/reporting/en </ref> and Impacts quantification of globalization (GLOBAL-IQ, 2011-2014)<ref> https://cordis.europa.eu/project/rcn/99713/reporting/en </ref>, are worth noting in this cluster but do not sufficiently cover UrbanA concepts of sustainability and justice in urban settings to be further elaborated upon here. See references for more information about the projects.		<u>Note:</u> Two approaches, Socio-technical strategy assessment (DESAFIO, 2013-2015)<ref> https://cordis.europa.eu/project/rcn/106722/reporting/en </ref> and Impacts quantification of globalization (GLOBAL-IQ, 2011-2014)<ref> https://cordis.europa.eu/project/rcn/99713/reporting/en </ref>, are worth noting in this cluster but do not sufficiently cover UrbanA concepts of sustainability and justice in urban settings to be further elaborated upon here. See references for more information about the projects.