This section provides an overview of the general process and methods used to develop the UESI framework and selection of indicators. Specific details regarding the calculation of indicators are included in each individual issue chapter and more information is also provided in the Methodological Appendix.
As of January 2020, the UESI includes 162 cities, from all continents and from a range of levels of development, that contain over 11,000 thousand districts.
The UESI Framework
The Index includes five categories of environmental concerns: Air Quality, Climate Change, Water and Sanitation, Urban Ecosystem, and Transportation (Figure 1). Each city is gauged on environmental performance indicators within these issues as well as how different demographic group (e.g., by income and sensitive populations) are affected. For instance, neighborhoods within each city receive scores for the amount of urban tree cover canopy as well as how equitable the access to the green space is. All data are made open and available through an online portal, which provides an interactive dashboard for citizens, policymakers, and urban managers to explore in more detail where they perform well and which areas may require greater attention.
Figure 1. Categories of environmental issues and indicators assessed in the Urban Environment and Social Inclusion Index (UESI).
The UESI uses primary and secondary data primarily from large-scale remote sensing datasets and open-source geospatial data, such as OpenStreetMap. We also collect census neighborhood and block-level data on population and income. Secondary datasets used in the UESI are restricted to the water stress indicator, which uses the Nature Conservancy‘s Urban Water Blueprint data to assess surface water stress (see Water Resource Management issue profile and Metadata for more details). The UESI applies a set of criteria to determine which datasets to select for inclusion (see Box 1: Guiding Principles for Indicator Selection in the UESI).
All sources of data are publicly available and include:
- Satellite data from remote sensing;
- Geospatial datasets, such as gridded global population1Center for International Earth Science Information Network – CIESIN – Columbia University. 2017. Gridded Population of the World, Version 4 (GPWv4): Population Density, Revision 10. Palisades, NY: NASA Socioeconomic Data and Applications Center (SEDAC). https://doi.org/10.7927/H4DZ068D. Accessed 12/8/2017., location of public transit stops2OpenStreetMap Foundation (OSMF) and OpenStreetMap contributors. Open Street Map. Retrieved from: https://www.openstreetmap.org/#map=4/38.01/-95.84.; and economic data.3Kummu, M., Taka, M., & Guillaume, J. H. (2018). Gridded global datasets for gross domestic product and Human Development Index over 1990–2015. Scientific data, 5, 180004.
- Official statistics measured and formally reported by governments through a census or other data collection effort.
Tiers of Calculation Methodology
For the socioeconomic, we prioritized the use of census and statistical income data, but for some cities, this data was not possible to obtain. To overcome this problem, primarily for the Equity calculations, we used a proxy for neighborhood-level income data, which is Gross Domestic Product (GDP) data from a globally generated dataset (Kummu et. al. 2018).4Kummu, M., Taka, M., & Guillaume, J. H. (2018). Gridded global datasets for gross domestic product and Human Development Index over 1990–2015. Scientific data, 5, 180004. While not equivalent to income per capita data, it is considered the best available proxy. For the sake of transparency, the UESI classifies cities as Tier I (i.e., cities that have income per capita data available) and Tier II (i.e., cities where we use GDP per capita) to designate potential differences when comparing these cities’ equity indicators. We encourage Tier II cities to get in touch with us with income per capita data if it is available.
Box 1: Guiding Principles for Indicator Selection in the UESI
The index incorporates spatially-explicit indicators as much as possible given data availability.
Because people do not experience cities uni-dimensionally, the UESI aims to incorporate spatial data as much as possible to examine patterns, trends, and differences in environmental performance throughout and between urban areas. Spatially-explicit indicators also expose variation within and between cities. For example, land surface temperature has been connected to population density 5Gémes, O., Tobak, Z., & Van Leeuwen, B. (2016). Satellite based analysis of surface urban heat island intensity. Journal of environmental geography, 9(1-2), 23-30., although this relationship is varied between cities 6Stone, B., Hess, J. J., & Frumkin, H. (2010). Urban form and extreme heat events: are sprawling cities more vulnerable to climate change than compact cities?. Environmental health perspectives, 118(10), 1425.. Identifying these differences can help urban environmental managers identify distributional impacts of various environmental policies. By combining spatially-explicit environmental and socioeconomic data, the UESI features indicators that allow us to assess how different communities are affected by environmental hazards and benefits.
2) Incorporate equity
We adopt equity and social justice as applicable end goals for our measurement framework and indicators, meaning we strive for our tools and analysis to shed light on socioeconomic drivers, patterns, and differences that can lead to disproportionate levels of environmental quality and performance. Exposing these differences requires detailed spatial data and we will continue seeking data and examples to determine how each city fares in terms of equity. In this sense, the commitment to equity echoes point 1 above.
3) Build on and support SDG 11 and other global goals
SDG 11 sets a goal to make cities “inclusive, safe, resilient, and sustainable,” which informs the design of our indicator framework as well as the selection of indicators. Because we aim for this tool to be useful for policymakers to track progress towards SDG 11 and other international policy goals, we have aligned our indicators and framework as much as possible.
Table 1. Sustainable Development Goal indicators related to the environmental metrics included in the UESI. Source: UN Sustainable Development Knowledge Platform.7UN Sustainable Development Knowledge Platform. Available: https://sustainabledevelopment.un.org/.
4) Focus on outcome, not process, indicators
Targeting policymakers, we designed our framework and indicators to measure outcomes where possible, allowing local managers to address areas of needed improvement in locally-sensitive ways. With the exception of the Climate Policy indicator, UESI indicators do not measure policy responses such as public expenditure on roads.
5) Reproducibility and transparency
We provide a clear and detailed report on the chosen datasets, methodology, framework, and indicators. Access to data in easily accessible formats will be important in ensuring transparency, reproducibility, and relationship building. Ensuring the data is accessible allows individuals and institutions to access the aggregated data to investigate their own research questions. All data are available on our web portal.
Constructing the UESI
The UESI framework was developed through a multi-step stakeholder engagement process that involved input from academic researchers, urban planners, practitioners, and city representatives. The starting point for the environmental issues we selected was drawn from the Environmental Performance Index (EPI)8Hsu et al. (2016). The 2016 Environmental Performance Index. Yale University: New Haven, CT. Available: www.epi2016.yale.edu., which is a biennial ranking that evaluates national environmental performance in 9 high-priority issue areas (e.g., Air and Water Quality, Biodiversity and Habitat Protection, Forests, Fisheries, Agriculture, Climate Change and Energy, Environmental Health). We then conducted a comprehensive literature review9Thomas, R., Hsu, A., and A. Weinfurter. Sustainable and Inclusive – Do existing urban sustainability indicators help measure progress towards SDG-11? Environment and Planning B: Urban Analytics and City Science, in review. of existing urban environmental sustainability metrics to evaluate prior work and where we could add value. This process also involved reviewing the environmental justice and social equity literature from a multi-disciplinary angle (see Equity and Social Inclusion issue profile). Expert practitioners and academic researchers were involved in the entire framework design and indicator selection process. In the end, the resulting UESI framework and indicators represent a trade-off between a scientific underpinning and available data, as we require neighborhood-scale data for most of the indicators (excluding the Climate Policy and Water indicators that are not applicable at the neighborhood scale).
Filtering primarily ‘non-urban’ neighborhoods
Due to the need to incorporate both environmental and socioeconomic data for the analysis, the UESI used the officially determined administrative divisions of a city as the spatial attributes for the calculation of the spatially explicit environmental indicator and later integration with its corresponding population and income information. However, it was identified that in some cases the defined boundaries of a city did not necessarily correspond exclusively to an urban area, including non-urban areas such as sparsely populated areas rural sections or even conservation areas.
As a result of this preliminary analysis, and due to the need to maintain a certain level of homogeneity within the districts analyzed within a city, we filter these neighborhoods from the analysis, particularly from the equity analysis. The filtering process was done initially using a clustering process based on a combination of land cover (Urban vs Non-urban land cover), population, income and extension data, to identify potentially groups of similar non-urban districts which were later manually reviewed and filtered out. By the end of this preliminary analysis we filtered 23 districts from all the UESI cities in cities such as Las Vegas, Oslo, Panama City, Salt Lake City and some others.
Use of Google Earth Engine for Computation
To scale up our analysis from the original 32 pilot cities to the 162 in this version of the UESI, we moved all our indicator calculations to Google Earth Engine, a cloud-computing platform for global scale geospatial analysis.10Gorelick, Noel, et al. “Google Earth Engine: Planetary-scale geospatial analysis for everyone.” Remote sensing of Environment 202 (2017): 18-27. This allows us to not only leverage Google’s computational capacity to process global data, but also provides a replicable method for incorporating additional cities in the future.
We present the UESI data in multiple formats recognizing that not all users experience information in the same way. Raw values in native units provide a largely unadulterated view of how neighborhoods and cities perform. Transformed values using a “proximity to target” (Box 2: Measuring Environmental Performance: Proximity-to-Target) method assesses how close or far neighborhoods and cities are to achieving an identified policy target.
The targets are high performance benchmarks defined primarily by international or national policy goals or established scientific thresholds. The benchmark for exposure to fine particulate pollution, for instance is 10 μg/m3, a threshold set by the World Health Organization as safe exposure.11World Health Organization. (2006). WHO Air quality guidelines for particulate matter, ozone, nitrogen dioxide and sulfur dioxide-Global update 2005-Summary of risk assessment, 2006. Geneva: WHO. A high-performance benchmark can also be determined through an analysis of the best-performing countries. Some of our indicators set benchmarks, for example, at the 95th percentile of the range of data. Scores are then converted to a scale of 0 to 100 by simple arithmetic calculation, with 0 being the farthest from the target and 100 being the closest (Box 2). In this way, scores convey analogous meaning across indicators, policy issues, and throughout the UESI.
Although the EPI aggregates the proximity-to-target scores into a single, weighted index that applies a series of statistical weights to each indicator and policy issue, the UESI does not combine all of the metrics into a single score. We keep the UESI scores disaggregated and sorted by issue based on the diversity of environmental performance observed at the neighborhood scale, and because the cities in our pilot effort represent a range of cities at different levels of development. Our aim in selecting these cities was to illustrate the range of urban sustainability solutions and challenges, not to “name and shame” or score cities. Future iterations of the UESI may provide an aggregate ranking, but at this stage the UESI intends to open a conversation to explore more in-depth variation in environmental performance and social inclusion throughout cities, which all stand to improve in each of the UESI’s categories.
Box 2: Measuring Environmental Performance: Proximity-to-Target
There are many ways to summarize and transform raw data to make it comparable. Targets are set by policy goals (e.g., in the case of the Tree Cover per capita target that uses a UN SDG goal of 15 meters per capita), established scientific thresholds (e.g., in the case of the PM2.5 indicator that uses the World Health Organization’s 10 microgram/m3 limit for exposure), or an analysis of the top performers (e.g., the top 5th percentile of the distribution of scores). Each indicator is transformed given a score from a scale of 0 (worst performer or those at the low performance benchmark) to 100 (best performer or those at the top performance benchmark).
Figure 2. Illustration of the proximity to target methodology (Source: adapted from Hsu et al., 2013).12Hsu, A., Johnson, L. A., & Lloyd, A. (2013). Measuring progress: a practical guide from the developers of the environmental performance index (EPI). New Haven: Yale Center for Environmental Law & Policy.