Measuring performance and urban impact

Measuring performance and urban impact

From technical delivery to outcome measurement in urban life

After the fountain is installed, new questions are faced by the employer: Does the complex work according to the plan? How do citizens use the surrounding space? Do visitors stay longer? What changes have the surrounding businesses experienced? What is the contribution of the fountain to the recognition of the place?

Answering these questions requires regular information. The opening images, the satisfaction of a few visitors or the cross-sectional increase of the crowd are part of the evidence; A complete evaluation should consider technical quality, user experience, costs, and changes in the surrounding environment together.

This chapter provides a framework for post-operational evaluation; A framework whose results can be used to modify the demonstration program, improve maintenance, and make decisions about future projects.

The indicators, tables and numbers of this chapter are educational. The evaluation method of each project should be coordinated with its objective, scale, resources and available information.

39–1. Defining success based on project goals

Evaluation should start from primary objectives.

If the main goal was to create a pause space in an urban square, the number and duration of the users' stop is important. If the fountain is designed for a commercial center, its relationship with the visiting experience, movement flow and activity of commercial units is also examined.

The goals can be divided into five groups:

In the introduction of OASE public space solutions, the creation of a gathering place, a pause and an urban sign are stated as the goals of the fountain design. These objectives can be turned into evaluation questions; Their realization in each project requires the evidence of the same place. Fountain solutions for public spaces OASE ^[267]

39–2. Separation of activity, output and effect

For a clear evaluation, it is necessary to distinguish between the work done and the result it produced.

For example, daily execution of several shows, the output can be recorded. Increased attendance is an outcome that should be measured. Increasing tourism in the city is also a broader claim and requires the examination of many factors.

The expected chain of the project should be written before operation:

Proper functioning of the fountain → favorable experience of presence → increasing desire to pause and return → possible effects on surrounding activities

At each stage, the necessary assumptions must be specified. Inadequate access, lack of seating or disruption to the show schedule can weaken this relationship.

Place Figure 39-1 — Project evaluation chain

Suggested form: relationship between technical performance, presence experience, user behavior and urban works, together with influencing factors such as accessibility, season and events.

Subtitle: "Each claim of effect must be supported by testable data and assumptions."

39–3. Record the status of the base before implementation

The value of the evaluation increases when there is information about the pre-project situation.

Base data can include:

• The number of entries or passes in certain time periods.

• Number of people pausing.

• Duration of attendance.

• The pattern of using different parts of the space.

• Opinions of users and surrounding businesses.

• Conditions of access and facilities of the place.

• Consumption status and cost in reconstruction projects.

• Simultaneous events and changes.

A single day's record is usually not representative of all conditions. Working day, holiday, season, weather and time of day should be seen in the sampling plan.

If pre-run data is not available, valid records, suitable archival images, existing counters or comparison sites can be used. The limitation of this information should be clear in the report.

The Magenta Book's guidance emphasizes planning the evaluation and collecting appropriate data for outcome and effect evaluation. Magenta Book Assessment Planning Guide ^[268]

39–4. Accurate definition of indicators and data quality

Each index must have a birth certificate.

"The number of passes", "the number of arrivals", "the number of visits" and "the number of unique people" are not the same. The report should name what was actually measured.

Counters and image methods also require accuracy control. Congestion, visibility and round trip counts can change the outcome.

For surveys and observations, the data required should be limited to the purpose of the assessment. Many space usage indicators can be measured without people's registration, face image or contact information.

39–5. Evaluation of technical quality and realization of operation plan

The urban effect of the fountain depends on the quality of its service. Therefore, the evaluation report should also show the technical status.

Appropriate indicators may include:

• The percentage of shows performed compared to the program.

• Hours of stoppage and its reason.

• Number of repeated failures.

• Water quality status within the framework of the approved program.

• The quality of the fountain, the light and the coordination of the show.

• Failure handling time.

• Water and energy consumption in the specified program.

Downtime due to wind, scheduled maintenance and breakdowns should be recorded separately. This separation is essential for analyzing design and operation decisions.

If the visitor encounters a silent fountain at the announced hour, his experience will be affected; Even if the equipment is healthy most of the other hours.

The method of calculating technical and consumption indicators has been discussed in previous chapters. In this chapter, the goal is to link them to real user experience.

39–6. Measuring the use of public space

To check the quality of use, counting the population alone is not enough. It should be determined in which departments people are present, for what activity and for how long.

Visible items are:

• Crossing the range.

• Stop to watch.

• Sitting and talking.

• Photography.

• Waiting for the show to start.

• Authorized use of the interactive part.

• Leaving the premises due to crowding, splashing or difficulty of access.

The observation map can show the location of the accumulation, the less used routes and the points of creating traffic nodes.

UN-Habitat's place-based assessment methods also examine the quality of public space in relation to stakeholder use and participation. This approach can be used to design the evaluation around the fountain. Introducing the location-based evaluation method of public space UN-Habitat ^[269]

Measurement of pause duration

The moment of entering and exiting the pause range should be defined. Average and median attendance can both be useful; Because the long presence of several people may change the average.

Individuals who were present before the start of the observation or who remain after it ends should be handled in a specific way in the analysis.

Figure 39-2 — User behavior map

Suggested image: area plan with assembly signs, crossing route and stopping points at different hours, without identity information of people.

Caption: "The space use map reveals successful locations and areas in need of improvement."

39–7. Survey of users and people who do not use the space

Questions should be short, clear and non-directive.

Examples of appropriate questions:

1. What is your main purpose of being in this area? 2. Before coming, did you know about the existence or program of the water fountain? 3. Approximately how long have you stayed in this range? 4. How do you evaluate the quality of water, light, sound and viewing space? 5. What makes it difficult for you to use this space? 6. What change would improve the attendance experience?

A question like "How much has the beautiful new fountain contributed to the attractiveness of the city?" It directs the answer.

In order to understand the barriers to use, the opinions of the surrounding residents and people who do not use the place are also important. A survey of only the audience present may not cover the experience of other groups.

Time, place, respondent selection method and number of complete responses are recorded. The results of the available sample should not be generalized to the entire population of the city without justification.

39–8. Effect estimation using comparison ranges

Post-opening increases may have coincided with holidays, promotions, improved transportation, or store openings.

One evaluation method is to compare project site changes with a similar range.

Educational example

Let's assume that the average number of daily arrivals in the same intervals is as follows:

Difference of changes:

$\Delta =(1100-800)-(720-600)=180$

In this example, the observed increase in the fountain range is 300 entries, but the difference is achieved by changing the comparison range by 180 entries.

Interpreting this number as the effect of the project depends on assumptions: previous trends must be comparable, no other significant specific change has occurred, and the project has not affected the comparison range. The two averages before and after do not prove these assumptions alone. The difference in differences method in the analytical appendix of Magenta Book ^[270]

A real analysis should use multiple data points, uncertainty analysis and appropriate statistical methods. The number in this example is not a definitive estimate for a real project.

39–9. Measurement of commercial works

In commercial centers or areas with businesses, the following indicators can be checked:

• Number of references to the commercial area.

• Number of buyers or transactions.

• Average purchase amount according to price changes.

• Hours of operation and change of reference pattern.

• What businesses think about customer flow.

• Additional costs associated with exploitation or congestion.

Educational example of the conversion rate of referral to purchase

Suppose in a comparable example:

• Before implementation: 600 clients and 120 buyers.

• After implementation: 800 clients and 160 buyers.

$\mathrm{نرخ\; تبدیل\; پیش\; از\; اجرا}=\frac{120}{600}\times 100=20\%$

$\mathrm{نرخ\; تبدیل\; پس\; از\; اجرا}=\frac{160}{800}\times 100=20\%$

The number of buyers increased, but the conversion rate remained the same in this example.

For such a calculation, the definition of client and buyer and the time range must be coordinated. The number of transactions may also differ from the number of buyers.

The increase in sales should be analyzed with price changes, discounts, seasons and opening of new units. Also, the increase in sales of one area may be a part of the transfer of purchases from another area; This issue is important for evaluating the effect at the city scale.

39–10. Assessing the role of fountains in tourism

Being next to a fountain alone does not determine a person's tourism status.

In the statistical definitions of tourism, travel outside the usual living environment and the distinction between daily visitors and overnight visitors are important. Definitions of tourism in the British Bureau of Statistics guide, based on international recommendations ^[271]

For a local assessment, the following information may be useful:

• Usual place of living, at the geographical level required for study.

• Purpose of travel or attendance.

• Existence or absence of overnight stay during the trip.

• The role of the fountain in choosing a destination or day plan.

• Visiting other attractions.

• A source of familiarity with the fountain.

The role of the fountain can be asked with options such as "main reason", "one of the reasons" and "casual visit".

The answer about motivation is the expressed opinion of the individual and does not alone prove the economic effect. The total cost of the trip should not be attributed to the fountain without a valid method.

Statistics of tours, visit programs and information of tourist centers can be used along with the survey.

39–11. Measuring identity and recognizing a place

To investigate the role of fountain in urban identity, the general concept of "branding" should be converted into specific indicators.

The suggested methods are:

• Ask about the known landmarks of the city, before showing the image of the fountain.

• Checking the recognition of the fountain from the nameless image.

• Asking about the characteristics that users attribute to the place.

• Investigating the use of fountains in the introduction of tourism destinations and programs.

• Studying published content with a clear definition of scope and period.

Unguided recognition and recognition after image presentation produce different results and should be reported separately.

The number of views of a video is not equal to the number of visitors. Advertising content, republishing and algorithmic changes should also be considered in the interpretation of cyberspace indicators.

For a fountain with a cultural theme, you can check if the audience has received the message of the design and what impressions they have of it.

Image location 39-3 — Measure of place recognition

Suggested image: sample of image cards without name and short answer registration form; Choose real images with similar display conditions.

Subtitle: "Spontaneous recognition and recognition after image presentation are two separate indices."

39–12. Examining the environmental consequences and the experience of the neighbors

The evaluation should cover the benefits and problems observed.

Topics to be reviewed include:

• Compensatory water and energy consumption.

• Spraying out of bounds.

• Water accumulation and slippage.

• Sound in sensitive areas.

• Glare and disturbing light.

• Congestion and disturbance in the crossing route.

• Cleanliness and waste.

• Ease of use of the routes and the viewing area.

If the objective of the project is to improve the thermal conditions, the measurement should be done with the appropriate place and time of comparison. Air temperature alone does not explain all dimensions of comfort; Humidity, wind, radiation and attendance conditions are also effective.

The opinion of nearby residents, employees, and businesses should be considered separately from that of the audience. The experience of groups may be different.

Filing complaints is useful, but the number of complaints also depends on people having access to the reporting route. For this reason, observation and direct questioning can provide additional information.

39–13. Compare the cost with the service provided

For cost management, it is possible to define indicators with a clear boundary:

$\mathrm{هزینه\; هر\; ساعت\; خدمت}=\frac{\mathrm{هزینه‌های\; منظورشده\; در\; دوره}}{\mathrm{ساعات\; خدمت\; ارائه‌شده\; در\; همان\; دوره}}$

In the report, it should be specified what expenses are included in the deduction: operation, maintenance, repair or a share of the capital cost.

Also, "the cost of each registered visit" is meaningful only when the calculation method and the intended costs are known. This index is different from the cost per unique visitor.

The social value of public space cannot always be limited to direct income. Free to use, quality of experience and accessibility to different groups should also remain in the report.

In economic calculations, sales increase, owner's income and social benefit are three different concepts. Adding them together without checking for overlap may result in recounting.

Chapter 35 Life Cycle Cost Analysis can be updated with actual data from this step.

39–14. Designing an evaluation schedule

Timing should separate the short-term effect of the opening and the more sustained performance.

The duration of each stage depends on the project. Opening day data is best reported as special event status.

At each turn, the weather, holidays, show schedule, advertisements and activities are recorded simultaneously.

The size of the survey sample should also be determined with the required accuracy, variety of users and sampling method. Increasing the number of responses alone does not remove the bias caused by inappropriate selection of respondents.

39–15. Management dashboard and converted to action

The dashboard should have a limited number of decisionable indicators.

For each indicator, the baseline value, current value, approved target, and data limit are included. "Insufficient data" status should be separate from "Unfavorable performance".

Example turned into action

"Observations show that a significant part of the audience leaves the venue before the show begins. The announced schedule and the interval between the shows should be checked and the result of the change should be measured in the next period.

This action must have a person in charge, a deadline and a method of measuring the result.

Place Figure 39–4 — Sample evaluation dashboard

Suggested form: a page including the display process, pause time, resource consumption and action items; The sample numbers are marked with the "hypothetical" label.

Subtitle: "Evaluation dashboard must be connected to decision and follow-up."

39–16. The structure of the evaluation report and the use of the results

The final report should have the following structure:

1. Project objectives and evaluation questions. 2. Specifications of the fountain and the scope of the study. 3. Collection and scheduling method. 4. Data quality and limitations. 5. Technical performance and costs. 6. Use of space and user comments. 7. Commercial and tourism findings. 8. Identity, access and environmental consequences. 9. Confidence in the conclusions. 10. Suggested actions and follow-up plan.

Claims must be made in proportion to the strength of the evidence. The sentence "in the investigated periods, the number of arrivals has increased" does not require the same evidence as the sentence "the fountain was the reason for the increase in arrivals".

In introducing GSO projects, information on fountain type, equipment and images can be supplemented with documented performance data and post-operational evaluation. Such data should be attributed to the same project, period and measurement method. Introducing projects and urban fountains of GSO ^[272]

Publication of results should include achievements, limitations and necessary corrections. This report can be the basis of the maintenance budget, changing the show program, improving the surrounding area and designing the next fountains.

The final version, together with the aggregated data, definition of indicators and method records, will be kept so that subsequent evaluations can proceed on a comparable basis.