Large scale public order police operations are a regular occurrence in almost every country, both in the context of sporting events and protest demonstrations.
Historically, investigations on the use of force by police have been conducted only by Commissions and interested groups as a response to a specific incident or series of publicized events. Well-known incidents that created an uproar and led to civil disturbances and police reform in the USA include the beating death of Arthur McDuffie in Miami (1979) and the beating of Rodney King (1991) in Los Angeles, and the beating death of Malice Green in Detroit (1996). As a consequence of these and other events, police administrators and researchers have looked critically and comprehensively at the use of force, its justifications, levels, and methods. (Smith & Alpert). Over the years, a number of these after-the-fact enquiries have been conducted all over the world (in the USA e.g. Kerner Commission, 1968; Eisenhower Commission, 1969; Scranton Commission, 1970; in the UK e.g. Popplewell, 1986; Scarman, 1981; Taylor 1990); in the Netherlands e.g. Nationale Ombudsman, 1993; COT, 1993, 1999; Enschedé, 1967). On occasion, people responsible for the management of public order were sacked (as happended e.g. after the recent riots during the G8 conference in Genoa) or had to resign after the conclusions of such an enquiry were presented, thus emphasising the potential for 'in the job trouble' (Waddington, 1998) resulting from public disorder.
The findings of the different enquiries contain many lessons and recommendations, not least for the policing of public order events. To avoid similar mistakes in the future, it is of course important to learn from previous experiences. Della Porta and Reiter (1998) point out that the police learn by analysing their failures: changes and learning processes of the police are initiated by an analysis of problematic public order interventions. However, Das (1984) noted that most training course in riot control in the US did not take into account lessons learned from past police riot control actions. Jefferson (1990) observed the operation of special patrol groups who deal with escalating public order incidents in the UK and noted that "critical reflection upon specific incidents was missing". In the Netherlands, Dijkhuis (1982) and Adang (1990) drew attention to the lack of systematic evaluations following public order incidents, a point which was corroborated by several police working groups as well (references in Adang, 1990). However, it is not only when things go wrong and incidents occur that lessons may be learnt (Adang, 1992). In fact, for purposes of organisational learning, it would be in the interest of the police to evaluate all instances of public order policing to be able to identify good practice and to exchange these with other police forces, as there are considerable variations in the frequency, variety and severity of the public disorder that different police forces are called on to manage. To achieve that aim, it is important for the police to develop its own evaluation capacity and not to be dependent on occasional external evaluations, where accountability for the past and fault finding often takes precedence over learning for the future. In this paper, we want to introduce a tested model for the police evaluation of public order management.
Adang (1994) proposed the introduction of evaluation teams to evaluate large scale police operations. The idea was to involve experienced police officers from other forces in the evaluation of large scale police operations. These officers would not have to investigate events after the fact, but would be involved in observing police operations in real time. In this way, the evaluation would not have to focus on incidents, but on the professional handling of public events. Having experienced police officers observe police deployment and the interaction of the police with the public was thought to have several advantages. The police force conducting the operation would be able to receive well-informed, independent feed-back in a safe way. Officers involved in the evaluation teams would have the opportunity to gain a lot of additional experience by observing how another force deals with the management of public order. By involving trainers as well, any lessons learned can be included in future training more easily.First experiments were carried out in 1994 and 1996 in the Netherlands (Adang & van Dam, 1996; Adang, 1997). On the basis of these experiences, a general model for the evaluation of large scale police operations in general was developed (Adang, 2000) and applied in practice during the Euro 2000 football championships held in Belgium and the Netherlands (Adang & Cuvelier, 2001). In the remainder of this paper, both the early experiments and the general model (and its application during Euro 2000) will be summarised.
Experiment 1: The Hague, 1994
On November 5, 1994 a large demonstration by students took place in The Hague. It was a demonstration with a past. A year earlier, a similar demonstration had led to a violent confrontation between the police and protesters, and a lot of negative publicity for the police. Three reports were published that were critical of the police operation (Adang & Standaar, 1993; COT, 1993; Nationale Ombudsman, 1993). The police in the Hague subsequently reconsidered its public order management practices and developed a "Plan for the policing of public manifestations". The demonstration of November 1994 was therefore a special one: it was felt the police had to prove it could handle such a big event properly. Both students and politicians needed to regain confidence in the police. As a result of that pressure, it was decided to pay extra attention to the evaluation of the event. The goal of the evaluation was to give a factual account of what had happened, to draw lessons for the future and facilitate internal and external accountability. Irrespective of the outcome of the event, it was agreed beforehand that every commanding officer should make an evaluation report.
At the pre-planning stage, an evaluation coordinator was appointed. During the event, police videoteams recorded the operation at his request. In addition, he recruited four teams of observers. Each team was composed of one police officer from the Hague and one external observer. Two external observers were provided by the Police Institute Public order and Safety (a national training institute); two others were provided by the organisers of the demonstration. Each observation team had complete freedom of movement. During a preparatory meeting, the teams agreed to record factual events, rather than their personal interpretations and judgements. It was stressed that the main aim of the evaluation was to promote learning and not to punish individuals. The observers also agreed that their role as observers precluded them from intervening, verbally or otherwise, in the course of events. All commanding officers were made familiar with the evaluation procedure, the observation teams and the use of external observers. Commanding officers were given the opportunity to propose specific points to be included in the evaluation. Some of them made use of this opportunity.The observation teams started their work at the day of the demonstration during the briefings of units. They continued their work troughout the day until the demonstrators (some 30.000) had left the Hague. All observations were recorded directly on portable audi-recorders or on paper. After the end of the observations, the observers gathered and reported their impressions to the evaluation coordinator. Three days later, each team sent a written observation report to the evaluation coordinator. The evaluation coordinator compiled all reports (both from the observation teams and the commanding officers) into an evaluation report for the Overall Commander of the operation.All observers were enthusiastic and very positive about their participation and the way in which they were able to observe events. The police officers from the Hague who took part in the observation teams, some of whom had many years of experience with large-scale police operations, reported it as quite a special experience. For them, it was a new and refreshing way to look at (the effects of) a police operation.
Experiment 2: Rotterdam, 1996
In 1996 several individuals, involved in large scale police operations in one way or another, gathered to discuss ways to implement evaluation teams more widely. In several brainstorm sessions, conditions were formulated which would make it possible for such a team to function. A high risk football match was chosen as an opportunity to test the concept further. Observers were recruited and a plan was put in writing. This plan was discussed with the Overall Commander of the operation. Two weeks prior to the match a training session was held. Observers were instructed about the proper way to conduct structured observations in general and during large scale police operations in particular. The observers assisted in writing down an observation plan, which was put to the Overall Commander.On the day of the match, six observers (divided into three teams) conducted observations according to the plan. They recorded their observations directly. Five of the observers were police officers: three from the Hague, two from Rotterdam (the first author was the sixth observer). As soon as the fans had dispersed after the match, the observers met and exchanged their impressions. Five days later, all observers had written observation reports. Together, the observers put togehether a chronology of observed events and an inventory of evaluation points. These were handed over to the Overall Commander. He indicated that the feed-back of the evaluation teams contained valuable information. He later used the evaluation report of the evaluation team to compile his own report.
Without exception, the observers experienced their activities as an "eye-opener". They reported that, when acting as a commanding officer within their own force, most of their attention necessarily had to go to their own organisations and units. Now they could focus on other aspects and saw other things, especially related to the interaction between the police and the public. In addition, seeing how colleagues in another force manage public order helped to reflect on one's own approach. The evaluation team formulated several points that they considered vital for a poper functioning of the teams. Clear arrangements and clear goals were considered important as well as a clear committment and involvement of the police force concerned. A good training and instruction of observers is essential. At the core of the concept is the fact that the feedback is being given by colleagues.
The model
Based on the results of the experiments, a general model was developed. As key elements of the model, several points of departure were formulated.
1.Responsability
Every organisation is responsible for its own evaluation, and ultimately this responsibility lies with the competent authority. Within each organisation, every commanding officer needs to evaluate the perfomance of the units under his command and to identify lessons to be learned for future actions. Every police organisation should have minimum standards for the evaluation of public order policing by those directly involved in the operation. If others (colleagues, trainers, researchers, members form the public) are involved it has to be clear under whose responsibility the evaluation will be done and how it will be reported.
2.Aim of the evaluation
The point of departure of any evaluation should be its aim, which should be determined in advance. Formulating the aim is essential to focus the evaluation. If the aim is not clearly formulated (as often occurs), it is not possible to focus the process of data gathering and it will be more difficult to draw meaningful conclusions. In determining the aim of the evaluation, one must distinguish between procedures at micro-level (the event itself and its approach), at meso-level (regarding the organisation in general, detached from the specific event) and at macro-level (the social context of the organisation and of the events). It is also important to be aware of the phasing in time: before, during and after the event. In principle, the evaluation could include elements from the decision-making and preparation up to the actual realisation and the results achieved. Depending on the focus of the evaluation, several standard questions will need to be dealt with, viz.:
-was the action performed as intended and agreed on?
-did the action have the assumed effect on the expected problems?
-which unforeseen circumstances occurred and why?
-how was this dealt with?
3.Structure of the evaluation
The minimum standards should include structured debriefings, which should form part of the operational plans. Time, form and contents of the debriefing are to be determined. This approach is especially appropriate to compare the plan and the execution at the micro-level as striking elements come to the fore even without the oriented gathering of information on specific points.The following items need to be arranged beforehand to structure the debriefing process:
-during the operation, each unit keeps a log, starting at the briefing;
-a time is set at which feedback resulting from the debriefing should be given;
-a structure for the debriefing is given.
In practice, it is often a good method to go through the operation in chronological order (starting at the briefing) by means of the approach plan (for the unit concerned) and the log. The first goal is to become clear on whether everybody has the same view on what actually happened. Combining information from different sources may complete each individual's knowledge. In case of widely differing views, a potential area of growth has been identified;- written feedback should be given to the evaluation co-ordinator or Overall Commander with the following information: unit, commander, time and place deployment, time of debriefing, nature of the actual operation, use of force, arrests made, persons injured, damage, functional plan of action, departures from plan of action, moot points and/or areas of growth, proposals for modification;The debriefing is also a kind of aftercare, and a way to establish where further emotional after-care is needed. It is also importnat for the evaluation to be part of a long-term learning process and that all lessons drawn from the evaluations are reported in a recoverable manner for the benefit of training and future operations.
4.Involvement of colleagues
Involving colleagues from outside police forces in the evaluation of large-scale police operations potentially has several advantages. Ideally, participants in the evaluation have several years of experience in large-scale operations, have a function at a tactical or strategic level in their own organisation and have followed a specific training preparing them for a review among colleagues. However, others may also be deployed on the basis of specific expertise as necessary. For a smooth proceeding of the review among colleagues it is important to set some starting points:
-the review among colleagues does not replace regular briefings, debriefings and accountability processes. The reporting of meetings, decision-making documents, approach plans, logs/journals, briefings, debriefings and so on needs to be taken care of by the organisation of each particular large-scale police operation;
-the evaluation does not aim at the appraisal of officials or their actions yet at providing insight into the relevant processes and decisions and their effects;
-it is determined in advance, in writing, what the evaluation specifically aims at, who is in charge of what, how the evaluation is carried out, who will be the recipient of the evaluation and which feedback moments (orally and in writing) are set;
-the reviewers do not interfere into an ongoing operation or hinder it in any way;
-the reviewers have access to all relevant information and to televant places where processes take place;
-the findings of the evaluation team are drawn up in a report that is presented to the Overall Commander. The report is and remains the property of the Overall Commander and is not public. The Overall Commander can put the report (either completed with remarks or not) at the disposal of the competent authority if desired. It can then be used as a basis for the administrative evaluation.
5.Plan the evaluation
To be effective, an evaluation should be properly planned, just as other aspects of the police operation. To properly plan an evaluation, an evaluation co-ordinator should be appointed who draws up an evaluation plan and co-ordinates the activieties of the evaluation team. The evaluation co-ordinator determines the areas of evaluation in consultation with the Overall Commander. The evaluation plan should always includes the starting time of the evaluation activities and the time at which the report should be ready, the aim of the evaluation including specific points to be evaluated as well as the criteria the evaluation must meet. The evaluation plan also states the organisation of the evaluation, identifies internal and external persons involved in the evaluation and evaluation instruments to be deployed. Facilities and means needed are made explicit as well as costs related to the evaluation. Evaluation activities to be carried out should be specified. These include the gathering and analysing of material (in writing, on audiotape, on videotape) about the event and the context in which the event takes place and the analysis and reporting.The evaluation report should always contain the following topics:
-the goal of the evaluation (the general aim will be to bring the main choices into the limelight and to place them in the right context, to pointing out what the consequences of these choices were and to identify possible areas of growth for the future);
-method followed;
-context of the event (social background, connection with previous events, current regulations, usual procedures, recent developments);
-an objectified representation in phases of events and decisive moments (before, during, after). The aim is not to make the "one and only correct" version but, if required, to indicate for which items the versions differ;
-an inventory of points of particular interest and areas of growth as they are identified by the persons concerned;
-conclusions, bearing in mind that these are observations of facts and not judgements;
-recommendations, explaining which criteria were used to come to those recommendations.
In drawing conclusions, the evaluation should use clearly identified and preferably generally accepted criteria wherever applicable such as:
-International legislation and regulations: European Treaty of Human Rights, New York Treaty, etc.;
-National legislation and regulations (e.g.: Constitution, Police law, Instruction on the use of force, Law on public demonstrations, Municipal law, Football law);
-National / interregional police agreements (in the Netherlands e.g.: frame of reference crisis and conflict management, final attainment levels training, professional code, Quality model);
Regional / local models and instructions.
Application of the model
The general model had been developed at the request of the Belgian- Dutch binational police project preparing for the Euro 2000 footballchampionships because it felt the need to preserve experiences for the benefit of all police forces that might be involved in the organisation of future events. The binational police project specifically wanted to use feedback from colleagues in this respect. The general model was subsequently adapted to the specific circumstances that surrounded Euro 2000.
1.Responsability
True to the principle that each organisation is responsible for its own evaluation, a distinction was made between local evaluation in the venue cities and a binational evaluation process. As a result, in four (out of eight) Euro 2000 venue cities, police forces supplemented their usual debriefing procedures by forming a local evaluation team. The binational evaluation process was coordinated by the first author, who worked with the cooperation of local police forces.
2.Aim
The binational evaluation process focused on aspects that had been arranged by the binational police project, viz. international police co-operation, central information management and the uniform police behavioural profile. Local evaluation teams made their own plans in consultation with the local Overall Commander. They provided feedback to him within 24h, thus enabling the local police organisation to make adjustments. In one city the local evaluation team and the binational observers worked closely together in this respect.
3.Structure
All binational observers conducted systematic and structured observations on match days, attended briefings and held ad hoc interviews with police officers, stewards or fans. Observations were conducted using previously developed observation procedures: one for observing the interaction between police and fans in the venue cities, one for observing site security management and another for observing information management at police information centres. Co-ordinators saw to it that data were collected in a uniform and accurate way. Within 24 h after each observation day, the teams summarised their observations. In addition, team members recorded their observations in individual diaries and computer files.
4.Involvement of colleagues
Police officers were involved in sevaral ways. Local evaluation teams were composed of police officers from the force concerned, as well as from outside police forces (in three venue cities). The binational evaluation process included an international monitoring team composed of four experienced police officers from Germany, France and the United Kingdom that visited all eight venue cities. In addition, in each venue city, a binational evaluation team of four observers was present. The feed-back of police officers from other countries who were deployed as spotters was used as well in the binational evaluation process. They completed questionnaires on police conduct and were interviewed afterwards.
5.Plan
Well before the start of Euro 2000, an evaluation plan was written according to the general model, observers were selected and trained and logistical as well as organisational preparations were made. The evaluation plan was communicated to all concerned, including police forces and comptetent authorities from the venue cities.
Conclusion
All data were gathered according to plan and without exception the observers felt they had learnt a lot by looking at police operations in a different police force, or even in a different country. A clear recognition that each organisation had its own responsibility as far as evaluation is concerned proved to be crucial. Consistent application of the principles of the model combined with a timely identification of evaluation points and evaluation instruments allowed for a climate in which the observers were welcome to do their work. All involved knew about the aims of the evaluation and knew that data would be gathered and divulged in a responsible way. Local police forces have benefited from the feed-back they received from the evaluation teams, as is indicated by the adjustments they made in response. Because of these benefits both in the short and in the long term, the recently established "Centre of expertise conflict and crisismanagenent" (at the Police Institute Public order and Safety, where all Dutch riot police officers receive their training) has taken it upon itself to set up and coordinate "evaluation teams" in the Netherlands. It also strives to serve as a long-term, institutional, memory and to get lessons from public order policing incorporated into training. Several police forces in the Netherlands are making "evaluation teams" an integral part of their operational plans for large scale police operations. A working group on football disorder (2001) in the United Kingdom recently concluded in a report submitted to Parliament that the evaluation yielded valuable results. In the mean time, the results of the binational evaluation report (Adang & Cuvelier, 2001) are being used to make amendments to the European Union "Handbook for international police co-operation and measures to prevent and control violence and disturbances in connection with international football matches".
It seems safe to conclude that the model has served its purpose during Euro 2000. The evaluation as carried out appears to be unique in several ways:
-the evaluation was carefully planned beforehand and was based on data gathered explicitly for the purpose of evaluating;
-the data were gathered in a systematic and structured way with a maximum of objectivity;
-the evaluation was aimed fully at learning for the future (rather than criticising or judging individuals). This was a crucial element;
-through the involvement of police officers, the evaluation directly contributed to learning processes within police organisations;
-by providing direct feed-back during the event itself, the evaluation aimed to contribute to the adequate policing of the event. These initiatives (for quick feedback) were considered beneficial and some (smaller) adjustments were actually made as a result of the feedback received;
-in this particular case, the evaluation had a European perspective and input from police officers from other European countries was explicitly sought. Observations by an independent team of international experts was useful in validating good practices independently and in helping to identify areas for improvement. Seeking feedback from foreign police officers involved in the operation was a simple and effective way to gather valuable information.
It is important to emphasise that the model presented here is aimed at increasing the learning capacity of police organisations and can work only if applied with the cooperation of the police organisations involved. In this respect it differs from other initiatives to "monitor" police activities from the outside (see e.g. Bryan & Jarman, 1999), although on occasion a combination of "inside" and "outside" monitoring could prove mutually beneficial.
It might be worthwhile to apply the model for the police evaluation of public order management presented in this paper more widely, to develop the concept of evaluation teams further and to involve many more officers that have a task in public order management. Outside the police, most organizations have found sooner or later that the nurturing of knowledge-based communities of practice (e.g. Wenger, 1999) is a sine qua non to enabling significant knowledge sharing to take place. Such communities are typically based on the affinity created by common interests or experience, where practitioners face a common set of problems in a particular knowledge area, and have an interest in finding, or improving the effectiveness of, solutions to those problems. In the long term, evaluation teams could assist in the creation of "communities of practice" of public order professionals, crossing organisational boundaries between different police forces.
References
Adang (1990) Geweld en politie-optreden in relsituaties. Universiteit Utrecht, Utrecht (unpublished report)
Adang, O.M.J. (1994). Evalueren is vooruitkijken. Het Tijdschrift voor de Politie, 21-24.
Adang, O.M.J. (1997). Intercollegiale toetsing van grootschalig optreden: opzet van evaluatieteams. PIOV, Hoogerheide (unpublished report)
Adang, O.M.J. (2000) Verslag fase 1 project politie expertise Euro 2000. LSOP, Apeldoorn (unpublished report)
Adang, O.M.J. & C. Cuvelier (2001) Policing Euro 2000. International police cooperation, information management and police deployment. Ubbergen, Tandem Felix
Adang, O.M.J., & C.J.L.van Dam (1996). Evaluatie van grootschalig optreden. Het Tijdschrift voor de Politie, 21-23
Adang, O.M.J., & H. Standaar (1993) Jongerendemonstratie Den Haag, 8 mewe 1993. PIOV, Hoogerheide (unpublished report)
Bryan, D. & N. Jarman (1999) Independent intervention. Monitoring the police, parades and public order. Democratic dialogue/ Community Development Centre, Belfast
COT (1993) Jongerendemonstratie Den Haag. Onderzoek en evaluatie van de gebeurtenissen van 8 mewe 1993. COT, Leiden/ Rotterdam
COT (1999) Voetbal en geweld. Onderzoek naar aanleiding van rellen en plunderingen bij een huldiging in Rotterdam (25 april 1999). Samsom, Alphen aan den Rijn
Das, D.K. (1984) Lessons from past police practices incorporated in state-mandated recruit training? Police Studies, 7 3, 175 184
Della Porta, D. & H. Reiter (1998) The policing of protest in Western democracies. In: Policing protest (Della Porta & Reiter, eds.) University of Minnseota Press, Minneapolis
Dijkhuis, G. (1982) Niet alleen met stok en steen. Een literatuuranalyse van niet-gewelddadige methoden van relbestrijding. Foss, Oegstgeest
Eisenhower, M. (1969) Report of the national commission on the causes and prevention of violence. Washington D.C., US Government printing office
Enschedé, J. (1967) Onderzoek naar de ordeverstoringen te Amsterdam op 13 en 14 juni 1966, 's-Gravenhage, Staatsuitgeverij
Jefferson, T. (1990) The case against paramilitary policing. Milton Keynes, Open University Press
Kerner, O. (1968) Report of the national Advisory Commision on civil disorders. New York, Dutton
Nationale Ombudsman (1993) Rapport nr. 93/ 630
Popplewell, O. (1986). Committee of enquiry into crowd safety and control at sports grounds. Final report. London: HMSO, 6.
Scarman, Lord (1981) The Brixton disorders. London, Penguin
Scranton, W. (1970) Report of the national commission on campus unrest. Washington D.C., US Government printing office
Smith, M.R. & G. Alpert (2000), Pepper spray. A safe and reasonable response to
suspect verbal resistance. Policing: An International Journal of Police Strategies & Management, 23, 2, 233 - 245
Taylor, P., Lord Justice (Chairman) (1990). Inquiry into the Hillsborough Stadium Disaster: Final Report . London: HMSO
Waddington (1998) Controlling protest in contemporary historical and comparative perspective. In: Policing protest (Della Porta & Reiter, eds.) University of Minnseota Press, Minneapolis
Wenger, E. (199) Communities of Practice: Learning, Meaning, and Identity. Cambridge University Press
Working group on football disorder (chaired by Lord Bassam) (2001): Report and recommendations. London, Home Office
Responsability
