Methodology: Part II | Facing Facts

Critical evaluation of the Journey and the Systems methods

Both graphics were used as tools for collaborative work during the second set of national workshops. This section identifies methodological themes arising from this work.

Workshop methods supported collaborative working:

Participants in the first set of workshops reported that they found it helpful to connect with colleagues from other organisations and institutions – often for the first time – and bring together information about how they each capture hate crime data, what gaps exist and the potential consequences of these gaps. Participants in all workshops reported that they learned new information about current reporting, recording and data collection systems. In several workshops, participants agreed specific actions on cooperation – for example arranging meetings to further discuss how hate crime recording and data collection might be improved between one or more institutions. Feedback from the interactive workshops included:

It is useful to see and compare peoples’ perceptions (CSO)

It was interesting to look another person in the eye and admit that the relationship could be improved. (public authority)

This way of working is not usual for me – the most positive thing is that it reflects many elements. (CSO)

It was quite rewarding, because even though we agreed, we also had discrepancies. (Public authority)

This feedback and agreements to work together following the workshops evidence that the interactive, victim-focused methods met its objective: to connect, to ‘get on the same page’ and to build relationships at the national level.

The need to allow for national context

The same methodology was used for all workshops except for the final consultation workshop for England and Wales. This was because:

The UK hate crime recording system is very complex. There was a risk that focusing on the entire ‘system’ in the national report and the half-day consultation workshop would simply repeat what is already known, without advancing knowledge or action
Interviews with change makers were taking the research in the direction of an in-depth consideration of third party reporting – its current state and possible future directions. In line with our design-driven participatory methodology, this research focus was prioritised as being most useful to those engaged in the issues at the national level

We therefore used the London consultation workshop to consider our findings on third party reporting and to seek agreement on possible recommendations. We did not conduct the ‘systems map’ exercise on the sticky wall or review the draft systems map in depth.[1]

Using the ‘sticky wall’[2]

A ‘sticky wall’ is a large piece of fabric that is sprayed with adhesive. It is mounted on a wall using blue/white tack or tape. It is used in a variety of ways as a facilitation tool for trainings and workshops. As the detailed activity guide describes below, during the first workshops, participants worked in small groups to plot the information and data that is captured and recorded at the reporting, investigation, prosecution and sentencing stages. Participants attached, removed and moved labels and paper as they wished, and often in negotiation with each other. During the second workshop, participants worked to create their own ‘systems maps’. They were asked to place red, yellow or green string to represent the strength of connections and relationships, again negotiating and debating across ‘divides’ to try to agree the final colour of the string. In all workshops, most participants took photos of the final sticky wall. Feedback on the tool was positive.

Feedback included the following themes:

The technique allowed participants to build and see a fuller picture of hate crime recording and data collection
It was interactive and fun to use
It was practical

From the public authority perspective in particular, these highly interactive and somewhat experimental approaches were very new and rarely practiced in the workshop, meeting and training contexts that they are used to. It was encouraging to see most participants taking part in using the sticky wall and being open to this new technique. However, several participants working in a public authority context reflected that the ‘novelty factors’ of the sticky wall and coloured string undermined the ‘seriousness’ of the activities. One participant suggested that these sorts of approaches can’t be introduced as a ‘one-off’ and needed to be regularly encouraged and used in more formal public authority contexts before they could be fully accepted as credible ways to engage. This is an important insight for future efforts.

Methodological limitations of national workshops

Not all workshops had full representation across relevant institutions, organisations and affected groups. As a result, key perspectives were missing. These gaps reflected a number of possible issues:

A lack of commitment by the invited institution to the importance of hate crime recording and data collection
A weakness in the relationship between the national partner and the institution or organisation concerned
A lack of capacity: a broad range of groups can be affected by hate crime yet not every group is effectively represented in national NGO monitoring activities. For example, while people with disabilities are a known target of hate crime across Europe, very few NGOs conduct effective recording and monitoring work in this area

The lack of involvement of key stakeholders in some workshops was countered by desk research and specific follow up when completing the systems maps.

National workshops and related outputs cannot reflect the diversity of regional and local approaches. As a result, a lot of important and good work is missing from national reports and systems maps. It is recommended that similar exercises are undertaken at the regional and local levels, following the activity guide set out below.

Strengths and challenges of the Journey of a Hate Crime graphic

The Journey image was shared during consultation workshops. Feedback included:

Taking a victim focused approach highlights the fact that our criminal justice system does not take this approach – (public authority)
The victim experience is actually more like a maze, where victims can bump into ‘walls’, feel stuck and not know where to go (academic)
It is really useful to see all agencies as part of the same picture (NGO)
I can use this in my trainings with colleague (public authority)
The graphic presents findings that could take many pages to present in narrative form (public authority)
The generally ‘hostile’ environment that many victims experience is not sufficiently conveyed by the image (public authority)
How can we make sure that all victim groups are represented in this graphic? (NGO and academic)
Make sure you can print out the image and use it in paper form for trainings and other activities (NGO)
The Journey graphic only represents three stages of the criminal justice process. The pre-reporting stage isn’t captured and the post sentencing stage is also invisible. This means that important local authority, probation and prison functions are not represented (combined feedback)

The potential versatility of the graphic as an engagement and training tool is highlighted in the feedback above. It is recommended that learners, trainers and workshop participants work with A3 colour printed versions of the graphic to insert information about the national context or to portray evidence about the victim experience. It is also recommended that further graphics are developed to represent other aspects of the hate crime experiences before reporting and/or after sentencing (e.g. probation) are developed. As a first step an instructional video on how to use the graphic as a training and development tool has been developed as part of the online learning for decision makers.

Challenges and limitations of national systems maps:

The systems maps took a lot of time and effort by the lead researcher, national partners and graphic designer to complete.[3] It was difficult to develop a framework that met our aims, which were to create a comprehensive and victim-focused assessment of the national situation, based on a concept of relationships that integrates a consideration of evidence of CSO-public authority cooperation, as well as evidence on the quality of CSO efforts to directly record and monitor hate crimes against the communities they support and represent. The elaborate process of creating national systems maps revealed that the current normative framework:

Does not oblige relevant stakeholders to implement a practical means of connection with each (e.g. shared definitions of hate crime as the basis of a flagging system, the technical facility to pass information and data from one agency to another)
Does not conceptualise relationships across stakeholders as being of fundamental importance when giving life to normative obligations

On reflection it is clear that, with the exception of the Victims’ Directive, the current normative framework has an institutional as opposed to a victim focus. While most international obligations focus on recording and collection of data by institution, the Directive obliges institutions to cooperate with each other and with CSOs to keep victims informed, safe and to collect data based on those obligations. The Findings section of this report proposes a victim-focused model for reporting, recording and data collection, which could inform the international, normative framework and possibly improve these sorts of assessments in the future.

Specific limitations of the systems maps:

The maps only represent the national level. It wasn’t possible to research include recording and data collection activity taking place at the local and regional levels. As a result, strong local and regional practice is missing, and the complexity of national law enforcement agencies and structures risks being oversimplified, particularly in federalised systems such as Spain and Italy or devolved contexts such as the United Kingdom[4]
Limitation of describing both ‘sides’ of a relationship with one ‘line’. in some relationships, one side does most of the ‘work’. For example, the flow of information and data might be positive from a CSO to a public authority but very limited in the other direction. This led to some workshop participants using two colours to describe the same relationship. In the online systems map, the line can only be one colour and the narrative of the self-assessment aims to describe where one ‘side’ has a stronger framework and/or is more ‘active’ than the other
Some relationships are naturally ‘one-way’. For example, there isn’t an expectation on victims to routinely share data and information with CSOs or with public authorities such as the police. However, there is an expectation that the police take action to ensure that all elements of a hate crime are captured, to communicate with victims and to encourage reporting
Some relationships don’t need to exist. For example the MoI might take the lead in coordinating data and information to IGOs. As a result, there is no need for there to be a direct relationship between law enforcement, the prosecution service or other government ministries and IGOs. This is also explained in national analyses
Over-complicating versus oversimplifying. On the one hand, the number of lines and relationships can make maps difficult to ‘read’. On other hand, several of the maps can’t reflect the depth and complexity of action in national contexts because the detail risked being overwhelming.[5] Efforts were made to achieve a balance, however there is still room for improvement. It would be interesting to explore technology that allows for certain relationships to be visually spotlighted and highlighted and/ or certain areas of the map to be ‘zoomed in’ for closer inspection within the broader national context
Missing information. While every effort was taken to work with national stakeholders to ensure that information and data is accurate, it is inevitable that some will be missing or possibly incorrect. The systems maps are designed to be dynamic and can be updated with corrections and as national situations change and evolve. National stakeholders will be able to work with Facing all the Facts to amend and update maps after publication
Presenting a comparative picture of diverse contexts. Some contexts are significantly ahead in their journey on hate crime recording and data collection, while others are at the beginning. For example, England and Wales has had a strategic approach to hate crime recording and data collection since 2008, whereas Ireland currently does not. The assessment framework gives ‘credit’ for both the framework (E+W will achieve higher score) and the action taken between stakeholders (Irish stakeholders’ efforts are reflected here). This approach better supports a comparative description
Description not ‘assessment’. The Facing all the Facts Project is not aiming to set norms and standards in the area of hate crime recording and data collection or to provide a formal ‘assessment’ of national efforts. Instead, it aims to develop transparent tools, based on existing standards, to bring key stakeholders together to co-describe, the current system and relationships; to co-diagnose areas for improvement and to co-prioritise what to work on in the short, medium and long-term. Therefore, the term ‘description’ is used and the importance of revisiting and revising the description over time is emphasised

Conclusions

The feedback and analysis of this project’s action-research methodology and outputs suggest that:

Taking the time to involve all relevant stakeholders in the process of describing and understanding the national hate crime and data collection ‘system’ can lead to better national understandings of the nature and impact of hate crime and who should and is doing what in this area
A focus on relationships and their varying strengths highlights the dynamic and complex nature of the system that needs to be understood and supported
The explicit focus on CSO/NGO-public authority relationships brought CSO’s from the periphery into this ‘system’ as central actors, and encouraged a victim focus

However, the process has been time consuming and quite messy at various stages, requiring a lot of input from national partners. This is partly because it was the first time that a normative framework, integrating a victim and community perspective has been created and applied to describe and diagnose national systems. Towards the end stages of the research, when country analyses were being finalised, the research team reflected that the process has been so difficult partly because the current framework does not specifically address the following issues:

That there should be ‘nodes’ of connection across institutions that can be created through technical and policy agreement such as inter-institutional definitions and hate crime flagging guidelines; electronic or manual databases that are connected;
That relationships, based on regular meetings, communication, and common goals, between institutions are important.

The implications of these gaps is that it is difficult to develop, describe and implement a victim-focused ‘system’. From the victim and community perspective protecting institutional boundaries – a preoccupation of many public authorities – can be in conflict with efforts to achieve the desired outcomes of safety, security and justice. Until this perspective is integrated into the international normative framework, it will remain difficult to most effectively describe, assess and prioritise. These points are addressed in more detail in the Findings and Recommendations sections.

Further, on a more practical note, one day and half-day workshops and the resulting national reports and systems maps will have limited impact if they are not revisited and revised. It is recommended that national stakeholders continue to use the tools produced by the project to re-assess their situation at the national regional and local levels and to agree specific actions and updates to their systems maps accordingly.

« THE SELF-ASSESSMENT FRAMEWORK BACK TO TABLE OF CONTENTS METHODOLOGY PART III »

[1] See ‘Connecting on hate crime data in England and Wales’.

[2] Thanks to the Institute of Cultural Affairs for inspiration on this facilitation method.

[3] See national reports for national systems maps.

[4] One approach to counter these weaknesses is to use the same methodology at the local and regional levels or with a tighter focus on one area of hate crime or one part of the system. This allows for more granularity in the context of the bigger, national picture offered in these reports.

[5] For example, there can be great complexity and details relating to the range of crime types and protected characteristics/bias motivations that are captured at the national level. Where there are gaps (e.g. LGBT+) these are mentioned, however there is not a detailed description of crime types and protected characteristics in each national context. It can be added at the national level at any time.