Summary of the Evaluation Report of RESHAPE

This article is a summary of RESHAPE’s evaluation process which generated a much larger report that used the prototype of 4D Evaluation created by members of RESHAPE’s Fair Governance Trajectory in collaboration with FARO learning community.

We are very grateful to all RESHAPE partners and communities for the opportunity of transforming the original prototype description into a real web based online tool that may have a life of its own, as well as the trust invested in our work, allowing us to perform this evaluation of RESHAPE’S first three years..

We thank enormously all the people who took part in the prototype development process who have dedicated endless hours in such a complicated year.

In general terms, the Evaluation prototype aims to value culture as a key element in our ecosystem for social transformation, through the creation of new applied metrics capable of attributing value to our cultural projects, focusing on the reformulation of future policies, public and private, towards a multidimensional paradigm shift.

Specific objectives:

1. Create a common language for a new methodology to design, manage and evaluate cultural and artistic projects.
2. Form a research unit on metrics that articulates qualitative and quantitative parameters through different perspectives.
3. Create a template - matrix to analyse cultural projects through a shared methodology.
4. Introduce the metrics to different institutions with the intention with the intention of further developing the methodology.

Introduction

About RESHAPE

RESHAPE (2018 - 2021) was a research and development project that brought together artistic and cultural organisations from Europe and the Southern Mediterranean region to reflect upon the biggest challenges in organisation of artistic and cultural fields and to work together on concrete solutions related to the production, distribution and presentation of contemporary art practices.

The aim of RESHAPE was to imagine an alternative to the European arts ecosystem by rethinking its instruments and collaborative models, placing them in line with artistic and social innovation and the principles of fairness, solidarity, geographic balance and sustainability.

RESHAPE developed and tested an experimental bottom-up method to construct new narratives and instruments aligned with the evolutions of the arts sector and the society. It was designed by 19 partners and associated partner organisations and coordinated by Onda — French office for contemporary performing arts circulation. The project was co-financed by the Creative Europe programme of the European Commission. Check the list of all involved organisations here.

The heart of RESHAPE’s development was a group of 40 practitioners (Reshapers) from Europe and Southern Mediterranean region that worked together in 5 thematic groups (Trajectories) mutually developing prototypes to address the burning questions of cultural and artistic production.

Alongside Reshapers, various different actors took participation in the process in different roles and with varying capacities: partners involved in the project, Advisors, Facilitators, members of local cultural and artistic scenes in different European and South Mediterranean cities and regions, speakers at conferences, Intensives and other public events, associates that have helped the creation of content within the project, be it through developing prototypes, or by participating in preparing the publication that was made as a part of this large project.

The project was conducted in a series of complementary formats (group workshops and larger gatherings, public events) aimed towards joint production of materials for dissemination (prototypes, other materials for publication). Due to the Covid 19 outbreak, a large proportion of project activities were shifted to digital formats.

The core methodological and thematic scope of the project was aimed at mutually envisioning concrete, realistic and sustainable ideas and tools to address the systemic and structural inequalities, along with other shortcomings that determine artistic and cultural production in Europe, Southern Mediterranean and on the global stage of contemporary cultural and artistic production.

Within this large-scale operation a great variety of actors were gathered in order to reflect the current state of the art in the chosen ecosystem, representing artists, cultural workers and activists, different forms of cultural organisations (traditional cultural institutions, new institutions and emerging models of organising collectives in cultural fields) and different types of cultural interlocutors (funding bodies, consultants, promoters and decision makers, researchers and other members of the academia).

As we have already mentioned, Reshapers, a group of enthusiastic practitioners were at the heart of the RESHAPE collective undertaking. In terms of focus, productive, effective and concrete changes, they were seated deeply into the ethos of the project.

RESHAPE was about experimenting and producing tools to change imbalances of the art world. So, how did it correspond with its communities? Did it facilitate certain processes? Where is it pointing to? Concentrating on RESHAPE as a process, we are evaluating its implementation, results and shared experiences from both, personal and general points of view.

Our Methodology

With this evaluation, the goal was to co-produce knowledge about the process we’ve been experiencing all together for the past 2 years while taking part in RESHAPE. In our journey we were guided with the following basic principle:

Evaluation is not an event or a method: we consider it as a transformative attitude towards change that is adopted as a collective state of mind.

The idea is to go beyond the usual indicators while working on a 4D Fluxonomy Matrix based on fractal thinking, involving thinking about Cultural, Social, Environmental and Transactional dimensions of the project.

The main elements of the process:

• Group formats with 2 variations: Focus Group and Workshops gathering groups of main actors of the RESHAPE universe.
• Interviews with the main actors of the RESHAPE universe (the online tool was used to process the interviews).
• Voluntary individual participation through the prototype’s online tool, which features detailed presentation of the 4D methodology as well as a questionnaire and a scalable surveying tool.
• Interim Evaluation Report, with quotes from the interviews organised according to RESHAPE’s objectives
• Final Report, a 70 pages analises of the evaluation using the 4D methodology and data visualisation
• Summary of the Report with main lessons learned, quotes extracted from the interviews, short extracts from Lala’s fluxonomic analyses (the annex document is included with full analyses), and a playful ending with a crossword puzzle to look into the future.

The methodology is based on the 4D Fluxonomy theory developed by Lala Dehenzelin, who has contributed to the report and evaluation, and on the evaluation prototype developed by the Fair Governance Evaluation sub-group in partnership with FARO - Learning Community during the RESHAPE process as one of its main tools.

Graphic visualisations were used to understand the 4D analysis and the way dimensions correspond to each other. We also used the same patterns during the interviews and to gather further data, see it on this link.

4D Analysis

Cultural Dimension

One of the most important aspects of the idea behind RESHAPE is its relevance, as it is clear that there is a need to open spaces for reflection, exchange and experimentation in an articulated way and through the convergence of different actors. RESHAPE found resonance with the needs of the communities involved and the prototypes have managed to broaden the concept of the trajectory they addressed, providing new formats and interactivity mediums that have contributed to various learning processes. Participants valued highly the fact that the project was designed to be learning oriented, to raise their own awareness and that of the wider cultural sector. RESHAPE is seen as a courageous and changemaking initiative as its scope differs from many other European projects: “it is not focused on producing art but on building communities to discuss important topics”.

Diversity and intercontextuality grounded the project and brought opportunities of gathering different actors, mostly from the creative sector, to work in open, experimental formats in order to address key issues with creative solutions. Reflections regarding the cultural ecosystem in terms of geographic and disciplinary diversity brought valuable insights and highlighted many differences and inequalities present in the cultural sector.

In terms of its geographic scope, the concept of being a broader European project involving people from all parts of the European continent and the Southern Mediterranean and having a constellation of associated actors coming from different organisations brings a variety of perspectives into the process. According to the group that envisaged the project, a comprehensive geographic balance was a necessity for the partner institutions and an opportunity for them to go through a learning process that they could later replicate and emulate:

“RESHAPE made us gain new insights into the knowledge being produced in unsuspected places. (...) I see that as a change. For me there is huge potential in this, and there was an acknowledgement that came from this.”

extract of Lala’s fluxonomic analyses

The point of attention here is a tendency to remain in the comfort zone of the known, but to evolve and find gateways we must adventure in lesser-known terrains. The suggestion is that in every stage of a project, focus should be put on how to go beyond the known. Regarding cultural workers, that means attention in what is tangible, for instance the transactional dimension and what has a more collective scope, going beyond the singularities of the cultural context.

Environmental Dimension

RESHAPE gathered a large number of stakeholders and players to contribute with their skills and resources. The project gave access to knowledge, experiences, expertise and contributed to the desire to interact with an even broader scope of fields beyond the arts and cultural worlds.

Participants valued the space dedicated to imagination and creativity and the fact that “there was a very big room for experimentation”. The project was created with the idea of giving Reshapers the option to be free to think about their goals and the methods to achieve them. In this sense, the openness of the process has been well valued, but the frequent adjustments of the framework were considered to have taken a lot of time and energy.

“Having to negotiate what you can do together across these different realities, sometimes led to processes being much more encompassing, or focusing on one reality, all of it was revealing (…) The process led them to consider realities that they would not consider normally.”

Intercontextuality is an important feature of RESHAPE’s cultural and environmental dimensions, the fact that many different actors were brought together to mutually find solutions for the most important challenges the cultural sector faces today was one of its main assets, but it also reflects great complexities and imbalances within the sector.

Mobility was a core element of RESHAPE, and most of the activities were designed for different groups to work in diverse contexts. As many transnational and especially European cooperation projects RESHAPE's design relied on travel and exchanges in physical spaces (especially pre-pandemic) whereas in times of great climate challenges there is an urge to rethink concepts of mobility and alternative models of work. In RESHAPE’s case, this would suggest finding greater balance between physical exchanges and focused work on the production of prototypes. RESHAPE’s communities appreciated workshops taking place in different cities as they served as inspiration. At the same time, the interviews showed that the development of desired outcomes and the benefits of meeting local actors should be better balanced.

The pandemic brought shock to the world and the project was impacted in a serious manner, with all physical gatherings cancelled and very fast adjustments needed. The groups and the project team had to come up with tools and solutions to facilitate prototypes production. As contents were getting more accessible online, people were experiencing extreme fatigue with long hours in front of the screens, but the switch was also seen as an opportunity, “especially when it comes to opening conversations, or experimenting within the organisation of the project itself, that might have not been possible otherwise.”.

extract of Lala’s fluxonomic analyses

There was geographical diversity of players but the majority of them played similar roles in their institutions. The answers reveal that more educational and therapeutic skills would be welcome, as well as design and technological ones. (...) The diversity and integration of partners as well as their close relationship with the Reshapers was highly praised by some participants (...) and the “sharing” aspect of the environmental dimension can bring viability not only through what the initiative already “possesses” but also through what it has “access to”.

Social Dimension

One of the focuses of RESHAPE was put on “building a network of mutual understanding” between professionals and institutions. The affective bonds between people that emerged from the process are seen as one of RESHAPE’s most highly valued impact. Local communities were included through various formats, although their role was mostly inspirational towards the achievement of project’s outcomes; in terms of impact in local communities involved, some Reshapers felt they had no “time, space and capacity” to be beneficial for other than direct participants, they felt “pressured to make a prototype and so busy trying to find a common ground.”.

Parallelly, the expectations between those who have envisioned the project and the ones who participated in it were somewhat different. The need to constantly re-negotiate roles and the positions between these roles gained importance in the internal logic of the process. Participants realised that the experimental nature of the project suggests that certain intentions do not necessarily result in expected outcomes. It seems that RESHAPE did not only reflect contradictions and frictions immanent to the actors gathered around this project and perhaps those of the larger artistic and cultural scene, but also reflected some frictions currently visible in the world.

At the beginning of the process, there were not enough opportunities to celebrate the fact that the project gathered all of these different actors together; with the necessity to work remotely brought by the pandemic, these occasions were further restricted and rushed. At the same time, many people within RESHAPE found a lot of comfort and support in continuous online group work.

extract of Lala’s fluxonomic analyses

Community is related to the players, the ecosystem, and the diversity of the participants, in which most of the players belong to the cultural sector, with less participation from environmental, social and financial actors. At the same time, in RESHAPE’s evaluation a strong indicator of success is the general feeling that continuity will happen organically due to the profound bounds that were created within its communities.

Transactional Dimension

RESHAPE is seen positively within the arts and culture, and creative fields, in particular as it provokes unusual ideas and self-questioning. Interviewees acknowledge that collaborations are still concentrated within the RESHAPE community, but see the potential for further and broader interactions and developments. Beyond the production of the prototypes, the potential for change resulting in personal learning and collective experiences was greatly appreciated.

“Now I am gaining much more understanding that the point was about being there, not so much about producing a prototype.”

The remuneration of all the participants involved, across all roles, was considered as a means for encouragement, and an important symbolic affirmation of the value of the work invested, one of the partners mentioned that “...the initiative to award people financially for participation was a change maker that should be standardised.”.

When resources for prototypes were discussed, participants pointed out that when travel was restricted due to the pandemic, there was flexibility to re-interpret, work and re-use the budget and additional remuneration was made available in the end.

Participants felt it was difficult to already measure or evaluate RESHAPE’s long-term impact and potential for change. This is due to the diversity of prototypes, timeframe and possible further developments and uses of the prototypes and other tools. The desire to have more exchanges between the Trajectories was also pointed several times, “cross-trajectory work” (coproduction of Tarot cards for instance) benefited Reshapers who have participated in it and there is an impression that the whole project could have benefited more from enhancing the circulation of knowledge generated across the Trajectories.

Regarding the project’s long term impact, it was mentioned that although the project might not have a lot of resonance outside the arts and cultural sector, the prototypes and personal impacts will lead to transformation. The fact that bigger players should be supporting and helping the information flow, was also a recurrent subject. Participants say that smaller groups and networks of synergies that were created will continue further.

“There are a lot of platforms for the tools that were created, and places where they can be disseminated.”

“I can see a long term interaction and collaboration. RESHAPE is a network, because we did something together and it has meaning because of the community's efforts to make this alive. I have no idea in which form but I feel it has the potential to live beyond the end and make other types of collaborations, the “RESHAPE feeling” will continue inhabitating us and give birth to more opportunities and possibilities.”

extract of Lala’s fluxonomic analyses

This pattern is usually observed while working with projects and players that act mainly in the cultural dimension: they have more difficulty dealing with the Transactional Dimension. And vice versa.

We nicknamed this the abyss between “projectland” (the ones engaged in creating solutions) and “businessland” (the ones engaged in implementing and scaling up them). The moment there is an understanding that they are not opposite to each other but complementary, the very urgent bridge between them will emerge. Considering that culture is the matrix of society it may then best reach its purpose of contributing to reshaping the future.

The 5th Dimension : Looking Forward

This is an epilogue of RESHAPE’s journey into the future inspired by a 5D evaluation of the project, along with texts and interviews from in and out of RESHAPE’s Workbook to Reimagine the Art World.

It is a contribution to a generous political imagination of a possible RESHAPED future.

Imagine yourself in a not-so-distant future, doing a crossword puzzleThis crossword puzzle was created by Katarina Pavic and Lívia Diniz inspired by the evaluation report written by Reshapers and FARO members Claire Zerhouni, Eduardo Bonito, Isabel Ferreira, Katarina Pavic, Lala Deheinzelin, and Livia Diniz, along with the “Transnational and Postnational Practices Manual” written by Martinka Bobrikova & Oscar de Carmen, Pau Catà, Petr Dlouhy, Heba El Cheikh, Gjorgje Javanovik, Marta Keil, Dominika Święcicka, Marine Thévenet, and Ingrid Vranken. Extracts from Dona Haraway’s book Staying with the Trouble and the video “Making kin beyond babies” made by just wondering… were also used and intertwined with extracts from the interview “Frictions and Detonations: An Approach to the Peripheral-Community of Cuirpoétikas & Pagoda Imaginaria” by journalist Alejandro Ortiz and the artists Numa Dávila and Esvin Alarcón Lam.. The answers are somewhere on this page.

1. Change is inevitable but how do we make it desirable? We already know that we can’t step twice into the same river and that the only constant in the way the world works is a shift, a change… we all participate in this perpetual journey through time, most of us not having a clear idea where our journey starts or where it ends. A great writer once said: everyone thinks about changing the world, but no one thinks of changing themselves. The Journey of RESHAPE was equally about realising how complex the world is and how changes occur but positive transformations are worthwhile of our efforts. We must desire to change… to “resist a lineal, progressive representation of movement forward to something better, or movement back to something purer.”- Kim TallBear

2. “What happens to the land, happens to the people.”

From a not-so-distant future….

Last centuries’ hyper-mobility has changed its shape. Humans have gained understanding of travelling through other dimensions, travellers can now move beyond physical boundaries, inwards their own bodies, and environments grow through them. Travellers now know where they are and are able to translate the world around and within themselves. Territories and spaces are fluid. Flexible and adaptable formats have generated creative approaches, spontaneous and interrelated connections.

3. A context defined by its multiple fissures, while “daily life is in constant friction provoking a complexity in need of interlocutors. Interlocution comes from learning processes, not necessarily from ‘artistic’ academies, but from everyday life itself, which is equally rich and violent“Frictions and Detonations: An Approach to the Peripheral-Community of Cuirpoétikas & Pagoda Imaginaria” by journalist Alejandro Ortiz and the artists Numa Dávila and Esvin Alarcón Lam..”

In terms of generating spaces of encounter; one must be questioned by the experiences of others and experiences that enable various areas of interaction, to allow and learn new ways…

4. From a not-so-distant future….

Relationships have been reshaped by dissolving power hierarchies through participatory processes, people have been constantly shifting roles so responsibilities are not accumulated, while power and resources have been distributed through collaborative actions in different formats and sizes.

Platforms and networks of dissemination are replenishable, continuous, with no pre-destined hierarchies. Subjective experiences of those who inhabit the spaces are considered and other forms of relationship between people linked to contemporary debates have been made possible through multiple intersections.

5. In a not-so-distant future….

We are at a time where ecological regeneration is unnegotiable. After many years of dedicated focus on nourishing and interspecies mutual care, generous ways of being are finally harvested in the world. Communities deal with weaknesses, and have been taking care of their broken parts, the levels of affection and empathy are high, thus communities are formed on their abilities to care for eachother.

6. In a not-so-distant future….

Al last, time has been understood as our most precious resource. Time to imagine, time to allow affections, to take breaks and suspend action, time to observe, to live moments with potential for chaos and disruption. Time to be together.

7. In anot-so-distant future….

People have desired to maintain pleasure as a basic necessity of life and to activate political imaginaries through the arts, conviviality, jollification… “we should find ways to celebrate new forms of making kinExtracts from Dona Haraway’s book Staying with the Trouble and the video “Making kin beyond babies” made by just wondering…”.

A Future Perspective

extract of Lala’s fluxonomic analyses

A few advisory perspectives

First, I would like to congratulate RESHAPE for its design and the outcomes and innovation it produced in all of the 4D: cultural, environmental, social and transactional. And celebrate the privilege of observing, contributing and learning from the experience of this Prototype inspired by FARO4s research.

Evaluation is actually sense-making: we learn while designing, applying and interpreting the data. This sense-making allows us to better design next steps and choose what and how they can be implemented and which goals they intend to reach. In the case of RESHAPE this second round will happen inside the initiatives of the partners and Reshapers who can benefit from this evaluation.

For an initiative to grow, it needs to jump from an operational stage, focused in materialising whatever it aims for, to a strategic one, where the focus is no longer in the “What” we do, but in “How” we do it. It will be strategic if this “How” is capable of bringing more value in all the 4D. When this happens, it opens up in a conic spiral with many rounds that can expand as a Fibonacci. An evaluation tool brings “credit” to the previous phase and points out possibilities for the next round.

Link to full text of Lala's Fluxonomic analises, data visualisation, brief evaluation of the prototype and credits.

Happy with what you’ve read? Send us more stuff like this!