3.2.3. Assesment criteria and indicators

For the full version of the original text go to Value and Indicators for Interdisciplinary Research by Anna Moreno, Mariona Moncunill and Reimund Fickert.

All the assesment criteria and indicators listed below include the following values:

- detection

- negotiation

- transparency and accessibility

- reviewability

- permeability

Based on these values, each interdisciplinary research group will have to agree on their own self-evaluation indicators, which can also be used later for external evaluation. At the very least, all processes must be recorded or documented and later archived, in order to ensure that indicators can be identified and evaluated.

Instituent Competencies

The process of identifying and studying competencies begins by discussing the motivation for setting up an interdisciplinary project. All members need to become aware of their own style of thinking and explicitly describe it to the group. This will allow them to communicate their motivations and concerns, and the tools and standards that are their point of departure, both within the process and in its transfer.

The group can then begin to negotiate and question legitimacy (agencies), to recognise the power relations intrinsic to each discipline, and what happens to these relationships when they combine in an interdisciplinary environment. The discussion should particularly take into account situations in which academic disciplines coexist with non-academic or instrumental disciplines.

The negotiation of competencies may differ according to the origins of each initiative. For example, a horizontal initiative may link together different disciplines from the zero point of the process, while other initiatives will begin with an invitation issued by one discipline to others. Interdisciplinary projects must at the very least guarantee that leadership will be transparent and that legitimacy will be questioned from the start, and that this will be reviewable over time.

b) Representation and Visualisation

Representation and visualisation applies to languages, formats and channels, be they disciplinary or interdisciplinary, academic and non-academic.

Detection: What standards are a starting point, what standards are respected, and what standards are created from scratch? The group has to identify these standards, and study their implications. Is there a common language/jargon, or are translation strategies required?

Negotiation: The standard or standards to be used must be consciously chosen and discussed by the members of the group at the start of the process and during the transfer.

Possibilities:

- Visualisation in various languages/formats/channels at the same time

- Distortion of existing standards at the symbolic level

- Creation of new or joint standards

c) Ethics

All interdisciplinary projects must include a negotiated, reviewable code of ethics. Discussions must include issues such as respecting the cognitive value of each member, the individual responsibilities within the group and in regard to the context, and the dissemination of the outcomes in terms of accessibility. Members must also agree on the sharing of possible benefits, intellectual property issues, and possible subsequent uses of the research.

d) Mediation

The standards mentioned above have to do with the process of recognition among members and disciplines of the group, and during the knowledge transfer. In terms of mediation, the standards must aspire to universality, but not to standardisation.

Mediation can be embodied in one or more individuals who perform this role within the group, or it may be distributed among all the agents, in which case the group must set aside spaces and times that favour it. Both cases require a capacity for abstraction, or the ability to “zoom out”. As a tool, mediation must enable dialogue and symmetry at all levels, and take advantage of conflicts and dissent as a source of content and a guarantee of replicability.

e) Community

Team: The members of the research team should share their interests, motivations, and affects. Ideally, once they have been pooled, the group will work with them to construct scalable, common interests, motivations, and affects, without cancelling out the original ones. Returning to mediation as a tool, the group should create spaces for social interaction that favour the creation of networks and affects.

Context: The group should identify the parties concerned (individuals or collectives) and the contextual connotations of different aspects of the project, and examine them from different points of view (ethical, and transfer-related). The group should consider whether a dialogue with the context is pertinent, and whether to include members of the community as an integral part of the group and as a decisive content generator.

f) Processes

If we accept negotiation and reviewability as constant elements that are present throughout the process, then said process must become exhaustive, non-linear, and open in nature, and it must be based on dynamics spread through different stages. As such, we can say that it is an iterative process.

The group has to negotiate the integration of elements such as contradictions, error, failure, the unknown, and the unpredictable, into the process. “Thinking by making” leads us to see the process as a result in itself. To this end, the documentation and recording of the process should focus on making it visible. When it forms part of this visualisation process, the documentation becomes performative. We believe that performativity is an important value to be taken into account during the research process and the knowledge transfer.

g) Transformation

Transdisciplinary research should be an iterative process of revision, analysis, learning and transformation. To this end, transdisciplinary projects should have a flexible, negotiable structure, and accept different degrees of change.

- At the personal (emotional) level.

- At the community level.

- At the disciplinary level.

- At the level of understanding other disciplines.

- At the level of transdisciplinary research itself.

The structure of the project must, at the very least, open up spaces or opportunities to detect, evaluate, incorporate, and visualise possible transformations. Ideally, the project will be permeable enough to totally transform itself, including its structure, its team, its objective, its processes, and any aspect of its content.

The group must create conditions of possibility for:

Interruption, which could be:

- Unexpected (due to the unknown, error): The group must decide whether or not to integrate an interruption of this kind.

- Provoked: The group must decide or detect whether the interruption is provoked from outside of the project (external agent, collaborator, etc), or through the process itself (due to occasional changes to the team structure, for example).

Dissent: Dynamics for contact and interaction among participants and disciplines (assumptions, presuppositions, and perspectives) can generate situations of dissent on four levels:

- Detection: Dissent can lead to changes in the recognition of the other and oneself. - Integration: Modification and generation of new action clusters and new positions. - Visualisation - Dissent as a strategy: dissent as a value in itself, which generates meaning and can be a means for its transfer.

Conflict: Unlike dissent, conflict brings into play questions of power, or reveals problems linked to the structure of the project or to transparency. Conflict must be managed through mediation in order to reach a consensus, or to determine whether the different positions can coexist on the same level without causing harm. In any case, the process must be made visible.

h) Terminating the Project and Risks

The group must contemplate the possibility of terminating the project, and create dynamics based on ongoing self-evaluation that allow its members to reflect on and define the situations or conditions in which the definitive termination of the project is inevitable and desirable (red traffic light).

The group should anticipate risks and express them verbally at the start of the project, no matter how abstract they may be, and accept them prudently, but also as a source of adrenalin. The risks to be considered include those that are intrinsic to the project, those that are related to the framing context, and those that are taken on by each member of the team, both at the professional and personal levels. Remember that identifying and evaluating these risks can be a source of content.