Mingei partners worked widely with diverse stakeholders in heritage crafts communities over the course of the project, through co-creation, research, documentation, digitalisation and much much more. Here we share good practice guidelines we have developed to guide engagement with heritage crafts communities in digital projects. This guidance is published in tandem with our ten safe-guarding steps.

1. Do your homework

Before engaging with a Heritage  Craft Community, do your homework. Gathering as much information as possible about who they are, what their background is, and what the cultural and socio-economical context in which they operate is. This information will not only save time and help in the preparation of the appropriate material for the activity but also may prevent possible awkward social interactions or even faux pas from happening (i.e., doing something that is against local traditions and customs or asking an inappropriate question). This aligns with the ‘Introspection’ section presented in the ten safe-guarding steps. 

2. Communicate clearly

Clear communication of activity goals, processes, and expectations is an important factor for a successful activity outcome. All parties involved should be on the same page as to who, why, how, and where the activity is going to be executed. Share this information well ahead of time so that enough time is allowed for the parties involved to ask for clarifications or express any concerns.

3. Be empathetic

Heritage Craft practitioners are highly skilled and understand their craft in a holistic, sometimes visceral way. However, the world of digitisation might be new to them and an engineer’s or programmer’s approach to their craft might be very different from how they view it themselves. Also, their age or (cultural) background might mean certain requirements need to be met that might not be commonly encountered by technologists. Sessions should be adapted to participants’ requirements and not the other way around. 

4. Be flexible

This refers to all aspects of planning and executing any collaborative activity with Heritage Crafts Communities or individual members. It means to be prepared to face unforeseen challenges and react accordingly. No matter how well prepared for the activity the team is, things can go different than expected. It is important to deal with such challenges promptly and accommodate changes to the original plans without compromising the value of the activity. 

5. Ease the fear of technology

In digital cultural heritage projects, not all partners involved are familiar with state-of-the-art technologies and applications. Terms like Virtual Reality, Augmented Reality, Mixed Reality, Avatars, etc., may be foreign to most people involved.  Moreover, their lack of technical expertise may make them reluctant to get involved in any technology-related decision making from fear that they have nothing of value to contribute to this aspect. To avoid such a situation from happening and encourage cultural heritage partners to engage fully in all stages of the project, the technical partners need to ease their fear of technology. Some of the ways to do that are to build technology or application demonstrators, build working or non-working prototypes that showcase the possibilities available, and to showcase existing examples of technologies that have been used in similar situations.

6. Establish a widely understood reporting medium

Craft understanding is an iterative process and alignment of all participants should take place prior to each new technology development iteration. A commonly understood reporting medium will provide insights on the outcomes of an iteration and allow further elaboration. For example, storyboards are useful for (a) illustrated scripts that decompose actions into simpler ones and (b) validating this transmitted information with the craft community, collecting feedback, and identifying parts of the process that may be underrepresented. 

7. Be considerate towards the needs of older participants

It is often the case that craft practitioners who represent an endangered craft are older people. When involving them in any type of project-related activity such as interviews, demonstrations, or co-creation workshops, there are a few points that the team needs to take into consideration to ensure a positive experience for them. These are:

• Duration of activity: keep the duration of the activity as short as possible and provide frequent breaks for refreshments, use of bathroom facilities, etc.
• Pace: people learn and think at different paces. Keep that in mind when planning and scheduling the activity and always include some buffer time to avoid rushing through the activity or running out of time.
• Envisioning abstract concepts: bear in mind that some people have no or limited prior experience in modern technologies. For example, it would be unrealistic to ask a group of people unfamiliar with digital technology to design or sketch an application/system from scratch without having a point of reference. We have found that it works best to start such activities by showing prototypes or other examples of technologies in similar contexts of use before asking for any type of input from them. Once people understand how a type of technology works and see examples of it in use, then they can then start envisioning how they can be of use for presenting the storyline of the craft they represent.

In the Mingei project, a key area of our impact work has been to focus on organisational impact experienced by our heritage crafts partners and technology partners alike. For a forward-reaching project in the digitisation and digital representation of heritage crafts, we can think of organisational impact as digital transformation. 

In the heritage sector, digital transformation has been a buzzword now for many years. In parallel with attempts to define digital transformation, like the definition developed by Europeana relating to the discoverability of heritage collections, we have seen more and more practical tools and targeted guidance emerge that aim to conceptualise, push forward and measure digital transformation. Yet there are few overviews of what was out there and what is designed for whom and when. 

To push forward our thinking on organisational impact in Mingei we conducted a snapshot review of what’s out there at the moment. We hope that this analysis will help to stimulate even more thinking about organisational impact and digital transformation, long after Mingei finishes in May 2022.

This is the second blog of a series dedicated to how Team-Based Inquiry (TBI) is being used in the Mingei project and what we are learning from it. In Mingei, each heritage partner and Waag have completed several TBI cycles, setting out a different research question, collecting and analysing the data, and implementing improvements each time. Each TBI cycle has examined different topics and led to new insights. Having introduced TBI already, we now explore some of the insights generated and what this means for organisational impact, as well as setting out recommendations to help you use TBI in your work. 

Using TBI to improve the museum visitor experience

The Mingei partners at the Conservatoire national des arts et métiers (CNAM) wanted to know why, of the many exhibitions that are part of the materials gallery (one of seven thematic galleries in the museum), the glass exhibitions were attracting so few visitors and how they could focus the visiting public’s attention more on the materials gallery. The question stemmed from the perception that the Mingei project’s focus on the digitisation of the intangible aspects of the craft of glassmaking would help to bring in more visitors, but that the layout of the museum itself, or something else, might be getting in the way.

They collected data through a series of interviews with mediators and demonstrators from the museum’s Public Department. These professionals had in-depth and direct knowledge about the visitors’ experiences that they could share. They also organised a visit with the person in the museum tasked with making changes to the museum layout. The data they collected led to practical changes being proposed: for example, a plan for renovating the flooring, improving audio-guide systems, and adding labelling and signals around the gallery. 

Asking TBI questions to help improve the visitor experience with museum digital applications

The Chios Mastic Museum investigated how instructions for three newly-installed digital applications could be improved for museum visitors. The TBI cycle was launched at the same time as these installations were being formally evaluated. One of the biggest challenges was how not to over-survey the audience. It was agreed to conduct the TBI survey once the preliminary evaluation data had been collected. Gathering data through three sessions of staff observation and surveys of museum visitors, they identified a number of areas where they could improve the instructions for the applications, including, for example, guidance on where to stand to activate a certain type of app. They also identified a clear need for museum professionals to remain on hand to answer questions and to guide visitors in how to use the apps. This is important, as it was observed that visitors often do not spend much time trying to learn how an app works. 

Colleagues at the Chios Mastic Museum also asked the question of ‘To what extent do the museum professionals understand how the digital applications work, feel comfortable using them and can explain their use to visitors and new colleagues?’. Danae Kaplanidi, scientific consultant at the Piraeus Bank Group Cultural Foundation (PIOP), reports on what they learned in a special blog post where she identifies that the question addresses a gap in the existing research literature, as well as helping the museum better train and supports its professionals in the use and demonstration of digital applications in the museum exhibition setting. 

Using TBI to harness good practice in communications and dissemination more effective

While Waag colleagues are coaching the heritage partners through the TBI cycles, we also undertook our own TBI cycles. Finding ourselves at a critical point in the project – our need to create impact through dissemination and communication – we dedicated ourselves to precisely that topic. We asked the question: what makes impactful and effective communication and dissemination in EU-funded projects?

We found eight problems with the current state of play with European funding projects communications, dissemination and exploitation, ranging from the short-term nature of project funding to project outputs (e.g. reports) not being designed appropriately for their audiences. We identified three solutions: 

• Know your target audience: talk to and with them
• Tell open, impactful, people-centred stories with substance 
• Plan long-term project legacy in a practical way

Each solution is presented alongside a series of tips and recommendations. You can read these in more detail in our blogs exploring what makes impactful communications and dissemination and how to measure the impact of communications, dissemination and exploitation in EU-funded projects. The findings are being widely shared across project and partner communication channels as well as on those channels where the original survey was published. 

Turning a challenge into an opportunity in data collection 

Data collection is an inevitable part of a TBI cycle, but what happens if you haven’t collected data from your audiences before? 

This was one challenge for the Haus der Seidencultur (HdS). Run entirely by volunteers, all of whom are retired, the museum lacks many of the resources other museums might enjoy. They asked the question of how to improve the experience of the non-guided museum visitor experience. Since re-opening after the first Covid lockdowns, they had been trying to informally collect more feedback from visitors, benefitting from the thoughts shared in the visitor book as well as the great personal connections made by guides with tour groups. Everything that was shared, however informally, was fed into the TBI cycle. The findings led the team to propose to capture and share more information about each museum exhibit with guests by creating QR codes as part of the Mingei pilot installation.

What was the organisational and professional impact of the Mingei TBI cycles? 

As a method of generating new knowledge and creating practical improvements, the TBI cycle has been a great asset in the Mingei project. We found three relevant themes that emerged as a result of the TBI cycles, no matter how different the context of the other heritage partners might be. 

The first was around informal data collection opportunities. The second was on the security of the digital devices that are being used in the exhibition pilots in the three museums. How can a museum ensure that its technology remains safely in place while creating the best and most enjoyable experience for visitors? The third was around how to train museum professionals on how to use the applications and new digital technologies. 

In addition to these very positive shared learnings, we can summarise some of the outcomes that have resulted from the Mingei TBI cycles as follows: 

• Attracting new users and those interested in heritage craft.
• Improved communication within the museum settings.
• Improved communication in the Mingei project setting.
• Potential future impact as a result of the wider awareness of the Mingei project’s developed resources.
• Better communications, dissemination and exploitation planning and delivery in future projects.
• New solutions to tricky problems (because the TBI cycles are a new tool in the professionals’ ‘toolbox’).
• Improved user experience for museum visitors using the digital applications and exploring the museum setting.

How can we improve the use and impact of TBI cycles in processes of digital transformation? 

‘The whole TBI process was not a procedure that we had implemented before. Nevertheless, it proved to be very useful when it comes to providing answers in a participative – bottom up approach and it has helped us in defining improvements in our decision making processes.’ heritage partner feedback

The cyclical and iterative nature of TBI, as well as its focus on answering the research question as a team, makes it suitable in a context of proactive partnership and skills development in processes of digital transformation, where the focus is not only on the results but the impact created through the process. TBI is a tool that can answer, in a simple way, much bigger questions relating to key stakeholders and key questions facing heritage crafts organisations in the contemporary, digital, context.

Tips and recommendations for the effective use of the TBI methodology 

• – Present clear instructions and an overview of the process from the beginning for all involved. 
• – Get people involved who have the agency to implement the changes that will be recommended through the TBI cycle.
• – Involve as many colleagues as possible in the introduction to and delivery of TBI cycles, so that everyone feels comfortable to take part. 
• – Start small and progress to bigger questions over time. 
• – Keep it lean and light-touch (this is a key attribute of the TBI cycle over other processes). 

Read more about team-based inquiry in the Mingei Hands-on Guide!

Team-Based Inquiry is a light-touch evaluation and research methodology specifically for teams that produce audience-facing and/or informal learning products. In this first of three articles about Team-Based Inquiry, we introduce the methodology and how we are using it in Mingei. In our follow-up article, we share what the heritage partners and Waag have learned from the process, and, in the final article, we’ll evaluate its use in the project and set out recommendations for others who wish to use it as a learning and impact methodology. 

The background to Team-Based Inquiry 

Team-Based Inquiry (TBI) has been around for more than a decade. It was created by the Nanoscale Informal Science Education Network (NISE Net), who have also produced a guide for museum educators on how to use the methodology. Much of how we apply the methodology in Mingei has been drawn from this guide, though we have also adapted it for our own use and published our own guide to support the use of TBI in heritage crafts contexts. 

TBI can be used for research and evaluation and improving processes and outputs in a focussed, low risk, light-touch, iterative and team-based setting. We decided to use TBI in Mingei in order to facilitate organisational learning and to give heritage partners control over the process of making impact for themselves (in terms of their digital transformation and involvement in the project) and for their audiences, the ultimate beneficiaries of their participation in the project. One of the key assets of the methodology is that it was designed to be light-touch and effective in bringing about tangible improvements, which is important if you’re already involved in a busy schedule of building apps and exhibitions, as is the case for the partners in Mingei.  

An overview to TBI: Question, Investigate, Analyse, Improve

As a method for evaluation and improvement, TBI makes use of a cycle of inquiry. This cycle of inquiry consists of four stages: Question, Investigate, Reflect, and Improve. 

Moving through these stages is a team effort. Some stages could be carried out by a smaller number of people; for other stages all team members should be involved. For digital heritage projects like Mingei, it’s great also to have technology partners involved where possible. TBI cycles can be conducted at any point in a project cycle but you should remember that the goal is to find actionable improvements. Be ready to make some changes!

We go through the four steps below, but encourage you to download the full guide (below) to find out more detail about the methodology.

1. Identify your question

We recommend kicking off by brainstorming around this prompt: What piece of information will really help us improve (the impact of) our project? If you can answer this, formulate a question that can help you find this information. According to NISE Net, your question should have at least three qualities: 

• You don’t know the answer
• It focuses on actionable and useful information
• It can be realistically investigated in the time and with the resources available

Once you have identified your question, it’s now time to get the data that could help you answer it. 

2. Investigate: find the answer to your question

As we already mentioned, TBI is a light-touch methodology. You might already have the data you need or you might be able to collect it in a very easy way. You should think about how much information (your sample) you need to collect to be able to answer your question. You should also think about the ethics of collecting this information and how to treat the data, and the participants from whom you collect the data, legally, ethically and respectfully. 

3. Reflect: analyse your information to answer your question

There are three steps to this stage: organise your data so that it’s easy to analyse, analyse the data for what they tell you about the research question, and interpret the patterns you have found in terms of what this means for your research question. You should also ask yourself: did you find anything unexpected, or did you find everything you thought you would? 

4. Improve

This is the point at which you should draw out some recommended improvements for your project. You might have to plan how to implement them as well as communicate the recommended improvements to your colleagues. 

Remember, you don’t need to make big changes to have a big impact. The beauty of TBI, as we’ve seen it in Mingei, is where some small changes can be made to really improve the experience of audiences using the digital applications in the exhibitions. 

How we use TBI in Mingei

In Mingei, each heritage partner has led between two to three TBI cycles. Each TBI cycle has taken on different topics, including how to find and work with volunteers; how to improve the experience of non-guided visitors to exhibitions; how to understand the levels of interest of museum visitors; and how to understand visitor movements in the museum. The Waag Mingei team has also conducted its own TBI cycle focussing on impactful and effective project communications and dissemination. 

Every three weeks, the three heritage partners, technology partners and Waag have met to discuss broader developments in the pilot exhibitions and to discuss and share what they have been learning in their TBI cycle. This moment of sharing what has been learned is like action research, even though this was not an original objective. We learned that however different the partners are, what they are learning is relevant to everyone. We share more about this in our next blog!

Read more about Team-Based Inquiry in our Hands-on Guide!

Participating in the Mingei project led to the Chios Mastic Museum in Greece installing advanced applications that preserve and present the tangible and intangible elements of the heritage craft of mastic production. At the same time, these applications are designed to trial new and exciting ways to learn more about crafts through digital applications. Chios Mastic Museum asked itself: how can we ensure a positive experience for our museum professionals who demonstrate this advanced technology to visitors? Meeting a gap in existing research, this case study reports on what was learned for the benefit of other museums that consider installing digital applications. 

About the Chios Mastic Museum

The Chios Mastic Museum is built in the Mastichohoria (mastic villages), a rural area in the south of the island of Chios. The museum replicates a mastic factory and consists of indoor and outdoor areas, including a mastic field. It is staffed by museum professionals and volunteers from the island. A typical day for the professionals responsible for the exhibition areas includes the opening of the exhibition (e.g. lights, audio-visual installations), keeping an eye on different exhibition areas, offering guided tours to groups and schools, and closing at the end of the day. 

About the mastic digital applications

Three applications were created and installed during the Mingei project, and each application had to be introduced to and internalised by the museum professionals. 

1. Airborne, developed by FORTH, is an immersive flight simulator allowing users to fly over various mastic villages of Chios. During the flyover, users can stop at each village and retrieve multimedia and text information related to those villages. The setup is very straightforward: it involves a desktop computer. 
2. Mastic Narratives, also developed by FORTH, is made up of four tablet devices located in four main spots of the museum. From each tablet, a specific area of the museum is covered and augmented through the camera of the tablet with ‘hotspots’. At each hotspot, a Virtual Human (VH) appears who is the digital twin of someone who used to work in that part of the factory. When the hotspot is selected the VH can be seen through the camera of the tablet narrating their life story and work at the factory. 
3. ARMINES developed Craft Training which demonstrates mastic cultivation activities through a more immersive experience. The installation consists of a personal computer and a monitor together with a depth sensor for tracking the user’s actions. The user stands in front of the installation and follows the instructions to mimic craft actions.

Researching the experience of museum professionals with digital applications 

We decided to research the experience of the museum professionals related to the digital applications recently installed in the museum through one of the Mingei Team-Based Inquiry cycles in March 2022. 

We investigated a number of different areas relating to the museum staff’s experience of the advanced digital applications described above, including their:

• understanding of how the applications work
• opinion about this addition to their everyday tasks
• comfort in using the applications
• thoughts on more efficient ways to implement new digital applications in the exhibition

We conducted a focus group interview with museum professionals working directly with the applications in the exhibition areas. 

What we learned

Experience and comfort in demonstrating using the applications

The results were very interesting and practical. The levels of comfort experienced by the museum professionals while using technology varied. All of them are open to technology but only one of them can be considered an expert and this is understandably the person that everyone turns to when there is a technical problem.

• The digital applications add value to the exhibition because they invite visitors to interact and learn about already existing information in a more playful way. 
• The applications need improvement regarding their technical stability and how their instructions were communicated to the audience. 
• They worry that during the summer they will face problems such as damage to the technical equipment and errors in the software because of overuse. 
• They worry also that they might not explain correctly the context and content of the applications. 
• Concerning their everyday tasks, there was no significant change and they are eager to learn more. 
• They expressed that they understand how the applications work and feel comfortable to transmit this knowledge.

Ideas to help museum professionals feel comfortable with current and future digital applications in future

• Hold a demonstration at the same time as when the applications are installed in the museum.
• Create an accompanying PDF file with user and installation instructions, troubleshooting information and a contact in case of emergency, cleaning requirements, a break-down of daily tasks, and background information on the context, content and development of the application.

Next steps

Getting to know the perspective of what museum professionals feel about digital applications is an under-researched topic in an era where, especially after COVID-19, digital transformation is a pressing matter. The museum professionals of the Chios Mastic Museum are not software developers but are nonetheless intimately involved in the concept development of the applications, because they have the experience of interacting directly with the audience and knowing more about their reactions and needs. We wished to research their perspective to make it easier for them to work with advanced digital applications. Looking strategically forward, we will take into account more often museum professionals’ perspectives as a source of insight on how to create effective digital activities that meet the audience’s expectations.

Waag and the Mingei partners have been collaborating intensely to optimise the project’s impact for the participating organisations and in wider heritage, crafts and technical communities (and beyond). In our previous blog, we shared key barriers and strategies to delivering impactful communication and dissemination. To move us forward, we now share ways for you to measure the impact of your communication, dissemination and exploitation (CDE), based on research conducted in the early months of Mingei and a more recent survey of professionals with experience in EU-funded projects. 

1. The impact of your communication 

To measure the success of online communication, the Mingei consortium followed the models and methods developed as part of the Let’s Get Real research project. In particular Let’s Get Real 1: How to evaluate online success, and Let’s Get Real 2: Measuring digital engagement. 

The list below presents a summary of some of these tips for you, alongside suggestions shared in our survey. 

• Analyse growth and interaction on social media platforms, including LinkedIn, Facebook, Twitter, Youtube and Instagram. The image above shows some specific indicators on some of these platforms. 
• Explore online analytics to get insight into who is engaging with your website content and how. Google analytics can help you collect data about the geographic locations of your visitors; how people find your content (e.g. through searching or social media); what people explore most often and for how long they stay; how your content is accessed (e.g. mobile or laptop); and the differences between the reach of organic and paid content, if you use this feature. 
• Assess if you have been mentioned on or by platforms that are key to your project: e.g. for Mingei, have we been featured on any crafts channels? 
• Assess to what extent your project hashtag(s) have been used across different channels (though it’s easier if this hashtag is only used in relation to your project, which might not always be the case). 
• What is your reach through your newsletters? Consider growth (and what encouraged growth) and compare interaction (through open and click-through rates) against other benchmarks, as well as your past results.
• Set a Google alert to track how often your project, activities or partners are discussed (and therefore contributing to sector or policy debates). Tools like Mention provide more extended versions of this functionality.
• If you have a platform people can sign-up to, assess how many users have signed up and when, and analyse how many of these users are active (e.g. regular users).

2. The impact of your dissemination

In Mingei, the consortium endeavours to offer open access to its scientific results reported in publications, to the relevant scientific data and to data generated throughout the project lifetime. 

Open access publishing can have more impact in terms of citations than publishing in closed-access journals. Where possible, we aim for “gold” open access. Gold means that the output is published in a fully open online archive for free and immediately. Wherever “gold” is not possible, “green” open access is pursued. This is when initial publishing may not be open, but free alternative access is provided by the author (e.g. in an institutional repository). The target is to maximise the impact on scientific excellence in ways that include publication in open access yet highly appreciated journals as well as blogs and publicly available White Papers. 

The impact of your publication relies to a great extent on the reputation of the journal in which you publish. Taylor and Francis suggest that you should consider the reputation of the editorial board in your community, the readership of the journal, and its traction with policymakers. 

When reporting on the impact of your dissemination activities, you might mention the impact factor of the journals in which you have published; the number of downloads or views of the research; and the number of citations. For the citation analysis, you might have to check several databases to review how often you have been cited. 

Mingei’s measurable outputs include:

1. Scientific impact:
   The consortium contributed with more than 35 scientific publications in prestigious journals and conferences in the area of Cultural Heritage. All the papers are in Gold or Green Open Access.
   All publications have been uploaded on OpenAir Zenodo. Furthermore, community pages were created in Zenodo and ResearchGate for the Mingei project and all publications are listed on these pages (besides the Mingei project website).
2. Tool adoptions by stakeholders: 10
3. Digital assets integrated: 10000
4. New digitisations: 2000
5. Heritage crafts digitised: 7
6. Adopted or curated content and digital assets in international repositories: 10 (but many in backlog)
7. Contribution to public knowledge (i.e. number of Wikipedia entries or edits): 3

You should also report how many conferences or events you are presenting at, whether physical or in person. You should be able to track how many people were in the (digital or physical) room when you presented and how many might now have awareness of your results by reading the conference programme (that is, the total conference sign-up). 

3. The impact of exploitation: look at impact in terms of outcomes and not just outputs

All the usual metrics (likes, comments […]) can be quite meaningless. Tell a story using some of those, but focus on impact and outcomes. 

A common theme in the responses to our survey was that the impact of CDE is not just about numbers (the outputs). Rather, impact can – where possible and where capacity exists – be measured more qualitatively, looking to quality metrics over quantity. It is generally thought that an impact narrative is strongest when you can ‘make a story out of it’ and when qualitative indicators are presented together with more quantitative results. 

Some more qualitative indicators that were suggested to us relating to exploitation include: 

• Assessing for how long, by whom and for what purposes the project resources are being used after the project ends.
• The number and type of project/research collaborations that begin as a result of the project.
• Measuring understanding and satisfaction of the users of the project resources.

Survey respondents also suggested indicators that could help you to think about the impact of the project’s resources and the exploitation of these, including, any changes in practice as a result of using the project resources and professional development amongst partners and participants. It is challenging to measure influence on projects on policy change as this sort of change takes time. Nonetheless, some examples were shared where materials published openly and for broad audiences were picked up and used by policy makers as well as in education. 

Was it useful for someone in the end? How many people have used it, quoted it, got inspired by it? 

Interestingly, one survey respondent reported project outputs being politicised and misused, but most commonly, we heard that it was usually difficult to discern usage and influence after the project ends – perhaps because people are already moving on to the next project and lack the capacity to reflect and measure change. 

Measuring the potential impact of exploitation in Mingei

‘Having [crafts] knowledge on the [Mingei] Platform is not the goal, but the goal is also to exploit it, meaning that the presentation modalities that we build can help automate the process of creating educational materials, creating demonstrations, and so on’. Xenophon Zabulis (FORTH) at the Mingei Day online seminar

In the Mingei project, we are focusing on measuring impact for the partners involved in the project. At the same time, we have also reported on the potential impact through exploitation in the following areas: 

• Capacity building for heritage professionals (where 83 professionals have already benefited from a curriculum developed for the DigiTraining project)
• Creative industries exploitation and impact (where we have seen how technology and crafts combine in this example of a digitally enhanced woven handbag)
• Future research projects and closer research collaboration (including two Greek crafts projects, close working with Europeana and the CRAFTED project, and plans to further extend the technology)
• The possibility to bring crafts to previously less-catered for audiences (including creating technology that can be reused for the benefit of those who are hearing disabled or blind, e.g. in the sign-language narration in the mastic pilot in Chios)
• Future heritage crafts education (where training materials was captured for apprentices in each of the Mingei crafts contexts)
• Immersive learning experiences for museum visitors (with the open-source technology in terms of crafts representation and its gamification being made available for further development)
• The reuse of the crafts content for any number of uses (because the content is now published in several open source platforms, including, for example, the Mingei Open Platform, Europeana and Zenodo). 

Such reuse has the potential to support sustainable tourism and destination management, and a stronger, more competitive Europe. 

Figure 3. Design pattern game (left) and punch card game (right), part of the silk pilot at HdS, Krefeld, Germany. 

Final impact measurement tips

We have learned that people are striving to tell more meaningful stories about the impact of their CDE activities and the project’s outputs and learnings. We finish this article with five top tips to help you measure and report on your project’s impact right from the start to the very end. 

1. Start off with a benchmark – that is to say, know where you started from and when! Capture change on a regular basis (e.g. every six months) or as often as you are required to report on your progress. 
2. Do your research right at the start. For example, what channels should you engage with? What hashtags can you use? What hashtag could you make for your project to assess its relevance?
3. Use a tool like the Europeana Impact Playbook to help you think about who you will have impact for and how you can measure this. 
4. Report to your funders in a standardised way – set out how you will report at the beginning of the project and follow-through with this so that they can easily see the progress in your communications and dissemination. 
5. Maintain a log of direct and indirect impact that all project partners can contribute to. This could be as simple as a Google document where you share examples of tweets, publications or citations related to your project outputs. 

Bibliography

Malde, S., Finnis, J., Kennedy, A., Ridge, M., Villaespesa, E. & Chan, S. (2013), Let’s Get Real 2. ‘A journey towards understanding and measuring digital engagement’. Report from the second Culture24 Action Research Project.

https://www.keepandshare.com/doc/6593572/lets-get-real-2-colour-pdf-11-2-meg?da=y (date accessed: 18/05/2020)

Finnis, J., Chan, S. & Clements, R. (2011), Let’s Get Real. ‘How to Evaluate Online Success?’ Report from the Culture24 Action Research Project.

https://www.keepandshare.com/doc/3148918/culture24-howtoevaluateonlinesuccess-2-pdf-september-19-2011-11-15-am-2-5-meg?da=y&dnad=y (date accessed: 18/05/2020)

Finne, H., Day, A., Piccaluga, A., Spithoven, A., Walter, P. & Wellen, D. (2011), A Composite Indicator for Knowledge Transfer. Report from the European Commission’s Expert Group on Knowledge Transfer Indicators. European Commission. https://ec.europa.eu/research/innovation-union/pdf/kti-report-final.pdf (date accessed: 18/05/2020)

As part of the Mingei project, we researched this important question by using a Teams-Based Inquiry (TBI) methodology. By looking at the available literature and collecting thoughts from our peers and partners on what works and what does not, we wanted to pull together recommendations on how we can create the most impact through communication and dissemination and, importantly, strengthen the final months of the Mingei project and its legacy.

In this post, we set out the problems we found, followed by some solutions with key tips on how and what to improve. Though we focus mostly on EU-funded culture, heritage and society-related projects, much (if not all) of what we learn can be taken on board by projects of any size and focus. In a second post, we share how you can best plan and measure the impact of your communication, dissemination and exploitation (CDE) activities – after all, this is a requirement for most European-funded projects!

Eight problems with the current state of play

I worked in 20+ projects but I wasn’t impressed by any of them in terms of communication or dissemination. Survey respondent.

1. Three or four year funding cycles are sometimes a risk for knowledge retention and knowledge sharing, limiting the project impact. 
2. Funding and capacity are not (always) available after the end of the project to promote the project outputs and project partners see the activities as finished once the planned activities stop. 
3. There is little reflection or evaluation of CDE, which means that improvements are not implemented and good practice is rarely shared.
4. Communication and dissemination are sometimes considered to be add-ons to the project. Public communication and wider dissemination were noted as some of seven key challenges in the interim evaluation of Horizon 2020 (European Commission, 2017). 
5. Responsibility for CDE might be left to the communications coordinator, who, being a communications professional, might have little insight into the research underway. 
6. On the other hand, responsibility might be left to a researcher with little to no experience of identifying and communicating to target audiences and who is unlikely to have time or the opportunity to stay up-to-date with trends and good practice in CDE, or have a network upon which they can draw. 
7. Projects often don’t identify target audiences at an early project stage. As a result, they don’t know how and when to create impact by sharing project updates and outputs with their audiences. How can you talk to your audience in the language that resonates with them if you don’t know who they are? 
8. Project outputs are often not designed for their audience. They might use internal language (‘why would any audience care about this?’) and be ‘inward-looking’. Materials may be badly designed and too complex, the language might be too ‘clinical’ or ‘corporate’ and the text too long. The media chosen might not be suitable for the audience they are trying to reach. 

Solution 1. Know your target audience: talk to and with them

All project partners should get involved in communication and dissemination activities. Each of them should identify a list of contacts from the target group of the project that they could engage from the beginning of the project. Survey respondent

From the very start, you should be leading a project that addresses a ‘real need’. When this is the case, CDE is very simple. In some cases, it might be that the need is technical innovation which is harder to pinpoint to one specific stakeholder beyond the scientific or tech sector. 

In either case, you need to think far beyond a social media plan. A comprehensive and successful CDE strategy is much more than Twitter! 

Tips for planning and strategy (before or at the beginning of the project)

• Ensure that you have wide project buy-in for your CDE strategy: get everyone involved right at the beginning to co-create the strategy and list of target stakeholders.
• Clearly set out the responsibilities and expectations regarding CDE for all partners.
• Ensure that your CDE strategy or plan is clear and easily understood.
• Brainstorm and plan moments and ways in which you can reach your target audiences during the project. 
• Invest in your branding and visual identity, including your project logo. 

Solution 2. Tell open, impactful, people-centred stories with substance 

Be down-to-earth when it comes to project description, precise with language and format, and include palpable results, not just perceived benefit to the final, end-customers. Survey respondent

Storytelling and narrative-building is key to communicating and disseminating your project outputs. This helps make what you share memorable. How you frame these stories is also important. We need to show ‘the human side of a project, organisation, brand’. Outputs should be open and reflective (sharing, for example, ‘learnings and little failures’). Don’t go ‘grandiose’ – keep your language simple and humble. 

Try to connect emotionally with your audience. You can start with telling the stories of those involved in the project and those benefiting from your work, for example. When talking about impact, ensure that you are really talking about impact. Share the data but also draw on testimonials and real-life experiences. What matters is that you share content with ‘substance’. We learned from our survey that there have been cases where projects got the visuals right but lacked the depth of content behind them. 

Tips for engaging with your stakeholders during the project

• Don’t expect good relationships with your audiences at the end of the project if they are not nurtured throughout. You need to plan regular communication throughout the project lifespan. 
• Take your project to your audiences. 
  • Plan demonstrations of your work and design these for the specific audiences. Think about how these could be delivered by ‘users who had their problems solved thanks to these results’. 
  • Plan get-togethers, online or onsite, where you can demonstrate and discuss project outputs. 
  • Consider how to safely bring people together in a room, where possible.
• Don’t only project your content to your audiences: get their perspectives and make it meaningful and a genuine conversation.
• Collect and share other relevant content, not just that related to your project. 
• Consider how you can avoid the ‘deliverables format and publish in documents that people like to read and share’.
• Encourage partners to act as ‘ambassadors’ and ensure that they remain active in communications through the project. 
• Invest in paid social media opportunities. 
• Connect with local stakeholders at a local level. 

Solution 3. Plan long-term project legacy in a practical way

There is a virtual graveyard of thousands of these websites of finished projects, not used because no one is taking care of them as project periods have ended. Survey respondent. 

All partners should be involved in discussions and planning for your project impact at the start of your project. You should already know for whom and how you’ll create impact but now you need to focus on practical ways to create impact as your project is finishing and then after the project finishes. The recommendations below cover these four stages. 

Create momentum at the close of a project

• Think personal. Send personal emails to those for whom the project is likely to have a big impact, e.g. specific contacts in cultural communities or policy-makers. They’re also more likely to take notice if the content is ‘transferable’.  
• Share your results in ways that make them scalable or adaptable. Publish your outputs using open licences (e.g. Creative Commons) so that they can be exploited and used by others. Being open alone isn’t enough – you must also think about how others will find the content (Campos and Codina, 2021). 
• Revamp your project website as the project archive. Bring the most important content to the fore and make it easy for the main outputs and ‘success stories’ to be found. 
• Think about your audience when designing your outputs, e.g. keep it short and concise for policy-makers. 
• Actively promote the project outputs via partner dissemination channels and not just through the project website and social media. 
• Consider if you’ll need an external mediator between you and policy makers – they could help ‘interpret’ your results and then make sure they are ‘appealing to the recipient’ (Rodari et al, 2012). 
• Consider how you can connect or include your final event (if one is planned) to an existing conference, where you might reach an even bigger audience and save time and money in terms of event planning. 
• Peer and sector dissemination channels can act as multipliers: harness your relationships and share your project outputs with them. 

If it is difficult to have partners commit to communication and dissemination during the project lifecycle, it is even more difficult after. Survey respondent. 

Create legacy and impact after the project finishes

• Actively target policy-makers on a longer-term basis if your recommendations include policy-change. Don’t just expect change to happen without some effort on your side.
• Make your outputs available on open repositories (e.g. Zenodo) and in particular, in those related to your field, if these exist. 
• Keep the website and social media alive. Consider sharing responsibility for managing these between partners in short chunks of time (e.g. three or six months). 
• Openly share your future project ideas or recommendations so that others can take these forward, even if you won’t be involved. 
• Consider drawing the good practice from related projects into one resource or add your results or resources into another project. One example of this is the Waag Co-Creation Navigator, which draws together content from multiple projects.

Bibliography

• Accelopment (n.d). Make an impact. Available from: https://accelopment.com/services/comm-diss-ex/ Accessed 4 April 2022.
• EIT Raw Materials, Project Communication and Dissemination Guideline (considers storytelling, asks good questions regarding dissemination, etc; print, online and events as the pre-Covid trio of dissemination activities). Available from: https://eitrawmaterials.eu/wp-content/uploads/2019/10/EIT-RawMaterials_Project-Communication-and-Dissemination-Guideline.pdf Accessed 4 April 2022.
• European Commission, Directorate-General for Research and Innovation, Key findings from the Horizon 2020 interim evaluation, Publications Office, 2017, https://data.europa.eu/doi/10.2777/708544 
• European Commission (2021), Communication, Dissemination and Exploitation. Why they all matter and what is the difference. Available from: https://rea.ec.europa.eu/system/files/2021-11/Communication%2C%20Dissemination%20and%20%20Exploitation-2021.pdf Accessed 4 April 2022.
• LEITAT (2019a). What makes a good communication, dissemination and exploitation plan of a research project? Part 1 – Communication. Available from: https://projects.leitat.org/what-makes-a-good-communication-dissemination-and-exploitation-plan-of-a-research-project-part-1-2/ Accessed 4 April 2022.
• LEITAT (2019b). What makes a good communication, dissemination and exploitation plan of a research project? Part 2 – Dissemination. Available from: https://projects.leitat.org/what-makes-a-good-communication-dissemination-and-exploitation-plan-of-a-research-project-part-2-dissemination/ Accessed 4 April 2022.
• LEITAT (2019c). What makes a good communication, dissemination and exploitation plan of a research project? Part 3 – Exploitation. Available from: https://projects.leitat.org/what-makes-a-good-communication-dissemination-and-exploitation-plan-of-a-research-project-part-3-exploitation/ Accessed 4 April 2022.
• Campos, A. and Codina, L. (2021) Communication, dissemination and exploitation strategy analysis in Horizon 2020: keys to multiply the impact of european projects. Prisma Social. 2021;32(1):293-319. Accessed on 12 January 2021. Available at https://repositori.upf.edu/bitstream/handle/10230/47027/Campos_PSJ_comm.pdf?sequence=1&isAllowed=y
• Rodari, P., Bultitude, K. and Desborough, K. (2017). Science communication between researchers and policy makers. Reflections from a European project, Jcom 11(03) (2012) C07. Accessed 12 January 2022. DOI: https://doi.org/10.22323/2.11030307
• Eureka (2021). Beste verfügbare Praktiken und Kanäle für die Kommunikation und Verbreitung in Projekten mit mehreren Akteuren. Available from: https://h2020eureka.eu/de//news/best-available-practices-and-channels-for-communication-and-dissemination-in-maps. Accessed on 5 April 2022.