About the Centre des Arts et Métiers

The Centre des Arts et Métiers (CNAM), Paris, France, hosts a museum of technological innovation and contains objects related to both the artistic and more industrial production of glass. Furthermore, it holds historic archives regarding the artefacts and techniques under study. All of these perspectives were harnessed in the Mingei pilot installation, which was open until the beginning of April 2022.

The pilot installation: training the public in glass blowing processes through re-enactment

The installation targeted craft presentation through an exploration of the workspace, as well as craft training through an interactive experience where users re-enact gestures of a glass master holding a tool and receiving audiovisual feedback on the accuracy of their performance. Preliminary evaluation results show high acceptance of the installation and good user interest.

Glasswork is a traditional craft that combines hand and body gestures and a thorough understanding of the material. It is a challenging craft because the material changes states from liquid to solid during production. While this complexity was not presented in the visitor-facing installation, in Mingei more broadly we pushed forward the technical means for capturing and conveying these sensory aspects of glasswork, that is to say, the requirements of dexterous aspects and tool manipulation in craft presentation and preservation. 

Learning and iterating: what we learned from user-experience evaluations

After the technical validation of the installation, we conducted a short preliminary evaluation with museum personnel. The first part of the preliminary evaluation was conducted with users from the education department of the museum who were invited to experience the installation and mimic the craftsperson actions using the bench and tools provided. What was learned led to changes to the user-interact (UI) to (a) provide real-time help to users to guide them through the training process and (b) enhance the feedback users get while using the app to better understand whether they are copying the movements right or wrongly. We fixed a glitch that meant that users sometimes thought they were doing it wrong because the feedback came too slowly. stopped with the application because they didn’t receive fast enough, and instead thought they were doing it wrong. 

A wider evaluation with visitors was conducted later. We asked a user-experience evaluator to monitor how users interacted with the installation. Minor issues with the UI were improved, including the addition of introductory screens to assist users to know when the presentation element had finished and when the training session was beginning (and when they were expected to get active). 

Responses from museum visitors

There were regular visitors to the installation, located as it was in part of the impressive church in the museum building, Saint-Martin-des-Champs. An audio component meant that the installation piqued the interest of those outside. 

Feedback collected via our post-interaction questionnaire showed that what seemed to impress visitors the most was the whole concept of being able to mimic the gestures, or as one of the visitors characteristically wrote “being in the shoes of the glassmaker” and receive feedback on the accuracy of the movement in real-time. Using a real-life workbench and glass blowpipe only added to the authenticity of the represented scene and further enhanced the whole user experience.

Find out more for yourself in the video below and explore the digital presentation of glass-blowing on the Mingei Open Platform.

International webinar Waag: Technology as a means of preservation

Online, four heritage experts along with moderator Nicole McNeilly conducted an international webinar focused on how technology can be utilised to preserve heritage crafts. During the presentation, the audience learned more about the three Mingei pilot projects, on glassblowing, silk weaving, and mastic growing, which will create tools for heritage craft presentation and guide future research.

The Mingei project platform and different technologies like 3D reconstructions, used to preserve and represent heritage craft, were also demonstrated. Following the presentation was a panel on various topics like the inclusion of AI in craft preservation, how the Mingei project can serve to pass on informal heritage craft knowledge to a broad audience, and how this knowledge of the past can serve to inform our future.

rewatch the webinar

Local session Waag: Fashion as a thread between past and present

At Waag, creative Director Dick van Dijk provided an overview of the Mingei project and introduced the attendees to keynote speaker and renowned fashion designer Antoine Peters’ work, saying that it ‘looks into the past and provides new context’ for the future. During the keynote, Peters discussed several of his projects including his collaboration with the Zeeuws Museum. For the museum, he reimagined a traditional nineteenth-century garment from Zeeland, the yak, as a modern garment: the Jaktrui. In creating the Jaktrui, Peters ‘wanted to communicate something from the past to the now and translate it in my own way’. The zero-waste folding technique was then used for economic reasons, but now is very relevant from a sustainability perspective.

Following his presentation was a workshop on the craft of repairing clothing. This workshop was designed based on the Reflow project aiming to share knowledge on how to rethink, repair, and revalue your wardrobe. During this workshop, attendees were encouraged to rethink items of their own clothing focusing on both aesthetic and technique in clothing repair.

• Rewatch the workshop
• Watch the instruction
• View the photos of the workshop (photo credit: Jimena Gauna)
• An impression of the local sessions from the European partners will be published on the Mingei website.
   

How heritage can shape the future

So how does the Mingei project serve to connect the past, present, and future? Inspired by the Mingei movement in Japan, which originally served as a response to Western mechanisation in the mid 1920s, the Mingei project today focuses on the digitalisation and accessibility of heritage craft, both tangible and intangible.

Through use of modern technologies like interactive Augmented Reality and Mixed Reality, Mingei seeks to tell stories not only about the craft objects themselves, but about the rituals, practice, and knowledge that accompany these objects. The application of modern technology to heritage craft can then serve to build a bridge between the past and present.

In regard to the connection between past and present, Antoine Peters notes that ‘a design or a translation now always has this reference captured in it. So you have these little bridges – in storytelling or in the visual part’. When Peters was researching the yak, he found that no documentation existed detailing its construction proces – namely, how to take one piece of fabric and fold it to create the jak. Instead, he learned the folding technique from 91-year-old craftswoman Mrs. Vos.

This mirrors a challenge that was discovered during the Mingei project: during a glassblowing pilot in Paris, there was no documentation that outlined the movements and rituals of past glassblowers. Similar to the work Peters did to understand the historical process of crafting the yak, those working on the glass pilot had to find alternate methods to learn craft heritage techniques and movements and were able to reverse-engineer steps required for glassblowing. Both Peters and the glass pilot help to further an understanding of the past while contributing valuable knowledge to the future.

Through work like the Mingei project and Peters’ collaboration with the Zeeuws Museum, modern concepts and technologies can be applied to the past in a way that creates bridges between the past and present. When talking about heritage, Peters noted that the past and present cannot be separated; that ‘it’s all connected’. Examining these connections allows us to see the thread that connects the present day with the past and tells us stories that can be leveraged to imagine the future.

Exhibition CNAM Paris

CNAM organised an exhibiton where the worlds of academics and professional activity come together. It is the only higher education establishment dedicated to life-long professional training. A dedicated space at the cathedral which is part of the museum invites you to experience the craft of glassblowing and use actual glassblowing tools.

Local webinar FORTH – Greece

FORTH organised two webinars for Mingei Day (videos are in Greek).

Mingei Day Geneva – Reenacting 3D craft people

But Mingei Day is not over yet. On 9 and 10 July MIRAlab is organising a local session for Mingei Day in Geneva during The Night of Science. The partners main goal is to assure the perennity of certain gestures and attitudes when former people were doing crafts. Through digital simulation, we can preserve the intangible heritage.

Miralab intend to present videos of the “making of” of the digital craft people who are reproducing the gestures of our 3 activities: Glass, Mastic and Silk. As well as the setup of the three pilots.

In the last four years, Waag and nine European partners and craftsmen have experimented in Mingei project by documenting and digitalising crafts, storytelling, interactive Augmented Reality (AR), Mixed Reality (MR) and motion capture.

During this webinar on Mingei Day we would like to share and discuss the results and knowledge we gained. In four online panel discussions, experts from all over Europe will discuss and share their views on how to preserve crafts and how we can keep improving this in the future. During the webinar, participants will also have the opportunity to ask questions online.

Join the conversation! The link to the meeting will be provided to you by email. This event will be recorded.

Programme

16.00 hrs – Welcome Mingei Day – host Nicole McNeilly (Impact Evaluation Advisor Waag)
16.05 hrs – Introduction to Mingei Project (Xenophon Zabulis – Project Coordinator Mingei)
16.15 hrs – Expert panel discussion will cover the following questions:

1. What is the urgency of preserving and documenting crafts?
2. What is the impact for the craft and heritage community, education and future generations?
3. What tech advances are helping us make steps in the preservation and documentation of crafts?
4. How could the Mingei platform be useful for future users to make the impact sustainable?

17.35 hrs – Q&A from participants and recap lessons learned
17.45 hrs – Closing

Panelists

• Xenophon Zabulis – Research Director FORTH, project coordinator Mingei
• Carlo Meghini – Research Director at CNR-ISTI and developer Mingei platform
• Arnaud Dubois – Research Associate at CNAM, social anthropologist Mingei
• Eirini Kaldeli – Researcher and AI expert involved in the Crafted Europeana project
• Marinos Ioannides – UNESCO chair Digital Cultural Heritage at Cyprus University of Technology

Have a look at this episode of Euronews (Europe’s leading international news channel) dedicated to one of the many innovative ideas of Mingei that engage today’s youth with past traditions.  

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement no. 822336.

Programme

19:15 – 19:30 hrs: Walk in
19:30 – 19:45 hrs: Introduction by Dick van Dijk (Creative Director at Waag)
19:45 – 20:15 hrs: Interview/presentation Antoine Peters
20:15 – 21:45 hrs: Workshop Reflow: Don’t let your textiles go to waste

Mingei Day

During the Mingei Day on March 10 we will provide insight into the research and applications of the Mingei project in an accessible manner. In the Mingei project, Waag works with European partners and craftsmen on ways to document traditional craft techniques. How can we use technology to preserve these crafts for the future? We do this, for example, by storytelling, interactive Augmented Reality (AR), Mixed Reality (MR) and motion capture. This way, the knowledge about the actions of traditional and industrial crafts can be preserved. During Mingei Day, passionate craftsmen show you what their craft is, and Waag shows you how you could document them.

Do you also want our crafts and clothing not to get lost? Come to Mingei Day on 10 March. Register if you want to be physically present in Waag’s Makers Guild (limited places available) and don’t forget your broken piece of clothing! You can also join the event from home. The link for the online live stream will be shared in the run-up to the event if you sign up.

Antoine Peters

The keynote of the evening is the worldwide famous fashion designer Antoine Peters. He has worked at Viktor & Rolf and worked with iconic brands such as Marcel Wanders, Moooi, United Nude, Quinze & Milan, Effio, Eastpak, Gsus Sindustries, EYE and Kidscase. Just like in Mingei, Antoine’s working method is characterised by his interest in traditional crafts.

In his work, Peters is concerned with the stories, historical development and conservation of these types of crafts. For example, he conducted intensive research into a nineteenth century yak for the Zeeuws Museum. He learned the craft of folding such a jacket, a technique that is more than two hundred years old, from a 92-year-old woman from Middelburg. She was one of the last wearers of the Walcheren regional dress. The uncomfortable fit of the yak gave the fashion designer the idea to use the traditional technique to make a sweater from soft recycled jersey.

Workshop

The necessity of preserving crafts, and the pleasure that these crafts can offer, are made clear in the workshop ‘Don’t let your textiles go to waste’. Did you know that in Amsterdam millions of kilos of textile end up in the wrong bin and are burned? And that people have an average of 170 pieces of clothing in their closet, 50 of which have not been used in the past year?

These workshops are designed to transfer knowledge on how to reuse, repair, reduce, rethink, recycle and revalue your wardrobe. During the workshops, developed within the Reflow project, you will learn how to repair holes in your clothing by rethinking the craft of clothing repair, and re-evaluating old garments.

Online streamers can already collect the following supplies:

An item of clothing you want to repair (socks, jumpers)
Wooden Embroidery Hoop
Wooden Darning Mushroom
Mixed colours of 100% Cotton Threads
Mixed colours of 100% wool Yarn
Chalk Pencil
A set of mixed needles and metal pins Darning needles
ruler and scissors

​

DigiTraining is specifically targeted to benefit small or midsize museums, as well as other tangible or intangible cultural heritage organisations, which include structures and facilities accessible to the general public.

Mingei Online Platform 

Mingei Online Platform (MOP) is an online authoring platform developed in the Mingei H2020 IA, maintained and used in additional projects by FORTH, The Foundation for Research and Technology Hellas in Greece. The Mingei Online Platform facilitates the representation of the socio-historic context through narratives.

The purpose is to:

• Document, represent, and preserve intangible dimensions along with objects and sites
• Contextualise presentation of tangible heritage
• Systematise and facilitate the presentation of socio-historical context
• Explore and promote world heritage, stimulate interest through educational and fascinating content.

Nowadays, not only the treasures of culture but also the stories, the values, and the collective memories of European citizens can be preserved and enhanced through digitalisation. This platform is a useful tool for preserving both tangible and intangible Cultural Heritage, as users are able to add  information in such an easy way, by registering both texts (historical events, dates) and multimedia material, such as images, videos, 360^ο videos, 3d reconstructions, which are interconnected, creating relevant stories and narratives. In addition, the digitalisation of Cultural Heritage in combination with the creation of stories by using Mingei Online Platform (MOP) will stimulate visitors’ interest in tangible and intangible Cultural Heritage, making museums and cultural organisations more attractive.

Pilot

After its pilot evaluation, the Mingei Online Platform was utilised by more than 80 cultural Heritage Institutes in the DigiTraining Creative Europe project, to create representations for and on their own. This activity provided immensely valuable feedback in the optimisation of its User Interface. The Mingei Online Platform (MOP) implements a protocol for the advanced digitisation of Institutes Cultural Heritage and socio-historical context, through narratives and process schemas. 

Structure of capacity building programme

The programme is structured in 3 different levels; the General Programme will provide many organisations with a combined training on digital and audiovisual technology, as well as on digital-related management. The Specific Programme will provide up to 6 organisations with mentoring through a specific training programme resulting in a strongly increased capacity in digital skills as well as in strategy and management tools to deal with them. Finally, through the Qualified Programme a maximum of 3 selected organisations will benefit at no cost from the production of a virtual or augmented reality audiovisual project specifically adapted to their mission and narrative, together with the strategic and managerial advice to best integrate it in their activities.

Combining digital technology research, execution and training

DigiTraining’s consortium is composed by an experienced team from five different countries with complementary skills and competencies to respond to the urgent need for providing specific support to the cultural and creative heritage sector. This team merges tested expertise in digital technology research, execution and training; in direct support on management and innovation for start-ups and midsize organisations; in media, communication and audience development in the cultural sector; and in audio-visual & virtual reality production for the arts and cultural heritage.

Mingei Day

During the Mingei Day webinar on 10 March the the team of FORTH will give a demo of the Mingei Online Platform. Sign up for the webinar and join the discussion! For more contact about the Digitraining Project, please leave your contact details here.

India has a long weaving tradition. Although the bulk of the country’s textile industry has been industrialised and moved to cities, the knowledge and skill of handweaving still exists in rural areas. Safeguarding this Heritage Craft takes a different form in India than it might in some European countries, for example, as a result of different heritage traditions, and unique challenges and opportunities that exist here. We spoke with Sumit Dang, director of Meraki Signature Private Limited, and Suket Dhir, fashion designer. Dang has been involved in various projects that safeguard weaving heritage through commercial practice. Dhir creates high-end menswear and womenswear designs, using primarily handwoven fabrics, sourced from and created in collaboration with expert weavers living in rural communities.

Same challenges, different contexts

Both Europe and India face the same biggest challenge in terms of Heritage Crafts. The numbers of Heritage Craft practitioners are dwindling, and the population is aging, as young people move to cities or find employment elsewhere. The big difference between Europe and India, however, is the context within which this challenge takes place. In India, this context consists of more significant rural isolation, greater socio-economic differences and cultural differences between rural communities and city dwellers. Where in Europe often local communities can maintain a craft as a hobby, or for example develop a heritage site that can facilitate the safeguarding of a Heritage Craft, this is not an option in most rural communities in India due to poverty, a lack of infrastructure and other socio-political structures. 

The solution surfacing in India is also different from that in most of Europe; it is commercial. For the case study at hand, ensuring weavers receive fair and regular pay for their heritage products is the best way to safeguard this particular Heritage Craft. A good example of a company working in this model is Jaipur Rugs, a company specialising in producing heritage rugs for a global market. This family business works closely with weaving communities and international designers. The needs and traditions of rural communities have shaped the company’s business model. 

Traditionally, rural weavers learn the craft within their community, often from a very young age. Weaving is generally one of various means of income, with agricultural work often being another important source of food and income for a family. Men and women weave at home alongside their other duties and children learn in this family context. However, contemporary child labour laws and the country’s experiences regarding this issue hinder this traditional way of learning how to weave. Children are not allowed to help out in the family business. As a result, the way families hand down knowledge and skills is interrupted. 

A cottage industry for a global market

Dang identifies two main requirements with regards to safeguarding hand weaving. First, hand weaving needs to provide attractive job prospects. Secondly, there needs to be a robust market for the products that are being produced. Needless to say, these two requirements influence each other. Of course, many Heritage Crafts have developed as a means of creating an income, but in the European heritage field this commercial side of Heritage Craft is sometimes problematised or even ignored. Taking a commercial approach, while respecting the heritage of a craft, can lead to different choices and solutions. 

For example, Dang emphasises the importance of focusing on those opportunities that are likely to yield commercial success. For a Heritage Craft to be monetised, it will need to be able to find a solid (international) market, as well as have a unique heritage quality. If there’s a good likelihood both can be achieved, it will be possible to gather the financial investment that is necessary to connect the producer(s) with the market. Therefore, it is important to understand the traditions and potential of the different Heritage Craft Communities. In addition, it is important to analyse the challenges they face when it comes to developing a market for their product. 

As many weavers live remotely, attention must be paid to creating infrastructures for knowledge exchange, source materials and finished products, and distributing finances. Jaipur Rugs has set up a small number of weaving hubs, where weavers can come together to weave, creating one physical space where information, materials and money can be exchanged. However, most weavers did not want to give up their traditional way of working, which traditionally is at home and planned around other activities and care responsibilities. This obviously greatly influences the infrastructure that is needed. Some weavers create their own designs, others work together with (international) designers. Designers need to understand the traditions, possibilities and boundaries of an individual weaver to create a suitable design. Weavers who work with designers can often earn more money, because they create a product that is in higher demand. When an end product is of higher quality, it often requires more time to produce, but it will also fetch a higher price.

Raising awareness

Handmade products cost more than those produced by machine and in India, where mass-produced woven products are readily available and where craft is often associated with charity, marketing Heritage Craft products can be challenging. Cutting out the ‘middleman’ or working with experienced distributors can help bring the cost down while maintaining fair wages for the weavers at the same time. However, most important, according to Dang, is to raise awareness amongst customers. Explaining the unique qualities of Heritage Craft products is key. The argument that handwoven materials are more expensive will not convince customers to pay the extra price. What are they getting ‘extra’ when they buy Heritage Craft textiles? 

In marketing Heritage Crafts, Dang suggests focusing on tradition, the human touch, the physical quality, the story behind the product and the fact it cannot be made by machine. Neither producers nor designers are necessarily good at building a brand. Therefore, it is important to invest in gaining this skill. A successful brand is not carried by one or two people, but exists within an ecosystem of people with different practical and commercial skills. It is crucial that the Heritage Craft Community is part of this ecosystem. Jaipur Rugs invests in training local community members and grassroots leaders to achieve this. Finally, creating a strong online presence, including background stories, images and videos can help people to familiarise potential customers with a brand.

Designing with handwoven fabrics

Suket Dhir sources all his textiles from India and 70% of the textiles he uses are woven on handlooms. This has a direct impact on the price of his products. His designs are high-end, but also designed to be worn for a lifetime, to be heirlooms and handed down. Or, as his grandmother used to say: “Buy little, buy good.” For this reason, Dhir focuses on creating designs that have personality, but can be worn to various occasions and are made to last. A certain romanticism can come with working with handwoven materials, but in order to build a successful business, pragmatism is also important. Therefore, Dhir focuses particularly on using handwoven materials when they can only be made using a handloom. Fabrics that can be produced by machines should continue to be made by machines. Handlooms would not be able to meet global demands and focusing on the techniques that are unique to handlooms helps preserve them.

The risk of guilt

When working with comparatively poor rural communities, it is often easy to act on a feeling of guilt. Dhir describes how the first time he worked with weavers from West Bengal, he was so focused on giving people a fair wage, he offered double the price they were used to getting. He was advised not to do it, but felt it was the right thing to do. He was surprised to find his first shipment was late and when he went to visit the weavers to see what was causing the delay, he was met by an angry wife. Now her husband earned twice as much for the same work, he only worked for half the time, meaning his production was falling behind and he had more spare time in which he could spend his money. For the family, this was not a positive change at all. This experience was an eye opener for Dhir and since that time he has noticed that guilt is often also used in sales further down the line. Paying a fair price for a product is often framed as ‘charity’. Many campaigns for fair fashion focus on the people making the clothes, and play into a feeling of guilt with the buyer: The poor weaver deserves a fair wage. But, as Dhir puts it: “There’s no improvement to the livelihood of the person who makes your clothes just because you see a picture of them.” What’s more, this approach ignores the mastery and skill a weaver has developed. This is why Dhir believes in focusing on the quality of the craft and creating high-end made-to-last pieces with high quality fabrics.

Many rural communities create their own distinctive weaves that can only be created on a handloom. Dhir sees potential to elevate these weaves and give them a cult-like status. Just like, for example, Japanese samurai swords have gained a recognisable status, resulting in people being willing to spend the money for an original, handmade sword. This is, as Dhir sees it, another way of building a ‘brand’. How can one present their traditional craft in a way that translates its traditional value into monetary value? 

At the same time, it is important to contemporise fabrics. Traditional weaving methods can be maintained while using more contemporary colours, for example. Dhir advocates celebrating the craft, the making, which he sees as distinctly different from the often-used trope of empathy, especially in the context of charity. The people making the products will come and go, but the craft, the art, will remain as long as it is passed down. Dhir believes focusing on the inherent quality of the craft can provide better chances of gaining a fair price for a product. If the craft itself is elevated to a cult-like status, there will always be people interested in learning it. The craft would not have to rely on children following in their parents’ footsteps, something that will inevitably become rarer over time. 

His biggest challenge now is to reach a global audience that might be interested in his designs. Where cheaper brands can rely more on online sales, handwoven materials need to be touched to convey their quality and value. There is an untapped potential in the longstanding heritage of handweaving in India which could fuel the birth of true luxury brands.

Based on an interview with Sumit Dang, director of Meraki Signature Private Limited, conducted on 30 September 2020, and Suket Dhir, fashion designer and winner of the   International Woolmark Prize 2015-16 on 12 April 2021.

The essence of an expertly executed craft is too nebulous to define. This is the reason why Mingei uses a variety of methods to record and present traditional crafts to the public. However, the difference between an expert and a novice is not their ability to adhere to strict guidelines and instructions, but rather how well they can decide when to break the conventional rules and create something unique. Even more interestingly, those decisions are made subconsciously. The expert can decide what motion will be the most efficient and perform it precisely, seemingly without any preparation. Since the motion of the body is the most important tool of performing a craft, Mingei uses motion capture to document all the subtle details of an expertly execution.

Motion capture

Motion capture (MoCap) is the recording of the joint angles of the human body during a task. In our previous article, we explained our first steps in digitizing the crafters’ movements. Unlike standard video, the only thing that is recorded with MoCap is the human skeleton and its associated motions. Even though general information about the environment and the appearance of the person is lost, this method gives the motion of each joint in detail for all three dimensions. This is why it has been widely adopted for movies and video games as well as medicine for many years. However, until now, its use for preservation of crafts is relatively limited.

The workshop as recording environment

Very early in the MoCap sessions of Mingei, a few differences became apparent when recording for cultural heritage. The first and most significant difference was the environment. In most MoCap sessions in either entertainment or medicine, the recording environment is heavily controlled. The sessions are most of the times indoors, in a room specifically used for MoCap, and whatever tool is required, is usually a stage prop. In contrast, within the Mingei project, all the recordings had to be done in the actual workshop or field while the expert interacted with the equipment as they normally would. As a result, the MoCap was done to be as unobstructive as possible to allow the expert the freedom to perform.

At the glass-blowing pilot for example, the recording did not stop from the moment the expert picked up the molten glass until he was finished. On the other hand, silk-weaving required the expert to use different equipment during the whole process and therefore the MoCap was segmented based on that. Mastic cultivation tasks are performed throughout the year, with weeks passing between them. Therefore, the most distinct tasks were selected and recorded consecutively.

Eye for detail

Another difference regarding MoCap for cultural heritage crafts is that the expert’s gestures are almost exclusively unique. Even when they perform the same task, they will almost always change their motions to account for small variations in the material they are working on. What is more interesting, is that small changes in motion are important because they highlight the expertise (i.e. a novice won’t do them). This is in stark contrast with motions of an industrial worker that operates machinery, or an actor who will perform the same motions many times, or even a patient who will try to keep their movement consistent. The point here is that in cultural heritage crafts, small variations in motion encapsulate proficiency, while in other cases, they are mostly random.

In conclusion, the experience with Mingei showed that the MoCap recordings have some unique requirements. The environment and the end-product will have a lot more impact on the recording protocol than in other cases. In a different context, it is more important to identify a pattern from accumulated data of multiple individuals with a varying level of expertise. In crafts however, each motion of the expert is “valued” more even if it appears only a few times.

Written by Dimitrios Menychtas (Armines)
Top image by FORTH, other images by Armines

The history of craft goes back almost to the history of man: sharpening stone, carving wood, weaving fabric were amongst the first things human beings had to master to make their way up to our days. Most of the objects that we use in our life today are still the results of highly-skilled craftsmanship. Even those objects that are made with the help of sophisticated machines, would not be possible without the masterful skills of people who designed the machines and their usage in the process.

Preserving crafts, even those that are no longer in use today, is paramount to preserve our history and the many wonderful techniques that we have invented to tame matter and make the world a pleasant and comfortable place to live in. Preserving and representing crafts is the mission of Mingei. One of the major challenges that the Mingei project is facing, is to use computer-based technologies to represent an exquisitely, intrinsically human activity. How do we digitize something that is so intangible and dynamic?

Representing crafts using state-of-the-art technology

To achieve this ambitious goal, we use a three-step approach at Mingei: we collect, we connect and we open up. In each step, we put Information Technology at the service of the step’s goals, employing a wide range of techniques to maximize the quality of the result.

We collect

Humans are the primary sources of the knowledge about craft that Mingei collects. We talk with craftsmen and craftswomen to hear their stories. What needs to be done in the craft they master, and when, and where, and how? People knowledgeable about the crafts are our primary source of knowledge. We run co-creation sessions with them, in which we all interact to obtain from them all they know about the craft they master. And we record what the people we work with say, using digital audio-visual tools and techniques to obtain the most from our interaction with them. But we also track their movements, placing sensors on key parts of their bodies. This way we are able to document the most minute details of their actions during crafting, both verbally and physically.

The secondary sources of knowledge that we look at are books, articles, images, movies, web sites and so on, that report knowledge relevant to Mingei. Our humanist scholars, including anthropologists, historians and sociologists, all with an expertise in craft, study and research to learn what are the most relevant of those sources. They then explore them, to extract the relevant stories and notions which they then encode in digital form. All this knowledge, gathered from primary and secondary sources in digital form is imported into the Mingei knowledge base, which is the digital repository where the project stores the knowledge that it needs to preserve crafts.

During the knowledge collection phase, the pilots first provided pre-existing digital content, such as photographic documentation of museum exhibits, video documentaries and curated literature. Next, we created new digitisations by photographic documentation of each pilot sites, including the museums, machines, pre-existing photographs, catalogues, workshops, etc. All kinds of objects, such as weaving looms, garments, mastic trees, mastic villages, tools, traditional clothing, and glass instruments were digitized with 3D reconstruction technology. With use of motion capture technology, we recorded the meticulous and skilled movements of crafts practitioners. This collection of knowledge on the three pilot crafts of Mingei is supplemented with the knowledge from open repositories and online resources.

The video above is an example of 3D reconstruction. Here we see a handheld machine that is used for the cultivation of mastic on Chios. The image at the top of this article shows another 3D reconstruction, featuring a woman during the process of cleaning mastic on Chios.

We connect

The knowledge collected in the first step is formed by many elements of diverse nature, each addressing some particular aspect of some particular craft. Precious as it is, this knowledge does form yet a set of stories that can be used to document crafts: the elements it consists of need to be connected into coherent wholes that convey meaningful messages to the Mingei user audience. Performing this connection is the objective of the second step. This step uses semantic information as a medium and narratives as the tool to connect the knowledge elements. That is, we use stories as coherent wholes that convey the knowledge about crafts, and in particular stories shaped as semantic networks, to make them as readable and as easy to understand as possible using today’s information technology.  Examples are stories about the Jacquard’s loom, the Krefeld textile industry, the history of Bontemps’ life, the construction of Crystal Palace, the narrative of Isidore of Chios, the story of mastic chewing gum, and many more.

Every story created by Mingei is a rich network, consisting of two basic elements: the schema of the craft, and a set of executions of the schema. The schema is a description of the activities needed to make that craft, and of the order in which these activities must be done, that is, which activities need to be done before, or after, or in parallel to which activity. Like a blueprint or a manual of a craft.

An execution of the schema represents an actual performance of the craft, as a set of real actions carried out by some craft master in a certain place and at a certain time, in the order prescribed by the schema. Both schema and executions are represented as semantic networks, that is set of RDF (Resource Description Framework) triples. Some of these triples link activities and events to the knowledge elements that document them. These networks are then lent to the third step of the Mingei approach.

We open up

The RDF triples produced in the second step encode knowledge in a way that is known only to the people which created them. To communicate this knowledge to the different types of users Mingei addresses, a non-negligible effort is required. This effort is done in the third step of the Mingei approach. Here again, we resort to co-creation: we run sessions with museum experts to design apps that will allow users to discover, access, understand and enjoy the knowledge about crafts we have mustered. This is the way Mingei opens knowledge about crafts to the outside world. At the moment, we are co-designing immersive digital experiences, such as an app that enables the virtual creation of patterns and textiles, a digital city exploration of Krefeld, a mastic cultivation training app, and a digital glass experience.

At the same time, we make sure our semantic networks will be safely preserved for long-term access in the future by applying digital preservation techniques to them. In essence, we add further knowledge to our network and we archive them in special archives, so that they will be accessible and usable for a long time after the Mingei project is over.

This video shows an example of the digital experiences that Mingei is currently developing. The game is set up in the past, in the physical landscape of Chios created through satellite depth maps and exploits the aerial 3D scans of Chios villages created by Mingei.

Timeline

In order to maximize its performance, Mingei is running the three steps in parallel, with the beginning of the steps scaled in time, so that each step is able to receive its input from the previous one. At present, Mingei has completed the first two steps on all three pilots, and has begun the third one. The first finalized digital experience will be launched soon, so stay tuned to hear the news!

Written by Carlo Meghini (CNR)

About movement sonification

Sonification is defined as the use of non-speech audio to convey information (Kramer 2016, 185–221), and the technique of rendering sound in response to data and interactions (Hermann et al. 2011, 1).

Bevilacqua et al. (2016, 385) describe the benefits of movement sonification in their paper as follows: “Movement sonification is considered the auditory feedback that is being created in relation to a movement and/or gesture performed. The idea of using auditory feedback in interactive systems has recently gained momentum in different research fields. In applications such as movement rehabilitation, sport training or product design, the use of auditory feedback can complement visual feedback. It reacts faster than the visual system and can continuously be delivered without constraining the movements.”

As such, sonification is currently used in a wide variety of fields, such as “driving a car or riding a bike blind, directly finding one’s way in an unfamiliar smoky environment or being able to improve the quality of one’s gestures in real-time” (Parseihian et al. 2016, 1).

In particular, movement sonification systems appear promising for educating complex techniques and skills in providing users with auditory feedback of their own movements. The purpose is guidance for improving movement performance during a learning process and the application can vary.

So, why researchers did put sound on purpose? Hermann et al. (2011, 3) describe that “the motivation to use sound to understand the world (…) comes from many different perspectives (…). Thus, the benefits of using the auditory system as a primary interface for data transmission are derived from its complexity, power, and flexibility.” In other words, the reason of distinguishing sound as relevant to communicate information is found in its inherent feature to have a resonance to one listener and diversity in the way it would be employed.

Considering that movement sonification can guide a user to reproduce an expert gesture, this method is proved suitable while focusing on enhancement of the learning experience and improving the movement performance.

The use case of glassblower’s craft

In the case of glassblowing, the key challenge is the development of a system that can at first track the human body, recognize the gestures that the person is executing and, in the end, compare them with the expert’s gestures and provide sound feedback. The sonification gives a motivation to the user to complete all his tasks/gestures by also reaching to a musical goal. Following the tempo of the sounds is also a helpful feedback on how well the gestures have been performed and in which way they need to be improved.

The researchers of Armines are currently applying movement sonification for vocational training as the most compatible method, taking into consideration the particularities of the craftsmanship. The motion capture of experts and their gestural skills related to glassblowing use case is getting implemented in collaboration of Armines and Cerfav, the National Innovation Center for Glass in France.

For this purpose, recording sessions of the gestural know-how of the glassblower are being organised within Cerfav with the use of high precision motion capture technologies. More precisely, this is done with a special suit equipped with sensors that the expert glassblower wears throughout the sessions of recordings. The data recorded are categorised, so it is specified which gestures are performed, where one gesture stops and another starts, and what are the tools used. This is what we call the “gesture vocabulary”.

The next step is to learn the gestures of a glassblower through gesture recognition and sonic feedback, where a user will try to reproduce the gestures of the professional glassblower. Whenever there is a deviation between the craftsman’s gesture and the apprentice’s gesture, a sound feedback is provided based on the pitch fluctuation of a predefined sound. In this way, the apprentice perceives the quality of the sound and knows if they performed a gesture in the same way as the expert, and if they have to repeat and correct the gesture.

In the video, the installation user is asked to perform one by one the gestures that the routine of the glassblower consists of. The sonification gives motivation to the user to complete all his tasks/gestures by also reaching a musical goal.

First, the user is invited to observe the expert gestures, to hear the correct sonification result of his movements. This is what we call here “the original sounds”. A mapping has been done between motion parameters and acoustic features: The tempo modality is affected by the movement of the right and left hand in the x-axis, while the panning of the sound is affected by the movement on the y-axis.

The gestures of the installation user are being recognized and also sonified. The quality of the sonification – in terms of how close the tempo or the pitch is to the original sounds – is the result of how well the gestures have been performed and recognized. The user is able to perform the gestures one by one, until he reaches the final gesture, thus the creation of the glass carafe. The sounds mapped to each one of the gestures are layered creating a complete music piece at the end of the gestural performance.

The learning process as an enjoyable user experience

Within this learning process developed in Mingei, one apprentice acquires the necessary skillset to perform and reproduce glassblower’s gestures and gradually improve their performance and technical prowess. Furthermore, the process itself is a positive enjoyable experience that brings pleasure and motivation as well as meaning and purpose for learning this craft.

Learning by receiving auditory feedback offers incentives for developing a natural and a non-intrusive experience, that informs the user in real time about their performance and gives them the stimulus to go ahead. Considering also that the end use for Mingei is setting up an installation within a museum, it is desirable that the experience of the visitor has a twist of entertainment and cultural engagement.

Encouraging the new uses of sound can prove to be an excellent, emergent method of embodied learning through learning by doing.

 Written by Ioanna Thanou (Armines)

References