Implementation timeline

Implementation timeline


Applying the Mingei protocol in the Glass pilot

STEP 1. Human resources and digital assets

In this step, we collect the documentation that will be eventually transformed into knowledge and history that we will digitally represent.

Human resources are invaluable in the description and explanation of craft practice and context. In this process, digital assets serve a range of purposes, from note-taking to reference practices by practitioners. Moreover, sometimes the contribution of a human resource may be available only through a recording.

Digital assets are carriers of documentation, information, and knowledge that required a comprehensive representation of craft. Moreover, digital assets that are appreciated by human observers (i.e., audio-visual) can encompass an insightful understanding of processes, environments and semiotics.

In this context, the following activities were conducted:

Knowledge collection and literature review

Craft understanding is initiated with the collection of existing knowledge of the following categories:

  • Basic knowledge including (a) curated text, (b) curated material, and (c) formal representation
  • Existing content includes (a) archives, (b) bibliographic research outcomes, (c) audiovisual material, and (d) digital representations
  • Contextual information (1) brief description of the craft (2) geographical location and range of the craft, (3) selected and representative communities concerning the craft, (4) craft workers / skilled workers / handicrafts education, (5) gender roles, (6) equipment, (7) craft products, (8) craft traditions incuding oral tradition / story telling / work songs / myths & legends, (9) social practices / social dimension, (10) economic dimension, (11) design dimension, (l2) artistic dimension, (13) rituals, (14) festive events, (15) religious dimension, (16) learning / education / transmission of knowledge, (17) geography of workshops, (18) emic (inside) Presentation
  • Narrative interviews with craft practitioners

Social and Historic Context

In 1842, Bontemps delivered more than fifty pieces, both tools (molds, and blowing apparatus) and finished objects. All pieces were dated from 1842 and originating from the Choisy-le-Roy glassworks. In 1868, Bontemps will make a further donation, in his name, objects relating to his famous Guide du Verrier, published that same year. Among them two sets of objects, revealing the process in a sort of «frozen state», a pattern much favored for in pedagogical dimension at the Musée des Arts et Métiers. The aim on selecting the carafe, in this work, was to document each manufacturing step, as exemplified by the «frozen» glass artifacts: the rough out foot and the finished one, the handle, etc. with a recording of the corresponding gestures and bring to life the objects. Furthermore, it was interesting to compare and document the historical tools with those still in use today.

Data collection

In this step, the components of the crafting process are digitally recorded, post-processed, and curated in an online platform.  

These digital assets record objects and actions, acquired from documenting photographically and in 3D materials, tools, products, and workspaces and recording practitioner crafting actions, in recording sessions. The organization of these sessions is facilitated by the vocabulary and storyboard, in identifying the objects, sites, and practitioner actions to be digitized. Moreover, some of the photographic and video assets may be recorded in combination with ethnographic fieldwork. A requirement relevant to the glasswork, but also relevant to other crafts, is that the viscous nature of hot glass prohibits the pause of the crafting process during some sequences of tasks. As such, motion files may contain multiple process steps and actions

Audiovisual recordings

Pertinent assets regard conventional audiovisual data acquired from the ethnography such as audio and video interviews photographic documentation, as well as documentation to be used for craft representation including photographs and video documentation of objects, spaces, and demonstrations.

Audiovisual recordings of the crafting process are important for overview and ethnographic study, as well as for documenting significant aspects of the process with a local spatial expression, such as tool gripping postures, detailed and minute manipulations, and so on.

Photographic documentation

Video documentation

Multiple and/or wide-angle bird view cameras were employed to capture the motion of multiple practitioners and their movement within the workshop. In the case of glass workshops, the latter is particularly important because glass workshops are shared by multiple practitioners and because the arrangement of workshop machinery is such that the practitioners have to walk from one locus to another during the crafting process.

Documentation of tools, machinery, and workspaces

Tools, machinery, and workspaces was documented using photographic documentation.

Motion capture

Motion Capture occurred during the ethnographic fieldwork at CERFAV, in September and October 2019. During this, fourteen carafes were produced in the workshop to observe and capture social and body interactions that take place during glasswork. Apart from the audiovisual documentation acquired the process was recorded using MoCap equipment and, in particular, the Nansense MoCap suit and Glove. The following gallery illustrates some of the tasks that were recorded for glassblowing.

Post processing of Digital Assets

Ethnographic documentation

Video analysis and identification of key operational sequences that signify craft actions and processes. Definition of a process flow diagram for the implementation of the glass carafe and connection of operational sequences with the flow diagram. Examples of the ethnographic documentation of such sequences are presented in the following gallery.

Visual tracking results

For visual tracking, the OpenPose library and the MocapNET body tracking software was used. On the Glass datasets we observe that, when all limbs are visible, motions tend to translate well into the armature. Failure frames mostly consist of several limbs being occluded and not visible in the camera observations. Invisible joints lead to missing 2D joint estimations and missing 2D estimations, in turn, mean empty lines and columns in the NSDM matrices of MocapNET and not enough structural information for a clearly defined solution.

Examples of the visual tracking outcomes are presented in the following gallery:

MoCap segmentation

After the correction and noise removal from the MoCap data, the data was segmented by gestures. The recordings initially were done by tasks, being one recording a whole task, these recordings were afterward segmented by gestures. One task could have only one gesture that is repeated several times or might have more gestures that are repeated through the task.

Examples of segmented MoCap data visualisation are presented in the following video:

Digitisation Outcomes

New digital assets

Glassblowing tools

The 3D documentation of metallic tools and glassware can be challenging. We distinguish between the potential historic significance of an object and its utility as a tool, wherein the latter solely their geometrical structure can be sufficient. In some cases, the 3D model of tools can be easily modeled or even provided by the manufacturer. The use of synthetic models can simplify the digitization tasks and significantly reduce scanning costs. The collection of tools for glasswork was developed in and is summarized in the following gallery.

Workshop equipment

Similarly, for the creation of 3D models of machinery, the acquired audiovisual documentation acquired during the ethnographic research was used and the machines were created from scratch using 3D modeling software.

Relevant Publications

  • Representation and Preservation of Heritage Crafts, (2020), X. Zabulis, C. Meghini, N. Partarakis, C. Beisswenger, A. Dubois, M. Fasoula, V. Nitti, S. Ntoa, I. Adami, A. Chatziantoniou, V. Bartalesi, D. Metilli, N. Stivaktakis, N. Patsiouras, P. Doulgeraki, E. Karuzaki, E. Stefanidi, A. Qammaz, D. Kaplanidi, I. Neumann-Janßen, U. Denter, H. Hauser, A. Petraki, I. Stivaktakis, E. Mantinaki, A. Rigaki, G. Galanakis, Sustainability, DOI:10.3390/su12041461.
  • What is needed to digitise knowledge on Heritage Crafts?, (2019), Zabulis, X., Meghini, C., Partarakis, N., D. Caplanidi, P. Doulgeraki, Karuzaki, E., Evropi Stefanidi, Evdemon, T., Metilli, D., Bartalesi, V., Fasoula, Maria, Tasiopoulou, E., Beisswenger, C., Memoriamedia 1: 1–25.

Applying the Mingei protocol in the Glass pilot

STEP 2. Knowledge Elements

In this step, basic craft knowledge or otherwise knowledge elements are formed and digitally represented. In plain words, this means that the digitisations are entered in the Mingei repository, properly classified and linked to the other entities in the repository. Collectively, the classifications and links formed in this step form the semantic metadata that have been previously introduced.

Forming a knowledge element requires a comprehensive understanding of the digitally represented asset and is regarded as a digital curation process. Digital assets are represented in the system by knowledge elements associated with the result of a curation process that yields metadata, annotations, and descriptions. In this process, the original meta-data (if any) that accompany the digital asset are useful.

Representation of knowledge on craft processes includes curated material which is also digitally represented. The basic elements of such representations are the tools, machines, workshops, etc. At the same time of importance is the social and historical context.

Social and Historical Context


Human resources are invaluable in the description and explanation of craft practice and context. In Mingei the profession, communal activity, and traditions of a place are of close relevant to the contextualisation of a craft. Of relevance is also the participation of persons in communities, guilds, social or artistic movements. In our case , we record and represent basic information on the life and times of a person, such as occupation, places of visits, time and place of life and death, and key events relevant to the topic of study, such as education, professional activity, collaborations with other persons and so on.


with 2 coordinates, latitude and longitude. Regions are polygonal representations upon the surface of the Earth. We call both “places”. e use a data base of location names (GeoNames).

More refined representation of locations may require addresses or even arbitrary user-defined locations on Earth that are not included in the database. The coordinates of these locations are provided through the digital curation process and supported through a GIS system.


We consider events as the changes of state in cultural, social or physical systems brought by phenomena or influenced by other events. Thus, the definition of an event requires at least two observations of this state in time.

Events are basic elements of stories in general and, in particular, to the type of “stories” we wish to capture in our craft representation. Knowledge elements of the event type are employed to represent:

  • Socio-historical context relevant to the expression of a craft instance.
  • Actions in craft processes

Craft Context

Digital assets

The most relevant digital data types are:

  • Text
  • Photographs
  • Video
  • Audio
  • 3D reconstruction
  • Human motion digitisation

Additional digitisation types of humans, matter, and actions (e.g. thermal, multispectral, X-ray), provide measurements of more material properties that photography can capture. Such properties are surface reflection and transmission properties, chemical composition, its degree of viscosity at a certain temperature, and other.

We provide some examples of primary digital assets from the Mingei dataset

  • Digital text of curated text, literature, and testimonies.
  • Photographs: historic photographs, photographic documentation of artefacts, printed matter, records.
  • Audio: interviews of glassmaker and mastic producer.
  • Video of craft practice.
  • 3D reconstruction of a workshop from a laser scan.
  • 3D reconstruction of an artefact from a handheld scanner.
  • Motion capture of craft practice.


3D Digitizations


Motion Vocabularies

3D Reconstructions

Motion Vocabulary Items

Relevant Publications

Open Datasets

Applying the Mingei protocol in the Glass pilot

STEP 3. Craft Representation

In this step, the individual entities represented in the previous steps are linked to each other into an organic representation of the craft instance. This linkage implements the semantic representation of the craft.

The scope of this representation covers the following craft dimensions:

  • Tangible elements, such as materials, tools, and products
  • Craft actions and processes
  • Contextual knowledge that provides an understanding of
    • Artefact usage
    • The CH of a region and its people embedded in the artefact
    • The historic, geographical, economic, and social dimensions of the associated craft instance.

Social and Historic context

  • Representation of Fabulae

    Regards the structuring of ‘Events’ in MOP to ‘Fabulae’ presenting the sequence of ‘Events’ that are the backbone of a ‘Narrative’. Connection between ‘Events’ is established by two kinds of relations:

    1. Mereological relating events to other events that include them as parts, e.g., the invention of the flying shuttle is part of the life of the Industrial Revolution.
    2. Causal dependencies, relating ‘Events’ that in are predicated to have a cause-effect relationship, e.g., “the Industrial Revolution resulted in a reduction of the number of weavers”.

    Through MOP ‘Event’ entries are transformed into ‘Fabulae’.

Craft Representation

To encode craft understanding, activity diagrams are borrowed from Unified Modeling Language (UML) and used in the following sense. While UML represents computational actions that transform data are represented, in this work, physical actions that transform materials are. The transition types Transition, Fork, Merge, Join, and Branch are adopted and denoted as in UML.
Activity diagrams can be refined hierarchically, allowing the increase of representation detail at later stages. Moreover, their visual nature was found to support the collaboration with practitioners. Activity diagrams are essential in representing transitions between process steps, in which distinct tools are employed and actions take place. and, in particular, decision points and parallel tasks in the process.
The progression of sequential steps is modeled by a Transition link. Forks are used to represent the initiation of two parallel tasks. In Merge transition, two or more control paths unite and Join connects steps that should be completed before the transitions to the next step. Merge and Join transitions are structurally similar, but a join is a synchronization across a set of parallel flows, while in a merge only a single flow is active. Finally, Branch transitions connect a step with a decision step that accepts tokens on one incoming edge and selects one outgoing alternative. Branch nodes control the flow of a process by selecting one of several alternatives, based on the outcome of a condition evaluation.

Process schemas

Activity diagram are transcribed into transition graphs. The MOP UI facilitates the structured instantiation of process schemas and their steps. Data fields are used to enter appellations, informal descriptions, and step order. Transitions are instantiated via dynamic UI components that adapt to transition type.


Processes representations account for the events that took place during the execution of a process schema. Intuitively, a process is an individual flow of events, out of those possible in the activity diagram. The MOP UI enables the instantiation of processes representation, via the entry and chronological ordering of events, accompanied by the recordings that document them.

Relevant Publications

  • Digitisation of traditional craft processes, (in press, JOCCH-21-0060), X. Zabulis, C. Meghini, A. Dubois, P. Doulgeraki, N. Partarakis, I. Adami, E. Karuzaki, A. Carre, N. Patsiouras, D. Kaplanidi, D. Metilli, V. Bartalesi, C. Ringas, E. Tasiopoulou, Z. Stefanidi, ACM Journal on Computing and Cultural Heritage, DOI:10.1145/3494675.

Open Datasets

Applying the Mingei protocol in the Glass pilot

STEP 4. Narratives

A digital narrative or, simply, a narrative is an abstraction that represents a set of facts that have happened in the world. These facts are connected to each other in a way that makes them a story.

This abstraction includes a narration, which is the way that a certain author has told the story. Individual narratives may be focused at a subject and include a pertinent subset of the events that are represented in the fabula.

The narration tells the story, using a certain medium. In Mingei multiple media are employed, starting from verbal and visual and reaching up to immersive and interactive narrations. A medium, and correspondingly, a media object may be multidimensional; for example the audio and visual components of a mobile phone “video” recording, or the combination of video and 3D MoCap recordings.

A narrator is a person or a program that presents a media object, i.e., verbally, by visual rendering, sonification, VR, etc. Besides multimedia and immersive presentations (discussed in the next step), conventional, verbal and visual, media objects and channels are of fundamental importance. A textual media object can be exported as a narrative for a person or an avatar to read, and a formatted document with illustrations, or an “infographic”, can output channels. In this context, Mingei aspires the export of more imaginative and interactive uses of conventional verbal and visual outputs, including scripts for theatrical re-enactment of stories and role-play on contextual dimensions of craft instance, such as skill, trade, and labour. The choice of narration medium is relevant to the use of the content, the intended audiences, and the experiences to be rendered these are discussed in the next step. In this step, we focus on the narrative representation.

A narrative can include additional information besides the connection of the narration to the facts that are narrated. In particular, a digital narrative consists of three main elements:

  1. A representation of the fabula made of representations of the events that the narration is telling.
  2. One or more media objects.
  3. A digital representation of the reference function that connects the (representations of the) events of a story with a fragment of a media object that describes them and allows to derive the plot.

Each narration of a fabula employs the use of one or more media objects, which have been authored by one or more narrator(s). Through the Mingei Online Platform, a tool for authoring and visualising narratives is provided.

Representation of Narratives

Relevant Publications

Applying the Mingei protocol in the Glass pilot

Step 5. Information tools

Implementation of Virtual Humans

For the needs of the demonstrations 3D virtual humans, with their corresponding garments and accessories, have been carried out based on cultural and historical information and sources. VHs were created using references from the ethnographic research. For the definition of the 3D meshes and the design of the skin surfaces a customized toolchain was employed that combines automatic generation methods, with manual editing and refining. The approach utilizes VH creation software and 3D modeling software for clothes and accessories. The final model inherits and encapsulates the required structural components, such as bone and skin attachment data. This allows the VH to be dynamically animated and to exhibit real-time animation capabilities.

Another important aspect for attaining a high level of realism is the motion, behavior, and natural interaction of the VH with users, to avoid the “uncanny valley” effect. The VH must have the ability for verbal as well as nonverbal communication skills, be intelligent, have natural communication with the users, perceive information from the user, and physically react with them. The deformation of the skinned characters during their movements must be realistic, smooth, and not contain any discontinuities. The VH that supports all these capabilities will be an intuitive craft master acting effectively as an elderly companion and assistant for elder people.

Garment design was also considered during the implementation of avatars. To ensure appropriate rigging, since the generated models are automatically rigged an additional checking was performed to ensure that the rig is applied correctly, and the bones are well adjusted to the 3D body.

Virtual Humans have been specially designed for the Glassblowing Pilot. Several datasets have been tested to ensure that the designed Virtual Humans will be capable of assuming the challenging poses in glassblowing.

Motion Re-targeting

The processing of the motion data has been done on Autodesk MotionBuilder software which is dedicated to animation and the direct integration of motion capture technologies. The process requires different steps: (a) creation of an “actor” in MotionBuilder with skeleton definition corresponding to the BVH hierarchy, (b) mapping of the received animations on the actor, and (c) synchronization of the VH with the actor by adjusting the 2 models so that the measurements match and the animations are correctly reproduced. Examples of the outcomes are presented below:

Web based information tools

The rich semantic representation of the craft is transformed to intuitive online presentations and delivered through the Mingei Online Platform.

Illustrated craft demonstrations

Please wait while flipbook is loading. For more related info, FAQs and issues please refer to DearFlip WordPress Flipbook Plugin Help documentation.

Relevant Publications

Open Datasets

  • Anastasia Rigaki, Nikolaos Partarakis, & Xenophon Zabulis. (2021, May 12). The making of the Bontemps carafe. Zenodo.
  • Anastasia Rigaki, Nikolaos Partarakis, & Xenophon Zabulis. (2021, May 11). The making of the Bontemps’ carafe. Zenodo.

Applying the Mingei protocol in the Glass pilot

Step 6. Experiences

Mixed Reality

The Mixed reality installation was deployed in the context of a periodic exhibition at the premises of the museum of CNAM, which is is where the worlds of academics and professional activity come together. It is the only higher education establishment dedicated to life-long professional training. In this context, a dedicated space at the cathedral which is part of the museum was dedicated for the exhibition. The space is a chapel of the cathedral located within the main exhibition of the museum. The installation is comprised of a special construction capable of hosting three large displays created through back projection in thin synthetic fabric. Furthermore, for craft training, a bench has been installed in front of the main display together with a glassblowing pipe to be used by visitors. The installation has integrated hosting spots for the glass carafes and pieces of the carafe created during the ethnographic fieldwork. Special lighting was integrated within to present the interaction of glass with light and to create atmosphere.

After the completion of the physical part of the installation, the software was installed. The scenario that was set up was that the presentation application sequence is executed first and then the main projection switches to the craft training applications, so that users test what they learned from the craft presentations.

Mobile AR

Relevant Publications

Please wait while flipbook is loading. For more related info, FAQs and issues please refer to DearFlip WordPress Flipbook Plugin Help documentation.