Future manufacturing will witness a shift in human-robot relationships towards collaboration,
compassion and coevolution. This will require seamless human-robot knowledge transfer.
Differences in language and knowledge representation hinder the transfer of knowledge
among humans and robots. Thus, a unified knowledge representation system that can be shared
by humans and robots is essential. Driven by this need in a product assembly scenario, we
propose the Human-Robot Shared Assembly Taxonomy (HR-SAT). With HR-SAT, any comprehensive
assembly task can be represented as a knowledge graph that both humans and robots can understand.
HR-SAT enables utilisation in various applications as a hierarchical, consistent and
information-rich representation
HR-SAT has a hierarchical structure which decomposes a task into primitive tasks
and primitive tasks into atomic actions. Definitions of task, primitive task
and atomic action are based on the functional state change of the manipulated object and
the interaction state change between the subject and manipulated object.
For better utilisation of HR-SAT, we also define nine atomic actions common in assembly processes.
Watch the video below for more information
The details of the nine atomics actions shown in the video and their action parameters can be found in the supplementary document below:
To flexibly describe the assembly task, we proposed the task description framework and modeled
it as a knowledge graph.
Use the interactive window below to see the modeled knowledge graph
The RDF/XML file of the task description framework modeled as a knowledge graph can be downloaded below.
We present two application case studies to demonstrate the utilisation and significance of HR-SAT in diverse applications.
This study presents an assistive assembly and quality assurance framework for a hydraulic valve assembly.
In this framework, HR-SAT can convert the disparate process information in the assembly instructions into
a unified process representation in the form of a knowledge graph. From the knowledge graph, the primitive
task and key atomic action nodes are extracted to obtain a unified instruction.
The generated unified instruction can then be used in to obtain personalised instructions, which is used to
assist the worker as well as to cross-check with the assembly process monitoring for quality assurance.
Watch the video below for more information
The valve assembly in Case Study 1 is based on the 304 Check Valve from Oasis Engineering Ltd.
The knowledge graph for the unified process information can be downloaded below as an .xml file.
The instructions used by the HR-SAT and the BoM of the valve can also be downloaded below.
This case study presents the human-robot collaborative assembly framework for the Generic Assembly Box (GAB).
In this framework, HR-SAT converts the disparate assembly process information (such as video demonstrations) into
a unified Subject-agnostic Task Graph, which provides a universal process description for any agent. From the
Subject-agnostic Task Graph, the primitive tasks and even atomic actions are allocated to agents to obtain the
HRC Task Graph. The allocation is based on the agent's capability and availability, guided by optimization
objectives such as overall efficiency and human wellbeing.
From the obtained HRC Task Graph, personalized instructions can be generated for human workers and executable
policies can be generated through motion planning for robots. During the collaborative task, the robot's policy
is updated dynamically by monitoring and predicting the human to ensure worker safety and collaborative efficiency.
Watch the video below for more information
Two task graphs are created in Case Study 2: the Subject-agnostic Task Graph (STG) and the HRC Task Graph (HTG).
The knowledge graph for the task graphs can be downloaded below as an .xml file.
The BoM of the Generic Assembly Box (GAB) can also be downloaded below.
The process of a worker assembling the Generic Assembly Box (GAB) with instructions can be seen in the video below
Video of worker performing assembly with instructionsWe would like to thank Oasis Engineering Ltd for giving us permission to use their 304 Check Valve in Case Study 2.
We would like to thank Vincent Link, Steffen Lohmann, Eduard Marbach, Stefan Negru, Vitalis Wiens for WebVOWL, which is used to visualise the modeled knowledge graph of the task description framework.
This project was funded by The University of Auckland FRDF New Staff Research Fund (No. 3720540)