The Display Design and Implementation of Augmented Reality based on play-based learning model in museum contexts

Meilun Tan¹ and Yan Guan²*

Published: October 27, 2022

DOI: 10.55162/MCET.03.087

View Pdf

Abstract  

In an era of rapid advances in information technology, augmented reality is becoming increasingly evident as a new technological medium for creating interesting learning experiences for the children's education sector. It uses computer-generated 3D information to embed virtual objects into the physical environment, providing children with an interactive, immersive, virtual-real environment, and it is one of the powerful tools for arousing children's interest in learning and enhancing their cognitive abilities. In this paper, we develop a museum learning service based on a PBL (play-based learning) model that combines AR technology, children's games and museum learning. The practice was eventually implemented in the Chinese Geological Museum.

Keywords: Augmented Reality; Learning based on games; Museum learning

Abbreviations

     PBL: Play-Based Learning; AR: Augmented Reality; UI: User Interface

 Introduction

     Museum learning is an interactive and informal learning process [1] that is an important way for children to acquire knowledge and enjoy a free learning environment. To promote children's learning experiences and enjoyment in museums, this paper considers the introduction of a Play-Based Learning (PBL) model to meet children's needs for learning in museum environments. Play-based learning (PBL) is a model that combines play and education to enable children to learn through play, valuing the process of children’s learning experiences. Influenced by Piaget's theory of cognitive development, the PBL learning model expects children to acquire concepts through active participation and interaction with the environment, and to construct their own knowledge through this exploration [2]. It is therefore necessary to design appropriate interactive experiences for museums through the use of technology in order to increase children's motivation and creativity towards museum learning.

     With the development of computer and communication technologies, more and more new technologies and interactive devices are being introduced into museum service functions to better enhance the fun and interactivity of children's visits to exhibitions and improve their experience of exhibiting. For example, research has found that augmented reality (AR) applications can provide children with a fun and immersive experience that can promote their motivation to learn and observe exhibits. The focus of this design practice, as a result, is applying augmented reality (AR) interactive design to museum exhibits so as to present abundant and comprehensive three-dimensional exhibit information to children via animation and human-computer interaction.

     In the last two years, the number of augmented reality (AR) research in China has soared, but there have been only a handful of applications in the field of children's education and culture. The current augmented reality (AR) interactive museum experience system in China has a single structural function and the experience is not yet mature [3]. In this paper, we apply augmented reality (AR) as the technical background, the specimens in the Prehistoric Biology Hall of the Chinese Geological Museum as the base content, andagame-based learning model (PBL) as the presentation form, to develop a more complete AR game interactive prehistoric biology museum experience for children, which can guide children to explore the mysteries of prehistoric biology by restoring biological models to the fossil exhibits.

Materials and Methods

     To allow children to experience the learning environment in a more immersive way, we provided an augmented reality (AR) scene consisting of a virtual dragon bird and a realistic exhibition hall through the user's mobile device and camera. Specifically, it incorporates animated displays, static observation and AR interaction elements of the museum experience. In order to increase children's engagement in the museum learning, we added UI (user interface) interaction, where children can make full use of their mobile devices to interact with the virtual model in real time through gestures, enabling interactive features such as feeding and petting, enhancing the interactivity and fun of the children's experience in the museum.

1. In this paper, we have designed an interaction logic framework based on the museum learning environment (Figure1). In order to achieve a natural human-computer interaction, we divided the user, the real environment and the virtual object into three modules, which are represented as follows. 1. when the user enters the exhibit case in the museum, he/she needs to take out the mobile device and scan the pre- designed recognition map;
2. After entering the AR interactive application, the user needs to place the mobile phone camera between the eyes and the exhibit case;
3. Through spatial location calculations, the virtual object appears in real time on the mobile device, bringing the dragon bird fossil to life in the form of a 3D model;
4. The final presentation is a model of the dragon bird superimposed on a realistic image to Provide feedback to the user.

     To generate interest and motivation in children, we will provide an interactive museum learning environment where children can learn about prehistoric dragon birds in an interactive play experience. A user interface is provided which enables the user to select specific interactive actions such as petting and feeding, and through gesture recognition these elements can be applied to the virtual dragon birds in the augmented environment. We also provide two modes, a viewing mode and an interactive mode, which can be switched depending on the demands of children. The viewing mode has designed animations and models, so children can not only learn about the prehistoric dragon birds' habits, but also simulate scenarios where they can interact with them.

Results and Discussion

     Based on the interaction logic designed in the previous section, in this section we have tried to build an interactive game with augmented reality features that combines spatial location tracking technology, game design, gestural interaction and augmented reality. Thanks to the game, some interesting results have been obtained.

Virtual mapping of the real space

     Firstly, we observed and measured the exhibition space of the museum in the field and made a floor plan based on the measured data (Fig 2); then a spatial model with the same dimensions as the real space was constructed in the modeling software based on the measured data and the floor plan (Fig 3). In order to superimpose the digital information more accurately in the real space, this practice applied the spatial location tracking registration technique. The different standsand exhibits in thee xhibition space are used as markers and enhancedw ith marker-based visual tracking technology to enhance the computer's measurement of the spatial location, thus achieving an accurate overlay of digital information. The digital construction of the virtual exhibition hall and the accurate measurement of spatial location will support the subsequent implementation of augmented reality games.

Interactive design of the augmented reality game

     With the technical support of the above section, we planned to design the interactive game based on the original tour route. As the museum itself has mature display conditions, the game design should avoid excessive modification of the real display space, which will help to protect the fossils and save costs. The game flow was designed and the animation resources used in the game were sorted out through storyboards. (Fig 4).

Designing the construction of fossil characters

     After designing the flow of the game, the content of the game needed to be designed. Due to the scientific nature of the museum, it was necessary to build a model of the Chinese dragon bird based on the fossil data and to design the animation of the model in conjunction with the study of its habits, so that it would not look too strange when displayed. To achieve it, firstly, we investigated the fossil skeleton of the Chinese dragon bird and recreated its skeletal model in virtual space based on its skeletal structure (Fig 5). We then used modeling software to add its body, eyes and hair to the skeletal model. Finally, a mapping was created for the model based on the data, so that a realistic-looking character model was obtained (Fig 6).

     Once the model was obtained, animations such as jumping and climbing trees were designed for the Chinese Dragon Bird dinosaur character, taking into account the advice of the museum advisor. These animations reproduced as closely as possible the characteristics of its activities during the Early Cretaceous period, and on top of this, feedback animations were designed for the user to touch it. The final design of this aspect of the game content was completed.

Augmented reality implementation mapping

     In this part we integrated the virtual space, game interaction and game resources from the previous stages through Unity and finally got a virtual reality game design output. Users can enter the game by scanning the QR code located in the booth through smart terminal devices and in the game a realistic Chinese dragon bird can be summoned through the Chinese dragon bird fossil on the booth. Through the device screen, the user can see its state of integration with the real space (Fig 7), such as climbing trees in the real space, jumping from one booth to another, etc. At the same time, the user can interact with the Chinese dragon bird through gestures, and it will make some movements to give feedback to the user.

Conclusion

     In this paper, we describe an augmented reality (AR) museum learning environment that enables children to learn about prehistoric creatures and experience interactive environments in an interactive, immersive and gamified environment. In the future, the use of augmented reality (AR) technology to build an AR interactive museum education experience model is an inevitable requirement for the development of the information age, and the digitalization and intelligence of museums will gradually become a development trend. Therefore, we plan to recycle user feedback on this design practice and make further adjustments to the evaluation results. F.7

Acknowledgements

     Tsinghua University, 2021 National Social Science Foundation of Art Major Project.

References

1. Hsu T-Y., et al. “Cobo Child: a blended mobile game- based learning service for children in museum contexts”. Data Technologies and Applications 52.3 (2018): 294-312.
2. M E and Boyer W. “Play-Based Learning: Evidence-Based Research to Improve Children’s Learning Experiences in the Kindergarten Classroom”. Early Childhood Educ J 48 (2020): 127-133.
3. J Riji Yu, Ming Cai and Shuai Jiang. “Research on the design and application of museum cultural education experience system based on mobile terminal and AR technology”. Chinese point education 31-35.
4. MEC Santos., et al. “Augmented Reality Learning Experiences: Survey of Prototype Design and Evaluation”. IEEE Transactions on Learning Technologies 7.1 (2014): 38-56.
5. Dunleavy M, Dede C and Mitchell R. “Affordances and Limitations of Immersive Participatory Augmented Reality Simulations for Teaching and Learning”. J Sci Educ Technol 18 (2009): 7-22.
6. S. Oh and Y-C Byun. “The Design and Implementation of Augmented Reality Learning Systems”. 2012 IEEE/ACIS 11th International Conference on Computer and Information Science (2012): 651-654.