Literature Review - NSU-SP21-CSE499-18/Group-01 GitHub Wiki
Augmented Reality in Science Classroom: Perceived Effects in Education, Visualization and Information Processing
1.In a nutshell this paper analyzed the effectiveness of an AR mobile app to aid students and teachers of K-12(secondary school) in learning/teaching chemistry. The aim was to improve students' learning efficiency with the inclusion of technology and reduce weekly school hours. This study utilized an action research design, PDSA (Plan-Do-Act-Study-Repeat) with a qualitative approach. Students(13-14 years old) and a trained teacher of a junior high school in Manilla, Philippines were taken as participants. Qualitative analysis was conducted through teacher reflections, journal entries from students and student focus group interviews.
The mobile AR app was marker based, which would show shapes, formations and interaction of different chemical elements when the camera of the phone was pointed to the card with a particular qr-code. Also the AR visuals would change when students changed the physical orientation and combination of the markers/cards.
Positive outcomes of the study
- Excitement among students to check out new technology.
- Increase of student-student interaction when troubleshooting the app.
- Increase class participation, students raised more questions and provided better answers to teacher’s pre-planned questions.
- Helping students build a realistic perception of atoms and molecules of various chemical elements through visualization.
- Aid teachers to better explain chemical element structures with 3D visualization rather than trying to draw those out in 2D.
Shortcomings of the study
- Each student needed to be provided with a tablet and fast internet connection in the classroom.
- Few students claimed the AR visuals/animations were too basic, that it was the same as watching a video or photo of the covered topics.
- Both teacher and students required to be taught of the app’s usage.
- The static 3D models couldn’t cover all the aspects of the subject. 3D models of only a few chemical elements (and a few of their combinations) were provided.
Augmented Reality in Education: A Study on Preschool Children, Parents, and Teachers in Bangladesh
2.In this study two android AR apps were designed to aid students of a preschool aged 5-6 in Dhaka, Bangladesh. The first app was called “AR Zoo App” which showed a 3D object of the animals when the camera was pointed towards the corresponding flashcard. “Bangla AR Book App” was designed for the book “Amar Boi”, which is mandated by the national curriculum textbook for preschool students of Bangladesh. Both the apps had a scored test feature for the students to take. The technology and toolsets used for developing both the apps were- Unity 3D, Vuforia(for marker based AR) and Maya (a tool for designing 3D objects).
Both quantitative and qualitative study was performed with the students, teachers and parents as participants. In the quantitative study portion, students were divided into two groups, one taught without any app and the other with AR apps. Then both the groups took tests and a 33% improvement of score was observed among the AR app group. Survey questions were also asked for the app’s performance evaluation where students, teachers and students, all answered in favor of the apps. In the qualitative portion of the study the participants were interviewed after the session took place. Students replied that they only knew to play games and watch cartoons on phones, so the AR aided study technique was very exciting for them. Parents exclaimed their concern for children using smartphones too much. Teachers suggested adding more animations and sound effects to the AR visuals, and a reward system with voice- “Very Good”, “Well done” to encourage students.
Positive outcomes of the study
- Performed in a real-life preschool classroom of Bangladesh.
- “Bangla AR Book App” was based on the existing NCTB mandated textbook for preschool students.
- Increased Excitement among students, who knew nothing about AR before.
- Interviewing the participants(students, teachers, parents) brought out new requirements such as- adding more animation to the AR visuals, voice based reward system.
Shortcomings of the study
- Teachers and students are required to be taught of the app’s usage.
- Classrooms need to be provided with smartphones, in the study a group of 5 students had to share one phone.
- They did not address the parents concern about children wasting time on phones.
- Quantitative study was conducted by evaluating test results of “preschool” students.
Augmented Reality for Education; AR Children’s Book
3.In summary, the study was about to offer Bangladeshi students of age 4-5 fundamental education of learning about Bengali and English alphabets more excitingly and enjoyably using AR-based application. The aim of developing this application was to encourage kids to learn and grow an enjoyable learning process. According to their specific pattern, the app's name was "AR Children's Book," which shows each alphabet's three-dimensional object. The application also offered the students to scan various fruits' pictures and display a 3D fruit object that can be viewed from multiple directions. The application also came with some interactive audio features that enhance the students learning ability. The tools and software which were used to build the application are as follows:
- AR- Vuforia Software Development Kit (SDK)
- Unity Game Engine (to develop 3D application)
- C Sharp (as a scripting language for Unity Engine)
- Blender 3D (for designing 3D objects)
The application provided users with two different user interfaces for both the English & Bangla version even though they have the same functionality. The app was mainly object-based, which means the user must scan almost similar-looking objects to view the 3D object in the app's camera interface.
Positive outcomes of the study
- Creates learning engaging and exciting through the application.
- Introduce children to new and advanced technologies from an early age.
- Enhances children's natural learning process and motivates them to learn and explore different things.
- Assures them that mobile phones have not only entertainment purposes but also learning objectives.
Shortcomings of the study
- Requires device with specific requirements to run the application.
- Students of that age group may face complications operating the application.
Inpresso AR: a generic Augmented Book
4.In this paper, the authors tried to portray a generalized approach in designing a system that can be used to enhance the reading experience of paper-printed books with the help of augmented reality. The authors tried to break down the necessary abstractions required in order to build a system that can provide AR experience for multiple books. They broke down the system into four separate modules:
Interface module (an android application):
- While designing the application the developers have to keep in mind the interface's compatibility with different device specifications.
- The layout of the app should be adaptive to the change in orientation of the device.
- In order to add a book the user will be prompted with a pinch gesture to select a cover for the book.
- The app shall be using an asset named SimpleScrollSnap that will allow the implementation of a slider of markers(book pages).
Communication Module( a server):
- The server will communicate with the app using the HTTP protocol.
- The communication starts as the app sends a POST request to the server with the picture of the front cover of the book in it.
- The server then finds the XML file associated with the book and reads all the contents(URLs, logic between trigger and content, types of interactions)in it.
- Upon reading the XML file the server converts it to a JSON file and responds back to the app with that JSON file using the Newtonsoft library. This process will be what causes the manipulation of information on the app.
Content download and instruction module:
- All contents and triggers are downloaded using the Unity native class ”WWW” and then stored in the application’s PersistentPath, with the name according to its ”id”. This is done through the use of the Trilib library.
- For all “object” type contents in the PersistentPath, a GameObject is created and all its properties are assigned to it(scale, position, rotation).
Data persistence module
- After receiving the JSON file the app will create a root directory for the book’s experience using the book id.
- As soon as the directory is created, the response from the server will be stored in it in the form of an encrypted file in the extension [book_id].dat using the AesEncryptor library.
- Every time the app starts, a book list is created in run-time by reading all the .dat files in the app’s PersistentPath. When the user starts to download the experience of a book, all the contents and triggers are downloaded and stored in the PersistencePath.
Augmented Reality in Education: Current Technologies and the Potential for Education
5.In this paper, the authors introduced AR technologies aspects and their encouraging potential in education and learning. In short, the authors described the technology thoroughly and the possible impact of AR technology in educational sectors. The combination of AR technology with educational content creates new types of automated applications. It acts to enhance the effectiveness and attractiveness of teaching and learning for students in real-life scenarios. Augmented Reality is a new medium, combining aspects from ubiquitous computing, tangible computing, and social computing. Augmented Reality has the power to change how we use computers and mobile devices.
AR Technology in a nutshell:
Augmented Reality provides a composite view for the user with a combination of the user's actual scene and computer-generated virtual scenes. It augments the real-world by engaging an ordinary place, space, thing, or event in a partly unmediated way. According to Azuma (author of 'A Survey of Augmented Reality" paper), Augmented Reality must have three characteristics: combining the real and virtual worlds, having real-time interaction with the user, and is being registered in a 3D space.
Technologies for Augmented Reality Systems:
- Head-Mounted Displays: This is a kind of display worn on the head or as part of a helmet. It has a small display optic in front of one or each eye.
- Handheld Displays: Handheld displays are mainly small computing devices such as mobile or tables.
- Pinch Gloves: Gestures can grab a virtual object and provide a reliable and low-cost method of recognizing natural gestures.
Augmented Reality in Education:
Augmented Reality has already been used in military, robotic, manufacturing, maintenance and repair applications, consumer design, etc. This implies that AR has so much potential in terms of human learning and education. An impressive application of this technology is in augmented reality textbooks. These books are printed typically but point a camera to the book brings visualizations and interactions designed. This technology allows any existing textbook to be developed into an augmented reality edition after publication. In this way, printed book pages, textbooks will become dynamic sources of information.
ARKit and ARCore in serve to augmented reality
6.In this paper, the authors talk about the strengths and the shortcomings of both Apple’s ARkit and Google’s ARcore when it comes to designing an augmented reality application using these libraries. The authors also talk about the functionalities included in these libraries that will aid in making an augmented reality application for both IOS and Android platforms. Some of the relevant features of both the libraries are listed below:
AR-Kit (library for IOS app):
- Tracking the smartphone’s position: The ARkit can detect the phone’s real-world position in real-time. This is done by using SDK in combination with the VIO (Virtual Internal Odometer) with data from the camera and the motion sensor of the smartphone.
- Understanding the environment: With the help of AR-Kit, iPhones and iPads can identify the real-world horizontal and vertical planes/surfaces in real-time. Which can be used for the realistic integration of virtual objects.
- Static 2D image recognition: 2D images can be stored in the application, later to be detected in the environment for displaying superimposed virtual information on these images.
- Plane Detection: ARkit can detect a plane by extracting and tracking the scene at the point of interest.
- Image Tracking: It can detect static or dynamic images in a scene. After detecting the image (trigger) it can render a 3d object over it. The 3d object shall be stored in the application beforehand.
- Tracking of moving images in the scene: This feature allows the virtual-object inserted on a detected image to be tracked. It can follow the moving image in the scene.
- Save a map of the environment: It allows the user to save the first augmented-reality experience in an unknown environment. All the experiences associated with the experience can be saved on a card. This card can later be used to reload the AR experiences when the user is in the same environment. This will cause the detection and tracking process to be optimized.
- AR-Kit 3D file support: AR-Kit supports several 3D formats. Some of which are: [.abc], [.usd], [.usda/.usdc], [.usdz], [.ply], [.stl]. Other than these 22 more file types are compatible with AR-Kit
AR-Core (library for Android app):
- Tracking the position of the phone: The AR-Core uses the IMU (Inertial Measurement Unit) sensor to determine the position and orientation of the device. It also uses the camera to detect the charismatics points of a scene for the proper positioning of the virtual objects.
- Understanding the environment: AR-Core uses multiple different sensors along with the sensors used in tracking the phone’s orientation to detect a horizontal surface.
- Brightness Estimator: This feature estimates the ambient light of the environment in order to ensure the perfect integration of 3D elements into the real world.
- Plane Detection: AR-Core searches for groups of key points in a scene that can be used to render horizontal and vertical planes. It can also detect the boundaries of each plane.
- Image Tracking: For this feature to work an image has to be pre-installed in the app so that when the camera hovers over that designated image, it can keep track of the image and render the associated virtual 3D object on it.
- Object Detection: AR-Core can only detect 2D markers whereas AR-Kit can detect both 2D and 3D markets.
Bornomala AR-bengali learning experience using augmented reality
7.Bornomala AR is an android application specifically designed for teaching children Bengali vowels (“Shorborno”). The study greatly focuses on conducting a qualitative analysis of the participants' feedback in answering two research questions-
- How much effort the working parents of Bangladesh put to teach the alphabets of their native language to their children (3 to 5 years old).
- How this AR Application can excel in the self-learning process of the native alphabets of this focus age group of children.
The application itself is very simple- pointing to an image of Bengali vowels it would show a 3D rendered view and an example object of the vowel and play a pre-recorded audio clip. Survey for the qualitative study was conducted in two stages. First, working parents (mostly rickshaw pullers, garment workers, day labourers) were interviewed on the overview of how they teach their children at home. Second, two groups of underrepresented preschool students were taught Bengali vowels- one with conventional way and the other with Bornomala App. The first survey showed that 75% of parents expect their children to learn Bengali vowels on their own and cannot make time to teach them. The second survey found that children performed 17% better with the inclusion of AR apps to learn on their own.
Positive outcomes of the study
- Qualitative study with the inclusion of minority communities (working parents- rickshaw pullers, garment workers, maids; students- less privileged preschool students) as participants.
- Focus on a particular goal- safeguarding Bengali language by making it easy for children to learn Bengali Vowels on their own with an AR app.
- Simple and lightweight AR app (Android Studio + Unity + Blender), that achieves the research goal gracefully.
Shortcomings of the study
- Although participants for the qualitative survey were from less privileged segments of society, the study did not take into account the availability of Augmented Reality supported smartphones among them.
An Interactive Mobile Augmented Reality Magical Playbook: Learning Number With The Thirsty Crow
8.In this study, an interactive mobile augmented reality application and playbook were offered for preschool children to learn numbers using old folklore literature. The application system mainly depends on four main components:
- Camera: For capturing the natural environment and tracking.
- Mobile Device: To display the virtual augmentation
- Natural Environment: Physical playbook in this case.
- Markers: For tracking the target.
The story page covered the storyline and illustration illustrating the story within that current page in the book. The marker page consists of a marker that representing each 3D character with animation within that current page. The AR application contains three display methods: HMD, handheld display (mobile devices), and monitor-based display. This application augments the 3D character, animation, and audio and provides the users with the narrator, which also works as guidance for the children. The user will interact with the book using their finger to count together with animation of the augmented 3D character on the book.
Positive outcomes of the study:
- An observational study was performed on the prototype to evaluate the research based on user feedback.
- It provides the children early literacy education, which helps develop their emotions and how they think and use the language.
- It establishes the foundation of reading and writing skills among the children.
- Using AR technology extends the utility of a traditional storybook or textbook. It also provides the student a different experience of getting primary education.
Shortcomings of the study
- More activity and learning part are needed to improve the efficiency of the book.
- Requires device with specific requirements to run the application. It also requires the exact storybook with markers to perform virtual augmentation.
Exploring in-service teachers’ acceptance of Augmented Reality
9.In this research, the acceptance level of AR in education among six secondary school teachers in Mumbai, India were evaluated. The three core constructs of the Technology Acceptance Model- perceived usefulness, perceived ease of use and behavioral intention to use were taken as the evaluation criteria. A simple AR app was designed for understanding different properties like- vertices, edges, faces of complex geometric shapes. Additionally, the app had an option for teachers to incorporate their own lesson plans by defining goals and steps(teacher action-student response) and guiding questions.
One-to-one interviews were performed with each teacher, asking questions regarding difficulties faced in their current method of teaching. Later the app was shown to the teachers and its perceived effectiveness was inquired. Although the teachers responded positively, valid concerns were also raised,
- Necessity of pointing the phone towards markers(text-book page) while interacting.
- Unavailability of resources- there were enough tablets for only two classrooms in the entire school.
- Not able to give AR based home works or practice at home because not all students are privileged enough to own an adequate smartphone.
Positive outcomes of the study:
- The study focused on teachers' acceptance rather than students, unlike most studies.
- Access to offline and flexible lesson plans for teachers within the app.
Shortcomings of the study
- Both the number and variation of participants was too narrow. Only 6 urban teachers who were well acquainted with using smartphones and apps.
- The interview questions were too leading, this may be the reason for such high positive feedback (100% teachers agreed to use the app right away, which is odd).
ScholAR: a collaborative learning experience for rural schools using Augmented Reality application
10.The research was conducted among students of a rural school in India with an AR app to visualize solid 3D geometric shapes. The markerless AR android app was built using Unity and ARCore SDK. The features of the app included- viewing and identifying different 3D shapes and their properties- faces, edges, vertices; building complex structure by using different shapes; quiz with audio response.
A primary investigation on three rural schools found that schools have scarcity of classrooms and study materials which made it compulsory for the students to collaborate. The school selected for this research is said to occasionally use projectors but mostly use blackboards only. In total 32 students were taken as participants, divided into two groups of 16 students. One group was taught using the AR app and the other in a traditional manner by the same teacher. Post-test was conducted with questions prepared with the help of the teacher, which tested the first three stages of Bloom’s Taxonomy- remember, understand and apply. The group taught using the AR app had better performance in the tests and also showed remarkable creativity in building structures using the taught 3D shapes with the app.
Positive outcomes of the study
- Participants were from minority group- rural school students.
- Primary investigation was conducted to understand the problems- scarcity of resources which prompted students to share resources and coming up with a solution- AR apps which are effective when applied in collaboration.
- Observational log was kept during the live study and by watching the recorded video. Later feedback of the participants was also taken.
Shortcomings of the study
- There were some inadequacies in the features- the app was in English which required researchers to translate to students continuously, 3D augmented shapes were not rotatable.
- Even though the participants were rural school students they were not under-privileged as the school had available smartphones and also said to occasionally use projectors.
Design and Development of Mobile Augmented Reality for Physics Experiment
11.In this research an AR app for playing out Physics experiments was designed. The goal was to allow students, grow a proper understanding of physics topics by witnessing experiments in front of their eyes, without the hurdles of maintaining a physics lab. The features for the AR app were decided by consulting with experienced physics teachers. Animated 3D model of three electromagnetism experiments were selected as the marker-based augmented content. Different parts of the models could be manipulated by users based on the experiment. The app also had a quiz feature with audio feedback. Upon completion of the app students and teachers both responded positively to the addition of AR mobile apps in physics laboratory experiments.
Positive outcomes of the study:
- Features of the app were decided upon consultation with experienced teachers.
- Allowed students to observe experiments anywhere anytime by pointing the phone to a marker, which also made it unnecessary for schools to maintain expensive labs.
Shortcomings of the study
- A fixed marker designed for the app was required to initiate the AR animation.
- Feedback from the participants was not taken in a systematic way like previous studies.
ARTitser: A Mobile Augmented Reality in Classroom Interactive Learning Tool on Biological Science for Junior High School Students
12.In this research an iOS based augmented reality app was created to aid junior high school students in biology by visualizing biological figures in real life. Features of the app were determined by interviewing two biology course teachers and one class adviser. The app was built for iOS using Xcode along with other tools- ARKit, SceneKit, and SiriKit. Some mention worthy features of the app were,
- Teachers could upload reading material, subsequent 3D models of a biology figure and a quiz to the students.
- From the student’s perspective the app acted as a virtual teacher which would read aloud the reading material, launch the augmented 3D model in front of their eyes and finally present a quiz.
The existing method of teaching biology and their inadequacies were inquired from the teachers and students. Even though teachers used presentations, videos and other visual aids; participants noted many issues like- limited slides in presentations, necessity of high bandwidth for videos, outdated textbooks etc. After using the app participants feedback was for evaluation via a questionnaire in accordance with ISO/IEC 9126 (a software quality evaluation standard). The response showed high acceptance among the respondents- 30 students, 3 teachers and 2 IT administrators.
Positive outcomes of the study:
- Features of the app were decided upon consultation with course instructors about inadequacies of the traditional teaching methods.
- Option for teachers to provide study materials- reading material, 3D model and quiz to the students.
- Feedback questionnaire was designed following ISO/IEC 9126, a renowned International software quality evaluation standard.
Shortcomings of the study
- Participants were well-acquainted with the use of technology for studying such as- presentation slides, online videos.
- App runs on iOS based devices only.
Fun Learning with AR Alphabet Book for Preschool Children
13.This study describes the design and evaluation of AR technology by offering an Augmented Reality based Alphabet book for teaching preschool children in a fun and interactive manner. Usually, teaching young children sometimes becomes very difficult as their focus on learning differs from that of the older adult. The AR Alphabet Book was created to enhance existing alphabet learning by utilizing AR technology and being different from the traditional interface method to grab students' attention to learn various things faster and in a natural manner. For developing the book, ARToolkit was used, which is a library software for developing AR applications. The user will require a computer to run the software, a webcam to capture and track images of the pattern markers, and a computer monitor to display the virtual model viewed as overlaid over the pattern markers. The AR Alphabet Book mainly comprises of two sections:
- Alphabet Introduction section: The users can view both letters' cases and learn simple vocabularies of words start with the respective alphabet letter in an augmented manner. This section also provides pronunciations of each alphabet and words.
- Exercise section: Two puzzle games were provided to guess a letter name so that students can review what they have learned. A class of 15 five-year-old students was recruited to perform the observational study, and the students were asked to rate on five short and simple questions survey.
Positive outcomes of the study:
- Draws the attention of the students towards learning by developing the traditional learning methods.
- Engage students with the beneficial side of the usage of different modern technologies.
Shortcoming of the study:
- The only drawback was that the application was limited to its platform, making the system less user-friendly.
MagicToon: A 2D-to-3D Creative Cartoon Modeling System with Mobile AR
14.This paper talks about a mobile AR system that allows children to creatively build 3D cartoon scenes from their own 2D drawings on paper. The 3D cartoon generated from the 2D drawing is automatically overlaid into the real world. This system also allows the user (children) to author more complex scenes by placing multiple registered drawings simultaneously. The whole system was built by keeping two things in mind.
- Generating 3D texture cartoon models by fully leveraging children’s drawing skills in the real world while requiring minimum inputs.
- Providing the children the opportunity to author their own creative cartoon scene in an AR environment easily through some simple multi-finger gesture.
Positive outcome of the study:
- The authors broke down the workflow of the system in section 3. Which can be used to understand how the whole system has been engineered to work.
- We can understand the step-by-step implementation process as it is described in detail in section 4.
- The study has all the theoretical knowledge required for conceptualizing and understanding the intricate workings of such a system.
- If we want to add the feature of letting our users interact with the generated 3D objects, we can use this paper as our reference.
Shortcomings of the study:
- The authors provide very little information about the tools that were used to build the system.
- They used vuforia SDK to detect and track the images, whereas we were planning to use ArCore.
Meta-Review of Augmented Reality in Education
15.In this paper, the authors presented an elaborate study on how Augmented Reality is enhancing multiple fields of education by reviewing eight different papers. Some of the fields of education mentioned are physics, library, ecology, and astrology. This paper also describes how the AR technology implementation in education has changed over the years, due to the very frequent research conducted in this field. It is mentioned in this paper that Marker-based AR systems are more stable and effective compared to a marker-less tracking system. One of the papers had a literature review that had some interesting findings, which is, AR learning games and learning contents seemed to have lessened the amount of interaction among the learners. There were many other useful findings the authors got by reviewing these papers, some of them are:
- A contradictory conclusion was found, while some literature claiming AR reduces cognitive load and others claiming AR increases cognitive load.
- Another study found that one of the limitations of AR education settings is the teacher's group. This is because most of the existing AR education based app does not have an easy way to modify or add new content.
Positive outcome of the study:
- Collective knowledge from multiple papers was presented in a single paper.
- Got familiar with a new method of conducting a literature review, named Systematic Literature Review (SLR)
Shortcomings of the study:
- Filtered knowledge from eight different papers was present, which automatically reduces the scope of understanding and knowledge one could have acquired from reading each of the papers separately.
- There was hardly any technical knowledge about making an AR system present in the paper.
Creating interactive physics education books with augmented reality
16.In this paper, a framework was presented for creating educational Augmented Reality books that overlay virtual content over actual book pages. The framework provides supports for:
- Distinct types of user interactions
- Model and texture animations
- Improved marker design for educational books
Using this framework, three books were also created to teach the concepts of magnetism based on Level 2 NCEA Physics material. The books were designed in a way that each page has text and diagrams related to the topic. A handheld AR device connected to a PC was needed to view the augmented contents of the book.
AR Framework
Mainly, the framework was developed to make the development of the AR book more effortless. It was written in C++ and designed to run on Windows operating system only. The critical components of the framework are listed below:
- Viewpoint Tracking: I handled the calculation of the position of the user's camera relative to the AR book pages and then used this to overlay the virtual graphics in the AR view with the correct perspective. ARToolkit was used in the framework for tracking.
- Graphics Rendering: It defines 3D scenes as a graph of different nodes. OpenSceneGraph (OSG) was responsible for graphics rendering in this framework.
- Interactivity: For interactivity features, the user can build or use build-in-interaction by extending the framework's 'Page' class.
- Scene Creation: Handles the AR scenes shown on the pages of the book or the targeted surface. The process consists of creating the 3D content in a 3D modeling program and then exporting it to one of the formats compatible with OSG. An observational study was performed where ten female secondary school-aged students from Year 10 (aged 13 to 15 years) participated in the study along with their science teacher.
Positive outcome of the study:
- The framework provides supports for massive AR application development effortlessly.
- Viewing 3D complex diagrams and interacting with them helps students understand and build a solid concept of that particular subject efficiently.
Shortcomings of the study:
- The framework is not platform-independent as it only works on Windows operating system.
- The observational study was performed on a minimal number of participants.
A Survey of Augmented Reality
17.This paper surveyed the field of augmented reality (AR), in which 3D virtual objects were integrated into a 3D natural environment in real-time. It also analyzed state-of-the-art augmented reality, described the work performed at many different sites and explains the issues and problems encountered when building augmented reality systems. As it was a survey or review paper, it did not present any new research results. The contribution came from consolidating existing information from many sources and publishing an extensive bibliography of papers in this field. The potential application of AR was divided into six classes:
- Medical Visualization: AR can be a training aid for surgery if patients data can be collected.
- Maintenance and Repair: Instructions might be easier to understand if they were available, not as manuals with text and pictures, but rather as 3D drawings superimposed upon the actual equipment, showing step-by-step the tasks that need to be done and how to do them.
- Robot path planning: Instead of controlling the robot directly, it may be more simplistic to maintain a virtual version of the robot instead.
- Entertainment: Several exhibitors of that time showed "Virtual Sets" that merge real actors with virtual backgrounds, in real-time and 3D.
- Annotation and Visualization: Hand-held display could provide information about the contents of library shelves as the user walks around the library.
- Military Aircraft Navigation and Targeting: Military aircraft and helicopters have used head-up displays (HUDs) and helmet-mounted sights (HMS) to superimpose vector graphics upon the pilot's view of the real world.
In this paper, several limitations of the augmented reality system were also discussed. The errors and limiting factors that were faced that time and still we may need to overcome:
- The Registration Problem
- Distortion in the Optics
- Incorrect Viewing Parameters.
- Errors in the Tracking System
- Mechanical Misalignments
- Dynamic Errors
Augmented reality was far behind virtual environments in maturity. Very few commercial vendors used to sell complete, turnkey virtual environment systems. However, no commercial vendor was available to sell HMD-based augmented reality systems. A few monitor-based "virtual set" systems were available, but systems were primarily found in academic and industrial research laboratories in the past AR. Now, the situation has changed, and AR technology has been matured enough to make them convenient for simple hand-held devices such as mobile phones.
Note: Generally, a survey paper does not present new results. This paper was published in 1997 by Ronald T. Azuma, and so far, it has been cited over 12000 times. Almost every paper we have summarized cited this paper.
Evaluating the usability and acceptance of an AR app in learning Chemistry for Secondary Education
18.In this paper, the authors tried to evaluate the usability and user experience of an augmented reality application made for teaching chemistry in high school. The authors claimed that the integration of text recognition enabled the application to process information at runtime to generate the AR contents in the context of the recognized text, made the application unique from the rest of AR-Chemistry books out there. According to the authors the application consisted of three modules.
- Electronic Configuration: It used a periodic table as the target. When the mobile device is focused on the target, a virtual image of the Periodic table would appear on the screen. Then the users can select any of the elements on the virtual periodic table to see an animated electronic configuration of the element.
- Element Detection: It uses a small window to focus on printed texts to check whether the text or the symbol of the element can be recognized by the preconfigured dictionary of elements. Upon the recognition of the element, the application displayed its name, atomic number, valence, element type and atomic configuration.
- Molecular Geometry: This module is based on Lewis Theory. The geometry is calculated using the theory and shown to the user. The user can then interact in 3D with the touchpad provided at the bottom of the screen.
The authors used four tasks to evaluate the usability and user experience:
- In the first task, the user had to focus on the target to trigger the periodic table to pop-u[p on the screen.
- The second task required the user to focus the mobile camera on a printed text that denotes the chemical name of the element in order for the screen to show the basic information about the element.
- The third task was about writing a chemical formula of a compound in the app, in order to see a 3D structure of the molecule on the screen along with other information.
- The fourth task required the user to focus the camera on some chemical formulas in order for the screen to show two molecular structures simultaneously. The users also had to rotate the structures 180 degrees along the Y-axis using their fingers.
Positive outcome of the study:
- The paper had extensive information and directives on how to conduct a study on usability and user experience.
- They used twelve high-school students to conduct the study where they had almost equal numbers of men and women. Which led to understanding that women were a bit faster than men while executing the tasks.
- A performance difference between IOS devices and Android devices was detected.
Shortcomings of the study:
- There was very little information present in the paper about the tools used to build the application.
Synchronized AR Environment for Multiple Users Using Animation Markers
19.In this paper, the authors talk about an AR system that allows multiple users to concurrently see and share AR contents triggered by proposed animation markers with time sequence information. This system requires synchronization in order for the users to concurrently see and sare AR contents and building such a system can be really complicated. Hence, the authors proposed an animation marker that can transmit temporal (frame) information among the users for visualizing a synchronized AR environment. The author proposed the animation marker to be on a large display (main panel) from which tablet PCs and smartphones (sub-panels) can get synchronized AR experience. This system does not require the users to transmit temporal information, position, and pose among themselves because the marker itself can transmit these pieces of information to the users.
Positive outcome of the study:
- Got to learn there are two ways of designing a synchronized AR environment.
- Was able to get a brief overview of a simple synchronized AR environment system.
- Reading this paper led me to believe if we were to integrate this feature into our application, according to this paper we would have to take the complicated approach. Where users themselves have to transmit the information such as transmit temporal information, position, and pose among themselves.
Shortcomings of the study:
- There was no information present in the paper about the tools used to build the application.
- The authors provided very little information on the other approach of making a Synchronized AR environment for multiple users.
- The study did not have elaborate information on the system architecture.