Introduction

Why we developed this project?

Recently, IOT-based equipment has been increasing rapidly and it is widely used in real life. In addition, large companies are enjoying steep growth with interest. For example, Samsung's "mirror display" shows that it provides tailor-made hair consulting and weather information services to its customers in the beauty salon where the product is installed. However, these smart mirror products have disadvantages in that they are not only experiential because they are limited to interaction with the user.

---

In order to overcome these disadvantages, a smart mirror using augmented reality (AR), which is natural and user-friendly, is needed. Especially, in the case of makeup, since the face type is different for each user, it is necessary to increase the personalized makeup experience effect, which is very suitable for the augmented reality environment.

---

In previous researches related to this, there was the result of the paper used in the user interface by detecting faces, extracting the facial landmark of the face, and detecting the pose. There are also studies to track faces and objects in real time. However, these are merely about detecting and recognizing faces on a computer, and they have not been realized through augmented reality. In addition, existing smart mirrors only provide services that recognize faces through real-time images and display relevant information such as images or weather.

---

In this study, we propose a smart beauty mirror system for makeup guide using augmented reality. The proposed system extracts facial features by using existing face detection and face tracking techniques and classifies facial shapes by using the tensor flow. Appropriate makeup guidelines were displayed on the eyebrows and near the lips for each of the classified face types. The development system allows users who are new to make-up to easily practice make-up through the guidelines displayed on the screen in real time.