Virtual Try-On with Augmented Reality

Where we are right now

Augmented reality is an incredibly valuable technology for the fashion industry, however only a few have used it for their business so far. Buying products such as clothes, shoes, glasses and accessories online has become the norm thanks to e-commerce. But what if they could also try the items on online? The industry is on the verge of taking that step.

Augmented reality (AR) is the technology that enables virtual try-ons. Unlike virtual reality (VR), AR doesn't block out reality, but extends (/augments) it. This is exactly what happens when a customer taps their smartphone and software colors her lips with a different lipstick in the camera image. At the moment, an entire industry is dreaming of the potential applications that this will bring.

Changing room are still the core element of offline shopping. With Virtual Try-Ons, that is now changing. The technology is already a reality today, apps like Snapchat have long been using AR in this context. Many brands in the fashion and retail sectors are ready to integrate virtual try-ons into their offerings. And the technology is currently accommodating them, because it convinces with an increasingly seamless integration of digital content into the users reality.

What Advantages does the Technology Offer?

Virtual try-on via augmented reality is a technology with many advantages for buyers and companies. Some of them are obvious, others are only apparent at second glance.

Better user experience: Trying on requires nothing more than a click on the try-on button. Customers can try products faster and get an overview of the range, all from the comfort of their own home.

Especially in the fashion sector, returns are a high cost factor. AR technology can solve one of the industry's most pressing problems.
What a customer has already tried on virtually, he is less likely to send back. After all, they get a more realistic impression of the product than is conveyed by photos alone.
Research in this area is currently gaining insights into a person's brain activity during virtual try-on - and surprisingly, according to initial findings, the same brain regions are active as during real try-on.

Personalization: Shoppers appreciate offers that are personally tailored to them. Virtual try-ons offer the opportunity to elegantly integrate such recommendations into the shopping experience. Once a customer has tried on an item that doesn't fit or they don't like, they are already looking for an alternative anyway. The online store should immediately have this ready and suggest it for the next fitting.

Simple integration: The integration into the existing web page design is done with a simple button. When the user clicks on it, he or she is taken directly to the virtual fitting room.

Higher engagement: Trying on requires significantly more activity from the user than clicking through product pages. This is also beneficial to the conversion rate. Statistics on the increase of engagement and conversion rate vary widely, ranging from 5% to 300% - it all depends on the quality of the solution, the industry and deployment type.

These hurdles still need to be overcome

In order to virtually try on products like clothes, glasses, handbags and shoes, they need to be in digital form. And preferably in photorealistic quality, so that customers get a realistic impression.

But not all companies have this data yet. Before they can use Virtual Try-On, they first have to digitize their products. This can take a long time for larger catalogs. If the data is already available, file-conversion is still required in many cases in order to be able to work with it in the AR apps.

Once the data is available, the next step is to integrate it into the existing e-commerce solution. However, these are often not yet prepared for AR apps. Integration is therefore problematic or at least very complex.

Performance is another aspect that companies have to keep in mind. Not everyone has an iOS smartphone of the latest generation. Android systems in particular are very broad in their performance and often cannot yet keep up with the high demands that AR makes.

Augmented reality has only become widespread thanks to the introduction of faster graphics processing units (GPUs) in mobile systems and neural processing units (NPUs). However, not every smartphone has NPUs to run the deep learning algorithms required for tasks such as real-time tracking and mapping fast enough. Since we are talking about video applications in this context, at least 25 frames per second should be displayable to enable a smooth user experience.

Three Real-World Examples

Three examples quickly illustrate what virtual try-ons can do. The technology is as versatile as the items that are for sale online.

vyking.io - Try on sneakers from an app

For the virtual fitting of sneakers, vyking.io presents a proprietary solution for tracking human feet via AR app. The sneakers can be visualized directly on the foot. The developer reports that over 40 retailers are already using the technology in their apps or on the website. What's interesting about this offering is its cross-platform compatibility. The software runs on systems with iOS, Android or even as a web application (WebAR).

The retailer can either provide their own 3D models or vyking.io takes care of the required scanning process. The latter should be interesting for smaller companies in the fashion sector that shy away from this effort.

Make-up-Try-On via YouTube AR

YouTube already introduced its own AR solution for trying out makeup in 2019, which is integrated directly into the YouTube app. Lipsticks from the MAC Cosmetics brand were the first to be tried out virtually as part of a corresponding video campaign.

Make-up tutorials on the platform are extremely popular with viewers, so the integration of such technology was an obvious choice for the Internet company. Users can only access the virtual try-on via their mobile device. When the function is activated, the app goes into split screen. Then, for example, lipsticks in different colors can be selected, and the crossfade over the lips is automatic. The high quality of the tracking is striking.

The cosmetics company Maybelline from New York also offers a pretty good experience in this area. You can also quickly try it out for yourself:

https://www.maybelline.de/virtual-try-on

Try on Shirts and Jackets virtually with Farfetch

Farfetch is an internationally active e-commerce company in the field of designer fashion. In cooperation with Snapchat, it offers a tool for trying on clothes virtually. This includes jackets from Virgil Abloh's brand Off-White. The technology accurately maps the garment onto the user's upper body. Added to this is a real-time fabric simulation to account for factors such as gravity. The aim is to make the item look as natural as possible. This technology is apparently already very advanced.

What the Future holds

Virtual Try-On already offers a compelling user experience. In the near future, we expect strong improvements in tracking. The increasing spread of depth sensors such as LIDAR scanners (Light Detection and Ranging) in newer smartphone generations that measures rooms, objects and bodies much more reliably than existing algorithms based on image data. This can significantly increase the immersive experience for the user.

In many cases, methods from the research field of artificial intelligence are used in virtual try-on experiences, e.g. in tracking or in the real-time modification of image or video data. Great progress is currently being made in this area, which is why an improvement in the user experience can also be expected in this respect.

There is also still some room for improvement in terms of the degree of realism or sizing. If the technology advances, trying on shoes, for example, should lead to the right size more reliably. There is a lot to expect in this area.

How about automatic foot measurement, for example, which immediately alerts the shopper to possible pressure and tension points? The app could directly make a recommendation as to whether it is a suitable product or not. AR thus leads over into the area of customer advice and opens up new possibilities here.

For the next development steps, it is crucial that the data quality improves. In most cases, good 3D data is the prerequisite for optimizing service. To achieve this, a 3D data strategy and a corresponding pipeline are required. After all, high-quality data is not a product of chance, but the result of a planned approach. Many companies in the fashion and interior-design industry are already looking for partners to help them develop such a strategy and implement the technology. This is the only way many of them can secure their success in this area or even expand it as pioneers.