IKEA launches AI-powered design experience (no Swedish meatballs included)

We look forward to presenting Transform 2022 in person again on July 19 and virtually from July 20 to 28. Join us for insightful conversations and exciting networking opportunities. Register today!


For IKEA, the latest digital transformation is all about home design powered by artificial intelligence (AI) — minus the home furnishings and décor retailer’s famous Swedish meatballs.

Today, IKEA has launched Creative, a design experience designed to bridge the e-commerce and in-store customer journeys, powered by the latest AI developments in spatial computing, machine learning and 3D mixed reality technologies . Available in the app and online, IKEA Creative’s core technology was developed by Geomagical Labs, an IKEA retail company acquired by Ingka Group (the holding company that controls 367 stores of 422 IKEA facilities) in April 2020.

IKEA Kreativ is the next step in IKEA’s long journey to digital transformation. According to the company, it is the first full-featured mixed reality self-service design experience for home retail for lifelike and accurate design of real-world spaces that are deeply integrated into the digital shopping journey. A user can use either IKEA inspiration images from the digital showroom or their own captured images to place furniture and fixtures, experiment with options, and more.

“The 3D and AI technology developed by Geomagical Labs is used to digitally highlight the uniqueness of IKEA,” says Phil Guindi, product manager at Geomagical Labs. “This is a pivotal moment in the IKEA transformation journey as we continue to evolve and innovate to meet customers and their needs where they are.”

How IKEA Creatively Offers AI-Driven Exploration

Home design can be tricky, Guindi said, because people often buy out of context and rely on their imaginations. “In fact, 87% of our customers say they want to feel good in their home, but only half of them know how,” he said.

IKEA Kreativ transforms room photos into stunning interactive “digital playrooms” that anyone can use to explore design ideas. “By thoroughly understanding a photographed scene—including the 3D geometry of the scene, the objects in the scene, the lighting in the scene, and the materials in the scene—clients can interactively design room images, allowing them to add furniture and wall decor and remove existing furniture digitally,” he said.

The core technology is bespoke and proprietary, he said. “Wherever possible, modern open standards and open source have been used rather than licensed or vendor services to maximize freedom for innovation and minimize lock-in,” he said.

Both mobile and web applications connect to a scalable, containerized, cloud-based platform of microservices and AI pipelines hosted on Google Cloud Platform.

According to Guindi, when a client scans a room, photo and sensor data is uploaded from the phone to the AI ​​pipeline in the cloud, where it’s processed in a highly parallelized GPU compute cluster to create wide-field spatial data images to deliver client To enable applications to interactively design and edit the scenes.

“By offloading complex calculations to the cloud, we enable less expensive, low-power mobile devices to run complex AI algorithms, reaching more IKEA customers,” he said.

Neural network technology similar to self-driving cars

The AI ​​for IKEA Creative was developed using the same visual AI neural network technologies as self-driving cars to understand living spaces using visualization at scale, according to the company’s press release.

Guindi explained that this is similar to self-driving cars, as robotic vehicles need to develop some understanding of the space around them, called “spatial awareness” or “scene awareness,” to operate safely.

“Robot vehicles use multiple technologies to understand the world around them, such as semantic segmentation to recognize and outline important objects in space,” he said.

Understanding an interior scene bears some similarities to a vehicle perceiving its surroundings. IKEA’s new digital experience draws inspiration from these methods – although interior design applications face unique and difficult challenges. For example, training neural networks for indoor vehicle applications does not work well because rooms and cities look very different and contain different objects.

“You need to specifically train networks with large amounts of indoor training data to give usable indoor results,” Guindi said.

Interiors and 3D computer vision

Interiors are also notoriously difficult to geometrically reconstruct by 3D computer vision algorithms, as many surfaces are carefully painted to be blank (with no traceable visual features), glossy (the specular reflections confused range estimators), repeating factory-made patterns (the visual confuse feature trackers) or fail because rooms are much darker than outdoor environments.

“As we implement a mixed reality solution to match virtual objects with real photography, we need to understand the lighting in a space much better,” explained Guindi. “We want to ‘erase’ objects from photos at home, so we need technology to be able to appreciate the geometry and imagery hidden behind furniture.” And while robotic vehicles can afford expensive sensor hardware like laser depth sensors, radar and multi-camera arrays, IKEA makes a point on accessibility, he added: “We want everyone to be able to use the applications on everyday smartphones without the need for exotic or specialized hardware.”

Once a view has been scanned and processed by the Cloud AI pipeline, consumers can design a space from anywhere on any device, be it their desktop web browser, laptop web browser and the IKEA mobile app.

“When they start a design experience, they are shown an immersive, wide-angle 3D photo of the scanned space,” Guindi said. Consumers can browse a catalog window of IKEA products and select any number of products to add to the space, appearing in the photo with realistic size, perspective, occlusion, and lighting. The user can move and rotate products, stack decor accents on top of other products, and hang wall art on the walls. Customers can also edit their original room by digitally removing items they may no longer need, like an old couch they want to replace.

IKEA Kreativ faced many challenges

IKEA Creative faced many challenges before launching, Guindi said. For one thing, it was a challenge to bring IKEA products to life in an inspiring and realistic way.

“This requires detailed estimates of scene lighting and geometry, and the use of 3D graphics rendering to render the materials,” he said. “And this requires the creation of storage-efficient, visually appealing 3D representations of the IKEA product range.”

In addition, IKEA must understand a photographed scene accurately enough for the customer to successfully design their space. “That means recognizing the presence and 3D position of floors, walls and surfaces that you can place furniture on so you can see them in the right size and perspective,” Guindi explained. This means, for example, that foreground objects in space will believably obscure objects you move behind them. This includes estimating the light sources in the scene to cast shadows and reflections.

improvements over time

AI scene perception technology is an actively researched topic and remains a very difficult challenge, Guindi continued. “Today’s product can feel magical, but it’s not flawless,” he said. “For example, it can become confused if it doesn’t see much floor or if the walls are completely empty.”

Still, IKEA Creative is “a huge leap forward for IKEA and for our customers, offering customers opportunities that weren’t possible before,” he said. “We see our customers achieve impressive results and have evaluated the product and technology in hundreds of thousands of different rooms where we have seen that the algorithms work well in the vast majority of cases.”

More improvements will be released over time, he added, including the ability to change wall colors in your own space and add wall- and ceiling-mounted furniture, as well as more collaborative ways to design a home with others.

VentureBeat’s mission is intended to be a digital marketplace for technical decision makers to acquire knowledge about transformative enterprise technology and to conduct transactions. Learn more about membership.

Leave a Comment