“Bibbidi-Bobbidi-Boo” — as Cinderella’s fairy godmother sang these three words and motioned the tip of her magic wand, everything turned into what she imagined them to be.

Yes, it’s magic in a film. The exact thing can’t happen in real life. But what about something like it?

With virtual reality, researchers at the Korean Advanced Institute of Science and Technology (KAIST) developed a new 3D sketching workflow that allows designers to create 3D sketches with both their hand motions and pen, singing a little bit of that magic spell themselves.

How it evolved: Turning 2D into 3D

Every product starts with conceptualizing an idea. While most designers first start with sketching their ideas by hand, at one point, they must all transform their hand sketches into 3D models.

While most designers currently use CAD (computer aided design), a design software that creates 3D models, the software is too complex for all designers to quickly become skilled users.

In response to this, in 2008, Seok-Hyang Bae, a professor of industrial design at KAIST, created the ILoveSketch system, which allows designers to sketch directly on concept 3D curve models.

Then, according to Bae, in 2016, his team incorporated hand-tracking sensor into their SketchingWithHands system to allow designers to sketch handheld product concepts by capturing whatever hand poses they want and drawing the details on top of the captured hand pose.

And recently, Bae and his team used virtual reality to further improve the sensor system to capture not only the static hand poses, but also the movement of the hand, which allows designers to easily scaffold in air, indicating the scale and proportion of the product.

“In particular, we recognized that existing 3D sketching systems have no analogous technique dedicated to ‘scaffolding,’ which is used when designers roughly set the desired scale and proportion of products in 2D sketching, and developed methods to utilize rapid hand motion as ‘3D scaffolding’ and combine it with delicate pen drawing,” said Bae and Yongkwan Kim, a doctoral candidate at KAIST and lead researcher.

Start with hand and finish with pen

Since drawing with hand motion is quick but inaccurate and drawing with pen is detailed but time consuming, the team combined the strengths of both mediums to take full advantage.

Called the Agile 3D Sketching with Air Scaffolding, the system identifies descriptive hand motions from transitory hand motions and extract only the intended shapes from unconstrained hand motions, based on air scaffolds from the identified motions.

First, designers use hand motions to air scaffold rough 3D shapes with accurate scale and proportion. Then, they can draw in details on the 3D shapes using a pen.

Chosen as one of the CHI (Conference on Human Factors in Computing Systems) 2018 Best Papers by the Association for Computing Machinery, the researchers believe that the technique will ease the way for 3D ideation, leading to efficient product design in terms of time and cost.

“There have been many attempts to encourage creative activities in various fields by using advanced computer technology,” Bae said in a statement.

“Based on in-depth understanding of designers, we will take the lead in innovating the design process by applying cutting-edge technology.”

Testing with real users

After testing with real-life product designers, the team found that the system is user-friendly, demonstrates good applicability, and most importantly, reduces design time while enhancing the accuracy of defining the proportion and scale of products.

According to Bae and Kim, the designers were new to their system, but showed competency after just 10 minutes of introduction and 30-60 minutes of self-training.

Although the completion time varied by designers because their design concepts varied widely, the researchers expect a skilled user to take 5-10 minutes to sketch a reasonable 3D product concept using their system.

“Sketching in the design process is accompanied by a continuous search for new ideas,” said Bae and Kim.

“Therefore, the time required for sketching depends on how each designer comes up with new ideas, tries out different alternatives, and develops the ideas with a desired level of detail.”

For every designer in every field

Eventually, the team hopes this tool will be applicable to various fields, including the automobile industry, home appliances, animations and the movie-making industry, and robotics.

It also can be linked to smart production technology, such as 3D printing, to make manufacturing processes faster and more flexible.

“3D sketching is a promising technology, but has not yet been applied to professional design practice,” said Bae and Kim.

“By developing a practical 3D sketching system that can be applied to the design practice, allowing field designers to author 3D design concepts with higher performance and quality, we would like to contribute to improving inefficient current design processes that are separated by 2D sketching and 3D CAD modeling.”

In addition to enhancing the applicability of 3D sketching in the design world, the team hopes to make their system so user-friendly that all of us can some day easily express and share ideas in 3D, according to Bae and Kim.

“I believe the system will enhance product quality and work efficiency because designers can express their 3D ideas quickly yet accurately without using complex 3D CAD modeling software,” Kim said in a statement.

“I will make it into product that every designer wants to use in various fields.”