How to Make a 3D Model of Your Head Using Blender and a RealSense Camera

I'm very excited about sensification of computing: an idea where smart things sense the environment, are connected, then become an extension of the user. Asan Intel Software evangelist (and a self-confessed hobbyist coder), I'm fortunate enough to get to experiment with these possibilities using Intel's cutting-edge technology. Previously, I had written about 3D-scanning the world around you in real time and putting a virtual object in it; however in this tutorial, I'd like to explain how to do the converse, by 3D-scanning yourself then adding that scan to the virtual world.

More specifically, we will:

1. Scan your head using a RealSense camera and the SDK.
2. Convert the 3D file from OBJ format to PLY format (for editing in Blender).
3. Convert the vertex colors to a UV texture map.
4. Edit the 3D mesh to give it a lower vertex and poly count.

We'll end up with a 3D, colored mesh of your head, ready to use in Unity. (In the next tutorial, we'll see how to actually get this into Unity, and what you could use this for in a game.)

You will need:

• An Intel RealSense camera (or computer with integrated RealSense camera)
• A computer with an Intel Core 4th gen processor (or better)
• The RealSense SDK (which is free to download)
• MeshLab (free)
• Blender (free)

Scanning Your Head

After installing the SDK, go to the Charm Bar (in Windows 8) or the Start Menu (in Windows 7 or 10) and search for RealSense SDK Sample Browser. Find it in the list, right-click it, and select Run as Administrator.

In the Sample Browser you will find a section called Common Samples. These are samples that will work on both the front-facing and rear-facing RealSense cameras. Find the sample called 3D Scan (C++).

How to Scan Your Head

• Select Run from the 3D Scan (C++) section of the sample browser.
• Put your head in front of the camera and you will see it in the view of the scanner. Hold it still until the second window pops up.
• Once the second window pops up, start to rotate in one direction (slowly), then reverse direction and try to go all the way around. 
• Once you've made a full loop around, tip your head back and then forward, then hold still. For the last few seconds. keep your eyes looking forward just over the camera, so that your eyes get set in a static position.
• When the scan is complete, a browser will open, allowing you to drag and rotate the model around. The model will be saved to your Documents folder as 3Dscan.obj.
• You can rerun the sample as many times as you like to get a scan that you're satisfied with.

Tips for Scanning

• The sample is designed to track a rigid 3D object in space as it rotates in front of the camera. This means you can scan it from multiple sides.
• If part of the object is rotating and another part is still, it will confuse the tracking algorithm and you'll end up with a bad scan. So, when scanning your head, do not just turn your neck to scan; instead, rotate your entire torso.
• You can tip forward, tip back, rotate left, rotate right, and turn around. With practice, it is possible to scan an object from all angles.
• You should scan in an area with even lighting. Avoid shadows and specular reflections, as they will get baked into your 3D texture.

Converting to PLY Format

The OBJ format is the default format you get from the scan. To use it in Blender (and later Unity), we'll convert it to PLY format using a free tool called MeshLab. So, download, install, and run MeshLab, if you haven't already.

From the File menu, select Import. Find the 3Dscan.obj file you created in the previous section.

After you've imported your object, it may be on its side. You can left-click at the bottom of the object and drag up to rotate the object and examine it. 

You should see the mesh textured with color. The color information is stored as vertex colors, meaning that each vertex is colored with the pixels obtained from the RGB camera when it scanned your head. 

Click File > Export Mesh As. In the dialog box that appears, select Stanford Polygon File Format (*.ply) from the Files of type drop-down:

Save the resulting PLY somewhere you'll be able to find it later.

Converting Vertex Colors to a UV Texture Map

The idea here is to import the 3D file to Blender, make a copy of it, clean up the copy, and map the vertex colors as a UV map for the new duplicate version.

Download, install, and run Blender, if you haven't already.

Import the Object to Blender

Create a new project in Blender, select Import > Stanford (.ply), then select the PLY file you just exported from MeshLab:

After importing it, you will see that the textures are not visible:

However, if you switch to Texture view, you will see that the RGB colors from the scan are now mapped to the Texture mode:

To see the colors properly, switch view mode again—this time, to Material. On the right-hand panel, select the Materials icon, and then create a new Material. 

In the panel editor, set Shading to Shadeless, and in the Options, turn on Vertex Color Paint. This successfully sets the vertex colors to the material of the object.

Now, this isn't yet a workable mesh for us. It's full of holes, it's rough, and the poly count is much higher than we'd like it to be, if we're going to use it in a game.

We need to create a version of this mesh that we can clean up, but map this vertex-colored texture as a UV map. That version is the version we will bring into Unity.

Create a Duplicate of the Mesh

Select Object > Duplicate Objects and hit Enter. Do not move your mouse.

Next, select Object > Move to Layer, and then select the square to the right of the current layer (the second square from the left in the top row).

Go to the Asset Browser and rename your meshes. Name one Original Mesh and the other Second Mesh. This will help you tell them apart.

Reduce the Poly Count of the Duplicate

Select the Second Mesh, select the relevant layer, and switch view mode from Material to Solid.

Then, from the right-hand panel, click the icon that looks like a wrench to add a modifier. We are going to reduce the poly count of this mesh and clean it up.

Select Tools > Remesh. In this dialog, set the Oct Tree to between 7 and 8. Set Mode to Smooth, and turn on Smooth Shading. This will cause you to lose the material mapping:

Click Apply. You now have a cleaner Mesh to which you can apply the vertex and UV map.

Apply the Vertex and UV Map

Go Into Edit mode on this Second Mesh. Click Select > Select All.

Create a new window and set it to UV/Image Editor mode to see the UV map.

In the 3D window, select Mesh > UV Unwrap > Smart UV Project.

In the UV Map window, select New to create a texture. Turn off Alpha. Name the texture the name you would like to give your final model.

Switch to Object mode. You will see a black model and a black UV map. That's okay!

Now select both layers by Shift-clicking them to make them both visible.

In the Asset Browser, select the Original Mesh. Click the camera icon.

In the Bake section, turn on Selected to Active. Make sure that Bake to Vertex Color is turned off.

In the Scene section, set Color Management to None.

In the Asset menu, select the Original Mesh. Then, hold down Shift and select the Second Mesh. (You must do it in that order.) Be sure that both layers are visible with the eye and camera icons.

Click the camera icon. In the Bake section, click Bake.

You should now have a texture mapped. To see this, turn off the first layer, so you are only seeing the layer with the second mesh. Select the Texture view and you will see your mesh textured with your UV Map. 

You will notice some funkiness where the image filled in missing areas of your mesh:

You can switch to Texture Paint mode to clean those up. You can also now switch to Sculpt mode to fill in missing areas, as well as to clean up or add features to your mesh. 

Export the Mesh

Once you've cleaned up your mesh, you can get ready to export it:

1. Make sure the UV map is named the same as the mesh, and is in the same directory as the mesh.
2. Go to the UV Map window, select Image > Save As, and navigate to the directory where you will save the mesh.
3. Select File > Export to .FBX, and enter the same name you gave the UV map.

Your model is complete!

Conclusion

You've scanned in your head, reduced the polycount, cleaned up the model, and exported it as a 3D model ready to use in other applications. In a future tutorial, we'll import this 3D mesh into Unity, and look at how it could be used in games.

The Intel® Software Innovator program supports innovative independent developers who display an ability to create and demonstrate forward looking projects. Innovators take advantage of speakership and demo opportunities at industry events and developer gatherings.

Intel® Developer Zone offers tools and how-to information for cross-platform app development, platform and technology information, code samples, and peer expertise to help developers innovate and succeed. Join our communities for the Internet of Things, Android*, Intel® RealSense™ Technology, Modern Code, Game Dev and Windows* to download tools, access dev kits, share ideas with like-minded developers, and participate in hackathons, contests, roadshows, and local events.