Chapters
Chapters

Chapters

Hide chapters

Metal by Tutorials

Fourth Edition · macOS 14, iOS 17 · Swift 5.9 · Xcode 15

Before You Begin

Section 0: 6 chapters
Show chapters Hide chapters

Section I: Beginning Metal

Section 1: 10 chapters
Show chapters Hide chapters

Section II: Intermediate Metal

Section 2: 8 chapters
Show chapters Hide chapters

Section III: Advanced Metal

Section 3: 8 chapters
Show chapters Hide chapters

Section IV: Ray Tracing

Section 4: 7 chapters
Show chapters Hide chapters

11. Maps & Materials
Written by Caroline Begbie & Marius Horga

Heads up... You're reading this book for free, with parts of this chapter shown beyond this point as scrambled text.

You can unlock the rest of this book, and our entire catalogue of books and videos, with a Kodeco Personal Plan.

Unlock now

In the previous chapter, you set up a simple Phong lighting model. In recent years, researchers have made great steps forward with Physically Based Rendering (PBR). PBR attempts to accurately represent real-world shading, where the amount of light leaving a surface is less than the amount the surface receives. In the real world, the surfaces of objects are not completely flat, as yours have been so far. If you look at the objects around you, you’ll notice how their basic color changes according to how light falls on them. Some objects have a smooth surface, and some have a rough surface. Heck, some might even be shiny metal!

Take for example, this sphere with a brick texture. The render on the left shows a simple color texture with the sun shining directly on it. The physically based render on the right is what you’ll achieve by the end of this chapter.

PBR render
PBR render

3D artists achieve real-world shading by creating materials for their models. Depending on the complexity of the surface, this material might be a texture, or it might be a numeric value to indicate the strength of the particular quality. You’ll create materials, and add textures where necessary, to improve the render.

Physically Based Rendering (PBR)

As its name suggests, PBR attempts physically realistic interaction of light with surfaces. Now that Apple Vision Pro is a reality, it’s even more important to render your models to match their physical surroundings.

Note: Just because you can make photo-realistic renders, it doesn’t mean that you always should. Disney uses stylized PBR, and you can change your fragment shaders to produce the result you desire. There is no “standard” PBR shader code and you can interpret the provided asset’s materials in any way you choose.

The general principles of PBR are:

  • Surfaces should not reflect more light than they receive.
  • Surfaces can be described with known, measured physical properties.

A Bidirectional Reflectance Distribution Function (BRDF) defines how a surface responds to light. There are various highly mathematical BRDF models for both diffuse and specular, but the most common are Lambertian diffuse; and for the specular, variations on the Cook-Torrance model (presented at SIGGRAPH 1981). This takes into account:

  • micro-facet slope distribution: The previous chapter briefly covered micro-facets and how light bounces off surfaces in many directions.
  • Fresnel: If you look straight down into a clear lake, you can see through it to the bottom, however, if you look across the surface of the water, you only see a reflection like a mirror. This is the Fresnel effect, where the reflectivity of the surface depends upon the viewing angle.
  • geometric attenuation: Self-shadowing of the micro-facets.

Each of these components have different approximations, or models written by many clever people. It’s a vast and complex topic. In the resources folder for this chapter, references.markdown contains a few places where you can learn more about physically based rendering and the calculations involved. You’ll also learn some more about BRDFs and Fresnel in Chapter 21, “Image-Based Lighting”.

Common PBR Material Properties

Poly Haven has some great 3D assets and textures. For example, this alarm clock model:

An alarm clock
Op itezs ywizr

Common texture maps
Metzig fewqeyu ciph

The Starter App

➤ In Xcode, open the starter project for this chapter.

A sphere with color texture
I vptopu kagf howaq tejqexa

Examining USD Files in Reality Composer Pro

You probably want to know how to change materials and add textures to your USD models.

The Sphere project
Mdu Bhfecu qjadutv

Reality Composer Pro
Vaevibs Jiqtenol Hxe

Material
Deloxaoh

Color, roughness and normal textures applied
Jixod, jeetkpaky eyr jubkij warnonij ippheob

Materials

Before adding more textures to your render, you’ll set up the basic default material values, so that your PBR shader can get to work.

typedef struct {
  vector_float3 baseColor;
  float roughness;
  float metallic;
  float ambientOcclusion;
} Material;

var material: Material

private extension Material {
  init(material: MDLMaterial?) {
    self.init()
    if let baseColor = material?.property(with: .baseColor),
      baseColor.type == .float3 {
      self.baseColor = baseColor.float3Value
    }
    ambientOcclusion = 1
  }
}

material = Material(material: mdlSubmesh.material)

MaterialBuffer = 14

var material = submesh.material
encoder.setFragmentBytes(
  &material,
  length: MemoryLayout<Material>.stride,
  index: MaterialBuffer.index)

constant Material &_material [[buffer(MaterialBuffer)]],

Material material = _material;

float3 baseColor = baseColorTexture.sample(
  textureSampler,
  in.uv * params.tiling).rgb;

if (!is_null_texture(baseColorTexture)) {
  material.baseColor = baseColorTexture.sample(
  textureSampler,
  in.uv * params.tiling).rgb;
}

float3 normalDirection = normalize(in.worldNormal);
float3 color = phongLighting(
  normal,
  in.worldPosition,
  params,
  lights,
  baseColor
);
return float4(color, 1);

// 1
float3 normal = normalize(in.worldNormal);
float3 diffuseColor =
  computeDiffuse(lights, params, material, normal);

// 2
float3 specularColor =
  computeSpecular(lights, params, material, normal);

// 3
return float4(diffuseColor + specularColor, 1);
PBR shading
QXK jyivexx

Surface Roughness

The smoother a surface is, the shinier it should be. So far, you haven’t set a roughness value in Material, so the roughness is zero. The surface is infinitely shiny.

material.roughness = 0.4;
Specular highlight
Lmuwiruy jepywojpy

Roughness texture
Piekmxumg moqsuze

var roughness: MTLTexture?

roughness = material?.texture(type: .roughness)

if let roughness = material?.property(with: .roughness),
  roughness.type == .float {
  self.roughness = roughness.floatValue
}

NormalTexture = 1,
RoughnessTexture = 2,
MetallicTexture = 3,
AOTexture = 4

encoder.setFragmentTexture(
  submesh.textures.roughness,
  index: RoughnessTexture.index)

texture2d<float> roughnessTexture [[texture(RoughnessTexture)]]

if (!is_null_texture(roughnessTexture)) {
  material.roughness = roughnessTexture.sample(
    textureSampler,
    in.uv * params.tiling).r;
}
Roughness texture applied
Loojxvurj kabduxo atzboem

Normal Maps

This is your desired final render:

An object rendered with a normal map
Ek uxgosk goqgoniz wevt u hovtos fod

A normal map texture
U bifyun suw zefdeso

Normals
Defraqq

Normal map channels
Pafric gux llikpidy

A flat normal map
O vpus rorvay cib

Creating Normal Maps

To create successful normal maps, you need a specialized app. You’ve already learned about texturing apps, such as Adobe Substance Designer and Mari in Chapter 8, “Textures”. Both of these apps are procedural and will generate normal maps as well as base color textures. In fact, the brick texture in the image at the start of the chapter was created in Adobe Substance Designer.

A cross photographed and converted into a normal map
A vracm ktafapruyxil asb qalkoqhuh itqe i jurjuh mor

Tangent Space

To render with a normal map texture, you send it to the fragment function in the same way as a color texture, and you extract the normal values using the same UVs. However, you can’t directly apply your normal map values onto your model’s current normals. In your fragment shader, the model’s normals are in world space, and the normal map normals are in tangent space. Tangent space is a little hard to wrap your head around. Think of a cube with all its six faces pointing in different directions. Now think of the brick’s normal map applied to it with all the bricks the same color on all the six faces.

Normals on a sphere
Pashoss uc e pvseke

Visualizing normals in world space
Bazuayikedh lamzery oq cesrs xceto

The TBN matrix
Swo WNH zizhay

Using Normal Maps

➤ In the Geometry group, open Submesh.swift, and add a new property to Submesh.Textures:

var normal: MTLTexture?

normal = material?.texture(type: .tangentSpaceNormal)

encoder.setFragmentTexture(
  submesh.textures.normal,
  index: NormalTexture.index)

texture2d<float> normalTexture [[texture(NormalTexture)]]

float3 normal;
if (is_null_texture(normalTexture)) {
  normal = in.worldNormal;
} else {
  normal = normalTexture.sample(
  textureSampler,
  in.uv * params.tiling).rgb;
}
normal = normalize(normal);
return float4(normal, 1);
The normal map applied as a color texture
Kka tutcen lel iwyjuoh uz i mibos yulpide

return float4(normal, 1);

1. Load Tangents and Bitangents

Model I/O will create tangent and bitangent attributes for you in new MTLBuffers. First, define these new buffer attribute and buffer indices.

Tangent = 3,
Bitangent = 4

TangentBuffer = 2,
BitangentBuffer = 3,

vertexDescriptor.attributes[Tangent.index] =
  MDLVertexAttribute(
    name: MDLVertexAttributeTangent,
    format: .float3,
    offset: 0,
    bufferIndex: TangentBuffer.index)
vertexDescriptor.layouts[TangentBuffer.index]
  = MDLVertexBufferLayout(stride: MemoryLayout<float3>.stride)
vertexDescriptor.attributes[Bitangent.index] =
  MDLVertexAttribute(
    name: MDLVertexAttributeBitangent,
    format: .float3,
    offset: 0,
    bufferIndex: BitangentBuffer.index)
vertexDescriptor.layouts[BitangentBuffer.index]
  = MDLVertexBufferLayout(stride: MemoryLayout<float3>.stride)

let (mdlMeshes, mtkMeshes) = try! MTKMesh.newMeshes(
  asset: asset,
  device: Renderer.device)

var mtkMeshes: [MTKMesh] = []
let mdlMeshes =
  asset.childObjects(of: MDLMesh.self) as? [MDLMesh] ?? []
_ = mdlMeshes.map { mdlMesh in
  mtkMeshes.append(
    try! MTKMesh(
      mesh: mdlMesh,
      device: Renderer.device))
}

mdlMesh.addTangentBasis(
  forTextureCoordinateAttributeNamed:
    MDLVertexAttributeTextureCoordinate,
  tangentAttributeNamed: MDLVertexAttributeTangent,
  bitangentAttributeNamed: MDLVertexAttributeBitangent)

2. Send Tangent and Bitangent Values to the GPU

➤ Open Rendering.swift, and in render(encoder:uniforms:params:), locate for mesh in meshes.

for (index, vertexBuffer) in mesh.vertexBuffers.enumerated() {
  encoder.setVertexBuffer(
    vertexBuffer,
    offset: 0,
    index: index)
}
Normal calculations are wrong
Didyux nedpufowiojt ote vrewk

3. Convert Tangent and Bitangent Values to World Space

Just as you converted the model’s normals to world space, you need to convert the tangents and bitangents to world space in the vertex function.

float3 tangent [[attribute(Tangent)]];
float3 bitangent [[attribute(Bitangent)]];

float3 worldTangent;
float3 worldBitangent;

.worldTangent = uniforms.normalMatrix * in.tangent,
.worldBitangent = uniforms.normalMatrix * in.bitangent

4. Calculate the New Normal

Now that you have everything in place, it’ll be a simple matter to calculate the new normal.

normal = normal * 2 - 1;

normal = float3x3(
  in.worldTangent,
  in.worldBitangent,
  in.worldNormal) * normal;
The sphere with a normal map applied
Nva mndame qufk i fuxgaz wex ujpgaih

Other Texture Map Types

Normal and roughness maps are not the only way of changing a model’s surface. You can replace material values with any texture. For example, you could create an opacity map that describes transparent parts of the surface. Or a reflection map that builds in reflected objects.

Challenge

Your challenge is to download and render the toy drummer model from Apple’s AR Quick Look gallery.

Rendering a toy drummer
Kuvtoyujp u qup mdizzik

The final render
Qne pahet gegpab

Where to Go From Here?

Now that you’ve whet your appetite for physically based rendering, explore the fantastic links in references.markdown, which you’ll find in the resources folder for this chapter. Some of the links are highly mathematical, while others explain with gorgeous photo-like images.

10. Lighting Fundamentals
12. Render Passes
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2024 Kodeco Inc.

You're reading for free, with parts of this chapter shown as scrambled text. Unlock this book, and our entire catalogue of books and videos, with a Kodeco Personal Plan.

Unlock now