ARKit: The View, the Scene and the Session


Hello!

This post is for the emerging ARKit developer that wants to know how to navigate the ARKit + SceneKit stack. It introduces the classes ARSCNView, SCNScene and ARSession. A basic understanding of what they are is essential if you want to avoid repeatedly reinventing the wheel and it sets you up to learn best practices easier as your skill grows.

viewimage

The View

The ARSCNView is the owner of the SCNScene and the ARSession. All interaction with them must go through the ARSCNView. To understand what the ARSCNView is one should read up on its superclasses. The ARSCNView is a subclass of SCNView which in turn is a subclass of UIView.

The View’s heritage

Grandfather UIView

The essential building block for presenting content on the screen for Iphones. If you want to show something to the user on the screen it must be put inside a UIView (or one of its subclasses below).

Father SCNView

As a subclass to the UIView it contains every method and property the UIView has and more. The most important addition is the .scene property. It is where you put a SCNScene object which contains 3D SceneKit content. In short, if you want to show SceneKit content to the user you must put it inside a SCNView.

.pointOfView is the second big property you need to know about in the SCNView. This contains the camera node which is the position of the user’s camera in the scene. If you wish to know the position of the user relative to other SCNNodes then ask this property for its position.

ARSCNView

Everything inside the SCNView exists inside the ARSCNView with some extensions. The most important extension that the ARSCNView introduces is an embedded ARSession object accessed through .session.

ARSCNViewDelegate

If you want to customize the synchronization between the SCNScene and the ARSession (both written about later in this post) with the ARSCNView then this delegate is what you’ll want to use. It provides you with many useful methods you can implement to start listening and react to changes in the SCNScene and/or in the ARSession. Adding ARSCNViewDelegate to our ViewController enables us to listen to event triggers from the SCNScene and the ARSession. Some examples are:

  • func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? - Everytime you add an ARAnchor to the ARSession this function is called. It creates a SCNNode that corresponds to the ARAnchor. Usually logic is added here to check which 3D model corresponds to the newly added ARAnchor, and then attaches it to the node. After the delegate method is called the resulting SCNNode is added to the SCNScene.
  • renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) - This method tells the ARSCNView that an ARAnchor has had its properties altered and a SCNNode in the scene has been updated to match this.
  • func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) - This is called exactly once every frame. Commonly used for real-time logic checks that alter the SCNScene instantly.

You’ll find all of its delegate methods here.

sceneimage

The Scene

This is a container for SCNNodes and some global properties that together form all you need to display a 3D scene. The virtual content of your ARKit application must be put inside a SCNScene if you want it displayed for the user.

SCNNodes

If you wish to place content in your scene then you must attach it to a SCNNode. As a programmer when interacting with SCNNodes you are typically either altering its transformation matrix or attaching an object to it.

The transformation matrix is a 4x4 matrix containing the following data:

  • The Position (3D vector)
  • The Rotation (4D vector)
  • The Scale (3D vector)

The three most common attachments you need to know about are:

  • .light - SCNLight. Light source that illuminates the scene.
  • .camera - SCNCamera. Possible point of view for the user.
  • .geometry - SCNGeometry. A 3D model.

Each attachment is optional.

The Root Node

All nodes are not equal. During initialization of a SCNScene a first SCNNode called the rootNode will be created. This is the mother of all future SCNNodes. If you wish to add a new SCNNode to the scene you must add it as a child to a preexisting SCNNode. The first SCNNode you add must be added as a child to the rootNode.

viewimage

The Session

This is the class that’s responsible for the augmented reality experience. It’s like the brain in the sense that it is constantly analyzing a wide range of inputs and tries to figure out what’s going on. The inputs are:

  • Motion sensing hardware (See the class Core Motion for details)
    • Accelerometer
    • Gyroscope
    • Pedometer
    • Magnetometer
    • Barometer
  • Images from the camera

An ARSession comes to life through calling its method .run(ARConfiguration). The ARConfiguration is a very important object and must be passed as an argument to the ARSession. It specifies how the ARSession tries to sync the virtual world with the real world.

ARWorldTrackingConfiguration - Arguably the most powerful ARConfiguration

If you want your application to focus on using the back-facing camera to track the position of the user’s device in the real world and you’re also interested in approximating the surfaces around you with planes and point clouds then this is the ARConfiguration you want.

ARFrame

ARSession.currentFrame contains an ARFrame with the last image processed by the ARSession and associated AR information. What is meant by AR information?

  • ARFrame.anchors - [ARAnchor] Positions of special interest for the ARSession. All the positions in the real world you wish to track extra carefully should be marked with anchors, and SCNNodes should be created with the delegate method mentioned earlier func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode?
  • ARFrame.rawFeaturePoints - ARPointCloud The current point cloud generated by the ARSession through analyzing recent images and motion sensor data.
  • ARFrame.lightEstimate - ARLightEstimate An estimate of lighting conditions in the current image. Useful if your goal is to display virtual content that has realistic lighting.

And there you have it. A basic description of the main classes you’ll use when building an ARKit application with SceneKit.

Back to blog