Basic application structure

This chapter introduces the minimum structure of an application built with the bg2 engine TypeScript API. The goal of this chapter is not to start rendering scenes, but to understand how a graphics application is organized, how the main loop works, and how input events are handled.

All course code is available on GitHub. Clone the repository and checkout the introduction_to_bg2e_typescript branch:

https://github.com/ferserc1/bg2e_learn

The TypeScript and C++ APIs of bg2 engine are open source and distributed under the GPLv3 license. The bg2-io library, which provides tools for 3D file format input/output, is distributed under the MIT license.

TypeScript/JavaScript: bg2 engine - TypeScript
C++/Vulkan: bg2 engine - native
3D file format input/output: bg2 input/output tools (MIT license)

1. Canvas DOM

In any graphics application web, regardless of the rendering backend (2D, WebGL, WebGL 2, WebGPU), we need a rendering surface or canvas. For this we use a <canvas> element. The first step in any application is to create the canvas. This part will be done in the HTML code:

...
<body>
    <canvas id="gl-canvas" width="800" height="600"></canvas>
    <script type="module" src="/src/main.ts"></script>
</body>

In this case, we have marked the canvas with an id attribute so we can obtain a reference in the TypeScript code. We need this reference for the graphics engine to configure the canvas to be used with the desired graphics backend, for example, WebGL.

Note: we can include a TypeScript file because we are using Vite. If we want to create a bg2 engine application directly for the browser without compilation, we need to use the JavaScript API.

2. Engine imports and modules

bg2 engine separates the classes and functions it provides into different packages. In this example, we are going to use elements from two packages:

The application core (app): lifecycle, events, main loop.
The rendering system (render): concrete graphics backends and rendering managers.

Although in this example we do not render a scene, we do need a renderer to be able to draw on the canvas.

import MainLoop, { FrameUpdate } from "bg2e-js/ts/app/MainLoop.js";
import Canvas from "bg2e-js/ts/app/Canvas.js";
import AppController from "bg2e-js/ts/app/AppController.js";
import WebGLRenderer from "bg2e-js/ts/render/webgl/Renderer.js";

Canvas: abstraction that encapsulates the HTMLCanvasElement and its graphics context.
WebGLRenderer: implementation of the graphics backend for web type WebGL.
AppController: base class where the application logic is defined.
MainLoop: coordinates events, time updates, and rendering.

bg2 engine provides its own typed events, independent of the DOM. Native browser events could be used, but instead three new event types adapted to the usage for the graphics engine are defined.

import Bg2KeyboardEvent from "bg2e-js/ts/app/Bg2KeyboardEvent.ts";
import Bg2MouseEvent from "bg2e-js/ts/app/Bg2MouseEvent.ts";
import Bg2TouchEvent from "bg2e-js/ts/app/Bg2TouchEvent.ts";

3. AppController: the application entry point

class MyAppController extends AppController { ... }

In graphics applications there is no classic sequential flow (run code and finish). The application stays alive and the system invokes callbacks when events occur:

user input,
time updates,
redrawing,
size changes.

The AppController is where this behavior is defined.

3.1. init(): initialization

async init() {
    console.log("Init");
}

It executes only once when the application starts.
It is used to initialize state, load resources, or configure the app.
It is asynchronous because in this function we will execute asynchronous actions, such as loading a scene from a URL.

3.2. reshape(): window size management

reshape(width: number, height: number) {
    console.log(`reshape - width:${width}, height:${height}`);
    const { gl } = this.renderer as WebGLRenderer;
    gl.viewport(0, 0, width, height);
}

In graphics, the size of the drawing surface directly affects the visual result.

gl.viewport defines which region of the framebuffer is used for rasterization.
It is called when the canvas size changes, when the window is resized, or when a mobile device is rotated.

3.3. frame(delta): frame update

async frame(delta: number) {
    console.log(`frame - elapsed time: ${ delta }`);
}

This function is used to execute code that needs to be synchronized with time, such as an animation.

delta represents the elapsed time since the last update.
It is the basis for animations, interpolations, and frame-rate independent movement.
Conceptually, here the application state is updated, not drawn. For example, in this function we do things like update the position of an element, move a camera, or change the color of a light.

3.4. display(): rendering

display() {
    const { gl } = this.renderer as WebGLRenderer;
    gl.clearColor(0, 0, 0, 1);
    gl.clear(gl.DEPTH_BUFFER_BIT | gl.COLOR_BUFFER_BIT);
    console.log("display");
}

The separation between update (frame) and render (display) is fundamental in real-time graphics. The display() function is exclusively responsible for drawing the scene. The code executed in this function depends on the rendering system being used and the backend (WebGL, WebGPU, etc.), while the code executed in the frame() function is the same regardless of the rendering system being used.

The display() function is called after frame(), and generally it is called continuously, as many times as possible, to obtain smooth animations and interactions.

In this example, the only thing we are doing is clearing the frame, i.e., preparing it to draw a frame.

3.5. Input event handling

keyDown(evt: Bg2KeyboardEvent) {
    console.log(`keyDown - key: ${evt.key}`);
}

keyUp(evt: Bg2KeyboardEvent) {
    console.log(`keyUp - key: ${evt.key}`);
}

mouseUp(evt: Bg2MouseEvent) {
    console.log(`mouseUp - mouse location: ${evt.x}, ${evt.y}`);
}

mouseDown(evt: Bg2MouseEvent) {
    console.log(`mouseDown - mouse location: ${evt.x}, ${evt.y}`);
}

mouseMove(evt: Bg2MouseEvent) {
    console.log(`mouseMove - mouse location: ${evt.x}, ${evt.y}`);
}

mouseOut(evt: Bg2MouseEvent) {
    console.log(`mouseOut - mouse location: ${evt.x}, ${evt.y}`);
}

mouseDrag(evt: Bg2MouseEvent) {
    console.log(`mouseDrag - mouse location: ${evt.x}, ${evt.y}`);
    this.mainLoop.postRedisplay();
}

mouseWheel(evt: Bg2MouseEvent) {
    console.log(`mouseWheel - mouse location: ${evt.x}, ${evt.y}, delta: ${ evt.delta }`);
    evt.stopPropagation();
}

touchStart(evt: Bg2TouchEvent) {
    console.log(`touchStart`);
}

touchMove(evt: Bg2TouchEvent) {
    console.log(`touchMove`);
}

touchEnd(evt: Bg2TouchEvent) {
    console.log(`touchEnd`);
}

Input events:

are not tied to rendering,
represent discrete user actions,

evt.stopPropagation() allows stopping the event propagation; in this specific case it is used to prevent the mouse wheel from scrolling the page.

4. On-demand redrawing

this.mainLoop.postRedisplay();

bg2 engine supports on-demand rendering. Generally, most graphics engines draw the scene continuously while the application is running. This simplifies architecture a lot when we have applications that will generate animations in the scene, but it is very inefficient if the scene is going to remain static. On-demand rendering consists of updating the scene only when explicitly requested.

If the application is configured with on-demand rendering, we must manually call the postRedisplay() function of the main loop so the scene is updated. In this case, we call postRedisplay() when any user event is generated.

The postRedisplay() function accepts parameters to configure the number of frames we want to add to the rendering queue, and a timeout to delay the first redisplay:

postRedisplay({
  frames = 10,
  timeout = 10 }
: { frames?: number, timeout?: number } = {}): void

5. MainLoop

The MainLoop is the core of a graphics application:

centralizes event management,
controls when frame() and display() are called,
manages time (delta),
applies synchronization and frame-lock.

In bg2 engine, the main loop is also where the application is configured: we need to indicate things like the desired rendering backend, which canvas of the DOM tree we want to draw the scene on, or which application controller will manage events.

To be able to configure the main loop, we need to obtain a reference to the HTML canvas. We will ensure that the canvas is already loaded and present on the page by adding the application initialization code in the window onload function.

window.onload = async () => {
    const canvasElem = document.getElementById('gl-canvas') as HTMLCanvasElement;
    if (!canvasElem) {
        console.error("Cannot find canvas element with id 'gl-canvas'");
        return;
    }
    ...
}

To connect the graphics engine with the HTML canvas, we use the Canvas class from the app package. This class receives as parameter the HTMLCanvasElement and the renderer of the desired graphics backend

window.onload = async () => {
    const canvasElem = document.getElementById('gl-canvas') as HTMLCanvasElement;
    if (!canvasElem) {
        console.error("Cannot find canvas element with id 'gl-canvas'");
        return;
    }
    const canvas = new Canvas(canvasElem, new WebGLRenderer());
    ...
}

To create the main loop we need the canvas we have created, and the instance of the application controller. Once we have created the main loop, we start it with the run() function, which is asynchronous, and it will keep running indefinitely.

window.onload = async () => {
    const canvasElem = document.getElementById('gl-canvas') as HTMLCanvasElement;
    if (!canvasElem) {
        console.error("Cannot find canvas element with id 'gl-canvas'");
        return;
    }
    const canvas = new Canvas(canvasElem, new WebGLRenderer());
    const appController = new MyAppController();
    const mainLoop = new MainLoop(canvas, appController);
    await mainLoop.run();
}

5.1. Frame lock and display synchronization

Displays have a fixed refresh rate (60 Hz, 120 Hz, 144 Hz, etc.).

If rendering is not limited or synchronized and we can potentially generate frames at a higher frequency than the display:

many frames never make it to screen,
CPU and GPU are wasted,
tearing can appear, where different parts of the screen belong to different frames,
power consumption increases without visual benefit.

Frame lock attempts to align rendering with the display refresh or with a reasonable limit, improving:

visual stability,
efficiency,
predictability of the temporal delta.

By default, the WebGL backend has frame lock enabled, but this is a web browser feature. In principle this cannot be configured from JavaScript/TypeScript code, but the browser might have some hidden configuration option. If frame lock is not working, check the browser settings.

5.2. FrameUpdate.AUTO vs FrameUpdate.MANUAL

mainLoop.updateMode = FrameUpdate.MANUAL;

The main loop’s updateMode attribute is what we will use to configure on-demand rendering. bg2 engine supports two update modes:

`FrameUpdate.AUTO`

Continuous rendering.
Classic graphics engine behavior.
Suitable for scenes with constant animations.

`FrameUpdate.MANUAL`

On-demand rendering.
To trigger a canvas redraw we must explicitly call postRedisplay().
Ideal for productivity-oriented applications and mobile devices, as it reduces CPU, GPU, and battery consumption.

To configure on-demand rendering mode you only need to set the value of the updateMode property. This can be done at any time. It is generally configured before running the main loop’s run() function, but the rendering mode can be modified at any time.

window.onload = async () => {
    ...
    mainLoop.updateMode = FrameUpdate.MANUAL;
    await mainLoop.run();
}