Audiokinetic Wwise Knowledge Base

The internal workings of the interactive music engine are fairly complicated. Here are few concepts that will help you understand how it behaves.

Active Segment

The first thing you need to understand is that there is one and only one segment that is truly active at a time. Active segments limits are defined by the synchronization points.

In playlist containers, synchronization points are always defined by the entry and exit cues of each segments, because transitions always occur from the exit cue of the source segment to the entry cue of the destination segment. Thus, if the current playing position is between the entry and exit cues of segment B, segment B is the active segment, even though the post-exit region of segment A is still playing.

A playlist can be represented as a chain of segments. When a switch container schedules a transition, it actually links a segment of a chain to the first segment of another chain. From the point of view of a switch container, the effective chain results from the connection of pieces of chains borrowed from its child playlists. There is still only one active segment at a time in a switch container's effective chain.

The active segment during the pre-entry of the first segment of a playlist is <nothing>. Likewise, <nothing> is the active segment during the post-exit of the last segment of a playlist, and beyond.

Scheduling of stingers and state changes is always based on to the effective chain of the top-level music object, that is, the music object which was the target of the Play action, triggered by the game.

Music switch transitions are based on the effective chain of the switch container that performs a transition. The transition rule is selected exclusively according to the active and the destination segment. If the transition cannot occur in the active segment, then the container attempts to schedule it from the next segment of the effective chain.

Playlist Stamping

The engine is intrinsically interactive, thus scheduling of segments is not determined a long time before they are actually played. When a playlist container is played, its playlist is queried (possibly selecting segments randomly, depending on the type of playlist group) to get only the first few segments that should play, and these segments are then scheduled in time. The process of querying segments from the playlist definition and scheduling them is called “playlist stamping”. Playlists are stamped little by little, not too much because scheduling could change at any moment, following a state change for example, but enough so that music transitions may be scheduled correctly.

Streaming Look-Ahead Time

Segments may contain audio clips that are streamed from disk, and thus involving some latency before they are heard. This latency would be unacceptable if it wasn’t taken into consideration when performing a musical transition, because segments must be playing in perfect synchronicity. To solve this issue, users of Wwise can specify a time value when marking a music object as streaming: the “look-ahead time” (in the Track's Property Explorer view). This value depends on disk bandwidth and required throughput, so it is left to users to experiment.

Before a music object can be scheduled correctly, the engine must query the look-ahead time of all the tracks of the first segment that was stamped, and pick the worst case in order to determine the global look-ahead time needed before the segment can start playing. After the segment is scheduled, low-level commands to open and read wave files are executed in order to prepare the audio data.

Refer to Interactive Music Streaming Look-Ahead Time and Prefetching for more details on streaming look-ahead time.

Computing a Music Transition

Now you know everything you need to understand how music transitions are scheduled.

Say you have a playlist Pa playing under a switch container, and the active segment is Sa1. The state group changes to state B, to which is associated playlist Pb.

Here is a description of what happens:

1. Revert previous pending transitions if possible (more about this in Transition Reversals below). After reverting transitions, the source segment is either the current segment (Sa1), or the first segment of the latest transition that was not reverted. Let's call it Sat.
2. Determine if a transition must be scheduled (read section "Continue to play on Switch change" Option). Bail out if it doesn't.
3. Instantiate and stamp playlist Pb. Obtain the first segment of the playlist, Sb1.
4. Select the appropriate rule: walk through the transition list in reverse order, and select the first rule that matches segments Sa1 and Sb1. We must take into account any rule that specify one of their ascendants (for example, a rule which destination object is playlist Pb would match).
5. If the rule specifies a “Jump To Playlist” destination behavior, flush everything that was stamped from playlist Pb and re-stamp it, forcing it to start at the required item.
6. Find the synchronization point:
• Compute the time that needs to elapse before being able to play the destination. This may include the length of the pre-entry, the streaming look-ahead time, and the fade in time before the synchronization point.
• b. Compute the time constraint on the source’s side. This may include the fade out time before the synchronization point.
• c. Find the first synchronization point that satisfies the transition rule's “Exit Source At” property, as well as the time constraints of the source and destination contexts.
7. If the synchronization point exists within segment SaT, the transition is scheduled. Otherwise, playlist Pa is dynamically stamped a little bit more, and we jump to step 3, starting with the following segment.

An important consequence on transitions behavior can be deduced from the algorithm described above: the switch container must take into account all timing constraints related to the transition rule, destination and source segments. For example, if you select "Next Beat" in the "Exit Source At" drop menu, the synchronization point will not necessarily be placed at "the" next beat, but instead at the first beat that satisfies all the timing constraints. At 120 BPM (a beat lasts 0.5 seconds), with a rule which specifies a fade-out of 5 seconds, synchronization will occur in more or less 11 beats (that is, at the first beat boundary that is found after 5 seconds of playback).

When a music switch container starts playing (via an event's Play action), a transition is executed between <nothing> and the first stamped segment of the container's child associated with the current switch/state.

Handling of Delayed State Changes

Keep in mind that States are global to a project. That is, a state group is always set to one and only one state at any given time, and affects all objects that are registered to it. The "Change occurs at" feature is in fact a way to delay the time at which the state will be changed. Here's the simplified algorithm of state changes:

1. The sound engine asks the music engine if the state change can be applied now or if it needs to be delayed.
1. The music engine queries all its top-level playing instances.
1. The playing instance finds its active segment.
2. The active segment's node is queried to see if it is registered to this state group, and if it is, if it requires a delayed state change (any option other than "Immediate"). The whole tree is searched: all the segment's ascendents, its child tracks, and all their output busses.
3. The possible outcomes are:
• a) No node is registered to the state group.
• b) At least one node is registered to the state group with "Immediate" property.
• c) At least one node is registered to the state group, but all nodes require a state change delay (no one has the "Immediate" property).
2. The music engine takes a decision:
• If at least one of the playing instances returned (b), state change should occur now.
• Otherwise if at least one of the playing instances returned (c), state change will be delayed, by a value corresponding to the smallest delay returned by all nodes search.
• Otherwise the state change should occur now.
2. If the delay is greater than one audio frame, the state change is delayed: the music engine takes control of the state change request, enqueues it and releases it later.

Notes:

• Note 1: State changes in the sound engine are not sample accurate, they always occur on an audio frame boundary. All the properties (volume, pitch, LPF) are always computed just before going through all the voices and moving data down to the audio driver.
• Note 2: In fact, the real algorithm is more complicated, because we need to handle the case where the "Next XXX" sync point cannot be found in the active segment. This occurs for example if the policy is "Next Bar" and we are currently passed bar 3 of a segment that has 3.5 bars between its Entry and Exit Cues. In such a case, the "next" segment is queried, if applicable. The "next segment" is the segment that is currently scheduled (stamped) to play next. This takes into account any segment change that could have been already scheduled by a music switch container transition. If a transition is reverted, or a new one is scheduled before the state change is applied, then it is rescheduled, but only in the "next" segment. If the sync point cannot be found anymore, then the change occurs at the boundary with the "second next" segment.
• Note 3: State change logic is not trivial. If you need to register a switch container to a game sync and you hesitate between a state group or a switch group, because you do not use the "Change occurs at" feature, then you should definitely choose to use a switch group instead, in order to avoid all these unnecessary computations.

Scheduling Stingers

The following logic applies when a Trigger is posted:

1. The active segment is queried to determine if it needs to respond to this trigger by launching a stinger.
• If it does not define an association, its ascendents are queried until one does.
2. If a stinger must be played, the engine tries to schedule it according to the "Play At" policy, taking the segment's pre-entry and look-ahead time into account.
• It is possible that the stinger cannot be scheduled, for example if playback of the current segment is too close from its end and the timing constraints of the stinger do not allow it, if a "Next Custom Cue" policy is selected and no custom cue could be found after the cursor, or simply if an "Entry Cue" policy is selected.
• If it cannot be scheduled, the outcome depends on the "Allow playing Stinger in next segment". If it is not checked, the stinger is not played. If it is checked, then we move to the next segment that was stamped in the scheduler and start this process again at step (1).

Notes:

• Note 1: Stinger scheduling is based on the active segment. Since an active segment can never be in its pre-entry region by definition, the "Entry Cue" Play At policy can only be scheduled in the next segment. If "Entry Cue" is selected but "Allow playing Stinger in next segment" is not, the stinger will never be played back.
• Note 2: Stingers that should play at "Exit Cue" are actually heard while the next segment is playing. Nevertheless, they are associated with the current segment.
• Note 3: <Nothing> segments (that is, empty state-child switch container associations, or end of playlists or segments) don't have meter info. Only "Immediate" stingers can play during <nothing>.
• Note 4: If a stinger was scheduled to play at some time, but a switch change occurs in between and kicks the stinger out of the active segment chain, the stinger is re-scheduled if and only if the "Allow playing Stinger in next segment" was selected. If it wasn't, the stinger is dropped. A stinger can only be scheduled in the next segment at most. Thus it can only be re-scheduled once, if it was scheduled in the current segment in the first place. This mechanism prevents the situation where a stinger would be re-scheduled over and over again because of switch changes. We thought it was preferable to avoid playing a stinger than to play one out of context.

You may limit stinger occurrence with the "Don't play this stinger again for X seconds". Note that this timer starts at the stinger's synchronization point (entry cue), and is exclusive. Say you have a stinger with a "Don't play again" property of 0 ms. From the moment the associated trigger T is posted until the stinger plays its entry cue, all subsequent triggers T are dropped. Consequently, it is impossible to have two identical stingers overlapping. Note that you may specify a different value for this property for any segment. The value that is used is the value that was specified in the segment (or its ascendents) over which the stinger was attached.