OBD:SNDD: Difference between revisions

From OniGalore
Jump to navigation Jump to search
m (removing trailing slash from URL, just in case)
m (→‎.raw part (MS ADPCM, PC demo): minor correction. Heading towards a complete rewrite of this thing.)
Line 67: Line 67:
====.raw part (MS ADPCM, PC demo)====
====.raw part (MS ADPCM, PC demo)====
For PC demo the .raw SNDD data is actually the same as for PC retail, but with the same short .dat header as on Mac. The ADPCM block size is 512 bytes for mono, and 1024 for stereo. The sample rate is 22.05 kHz.
For PC demo the .raw SNDD data is actually the same as for PC retail, but with the same short .dat header as on Mac. The ADPCM block size is 512 bytes for mono, and 1024 for stereo. The sample rate is 22.05 kHz.
:(Note, however, that the "electric" SNDDs are sampled at 44.1 kHz in PC retail, with 1024 bytes per block even though it's mono. Therefore, for these sounds, the .raw data ''is'' different on PC retail and PC demo.)
====.raw part (IMA4 ADPCM, Mac)====
====.raw part (IMA4 ADPCM, Mac)====
For an overview of the IMA ADPCM algorithm and IMA4 header (if interested), see [https://wiki.multimedia.cx/index.php/Apple_QuickTime_IMA_ADPCM HERE]
For an overview of the IMA ADPCM algorithm and IMA4 header (if interested), see [https://wiki.multimedia.cx/index.php/Apple_QuickTime_IMA_ADPCM HERE]

Revision as of 13:22, 14 June 2021

ONI BINARY DATA
QTNA << Other file types >> StNA
SNDD : Sound Data
switch to XML:SNDD page
Overview @ Oni Stuff
OBD.png
For metadata instances used to group sounds together, randomize them, adjust their volume or frequency, etc, see OSBD and its subtypes: OSAm, OSIm and OSGr.

SNDD instances is where Oni stores sound data. Sounds can be either mono or stereo waveforms (with sampling frequencies of either 22.05 kHz or 44.1 kHz), and they are typically compressed to save on storage space. Both the PC and Mac versions use a form of ADPCM compression (Adaptive Differential Pulse-Code Modulation), where 16-bit sound samples are encoded as 4-bit "nibbles" (resulting roughly in a 4:1 compression ratio as compared to uncompressed 16-bit PCM).

  • On PC (both retail and demo), sounds are encoded using Microsoft's ADPCM algorithm described HERE. FFmpeg lists this codec as adpcm_ms.
  • On Mac, sounds are encoded using the IMA4 algorithm described HERE. FFmpeg lists this codec as adpcm_ima_qt.
Key shortcomings of the PC demo and Mac SNDDs as compared to PC retail SNDDs
  • PC demo and Mac SNDDs have a "short" .dat part that specifies only the frame count (animation length in game ticks) and number of channels (mono or stereo). The waveform data is always assumed to be sampled at 22.05 kHz, and compressed into 4-bit ADPCM (either IMA4 or MS ADPCM).
  • PC retail supports arbitrary sample rate, which allows for crisper high frequencies: specifically, 44.1 kHz (CD quality) is used for the 46 electric spark sounds ap_hit_shld and zap##. Possibly PC retail supports uncompressed PCM waveforms as well.
Key shortcomings of Mac SNDDs as compared to PC SNDDs (both retail and demo)
  • At 22.05 kHz, the storage size of Mac SNDDs (IMA4) is about 5% larger than for PC equivalents (MS ADPCM), because of a much smaller block size in the .raw part (the smaller .dat part of Mac SNDDs doesn't help).
  • Mac SNDDs have encoding/editing artifacts at the ends of looping segments (music and ambient tracks). PC SNDDs (retail or demo) have no such artifacts and allow nearly seamless playback. See BELOW for details.

Oni storage

PC retail

Below is the .dat file part used in the PC retail version.

Sndd all.gif


Offset Type Raw Hex Value Description
0x00 res_id 01 D7 08 00 2263 02263-comguy_dth2.aif.SNDD
0x04 lev_id 01 00 00 06 3 level 3
0x08 int32 08 00 00 00 8 flags
1 - (never used in Vanilla Oni)
2 - (never used in Vanilla Oni)
4 - (never used in Vanilla Oni)
8 - ADPCM compressed
0x0C block[50]     wav header (corresponds to the "fmt " section of a RIFF WAVE file)
0x3E int16 37 00 55 duration in 1/60 seconds (game ticks)
0x40 int32 56 28 00 00 10326 size of the part in the raw file in bytes
0x44 offset 20 10 59 00 0x591020 at this position starts the part in the raw file
0x48 char[24] AD DE dead 24 unused bytes (padding)

The .raw data contains the actual audio sample blocks without any other headers (other than ADPCM block headers).

.raw part (MS ADPCM)

For a detailed overview of the ADPCM algorithm (if interested), see HERE. For an actual implementation example, see, e.g., FFmpeg.

The MS ADPCM .raw data has 512- or 1024-byte blocks (512 bytes for 22.05 kHz mono, 1024 bytes for 22.05 kHz stereo and 44.1 kHz mono)
Each block consists of a 7- or 14-byte header (7 bytes for mono, 14 bytes for stereo), which includes the block's first two samples.
The remaining 505, 1010 or 1017 bytes of each block consist of nibbles (half-bytes), with left-right interleaving in the case of stereo.
(that's 1010 more samples in the case of 22.05 kHz mono or stereo, and 2034 more samples in the case of 44.1 kHz mono)
Thus the total number of samples per block (including the two in the header) is 1012 for 22.05 kHz (mono or stereo) and 2036 for 44.1 kHz mono.
The final block in the file can be incomplete (the decoder can infer this from the block size and raw data size).

PC demo and Mac

The Mac version and the PC demo version use a simpler format, with no support for different sample rates (all sounds are sampled at 22050 Hz).

Sndd alm.gif

Offset Type Raw Hex Value Description
0x00 res_id 01 D6 08 00 2262 02262-comguy_dth2.aif.SNDD
0x04 lev_id 01 00 00 06 3 level 3
0x08 int32 01 00 00 00 1 flags
1 - (unknown; always the same in Vanilla Oni)
2 - stereo (mono if disabled)
0x0C int32 37 00 00 00 55 duration in 1/60 seconds (game ticks)
0x10 int32 5E 2A 00 00 10846 size of the part in the raw file in bytes
0x14 offset 00 B1 01 00 0x1B100 at this position starts the part in the raw file
0x18 char[8] AD DE dead 8 unused bytes (padding)

The .raw data contains the actual audio sample blocks without any other headers (other than ADPCM block headers).

.raw part (MS ADPCM, PC demo)

For PC demo the .raw SNDD data is actually the same as for PC retail, but with the same short .dat header as on Mac. The ADPCM block size is 512 bytes for mono, and 1024 for stereo. The sample rate is 22.05 kHz.

(Note, however, that the "electric" SNDDs are sampled at 44.1 kHz in PC retail, with 1024 bytes per block even though it's mono. Therefore, for these sounds, the .raw data is different on PC retail and PC demo.)

.raw part (IMA4 ADPCM, Mac)

For an overview of the IMA ADPCM algorithm and IMA4 header (if interested), see HERE

The IMA4 ADPCM data has 34-byte blocks (in the case of stereo, there is an even number of such blocks, because Left and Right blocks are interleaved).
The first two bytes of each block are used to set the initial predictor (upper 9 bits) and step (lower 7 bits) for decoding the block's samples.
The other 32 bytes consist of 64 samples stored as nibbles (half-bytes). In the case of stereo, all the nibbles in a block belong to the same channel (either all Left or all Right).
Unlike for MS ADPCM, incomplete trailing blocks (if any) are not indicated in any way: the final blocks are stored in their entirety, with no way to tell how much of it is actual data.
For this reason, identical sounds do not have the same sample count on PC (both retail and demo) and on Mac. As an example, here are the stats for the main menu music:

By looking at the end of the Mac SNDDs (or the exported AIFF files), it can be confirmed that the extraneous samples are actually there, at the end of the last two 34-byte blocks (last Left block and last Right block), with no way to interrupt playback upon reaching these trailing samples - because they are no different from regular samples.

Also, from a careful examination of the sound stream that is actually played back by Oni in the main menu, it is clear that all the Oni engines (both Mac and PC) play back all the available data (including the "padding" of the fixed-size IMA4 blocks) before switching to the next segment. The frame count (number of game ticks) is ignored or used only as an indication (e.g., for approximate cueing in BSL).

This uninterrupted playback of fixed-size IMA4 blocks is one of the aspects that impact seamless playback of sound sequences in Mac Oni (music or ambient tracks). See "Looping Issues" below.

NOTE
Musically, the two segments of the main menu music correspond to the same duration (four bars of a 4:4 beat). However, somewhat suprisingly, the two segments don't have the same sample count - or even the same frame count (in game ticks) -, not even when comparing the two sounds on the same platform. That means that, even on PC where the playback is nearly seamless, we are actually hearing musical loops of unequal length, ending 10 milliseconds early or late, and it still sounds OK.

Exporting and importing tips

To create a wav/aif file one needs to write a file header like below and then write the contents of the raw data part.

WAV files (from PC retail/demo SNDDs)

  • Write "RIFF"
  • add the size of the part in the raw file + 70 bytes
  • write "WAVE"
  • write "fmt "
  • write 50
  • write the wav header (for PC demo, the wav header is not present in the .dat part, and has to be deduced)
  • OPTIONAL/RECOMMENDED: compute the number of samples and add a "fact" section announcing it
  • write "data"
  • add the size of the part in the raw file OPTIONAL/RECOMMENDED: increase the size if the last sample block is incomplete
  • add the raw file data OPTIONAL/RECOMMENDED: add padding to the last sample block if it is incomplete
  • save it as a wav file.

Sndd wav.gif

Offset Type Raw Hex Value Description
Complete ADPCM wav format header (black outline)
0x00 char[4] 52 49 46 46 RIFF identifier for the "IBM/Microsoft RIFF" standard
0x04 int32 9C 28 00 00 10396 size of the file from 0x08 to the end (= size of the .raw part + 70 bytes)
0x08 char[4] 57 41 56 45 WAVE identifier for the "WAVE" format
0x0C char[4] 66 6D 74 20 "fmt " identifier announcing the following wav format header
0x10 int32 32 00 00 00 50 wave format header size
0x14 block[50]     wav header
0x46 char[4] 64 61 74 61 data identifier announcing the following wav data
0x4A int32 56 28 00 00 10326 size of the following wav data in bytes (= size of the .raw part)

The above is not 100% consistent with the WAVE storage rules, because it allows for a completely arbitrary "data" size. Microsoft ADPCM data is supposed to be stored as a number of fixed-size blocks (in Oni, each block is either 512 bytes for 22.05 kHz mono, or 1024 bytes for 22.05 kHz stereo and 44.1 kHz mono). Thus, according to the standard, the last block - even if incomplete - must be stored in its entirety, and the "data" size must be a multiple of the block size. In the above example, since the format is 22.05 kHz mono, the "data" size should be increased from 10326 to 10752=21x512, and 426 empty bytes should be added as padding, so that there are 21 complete data blocks.

The standard way to deal with incomplete blocks is to specify not just the data size, but the actual number of samples, by adding a "fact" section to the WAVE header, like this:

Offset Type Raw Hex Value Description
Complete ADPCM wav format header
0x00 char[4] 52 49 46 46 RIFF identifier for the "IBM/Microsoft RIFF" standard
0x04 int32 9C 28 00 00 10396 size of the file from 0x08 to the end (= size of the .raw part + 70 bytes)
0x08 char[4] 57 41 56 45 WAVE identifier for the "WAVE" format
0x0C char[4] 66 6D 74 20 "fmt " identifier announcing the following wav format header section
0x10 int32 32 00 00 00 50 wave format header size
0x14 block[50]     wav header
0x46 char[4] 66 61 63 74 fact identifier announcing the following "fact" section
0x4A int32 04 00 00 00 4 size of the following "fact" section in bytes
0x4E int32 B0 4F 00 00 20400 actual number of samples (see below for calculation)
0x52 char[4] 64 61 74 61 data identifier announcing the following wav data
0x56 int32 00 2A 00 00 10752 size of the following wav data in bytes (= size of the .raw part + 426 empty bytes)

The actual number of samples is implied from the actual data size (size of the .raw part) and wav header properties as follows:

  • n_whole_blocks = floor(raw_size/block_size); // EXAMPLE: floor(10326/512) = 20
  • last_block_size = raw_size - whole_blocks*block_size; // EXAMPLE: 10326 - 20x512 = 86
  • last_block_samples = (last_block_size - 7*n_channels)*(8/bits_per_sample/n_channels) + 2; // EXAMPLE: (86 - 7)*(8/4) + 2 = 160
  • n_samples = n_whole_blocks*samples_per_block + last_block_samples; // EXAMPLE: 20*1012 + 160 = 20400

AIF files (from Mac SNDDs)

  • Write "FORM"
  • add the size of the part in the raw file + 50 bytes
  • write "AIFC"
  • write "COMM "
  • add the aif header (after filling in in, the number of channels, the sample rate - always 22.05 kHz -, the bits per sample - always 16 - and the number of sample frames/blocks)
  • write "SSND"
  • add the size of the part in the raw file + 8 bytes
  • add 8 zero bytes (custom "offset" and "block size" fields)
  • add the raw file data and save it as an aif file.

Note the Big Endian order

Sndm aif.gif

Offset Type Raw Hex Value Description
Complete aif format header (black outline)
0x00 char[4] 46 4F 52 4D FORM identifier for the "EA IFF 85" standard
0x04 int32 00 00 2A 90 10896 size of the file from 0x08 to the end (= size of the .raw part + 50 bytes)
0x08 char[4] 41 49 46 43 AIFC identifier for the "AIFC" format (compressed aif file)
0x0C char[4] 43 4F 4D 4D COMM identifier announcing the following aif format header
0x10 block[26]     aif header
0x2A char[4] 53 53 4E 44 SSND identifier announcing the following aif data
0x2E int32 00 00 2A 66 10854 size of the file from 0x32 to the end (= size of the .raw part + 8 bytes)
0x32 int32 00 00 00 00 0 offset; determines where the first sample in the data starts; use zero
0x36 int32 00 00 00 00 0 block size; used in conjunction with offset for block-aligning data; use zero


Looping issues

As detailed above, ADPCM data is stored in blocks, but the actual sound data does not necessarily end exactly at the end of a block. This is true both for MS ADPCM (PC retail or demo) and IMA4 ADPCM (Mac), but is especially noticeable for the comparatively large blocks of MS ADPCM, where the padding can be as large as ~1010 samples, i.e., a ~46-millisecond silence in the case of 22.05 kHz (for IMA4, the biggest possible gap is 63 samples, or ~3 milliseconds).

MS ADPCM

Although the final block of a MS ADPCM SNDD file (PC retail) is stored in incomplete form (with only the actual samples and no padding), the standard decoding behavior when loading an ADPCM-compressed WAV (e.g., in a non-destructive audio program) is to assume full-sized blocks, with padding up to the end of the last block. Depending on the audio program, this can create a silence or some "bad data" at the end of the imported audio, which can be a problem if one wants to join SNDDs that are supposed to play seamlessly one after another (e.g., a musical or ambient sequence).

As a workaround, one can preprocess .wav files with some tools that can handle incomplete MS ADPCM blocks and convert to a less ambiguous format:

  • For Sox, padding is disabled by default when joining several files.
  • For FFmpeg, padding can be disabled as an optional setting.

So, you'd either join the .wav files in Sox or FFmpeg, or convert them, e.g., to uncompressed PCM, and then import them into a fancy audio tool.

As an actual solution, the .wav file should be made compliant with RIFF WAVE standards, i.e., the last block should be padded to its full size, and a "fact" section should be used to specify the actual number of samples. This is implemented in OniSplit v 0.9.###

Slight distorsions are sometimes observed near the ends of looping SNDDs (music and ambient tracks). These artifacts were likely caused by Bungie's audio tools, and can not be undone automatically. Barely noticeable, they can be healed by manually editing audio samples near the seams.


IMA ADPCM

Padding

In the case of IMA ADPCM, the padding is actually present in the stored audio, so it is impossible (both for OniSplit and for a third-party converter) to automatically trim it down to just the relevant audio data. In fact, just by looking at the Mac SNDD itself, there is no way to tell how many of the trailing samples need to be cut for a truly seamless transition (for one thing, the trailing samples are not flat zero).

As a workaround/solution, the correct sample count of a Mac SNDD can be looked up in a PC counterpart (always available, since we're only talking of music/ambients/sirens, which are neither localized nor sampled at 44.1 kHz), and then used to trim the .aif file in FFmpeg, while converting to .wav (either PCM or ADPCM). However, it's easier (and more reliable) to just grab a PC retail copy of Oni and extract the MS ADPCM sounds.


Initial transient

The biggest problem with seamless playback of Mac SNDDs (for music and ambient tracks) is that - even if you figure out the correct length of each segment - the waveform of each next segment does not pick up where the previous segment left (or should have left) - instead it builds up from zero over ~7 samples. This introduces about 0.3 milliseconds of silence, and an audible discontinuity in the waveform, even if the two segments are lined up properly.

The values of those initial samples is not recoverable (unlike the padding at the end of SNDDs, which can be trimmed down). Therefore, if working with sound samples extracted from Oni, it is recommended to turn to a PC version's SNDDs.


PCM export and PC demo detection

OniSplit v0.9.### implements export to uncompressed PCM (signed 16-bit linear) from both the PC retail and the Mac SNDD format: use -extract:pcm instead of either -extract:wav or -extract:aif. As compared to ADPCM, linear PCM is a much more straightforward format (almost human readable), and makes it easier to analyze artifacts.

Since the only difference between PC demo and Mac is the actual storage format of SNDD files (in the .raw part), and the template checksum is the same, OniSplit has no way of determining which ADPCM algorithm to use, other than by actually scanning and validating the data as IMA4 (or not). Starting with OniSplit v0.9.###, this automatic check is implemented, allowing both -extract:wav and -extract:pcm on PC-demo SNDDs. It is very unlikely that any Mac SNDDs will be falsely identified as MS ADPCM (or, rather, invalidated as IMA4). If it ever happens, do as instructed by the following warning: "PC-demo MS ADPCM detected; use -nodemo flag to treat as IMA4."

Note that transcoding (between IMA4 and MS ADPCM) and encoding is not implemented at this point. So -extract:aif will not work on PC SNDDs, -extract:wav will not work on Mac SNDDs, and -create will only work on sound files that use the correct codec and sample rate supported by the PC retail/demo or Mac Oni engine.

Importing SNDDs for PC demo (with a short .dat part and MS ADPCM in the .raw part) is also not implemented yet. Last but not least, PC retail apparently supports uncompressed PCM sounds, but they need to be tested in the engine first. Possibly ADPCM encoding/transcoding will be implemented at some point, too.



ONI BINARY DATA
QTNA << Other file types >> StNA
SNDD : Sound Data
Generic file