OBD:Raw and separate file formats

Unless mentioned otherwise, this is about the PC retail .raw files.

When mentioned otherwise, this can be about the .sep files as well.

Chunk offsets

This applies for .sep files as well

Alignment

Although this may not be strictly required by the engine, the offsets of all original chunks start at 32-byte multiples.

This implies a slight loss of space, but the storage is somewhat tidier that way. Easier to manipulate in hex editors ^^

OUP will probably continue to store data that way. Storing things the way Bungie did is a way to avoid surprises.

Null offset

At the start of the .raw or .sep file, there are 32-bytes of garbage

That way, all meaningful raw/sep chunks start at offsets strictly greater than 0.

If that wasn't the case, the null value of a raw/sep link would be ambiguous.

---

Differences across versions

Different content

Language versions

In this case the main difference in the .raw is due to the SNDD files (and SUBT for level0).

Extra level0 textures

The level0_Final.raw of older versions stores a few megabytes worth of "obsolete" textures.

Texture compression

Depending on the version, the amount of textures stored in a lossy compressed format can be "most" to "almost none".

This leads to dramatic variations in size for the .raw (over 10% of the total amount of game content).

Mac vs. Windows

.dat files

These files are mostly identical between Mac and Windows, but Mac SNDD resources are 32 bytes long, whereas Windows and Windows demo SNDD resources (wrappers for the actual sound data in the .raw files) are 96 bytes long; this is because of the change in SNDD format noted below under ".raw files".

.raw files

In Windows, these files contain the core data of the AGDB, AKVA, BINA, OSBD, SNDD, SUBT, TRAM and TXMP resources.

On Mac and in the Windows demo, these files only contain the core data of the AKVA, SNDD, SUBT and TRAM resources; the rest have been moved to the .sep files (except for AGDB resources, which were omitted from Macs and the Windows demo because this data is unused by Oni).

In terms of differences in the actual data formats, the most notable change is that Macs use SNDDs in the AIFF format, and Windows (retail and demo) uses WAV sound data.

.sep files

Windows retail Oni lacks these files, all the relevant data being in the .raw files. On Macs and in the Windows demo, .sep files contain the BINA, OSBD and TXMP resources. For TXMPs in particular, only the "wrapper" for the texture data is stored in a Mac's .raw file, with a link to the actual image data where it is found in the level's .sep file.

---

Gaps

The original .raw are not packed as a compact sequence of 32-byte-aligned chunks.

There are gaps of various sizes in every level, totalling up 25 megabytes for the whole game.

Interestingly, those gaps seem to be the same across version, even if other content differs.

The content of those gaps is detailed below.

Skybox mipmaps

Those gaps occur in every level after each of the 5 skybox textures. They're filled with zeroes.

Their size matches that required to store MIP map generations of the preceding TXMP chunk.

Orphaned chunks

Those seem to be mostly .raw parts of existing resources, orphaned at some point by the appending of a new version to the .raw.

The pointer in the .dat got redirected to the new version, and the old version was lost.

So, either duplicates of existing resources, or obsolete versions reflecting development.

OSBD

A few of those in level 0. Reauthored duplicates of existing OSAm/OSGr pairs: for every pair, there are (minor) changes either in the OSAm or in the OSGr

SNDD

Those make the most of the gap space. Some of the sounds were actually reauthored (higher sample rates, shorter durations...)

Most of them, however, are duplicates, apparently due to a bug in Bungie's packing system.

Adding an overview later...