OBD:Oni2AS
The Oni2 dev disc from Angel Studios contains:
- the main ELF executable SCPS_123.45
- a plain-text configuration file SYSTEM.CNF
- a SYSTEM folder containing fifteen ELF binaries (*.IRX) and a file called IOPRP255.IMG
- the three "binary data" files BANKS.DAT, RB.DAT and STREAMS.DAT
DAT file format (file table)
The common format of the three "binary data" files is as follows (illustrated on the example of RB.DAT).
| Offset | Type | Raw Hex | Value | Description |
|---|---|---|---|---|
| 0x00 | 4CC | 44 41 56 45 | "DAVE" | a control code |
| 0x04 | int32 | 18 41 00 00 | 16664 | number of file table entries (files or folders) |
| 0x08 | offset | 00 18 04 00 | 0x41800 | offset to start of file name block, from 0x800, in bytes |
| 0x0C | offset | 00 30 0A 00 | 0xA3000 | offset to start of file data block, from 0x800, in bytes |
| 0x10 | char[2032] | 00 ... | 0 | padding |
| File table, first entry | ||||
| 0x800 | offset | 00 00 00 00 | 0x00 | offset to file name, from start of file name block, in bytes
|
| 0x804 | offset | BC D9 1F 02 | 0x21FD9BC | absolute offset to file data, from start of DAT file, in bytes |
| 0x808 | int32 | 00 08 00 00 | 2048 | uncompressed file size, in bytes |
| 0x80C | int32 | 94 00 00 00 | 148 | compressed file size, in bytes |
| File table, second entry | ||||
| 0x810 | offset | 0B 00 00 00 | 0x0B | offset to file name, from start of file name block, in bytes
|
| 0x814 | offset | 22 D3 1D 02 | 0x21DD322 | absolute offset to file data, from start of DAT file, in bytes
|
| 0x818 | int32 | 00 00 00 00 | 0 | uncompressed file size (zero; it's a folder) |
| 0x81C | int32 | 00 00 00 00 | 0 | compressed file size (also zero) |
| File table, third entry | ||||
| 0x820 | offset | 12 00 00 00 | 0x12 | offset to file name, from start of file name block, in bytes
|
| 0x824 | offset | 22 D3 1D 02 | 0x21DD322 | absolute offset to file data, from start of DAT file, in bytes
|
| 0x828 | int32 | 00 00 00 00 | 0 | uncompressed file size (zero; it's a folder) |
| 0x82C | int32 | 00 00 00 00 | 0 | compressed file size (also zero) |
| File table, fourth entry | ||||
| 0x830 | offset | 1F 00 00 00 | 0x1F | offset to file name, from start of file name block, in bytes
|
| 0x834 | offset | 00 00 53 06 | 0x6530000 | absolute offset to file data, from start of DAT file, in bytes |
| 0x838 | int32 | 90 09 03 00 | 199056 | uncompressed file size, in bytes |
| 0x83C | int32 | 13 85 02 00 | 165139 | compressed file size, in bytes |
| ...etc (rest of the file table) | ||||
The file table format is apparently a PS2 standard, and there are free tools like Game Extractor (basic version) that can display the folder structure and file sizes, and extract uncompressed files. However, while Game Extractor correctly identifies the compressed and uncompressed file sizes, it apparently does not know what the compression algorithm is (at least in the free version, the compressed files are extracted as-is, and are thus unusable). Only very short files are packed without compression (mostly trivial-looking stub files of little interest).
After some forum lurking, it becomes clear that the compression algorithm is ZIP, or rather "deflate", in its most common implementation (open-source ZLIB library). However, the compressed files have no ZIP header, so the decompressor/compressor must use the right default settings for this to work. The correct setup is as follows:
z_stream infstream; // initialization of the "inflate" stream
infstream.zalloc = Z_NULL;
infstream.zfree = Z_NULL;
infstream.opaque = Z_NULL;
infstream.avail_in = (uInt)(size_comp); // size of compressed input
infstream.next_in = (Bytef *)file_comp; // input file (char array)
infstream.avail_out = (uInt)(size_orig); // size of decompressed output
infstream.next_out = (Bytef *)file_orig; // output file (char array)
// the actual DE-compression work.
inflateInit2(&infstream,-15);
inflate(&infstream, Z_NO_FLUSH);
inflateEnd(&infstream);
The important line, which defines the decompression rules, is inflateInit2(&infstream,-15). -15 means maximum compression and no header. For more details, c.f. the ZLIB documentation.
With proper decompression, the DAT files extract as follows: BANKS.DAT RB.DAT STREAMS.DAT
- STREAMS.DAT holds 22 *.stm files (21 vocalization sounds and 1 musical piece). See below for the STM format and a link to the converted sound files.
- BANKS.DAT holds 13 sound sets, each of which is a pair of a *.hd file (similar to Oni's OSGr?) and a corresponding *.bd file (similar to a group of Oni's SNDD bundled together). The *.bd files in BANKS.DAT are identical to the ones stored in RB.DAT (in the Audio/banks/ folder), and possibly redundant. The *.hd files are not duplicated in RB.DAT, which instead has plain-text *.td files (apparently all three file types are supposed to work together, but perhaps *.hd is just a binary version of *.td with some additional features)
- RB.DAT holds all the rest of the game data: textures, levels, characters, animations, scripts, etc.
Commonplace file formats
Some of the files in RB.DAT are either in plain text or in otherwise widely known formats that can be opened by third-party programs. Here is the list of such commonplace formats used in Oni2.
- *.txt : Text files. Some in Windows format (CR+LF at EOL), some in Unix format (only LF at EOL)
- *.mb : binary file used by Maya 3D program (work-in-progress models of some characters and props)
- *.wav : wp:WAV. a common format for the uncompressed storage of sounds (through pulse-code modulation)
- *.tga : wp:Truevision_TGA. a versatile texture format that supports RLE compression, MIP map storage, indexed as well as RGB color schemes, transparency (and therefore is widely used in games).
- *.xml : wp:XML.
- *.cue : wp:Cue_sheet_(computing). Occurs only once, as rb.cue at the root of RB.DAT
- *.m2v : wp:MPEG-2. Video files. Occur in the form of movies/angel.m2v and movies/rockstarlogo.m2v
- ...
Plain text file formats
Apart from XML files (and some TXT files; mostly debug logs), Oni2 has many custom plain-text files, listed below.
Save files - occurs only in the form of sample.oni2save at the root of RB.DAT
Template files - occurs only in the form of template/rb.template and a couple of layout.template files (see Layout files below)
Layout files - files that specify the layout of a level. They're called names like layout.et (entities), layout.graphs, layout.lights, layout.paths
Script files - files ending in .oni, rather similar to BSL. Can be compiled into binary *.onc files.
Model files - files ending in .mod or .xmod. Version 1 format is plain-text, version 2 is binary (see below for both).
Pathfinding files - files ending in .room, .bsp or .mesh (collection of interconnected pathfinding meshes)
Fighting state machines - files ending in .fsm, found in Statemachine/
Attack patterns - files ending in .atk, found in Statemachine/ and layout/fightai_grapple/statemachine_backup/
...
STM file format (sound stream)
There are some regular WAV files in Audio/banks/wav/ [and some audio streams encoded in the *.bd' files, not yet reverse engineered at the time of writing geyser (talk) 01:40, 6 February 2018 (CET) ], but there is also a WAV-like format called STM (unrelated to the MIDI-like STM file mentioned on Wikipedia).
...
TEX file format (texture)
A texture format similar to Oni's TXMP.
...
MOD file format (plain text)
Most of Oni2's models are in a plain text format identified as "version: 1.10".
...
MOD file format (binary)
The more recent model format is a binary format identified as "version: 2.10". Apart from the more compact binary storage, it seems equivalent to the plain-text 1.10 format described above.
...
ANIM file format
...