OBD:Oni2AS

The Oni 2 dev disc from Angel Studios contains:

• the main ELF executable SCPS_123.45
• a plain-text configuration file SYSTEM.CNF
• a SYSTEM folder containing fifteen ELF binaries (*.IRX) and a file called IOPRP255.IMG
• the three "binary data" files BANKS.DAT, RB.DAT and STREAMS.DAT

"RB" possibly stands for "Rebirth", one of Oni2's working (sub)titles.

"DAT" probably does not stand for "Death and Taxes", another of Oni2's working (sub)titles. ^_^

DAT file format (file table)

The common format of the three "binary data" files is as follows (illustrated on the example of RB.DAT).

Offset	Type	Raw Hex	Value	Description
0x00	4CC	44 41 56 45	"DAVE"	a control code
0x04	int32	18 41 00 00	16664	number of file table entries (files or folders)
0x08	offset	00 18 04 00	0x41800	offset to start of file name block, from 0x800, in bytes
0x0C	offset	00 30 0A 00	0xA3000	offset to start of file data block, from 0x800, in bytes
0x10	char[2032]	00 ...	0	padding
File table, first entry
0x800	offset	00 00 00 00	0x00	offset to file name, from start of file name block, in bytes in this case it points to 0x800+0x41800+0x00=0x42000 the string at that address is actlog.txt (null-terminated)
0x804	offset	BC D9 1F 02	0x21FD9BC	absolute offset to file data, from start of DAT file, in bytes
0x808	int32	00 08 00 00	2048	uncompressed file size, in bytes
0x80C	int32	94 00 00 00	148	compressed file size, in bytes
File table, second entry
0x810	offset	0B 00 00 00	0x0B	offset to file name, from start of file name block, in bytes in this case it points to 0x800+0x41800+0x0B=0x4200B the string at that address is Audio/ (null-terminated)
0x814	offset	22 D3 1D 02	0x21DD322	absolute offset to file data, from start of DAT file, in bytes the offset 0x21DD322 points to no actual data in this case is it common to folders (as in this case) and empty files
0x818	int32	00 00 00 00	0	uncompressed file size (zero; it's a folder)
0x81C	int32	00 00 00 00	0	compressed file size (also zero)
File table, third entry
0x820	offset	12 00 00 00	0x12	offset to file name, from start of file name block, in bytes in this case it points to 0x800+0x41800+0x0B=0x42012 the string at that address is Audio/banks/ (null-terminated)
0x824	offset	22 D3 1D 02	0x21DD322	absolute offset to file data, from start of DAT file, in bytes the offset 0x21DD322 points to no actual data in this case is it common to folders (as in this case) and empty files
0x828	int32	00 00 00 00	0	uncompressed file size (zero; it's a folder)
0x82C	int32	00 00 00 00	0	compressed file size (also zero)
File table, fourth entry
0x830	offset	1F 00 00 00	0x1F	offset to file name, from start of file name block, in bytes in this case it points to 0x800+0x41800+0x1F=0x4201F the string at that address is Audio/banks/attack.bd (null-terminated)
0x834	offset	00 00 53 06	0x6530000	absolute offset to file data, from start of DAT file, in bytes
0x838	int32	90 09 03 00	199056	uncompressed file size, in bytes
0x83C	int32	13 85 02 00	165139	compressed file size, in bytes
...etc (rest of the file table)

The file table format is apparently a PS2 standard, and there are free tools like Game Extractor (basic version) that can display the folder structure and file sizes, and extract uncompressed files. However, while Game Extractor correctly identifies the compressed and uncompressed file sizes, it apparently does not know what the compression algorithm is (at least in the free version, the compressed files are extracted as-is, and are thus unusable). Only very short files are packed without compression (mostly trivial-looking stub files of little interest).

After some forum lurking, it becomes clear that the compression algorithm is ZIP, or rather "deflate", in its most common implementation (open-source ZLIB library). However, the compressed files have no ZIP header, so the decompressor/compressor must use the right default settings for this to work. The correct setup is as follows:

z_stream infstream; // initialization of the "inflate" stream
infstream.zalloc = Z_NULL;
infstream.zfree = Z_NULL;
infstream.opaque = Z_NULL;

infstream.avail_in = (uInt)(size_comp); // size of compressed input
infstream.next_in = (Bytef *)file_comp; // input file (char array)
infstream.avail_out = (uInt)(size_orig); // size of decompressed output
infstream.next_out = (Bytef *)file_orig; // output file (char array)
    
// the actual DE-compression work.
inflateInit2(&infstream,-15);
inflate(&infstream, Z_NO_FLUSH);
inflateEnd(&infstream);

The important line, which defines the decompression rules, is inflateInit2(&infstream,-15). -15 means maximum compression and no header. For more details, c.f. the ZLIB documentation.

With proper decompression, the DAT files extract as follows: BANKS.DAT RB.DAT STREAMS.DAT

• STREAMS.DAT holds 22 *.stm files (21 vocalization sounds and 1 musical piece). See below for the STM format and a link to the converted sound files.
• BANKS.DAT holds 13 sound sets, each of which is a pair of a *.hd file (similar to Oni's OSGr?) and a corresponding *.bd file (similar to a group of Oni's SNDD bundled together). The *.bd files in BANKS.DAT are identical to the ones stored in RB.DAT (in the Audio/banks/ folder), and possibly redundant. The *.hd files are not duplicated in RB.DAT, which instead has plain-text *.td files (apparently all three file types are supposed to work together, but perhaps *.hd is just a binary version of *.td with some additional features)
• RB.DAT holds all the rest of the game data: textures, levels, characters, animations, scripts, etc.

Commonplace file formats

Some of the files in RB.DAT are either in plain text or in otherwise widely known formats that can be opened by third-party programs. Here is the list of such commonplace formats used in Oni 2.

• *.txt : Text files. Some in Windows format (CR+LF at EOL), some in Unix format (only LF at EOL)
• *.mb : Binary file used by Maya 3D program (work-in-progress models of some characters and props)
• *.mb.swatches : Maya swatches, whatever they are. There are also *.mel files (plain text)
• *.wav : WAV. A common format for the uncompressed storage of sounds (through pulse-code modulation)
• *.tga : Targa. A versatile texture format that supports RLE compression, MIP map storage, indexed as well as RGB color schemes, and transparency (and therefore is widely used in games).
• *.xml : XML.
• *.cue : Cue sheet. Occurs only once, as rb.cue at the root of RB.DAT
• *.m2v : MPEG-2. Video files. Occur in the form of movies/angel.m2v and movies/rockstarlogo.m2v
• *.doc : Microsoft Word file
• *.xls : Microsoft Excel file
• *.psd : Adobe Photoshop file
• *.tif : TIFF image file
• *.lnk : Microsoft shortcut

Plain text file formats

Apart from XML files (and some TXT and *.log files; mostly debug logs), Oni 2 has many custom plain-text files, listed below.

Save files - occurs only in the form of sample.oni2save at the root of RB.DAT

Template files - occurs only in the form of template/rb.template and a couple of layout.template files (see Layout files below)

Graph files - files ending in .graph, found in layout/*/

Layout files - files that specify the layout of a level. They're called names like layout.actors, layout.et (entities), layout.graphs, layout.groups, layout.lights, layout.paths, layout.fog, layout.atktab (attack table), layout.template, layout.campack (also layout.campacknew), layout.bnds (bounds)

Environment settings - occurs only in the form of default.environment files in various layout/*/ folders

"rb" files - core settings of some sort found in Settings/: rb.ammo, rb.expl (explosions), rb.formations, rb.fx (effects), rb.gamedata, rb.gt, rb.inventory, rb.patterns, rb.proj (projectiles), and rb.weap (weapons). Also Audio/rb.audiopackages and fx/rb.fxe, as well as the aforementioned template/rb.template. ("rb" possibly stands for "Rebirth", one of Oni2's working (sub)titles.)

UI files - two files ending in .ui, found in Settings/: rbfrontend.ui and rbgame.ui

Tuning files - two files ending in .tune, found in Settings/: pad.tune and pad_before_massive_tuning.tune

Script files - files ending in .oni, rather similar to BSL. Can be compiled into binary *.onc files.

Model files - files ending in .mod (or .mod.bak or .mod.old) or .xmod. Version 1 format is plain-text, version 2 is binary (see below for both).

Pathfinding files - files ending in .room and .bsp, found in Level/*/

Mesh files - files ending in .mesh, found in Level/*/ (pathfinding) or Entity/*/ (explosions and gun flares)

Fighting state machines - files ending in .fsm, found in Statemachine/

Attack patterns - files ending in .atk, found in Statemachine/ and layout/fightai_grapple/statemachine_backup/

Formation files - files ending in .frm, found in Settings/, apparently something to do with the GUI page layout.

HUD setting files - occurs only in the form of Settings/HUD_Settings.hud

Control mapping files - occurs only in the form of Settings/control.map

Particle files - files ending in .ptx, found in Settings/

Shader files - files ending in .shader, used as an intermediate between models and their textures.

"Neat" files - two files ending in .neat in Entity/kno/ (apparently improved .shader files)

Attack FX tables - files ending in .attackfxtable, found in fx/

Effect lists etc - files ending in .fxl and .fxm, found in fx/

Jump files - files ending in .jump, found in entity.tune/animpkg/nav/

Locomotion files - files ending in .loco, found in entity.tune/animpkg/nav/, and in the respective folders of some character entities.

Blocking files - files ending in .blk, found in entity.tune/*/, and in the respective folders of some character entities.

Reaction files - files ending in .rct, found in entity.tune/*/

Animation packages - files ending in .apkg (and .apkg.txt), found in entity.tune/animpkg/. Also, template file entity.tune/animpkg/nav/nav.tpkg

Animation collections - files ending in .anims or .Anims, found in entity.tune/*/, and in the respective folders of some animated entities.

Attack data files - files ending in .atdt (or .atdt.bak), found in entity.tune/*/, and in the respective folders of some character entities.

Attack collections - files ending in .attacks, found in entity.tune/*/, and in the respective folders of some character entities.

Attachment files - occurs only in the form of entity.tune/erc/erc.attach

Fighter data - occurs only in the form of entity.tune/kno/kno.ft

Weapon data - files ending in .weap, found in entity.tune/*/. Possibly related to Settings/rb.weap

Sound library - files ending in .sndlib, found in entity.tune/*/

BFG files - files ending in .bfg, found in layout/*/ and Entity/*/

MMB files - files ending in .mmb, found in layout/*/

Entity type - files ending in .type, found in Entity/*/ (mostly entity.type or Entity.type)

Bounds and physics - files ending in .bnd (mostly Bound.bnd or Bound#.bnd) and .phys (either Bound.phys or Bound#.phys), found in Entity/*/

Shape files - files ending in .sha, found in Entity/*/

Skeleton files - files ending in .skel, found in Entity/*/ for rigged/animated models

Edge files - files ending in .em, found in Entity/*/ (for character collision?)

Gait files - files ending in .gait (or .gait.bak), found in Entity/*/ for characters (a gait is a "manner of walking")

TC files - files ending in .tc, found in Entity/*/ (apparently for animated textures; "texture collection"?)

Playback files - files ending in .play, found in Entity/jae/ and Entity/krk/

"lot" files - files ending in .lot, found in Entity/ (lists of *.mb files). Also Entity/back.lot.withTemplates

"ini" files - occurs only in the form of Entity/button.ini

"mel" files - apparently Maya project definition (plain text)

"attr" and "geom" - only occur as Entity/proxy/exportTMP/exportTmp/set_temp1.attr and Entity/proxy/exportTMP/exportTmp/static.geom

Some plain-text files do not have an "extension". Usually those are either empty files, or dummy files of an identifiable type. Examples are tim_atk_FCH1_rht_pch_uprcutreadme (a plain-text "readme" file), trashpath and guardpos (unused stubs of .oni-like scripts), ack (some kind of skeleton data for Konoko) and pants

STM file format (sound stream)

There are some regular WAV files in Audio/banks/wav/ [and some audio streams encoded in the *.bd' files, not yet reverse engineered at the time of writing geyser (talk) 01:40, 6 February 2018 (CET) ], but there is also a WAV-like format called STM (unrelated to the MIDI-like STM file mentioned on Wikipedia).

To be documented later. The WAV files obtained after conversion can be found HERE

TEX file format (texture)

A texture format similar to Oni's TXMP.

To be documented later. The TGA images obtained after conversion can be found HERE

Some of the textures are temporary or alternative copies, and so their filenames don't end in .tex but .tex.old or .tex.highRes

MOD file format (plain text)

Most of Oni 2's models are in a plain text format identified as "version: 1.10".

...

MOD file format (binary)

The more recent model format is a binary format identified as "version: 2.10". Apart from the more compact binary storage, it seems equivalent to the plain-text 1.10 format described above.

...

SNE file format (binary)

...

HD and BD file format (binary)

...

ANIM file format

All of the .anim files in Oni2 are uncompressed, and store translation, rotation and scaling values as regular IEEE floats. The meaning of each float (whether it is a translation, a rotation or a scaling, what its axis is, and which bone it corresponds to) is described in the .skel file (skeleton) of the mesh: the transforms are parsed in the same order as they appear in the .skel file. The link between the skeleton and the animation is established at the level of the "entity", and is not detailed here.

For rotations, most commonly the full triplet of Euler angles is stored, and for translations the full position vector. For a few UI elements the pose is limited to a rotation around 1 axis (indicated in the .skel file), and therefore a single float is enough to describe the pose. Sometimes a rotation axis is present in the .skel but locked at a fixed value - in that case the keyable values are reduced to the remaining rotation axes. Scaling triplets are present, e.g., for the "Buco" robots.

There is no time compression. The complete pose (i.e., all the keyable translations/rotations/scalings listed in the .skel) is stored for each consecutive frame in the animation (i.e., if the transformation for a bone does not change at all for a certain time range, or during the whole animation, it is stored for each frame anyway). The frame rate is 30 fps.

Apart from the floats (translations and rotations for each pose/frame), each .anim file has a header, which announces, among other things, the number of transforms per frame and the number of frames. The header can be of the following three types:

Offset	Type	Raw Hex	Value	Description
0x00	4CC	41 4E 49 31	"ANI1"	animation type "ANI1" (example file: kno_atk_comb_bak_bakkckCH.anim)
0x04	bitset	00 00 00 80	0, 0, 0, 128	flags; the following bits occur in the original files (values in hex): 0x01 00 00 00 - unknown 0x00 00 00 80 - has detailed (per-frame) root motion data
0x08	int32	21 00 00 00	33	number of frames
0x0C	int32	78 00 00 00	120	number of transforms per frame (must be consistent with the number of keyable values in the .skel file). In this example the skeleton is kno.skel, which has 39 bones; all bones have an XYZ rotation triplet (Euler angles), and the root bone also has an XYZ translation triplet; hence there will be 120 values in total (plus the detailed root motion).
0x10	float	BD 97 4D BD	-0.0502	total root motion along X (matches the cumulated X values of the detailed root motion track, if any)
0x14	float	00 00 00 00	0.0	total root motion along Y (matches the cumulated Y values of the detailed root motion track, if any)
0x18	float	63 A7 03 BF	-0.514	total root motion along Z (matches the cumulated Z values of the detailed root motion track, if any)

---

Offset	Type	Raw Hex	Value	Description
0x00	4CC	61 6E 69 00	"ani"+null	animation type "ani" (example file: kno_atk_BLH1_lft_punch.anim)
0x04	bitset	01 00 00 00	1, 0, 0, 0	flag(s); the following bits occur in the original files (values in hex): 0x01 00 00 00 - unknown
0x08	int32	51 00 00 00	81	number of frames
0x0C	int32	A2 00 00 00	162	number of transforms per frame (must be consistent with the number of keyable values in the .skel file). In this example the skeleton is the (missing) xkno.skel, which has 53 bones; all bones have an XYZ rotation triplet (Euler angles), and the root bone also has an XYZ translation triplet; hence there will be 162 values in total.
0x10	float	7D 18 7D BD	-0.0618	total root motion along X (matches the keyed motion of the root bone)
0x14	float	00 00 00 00	0.0	total root motion along Y (matches the keyed motion of the root bone)
0x18	float	00 00 00 00	0.0	total root motion along Z (matches the keyed motion of the root bone)
0x1C	byte	01	1	flag(s); the following bits occur in the original files (values in hex): 0x01 00 00 00 - unknown

---

Offset	Type	Raw Hex	Value	Description
0x00	4CC	00 00 00 00	null chars	"short" animation type (example file: kno_BR2H1_HighPunchBackRight.anim)
0x04	int32	51 00 00 00	81	number of frames
0x08	int32	A2 00 00 00	162	number of transforms per frame (must be consistent with the number of keyable values in the .skel file). In this example the skeleton is the (missing) xkno.skel, which has 53 bones; all bones have an XYZ rotation triplet (Euler angles), and the root bone also has an XYZ translation triplet; hence there will be 162 values in total.
0x0C	float	AA 9D 00 3E	0.126	unknown
0x10	byte	01	1	flag(s); the following bits occur in the original files (values in hex): 0x01 00 00 00 - unknown

Immediately after the header begin the keyed values (floats), grouped by frame: first all the keyed values for frame 0, then for frame 1, etc.

For each frame, the keyed values correspond to the ones listed in the .skel and occur in the same order. The typical order is:

X translation of "root"; Y translation of "root"; Z translation of "root"

X rotation of "root"; Y rotation of "root"; Z rotation of "root"

X rotation of 1st child of "root", typically "spine_01"; Y rotation of "spine_01"; Z rotation of "spine_01"

... XYZ rotation for all the children of "spine_01", in hierarchical order (same as in the .skel)

X rotation of 2nd child of "root", typically "pelvis"; Y rotation of "pelvis"; Z rotation of "pelvis"

... XYZ rotation for all the children of "pelvis", in hierarchical order (same as in the .skel)

Detailed XYZ root motion (if any; can only occur in ANI1 type animation). Stored as single-frame increments.

Rotations are stored as XZY Euler angles (first rotation is around bone's Y, then around the new Z, and finally around the latest X). Angle values are in radians.