Jump to content

OBD:Oni2AS: Difference between revisions

OBD:Oni2AS (view source)

Revision as of 19:25, 23 January 2020

9,631 bytes added ,  23 January 2020
Undo revision 28817 by Geyser (talk)
(documenting)
(Undo revision 28817 by Geyser (talk))
Line 1: Line 1:
==ANIM file format==
The [[Oni2AS|Oni 2 dev disc]] from Angel Studios contains:
All of the '''.anim''' files in Oni2 are uncompressed, and store translation, rotation and scaling values as regular IEEE floats. The meaning of each float (whether it is a translation, a rotation or a scaling, what its axis is, and which bone it corresponds to) is described in the '''.skel''' file (skeleton) of the mesh: the transforms are parsed in the same order as they appear in the '''.skel''' file. The link between the skeleton and the animation is established at the level of the "entity", and is not detailed here.
*the main ELF executable '''SCPS_123.45'''
*a plain-text configuration file '''SYSTEM.CNF'''
*a '''SYSTEM''' folder containing fifteen ELF binaries ('''*.IRX''') and a file called '''IOPRP255.IMG'''
*the three "binary data" files '''BANKS.DAT''', '''RB.DAT''' and '''STREAMS.DAT'''
:"RB" possibly stands for "Rebirth", one of Oni2's working (sub)titles.
:"DAT" probably does ''not'' stand for "Death and Taxes", another of Oni2's working (sub)titles. ^_^


For rotations, most commonly the full triplet of Euler angles is stored, and for translations the full position vector. For a few UI elements the pose is limited to a rotation around 1 axis (indicated in the '''.skel''' file), and therefore a single float is enough to describe the pose. Sometimes a rotation axis is present in the '''.skel''' but locked at a fixed value - in that case the keyable values are reduced to the remaining rotation axes. Scaling triplets are present, e.g., for the "Buco" robots.
==DAT file format (file table)==
The common format of the three "binary data" files is as follows (illustrated on the example of '''RB.DAT''').


There is no time compression. The complete pose (i.e., all the keyable translations/rotations/scalings listed in the '''.skel''') is stored for each consecutive frame in the animation (i.e., if the transformation for a bone does not change at all for a certain time range, or during the whole animation, it is stored for each frame anyway). The frame rate is 30 fps.
Apart from the floats (translations and rotations for each pose/frame), each '''.anim''' file has a header, which announces, among other things, the number of transforms per frame and the number of frames. The header can be of the following three types:
{{table}}
{{table}}
{{OBDth}}
{{OBDth}}
{{OBDtr| 0x00 | 4CC        | FF0000 | 41 4E 49 31 | "ANI1" | animation type "ANI1" (example file: kno_atk_comb_bak_bakkckCH.anim) }}
{{OBDtr| 0x00 | 4CC        | FF0000 | 44 41 56 45 | "DAVE" | a control code }}
{{OBDtr| 0x04 | bitset     | FFFF00 | 00 00 00 80 | 0, 0, 0, 128 | flags; the following bits occur in the original files (values in hex):
{{OBDtr| 0x04 | int32      | FFFF00 | 18 41 00 00 | 16664  | number of file table entries (files or folders) }}
:0x'''01''' 00 00 00 - unknown
{{OBDtr| 0x08 | offset     | 00FF00 | 00 18 04 00 | 0x41800 | offset to start of file name block, from 0x800, in bytes }}
:0x00 00 00 '''80''' - has detailed (per-frame) root motion data
{{OBDtr| 0x0C | offset    | 00FFFF | 00 30 0A 00 | 0xA3000 | offset to start of file data block, from 0x800, in bytes }}
}}
{{OBDtr| 0x10 | char[2032] | FFFFFF | 00 ... |0|padding}}
{{OBDtr| 0x08 | int32     | 00FF00 | 21 00 00 00 | 33  | number of frames }}
{{OBDtrBK|File table, first entry}}
{{OBDtr| 0x0C | int32     | 00FFFF | 78 00 00 00 | 120 | number of transforms per frame (must be consistent with the number of keyable values in the .skel file). In this example the skeleton is kno.skel, which has 39 bones; all bones have an XYZ rotation triplet (Euler angles), and the root bone also has an XYZ translation triplet; hence there will be 120 values in total (plus the detailed root motion).
{{OBDtr| 0x800 | offset |FFC8C8| 00 00 00 00 | 0x00      | offset to file name, from start of file name block, in bytes
}}
*in this case it points to <nowiki>0x800+0x41800+0x00=0x42000</nowiki>
{{OBDtr| 0x10 | float    | 00FF00 | BD 97 4D BD | -0.0502  | total root motion along X (matches the cumulated X values of the detailed root motion track, if any) }}
*the string at that address is '''actlog.txt''' (null-terminated) }}
{{OBDtr| 0x14 | float    | 00FF00 | 00 00 00 00 | 0.0     | total root motion along Y (matches the cumulated Y values of the detailed root motion track, if any) }}
{{OBDtr| 0x804 | offset |FFFFC8| BC D9 1F 02 | 0x21FD9BC | absolute offset to file data, from start of DAT file, in bytes }}
{{OBDtr| 0x18 | float    | 00FF00 | 63 A7 03 BF | -0.514 | total root motion along Z (matches the cumulated Z values of the detailed root motion track, if any) }}
{{OBDtr| 0x808 | int32  |C8FFC8| 00 08 00 00 | 2048 | uncompressed file size, in bytes }}
{{OBDtr| 0x80C | int32  |C8FFFF| 94 00 00 00 | 148  | compressed file size, in bytes }}
{{OBDtrBK|File table, second entry}}
{{OBDtr| 0x810 | offset |FFC8C8| 0B 00 00 00 | 0x0B        | offset to file name, from start of file name block, in bytes
*in this case it points to <nowiki>0x800+0x41800+0x0B=0x4200B</nowiki>
*the string at that address is '''Audio/''' (null-terminated) }}
{{OBDtr| 0x814 | offset |FFFFC8| 22 D3 1D 02 | 0x21DD322 | absolute offset to file data, from start of DAT file, in bytes
*the offset 0x21DD322 points to no actual data in this case
*is it common to folders (as in this case) and empty files }}
{{OBDtr| 0x818 | int32 |C8FFC8| 00 00 00 00 | 0        | uncompressed file size (zero; it's a folder) }}
{{OBDtr| 0x81C | int32 |C8FFFF| 00 00 00 00 | 0        | compressed file size (also zero) }}
{{OBDtrBK|File table, third entry}}
{{OBDtr| 0x820 | offset |FFC8C8| 12 00 00 00 | 0x12        | offset to file name, from start of file name block, in bytes
*in this case it points to <nowiki>0x800+0x41800+0x0B=0x42012</nowiki>
*the string at that address is '''Audio/banks/''' (null-terminated) }}
{{OBDtr| 0x824 | offset |FFFFC8| 22 D3 1D 02 | 0x21DD322 | absolute offset to file data, from start of DAT file, in bytes
*the offset 0x21DD322 points to no actual data in this case
*is it common to folders (as in this case) and empty files }}
{{OBDtr| 0x828 | int32  |C8FFC8| 00 00 00 00 | 0         | uncompressed file size (zero; it's a folder) }}
{{OBDtr| 0x82C | int32  |C8FFFF| 00 00 00 00 | 0         | compressed file size (also zero) }}
{{OBDtrBK|File table, fourth entry}}
{{OBDtr| 0x830 | offset |FFC8C8| 1F 00 00 00 | 0x1F     | offset to file name, from start of file name block, in bytes
*in this case it points to <nowiki>0x800+0x41800+0x1F=0x4201F</nowiki>
*the string at that address is '''Audio/banks/attack.bd''' (null-terminated) }}
{{OBDtr| 0x834 | offset |FFFFC8| 00 00 53 06 | 0x6530000 | absolute offset to file data, from start of DAT file, in bytes }}
{{OBDtr| 0x838 | int32  |C8FFC8| 90 09 03 00 | 199056    | uncompressed file size, in bytes }}
{{OBDtr| 0x83C | int32 |C8FFFF| 13 85 02 00 | 165139    | compressed file size, in bytes }}
{{OBDtrBK|...etc (rest of the file table)}}
|}
|}
----
{{table}}
{{OBDth}}
{{OBDtr| 0x00 | 4CC        | FF0000 | 61 6E 69 00 | "ani"+null | animation type "ani" (example file: kno_atk_BLH1_lft_punch.anim) }}
{{OBDtr| 0x04 | bitset    | FFFF00 | 01 00 00 00 | 1, 0, 0, 0 | flag(s); the following bits occur in the original files (values in hex):
:0x'''01''' 00 00 00 - unknown
}}
{{OBDtr| 0x08 | int32    | 00FF00 | 51 00 00 00 | 81 | number of frames }}
{{OBDtr| 0x0C | int32    | 00FFFF | A2 00 00 00 | 162 | number of transforms per frame (must be consistent with the number of keyable values in the .skel file). In this example the skeleton is the (missing) xkno.skel, which has 53 bones; all bones have an XYZ rotation triplet (Euler angles), and the root bone also has an XYZ translation triplet; hence there will be 162 values in total.
}}
{{OBDtr| 0x10 | float    | 00FF00 | 7D 18 7D BD | -0.0618 | total root motion along X (matches the keyed motion of the root bone) }}
{{OBDtr| 0x14 | float    | 00FF00 | 00 00 00 00 | 0.0    | total root motion along Y (matches the keyed motion of the root bone) }}
{{OBDtr| 0x18 | float    | 00FF00 | 00 00 00 00 | 0.0    | total root motion along Z (matches the keyed motion of the root bone) }}
{{OBDtr| 0x1C | byte      | 00FF00 | 01          | 1      | flag(s); the following bits occur in the original files  (values in hex):
:0x'''01''' 00 00 00 - unknown
}}
|}
----
{{table}}
{{OBDth}}
{{OBDtr| 0x00 | 4CC      | FF0000 | 00 00 00 00 | null chars | "short" animation type (example file: kno_BR2H1_HighPunchBackRight.anim) }}
{{OBDtr| 0x04 | int32    | 00FF00 | 51 00 00 00 | 81 | number of frames }}
{{OBDtr| 0x08 | int32    | 00FFFF | A2 00 00 00 | 162 | number of transforms per frame (must be consistent with the number of keyable values in the .skel file). In this example the skeleton is the (missing) xkno.skel, which has 53 bones; all bones have an XYZ rotation triplet (Euler angles), and the root bone also has an XYZ translation triplet; hence there will be 162 values in total.
}}
{{OBDtr| 0x0C | float    | 00FF00 | AA 9D 00 3E | 0.126  | unknown }}
{{OBDtr| 0x10 | byte      | FFFF00 | 01          | 1      | flag(s); the following bits occur in the original files  (values in hex):
:0x'''01''' 00 00 00 - unknown
}}
|}
Immediately after the header begin the keyed values (floats), grouped by frame: first all the keyed values for frame 0, then for frame 1, etc.


For each frame, the keyed values correspond to the ones listed in the '''.skel''' and occur in the same order. The typical order is:
The file table format is apparently a PS2 standard, and there are free tools like [http://www.watto.org/game_extractor.html Game Extractor (basic version)] that can display the folder structure and file sizes, and extract uncompressed files. However, while Game Extractor correctly identifies the compressed and uncompressed file sizes, it apparently does not know what the compression algorithm is (at least in the free version, the compressed files are extracted as-is, and are thus unusable). Only very short files are packed without compression (mostly trivial-looking stub files of little interest).
:X translation of "root"; Y translation of "root"; Z translation of "root"
 
:X rotation of "root"; Y rotation of "root"; Z rotation of "root"
After some forum lurking, it becomes clear that the compression algorithm is ZIP, or rather "deflate", in its most common implementation (open-source [https://www.zlib.net/ ZLIB] library). However, the compressed files have no ZIP header, so the decompressor/compressor must use the right default settings for this to work. The correct setup is as follows:
:X rotation of 1st child of "root", typically "spine_01"; Y rotation of "spine_01"; Z rotation of "spine_01"
z_stream infstream; // initialization of the "inflate" stream
::... XYZ rotation for all the children of "spine_01", in hierarchical order (same as in the '''.skel''')
infstream.zalloc = Z_NULL;
:X rotation of 2nd child of "root", typically "pelvis"; Y rotation of "pelvis"; Z rotation of "pelvis"
infstream.zfree = Z_NULL;
::... XYZ rotation for all the children of "pelvis", in hierarchical order (same as in the '''.skel''')
infstream.opaque = Z_NULL;
:Detailed XYZ root motion (if any; can only occur in ANI1 type animation). Stored as single-frame increments.
infstream.avail_in = (uInt)(size_comp); // size of compressed input
infstream.next_in = (Bytef *)file_comp; // input file (char array)
infstream.avail_out = (uInt)(size_orig); // size of decompressed output
infstream.next_out = (Bytef *)file_orig; // output file (char array)
   
// the actual DE-compression work.
inflateInit2(&infstream,-15);
inflate(&infstream, Z_NO_FLUSH);
inflateEnd(&infstream);
The important line, which defines the decompression rules, is '''inflateInit2(&infstream,-15)'''. -15 means maximum compression and no header. For more details, c.f. the ZLIB documentation.
 
With proper decompression, the DAT files extract as follows: [http://geyser.oni2.net/BANKS.DAT.ZIP BANKS.DAT] [http://geyser.oni2.net/RB.DAT.ZIP RB.DAT] [http://geyser.oni2.net/STREAMS.DAT.ZIP STREAMS.DAT]
 
*'''STREAMS.DAT''' holds 22 '''*.stm''' files (21 vocalization sounds and 1 musical piece). See below for the STM format and a link to the converted sound files.
*'''BANKS.DAT''' holds 13 sound sets, each of which is a pair of a '''*.hd''' file (similar to Oni's [[OSGr]]?) and a corresponding '''*.bd''' file (similar to a group of Oni's [[SNDD]] bundled together). The '''*.bd''' files in '''BANKS.DAT''' are identical to the ones stored in '''RB.DAT''' (in the '''Audio/banks/''' folder), and possibly redundant. The '''*.hd''' files are not duplicated in '''RB.DAT''', which instead has plain-text '''*.td''' files (apparently all three file types are supposed to work together, but perhaps '''*.hd''' is just a binary version of '''*.td''' with some additional features)
*'''RB.DAT''' holds all the rest of the game data: textures, levels, characters, animations, scripts, etc.
 
==Commonplace file formats==
Some of the files in '''RB.DAT''' are either in plain text or in otherwise widely known formats that can be opened by third-party programs. Here is the list of such commonplace formats used in Oni 2.
*'''*.txt''' : [[wp:Text_file|Text file]]s. Some in Windows format (CR+LF at EOL), some in Unix format (only LF at EOL)
*'''*.mb''' : Binary file used by Maya 3D program (work-in-progress models of some characters and props)
*'''*.mb.swatches''' : Maya swatches, whatever they are. There are also '''*.mel''' files (plain text)
*'''*.wav''' : [[wp:WAV|WAV]]. A common format for the uncompressed storage of sounds (through pulse-code modulation)
*'''*.tga''' : [[wp:Truevision_TGA|Targa]]. A versatile texture format that supports RLE compression, MIP map storage, indexed as well as RGB color schemes, and transparency (and therefore is widely used in games).
*'''*.xml''' : [[wp:XML|XML]].
*'''*.cue''' : [[wp:Cue_sheet_(computing)|Cue sheet]]. Occurs only once, as '''rb.cue''' at the root of '''RB.DAT'''
*'''*.m2v''' : [[wp:MPEG-2|MPEG-2]]. Video files. Occur in the form of '''movies/angel.m2v''' and '''movies/rockstarlogo.m2v'''
*'''*.doc''' : Microsoft Word file
*'''*.xls''' : Microsoft Excel file
*'''*.psd''' : Adobe Photoshop file
*'''*.tif''' : [[wp:TIFF|TIFF]] image file
*'''*.lnk''' : Microsoft shortcut
 
==Plain text file formats==
Apart from XML files (and some TXT and '''*.log''' files; mostly debug logs), Oni 2 has many custom plain-text files, listed below.
 
'''Save files -''' occurs only in the form of '''sample.oni2save''' at the root of '''RB.DAT'''
 
'''Template files -''' occurs only in the form of '''template/rb.template''' and a couple of '''layout.template''' files (see '''Layout files''' below)
 
'''Graph files -''' files ending in '''.graph''', found in '''layout/*/'''
 
'''Layout files -''' files that specify the layout of a level. They're called names like '''layout.actors''', '''layout.et''' (entities), '''layout.graphs''', '''layout.groups''', '''layout.lights''', '''layout.paths''', '''layout.fog''', '''layout.atktab''' (attack table), '''layout.template''', '''layout.campack''' (also '''layout.campacknew'''), '''layout.bnds''' (bounds)
 
'''Environment settings -''' occurs only in the form of '''default.environment''' files in various '''layout/*/''' folders
 
'''"rb" files -''' core settings of some sort found '''in Settings/''': '''rb.ammo''', '''rb.expl''' (explosions), '''rb.formations''', '''rb.fx''' (effects), '''rb.gamedata''', '''rb.gt''', '''rb.inventory''', '''rb.patterns''', '''rb.proj''' (projectiles), and '''rb.weap''' (weapons). Also '''Audio/rb.audiopackages''' and '''fx/rb.fxe''', as well as the aforementioned '''template/rb.template'''. ("rb" possibly stands for "Rebirth", one of Oni2's working (sub)titles.)
 
'''UI files -''' two files ending in '''.ui''', found in '''Settings/''': '''rbfrontend.ui''' and '''rbgame.ui'''
 
'''Tuning files -''' two files ending in '''.tune''', found in '''Settings/''': '''pad.tune''' and '''pad_before_massive_tuning.tune'''
 
'''Script files -''' files ending in '''.oni''', rather similar to [[BSL]]. Can be compiled into binary '''*.onc''' files.
 
'''Model files -''' files ending in '''.mod''' (or '''.mod.bak''' or '''.mod.old''') or '''.xmod'''. Version 1 format is plain-text, version 2 is binary (see below for both).
 
'''Pathfinding files -''' files ending in '''.room''' and '''.bsp''', found in '''Level/*/'''
 
'''Mesh files -''' files ending in '''.mesh''', found in '''Level/*/''' (pathfinding) or '''Entity/*/''' (explosions and gun flares)
 
'''Fighting state machines -''' files ending in '''.fsm''', found in '''Statemachine/'''
 
'''Attack patterns -''' files ending in '''.atk''', found in '''Statemachine/''' and '''layout/fightai_grapple/statemachine_backup/'''
 
'''Formation files -''' files ending in '''.frm''', found in '''Settings/''', apparently something to do with the GUI page layout.
 
'''HUD setting files -''' occurs only in the form of '''Settings/HUD_Settings.hud'''
 
'''Control mapping files -''' occurs only in the form of '''Settings/control.map'''
 
'''Particle files -''' files ending in '''.ptx''', found in '''Settings/'''
 
'''Shader files -''' files ending in '''.shader''', used as an intermediate between models and their textures.
 
'''"Neat" files -''' two files ending in '''.neat''' in '''Entity/kno/''' (apparently improved '''.shader''' files)
 
'''Attack FX tables -''' files ending in '''.attackfxtable''', found in '''fx/'''
 
'''Effect lists etc -''' files ending in '''.fxl''' and '''.fxm''', found in '''fx/'''
 
'''Jump files -''' files ending in '''.jump''', found in '''entity.tune/animpkg/nav/'''
 
'''Locomotion files -''' files ending in '''.loco''', found in '''entity.tune/animpkg/nav/''', and in the respective folders of some character entities.
 
'''Blocking files -''' files ending in '''.blk''', found in '''entity.tune/*/''', and in the respective folders of some character entities.
 
'''Reaction files -''' files ending in '''.rct''', found in '''entity.tune/*/'''
 
'''Animation packages -''' files ending in '''.apkg''' (and '''.apkg.txt'''), found in '''entity.tune/animpkg/'''. Also, template file '''entity.tune/animpkg/nav/nav.tpkg'''
 
'''Animation collections -''' files ending in '''.anims''' or '''.Anims''', found in '''entity.tune/*/''', and in the respective folders of some animated entities.
 
'''Attack data files -''' files ending in '''.atdt''' (or '''.atdt.bak'''), found in '''entity.tune/*/''', and in the respective folders of some character entities.
 
'''Attack collections -''' files ending in '''.attacks''', found in '''entity.tune/*/''', and in the respective folders of some character entities.
 
'''Attachment files -''' occurs only in the form of '''entity.tune/erc/erc.attach'''
 
'''Fighter data -''' occurs only in the form of '''entity.tune/kno/kno.ft'''
 
'''Weapon data -''' files ending in '''.weap''', found in '''entity.tune/*/'''. Possibly related to '''Settings/rb.weap'''
 
'''Sound library -''' files ending in '''.sndlib''', found in '''entity.tune/*/'''
 
'''BFG files -''' files ending in '''.bfg''', found in '''layout/*/''' and '''Entity/*/'''
 
'''MMB files -''' files ending in '''.mmb''', found in '''layout/*/'''
 
'''Entity type -''' files ending in '''.type''', found in '''Entity/*/''' (mostly '''entity.type''' or '''Entity.type''')
 
'''Bounds and physics -''' files ending in '''.bnd''' (mostly '''Bound.bnd''' or '''Bound#.bnd''') and '''.phys''' (either '''Bound.phys''' or '''Bound#.phys'''), found in '''Entity/*/'''
 
'''Shape files -''' files ending in '''.sha''', found in '''Entity/*/'''
 
'''Skeleton files -''' files ending in '''.skel''', found in '''Entity/*/''' for rigged/animated models
 
'''Edge files -''' files ending in '''.em''', found in '''Entity/*/''' (for character collision?)
 
'''Gait files -''' files ending in '''.gait''' (or '''.gait.bak'''), found in '''Entity/*/''' for characters (a gait is a "manner of walking")
 
'''TC files -''' files ending in '''.tc''', found in '''Entity/*/''' (apparently for animated textures; "texture collection"?)
 
'''Playback files -''' files ending in '''.play''', found in '''Entity/jae/''' and '''Entity/krk/'''
 
'''"lot" files -''' files ending in '''.lot''', found in '''Entity/''' (lists of '''*.mb''' files). Also '''Entity/back.lot.withTemplates'''
 
'''"ini" files -''' occurs only in the form of '''Entity/button.ini'''
 
'''"mel" files -''' apparently Maya project definition (plain text)
 
'''"attr" and "geom" -''' only occur as '''Entity/proxy/exportTMP/exportTmp/set_temp1.attr''' and '''Entity/proxy/exportTMP/exportTmp/static.geom'''
 
Some plain-text files do not have an "extension". Usually those are either empty files, or dummy files of an identifiable type. Examples are '''tim_atk_FCH1_rht_pch_uprcutreadme''' (a plain-text "readme" file), '''trashpath''' and '''guardpos''' (unused stubs of '''.oni'''-like scripts), '''ack''' (some kind of skeleton data for Konoko) and '''pants'''
 
==STM file format (sound stream)==
There are some regular WAV files in '''Audio/banks/wav/ ''' [and some audio streams encoded in the ''*.bd''' files, not yet reverse engineered at the time of writing [[User:Geyser|geyser]] ([[User talk:Geyser|talk]]) 01:40, 6 February 2018 (CET) ], but there is also a WAV-like format called STM (unrelated to the MIDI-like STM file mentioned on Wikipedia).
 
To be documented later. The WAV files obtained after conversion can be found [http://geyser.oni2.net/STM2WAV.ZIP HERE]
 
==TEX file format (texture)==
A texture format similar to Oni's [[TXMP]].
 
To be documented later. The TGA images obtained after conversion can be found [http://geyser.oni2.net/TEX2TGA.ZIP HERE]
 
Some of the textures are temporary or alternative copies, and so their filenames don't end in '''.tex''' but '''.tex.old''' or '''.tex.highRes'''
 
==MOD file format (plain text)==
Most of Oni 2's models are in a plain text format identified as "version: 1.10".
 
...
 
==MOD file format (binary)==
The more recent model format is a binary format identified as "version: 2.10". Apart from the more compact binary storage, it seems equivalent to the plain-text 1.10 format described above.
 
...
 
 
==SNE file format (binary)==
 
...
 
 
==HD and BD file format (binary)==
 
...


Rotations are stored as XZY Euler angles (first rotation is around bone's Y, then around the new Z, and finally around the latest X). Angle values are in radians.
==ANIM file format==


...
[[Category:Modding information]]
[[Category:Modding information]]
Retrieved from "https://wiki.oni2.net/Special:MobileDiff/28818"