OBD:Oni2AS: Difference between revisions

From OniGalore
Jump to navigation Jump to search
(a page for REing the game data of Oni 2 (Angel Studios))
 
(Basic extraction procedure)
Line 1: Line 1:
[[Category:Modding information]]
[[Category:Modding information]]
The Oni2 dev disc from Angel Studios contains:
*the main ELF executable '''SCPS_123.45'''
*a plain-text configuration file '''SYSTEM.CNF'''
*a '''SYSTEM''' folder containing fifteen ELF binaries ('''*.IRX''') and a file called '''IOPRP255.IMG'''
*the three "binary data" files '''BANKS.DAT''', '''RB.DAT''' and '''STREAMS.DAT'''
The common format of the three binary files is as follows (illustrated on '''RB.DAT''').
{{table}}
{{OBDth}}
{{OBDtr| 0x00 | 4CC        | FF0000 | 44 41 56 45 | "DAVE"  | a control code }}
{{OBDtr| 0x04 | int32      | FFFF00 | 18 41 00 00 | 16664  | number of files }}
{{OBDtr| 0x08 | int32      | 00FF00 | 00 18 04 00 | 0x41800 | offset to start of file name block, from 0x800, in bytes }}
{{OBDtr| 0x0C | int32      | 00FFFF | 00 30 0A 00 | 0xA3000 | offset to start of file data block, from 0x800, in bytes }}
{{OBDtr| 0x10 | char[2032] | FFFFFF | 00 ... |0|padding}}
{{OBDtrBK|File table, first entry}}
{{OBDtr| 0x800 | int32 |FFC8C8| 00 00 00 00 | 0x00      | offset to file name, from start of file name block, in bytes
*in this case it points to <nowiki>0x800+0x41800+0x00=0x42000</nowiki>
*the string at that address is '''actlog.txt''' (null-terminated) }}
{{OBDtr| 0x804 | int32 |FFFFC8| BC D9 1F 02 | 0x21FD9BC | absolute offset to file data, from start of DAT file, in bytes }}
{{OBDtr| 0x808 | int32 |C8FFC8| 00 08 00 00 | 2048 | uncompressed file size, in bytes }}
{{OBDtr| 0x80C | int32 |C8FFFF| 94 00 00 00 | 148  | compressed file size, in bytes }}
{{OBDtrBK|File table, second entry}}
{{OBDtr| 0x810 | int32 |FFC8C8| 0B 00 00 00 | 0x0B        | offset to file name, from start of file name block, in bytes
*in this case it points to <nowiki>0x800+0x41800+0x0B=0x4200B</nowiki>
*the string at that address is '''Audio/''' (null-terminated) }}
{{OBDtr| 0x814 | int32 |FFFFC8| 22 D3 1D 02 | 0x21DD322 | absolute offset to file data, from start of DAT file, in bytes
*the address 0x21DD322 points to no actual data in this case
*is it common to folders (as in this case) and empty files }}
{{OBDtr| 0x818 | int32 |C8FFC8| 00 00 00 00 | 0        | uncompressed file size (zero; it's a folder) }}
{{OBDtr| 0x81C | int32 |C8FFFF| 00 00 00 00 | 0        | compressed file size (also zero) }}
{{OBDtrBK|File table, third entry}}
{{OBDtr| 0x820 | int32 |FFC8C8| 12 00 00 00 | 0x12        | offset to file name, from start of file name block, in bytes
*in this case it points to <nowiki>0x800+0x41800+0x0B=0x42012</nowiki>
*the string at that address is '''Audio/banks/''' (null-terminated) }}
{{OBDtr| 0x824 | int32 |FFFFC8| 22 D3 1D 02 | 0x21DD322 | absolute offset to file data, from start of DAT file, in bytes
*the address 0x21DD322 points to no actual data in this case
*is it common to folders (as in this case) and empty files }}
{{OBDtr| 0x828 | int32 |C8FFC8| 00 00 00 00 | 0        | uncompressed file size (zero; it's a folder) }}
{{OBDtr| 0x82C | int32 |C8FFFF| 00 00 00 00 | 0        | compressed file size (also zero) }}
{{OBDtrBK|File table, fourth entry}}
{{OBDtr| 0x830 | int32 |FFC8C8| 1F 00 00 00 | 0x1F      | offset to file name, from start of file name block, in bytes
*in this case it points to <nowiki>0x800+0x41800+0x1F=0x4201F</nowiki>
*the string at that address is '''Audio/banks/attack.bd''' (null-terminated) }}
{{OBDtr| 0x834 | int32 |FFFFC8| 00 00 53 06 | 0x6530000 | absolute offset to file data, from start of DAT file, in bytes }}
{{OBDtr| 0x838 | int32 |C8FFC8| 90 09 03 00 | 199056    | uncompressed file size, in bytes }}
{{OBDtr| 0x83C | int32 |C8FFFF| 13 85 02 00 | 165139    | compressed file size, in bytes }}
{{OBDtrBK|...etc (rest of the file table)}}
|}
The file table format is apparently a PS2 standard, and there are free tools like [http://www.watto.org/game_extractor.html Game Extractor (basic version)] that can display the folder structure and file sizes, and extract uncompressed files. However, while Game Extractor correctly identifies the compressed and uncompressed file sizes, it apparently does not know what the compression algorithm is (at least in the free version the compressed files are extracted in their compressed form and are thus unusable). Only very short files are packed without compression, for example minor shader or animation scripts.
After some forum lurking, it becomes clear that the compression algorithm is ZIP, or rather "deflate", in its most common implementation (open-source [https://www.zlib.net/ ZLIB] library). However, the compressed files have no ZIP header, so the decompressor/compressor must use the right default settings for this to work. The correct setup is as follows:
z_stream infstream; // initialization of the "inflate" stream
infstream.zalloc = Z_NULL;
infstream.zfree = Z_NULL;
infstream.opaque = Z_NULL;
infstream.avail_in = (uInt)(size_comp); // size of compressed input
infstream.next_in = (Bytef *)file_comp; // input file (char array)
infstream.avail_out = (uInt)(size_orig); // size of decompressed output
infstream.next_out = (Bytef *)file_orig; // output file (char array)
   
// the actual DE-compression work.
inflateInit2(&infstream,-15);
inflate(&infstream, Z_NO_FLUSH);
inflateEnd(&infstream);
The important line, which defines the decompression rules, is '''inflateInit2(&infstream,-15)'''. For the meaning of the -15 value, and the detail of the algorithm, read the ZLIB docs.
With proper decompression, the DAT files extract as follows: [http://geyser.oni2.net/BANKS.DAT.ZIP BANKS.DAT] [http://geyser.oni2.net/RB.DAT.ZIP RB.DAT] [http://geyser.oni2.net/STREAMS.DAT.ZIP STREAMS.DAT]

Revision as of 18:22, 5 February 2018

The Oni2 dev disc from Angel Studios contains:

  • the main ELF executable SCPS_123.45
  • a plain-text configuration file SYSTEM.CNF
  • a SYSTEM folder containing fifteen ELF binaries (*.IRX) and a file called IOPRP255.IMG
  • the three "binary data" files BANKS.DAT, RB.DAT and STREAMS.DAT

The common format of the three binary files is as follows (illustrated on RB.DAT).

Offset Type Raw Hex Value Description
0x00 4CC 44 41 56 45 "DAVE" a control code
0x04 int32 18 41 00 00 16664 number of files
0x08 int32 00 18 04 00 0x41800 offset to start of file name block, from 0x800, in bytes
0x0C int32 00 30 0A 00 0xA3000 offset to start of file data block, from 0x800, in bytes
0x10 char[2032] 00 ... 0 padding
File table, first entry
0x800 int32 00 00 00 00 0x00 offset to file name, from start of file name block, in bytes
  • in this case it points to 0x800+0x41800+0x00=0x42000
  • the string at that address is actlog.txt (null-terminated)
0x804 int32 BC D9 1F 02 0x21FD9BC absolute offset to file data, from start of DAT file, in bytes
0x808 int32 00 08 00 00 2048 uncompressed file size, in bytes
0x80C int32 94 00 00 00 148 compressed file size, in bytes
File table, second entry
0x810 int32 0B 00 00 00 0x0B offset to file name, from start of file name block, in bytes
  • in this case it points to 0x800+0x41800+0x0B=0x4200B
  • the string at that address is Audio/ (null-terminated)
0x814 int32 22 D3 1D 02 0x21DD322 absolute offset to file data, from start of DAT file, in bytes
  • the address 0x21DD322 points to no actual data in this case
  • is it common to folders (as in this case) and empty files
0x818 int32 00 00 00 00 0 uncompressed file size (zero; it's a folder)
0x81C int32 00 00 00 00 0 compressed file size (also zero)
File table, third entry
0x820 int32 12 00 00 00 0x12 offset to file name, from start of file name block, in bytes
  • in this case it points to 0x800+0x41800+0x0B=0x42012
  • the string at that address is Audio/banks/ (null-terminated)
0x824 int32 22 D3 1D 02 0x21DD322 absolute offset to file data, from start of DAT file, in bytes
  • the address 0x21DD322 points to no actual data in this case
  • is it common to folders (as in this case) and empty files
0x828 int32 00 00 00 00 0 uncompressed file size (zero; it's a folder)
0x82C int32 00 00 00 00 0 compressed file size (also zero)
File table, fourth entry
0x830 int32 1F 00 00 00 0x1F offset to file name, from start of file name block, in bytes
  • in this case it points to 0x800+0x41800+0x1F=0x4201F
  • the string at that address is Audio/banks/attack.bd (null-terminated)
0x834 int32 00 00 53 06 0x6530000 absolute offset to file data, from start of DAT file, in bytes
0x838 int32 90 09 03 00 199056 uncompressed file size, in bytes
0x83C int32 13 85 02 00 165139 compressed file size, in bytes
...etc (rest of the file table)

The file table format is apparently a PS2 standard, and there are free tools like Game Extractor (basic version) that can display the folder structure and file sizes, and extract uncompressed files. However, while Game Extractor correctly identifies the compressed and uncompressed file sizes, it apparently does not know what the compression algorithm is (at least in the free version the compressed files are extracted in their compressed form and are thus unusable). Only very short files are packed without compression, for example minor shader or animation scripts.

After some forum lurking, it becomes clear that the compression algorithm is ZIP, or rather "deflate", in its most common implementation (open-source ZLIB library). However, the compressed files have no ZIP header, so the decompressor/compressor must use the right default settings for this to work. The correct setup is as follows:

z_stream infstream; // initialization of the "inflate" stream
infstream.zalloc = Z_NULL;
infstream.zfree = Z_NULL;
infstream.opaque = Z_NULL;

infstream.avail_in = (uInt)(size_comp); // size of compressed input
infstream.next_in = (Bytef *)file_comp; // input file (char array)
infstream.avail_out = (uInt)(size_orig); // size of decompressed output
infstream.next_out = (Bytef *)file_orig; // output file (char array)
    
// the actual DE-compression work.
inflateInit2(&infstream,-15);
inflate(&infstream, Z_NO_FLUSH);
inflateEnd(&infstream);

The important line, which defines the decompression rules, is inflateInit2(&infstream,-15). For the meaning of the -15 value, and the detail of the algorithm, read the ZLIB docs.

With proper decompression, the DAT files extract as follows: BANKS.DAT RB.DAT STREAMS.DAT