Talk:OniSplit

Talk page archives: #1

OniSplit 1.0 release notes

Really? Yes, really. Get the nightly HERE.

  0. Possible introduction of aliases for the notoriously confusing -export and -import (easily mistaken for the -extract/-create functionality).
     Historical synonyms for -export and -import are "unpack" and "pack", but I am leaning towards new aliases, most probably -split and -link.
     OniSplit itself may be rebranded as GameDataTool, and a lightweight app (limited to the split/link functionality) may be dubbed (Oni)SpLink.

1. The required .NET framework is changed back to .NET 2.0 (a few past versions of OniSplit were built for .NET 4.0 for some reason, but it appears that 2.0 is sufficient).
2. A new packing format called -import:onix was added, which is the same as -import:sep (PC demo-like) but suppresses template checksums and replaces them with a new versioning system.
   The produced .dat/.raw/.sep files are marked VR33 instead of VR31, only work with the OniX engine, and allow for new features in the game data format - provided that the OniX engine implements their support as well.
3. The misguided implementation of 32-bit transparent textures has been remastered, with a distinction between "rgba" and "bgra32" import formats: "rgba" corresponds to TXMP type 11 (directly supported by all Oni engines); "bgra32" is TXMP type 7, mistakenly used by previous versions of OniSplit (and Daodan/Mac patches) as a substitute for the overlooked type 11.
   Old instances of type 7 TXMPs are automatically detected and repaired (replaced with type 11) at the first opportunity.
   New TXMPs created with -format:bgra32 are stored as legitimate type 7, which will only be handled properly by the new OniX engine (or by an amended Daodan DLL).
4. Sound conversion is now (almost) fully featured, although with a strong bias towards the Windows platform (a.k.a. "PC") and MS ADPCM compression.
   In addition to the previously available stream copying, you can now -extract:wav from a Mac SNDD (-extract:aif from a PC SNDD may be added later).
   When using -extract:wav, an additional option -pcm decompresses the waveform and produces a .wav file with raw PCM storage.
   Compressed ADPCM .wav-files (exported through -extract:wav) are now standard-compliant, and the importing of a .wav preserves custom (AD)PCM settings fully as well, meaning that third-party WAV files can now be roundtripped in and out of Oni (PC retail) without any issues. (However, the validation of corrupt third-party WAVs has not been thoroughly implemented or tested.)
   In addition to the automatic creation of PC or Mac SNDDs through the -create command (depending on whether the input is a .wav or an .aif file), the -demo option can be used to generate a PC demo-like SNDD (short header, no custom sample rate or compression) both from a .wav and from an .aif file (useful when mass-converting sounds for VR33/OniX).
   By default all SNDDs made with -create are ADPCM-compressed SNDDs even if the input is an uncompressed .wav file. The -pcm option allows uncompressed/decompressed storage for PC SNDDs, but this is not recommended: PCM sounds take up 4 times as much space as ADPCM, and their playback is currently broken for the PC demo and OniX engines.
   Considering that, as of now, 18:55, 14 December 2021 (CET), all Oni engines play back SNDDs as 22.05 kHz waveforms (the PC retail engine only pretends to support custom sample rates, see HERE), it is now OniSplit's duty to report any waveforms that are not sampled at 22.05 kHz and to suggest the custom playback speed that should be used at OSGr level. 44.1 kHz sounds will automatically be downsampled to 22.05 kHz except when doing so would involve an additional round of decoding and reencoding; this potentially lossy operation must be explicitly requested with -forcestd.
   When creating short-header SNDDs for the PC demo/OniX (using the -demo tag), incoming MS ADPCM files are ultimately required to have a standard block size of 512 bytes per channel. Here too, a potentially lossy reencoding must be explicitly requested through -forcestd, otherwise importing will fail with a report of the non-conformant block size.
   
5. Subtitle files now have a custom XML format as requested by Script10k. You go from SUBT*.oni to XML using -extract:xml, and you import using -create (both the old .txt subtitles and the new XML format are accepted as input). Vanilla English SUBTs roundtrip exactly both through TXT and through XML. Non-Vanilla SUBTs and other language versions not tested.
6. The -fullname option is available for -extract:dae operations, prepending the 4-character template tag to the filename (TRBS, ONCC, AKEV, etc).
7. Level creation works again. The <Import> feature of Physics.xml has been repaired (it is the feature that lets you import several OBOA entries and their OBANs from the same Collada file) and enhanced with a couple of options, like explicit script IDs for each animated object and custom prefixes/suffixes for animations. As a minor convenience, all _marker textures are now imported automatically.
8. Interaction with Blender has been consolidated through the -blender option, available both when -extracting and -createing.
   The primary use of -blender is to control the transition between Oni's and Blender's axis conventions, both in terms of scene orientation (Y-up for Oni, Z-up for Blender) and rotation order (X-then-Y-then-Z for Oni, Z-then-Y-then-X for Blender).
   Another effect of -blender, exclusive to -extract, is to replace all instance of <technique_common> with <technique sid="common"> in the exported Collada files. when importing a Collada file through -create, both <technique_common> and <technique sid="common"> are accepted.
9. Characters/animations have been consolidated. Among minor improvements, quaternion math is more accurate (preventing "upside-down" pelvis animation and other artifacts) and the identification of animation curves in Collada input is more robust.
   A new -keepkeys option is available for TRAM operations: for -extract it preserves Oni's sparse animation data instead of keying every frame; for -create it bypasses the keyframe reduction pass and uses the same keys as in the Collada file.
   When -createing a TRBS, non-standard skeleton hierarchy is still allowed (controlled by the node tree in the Collada file), but standard sibling order is enforced for "thigh" and "shoulder" bones (if detected).
   At this point there is still no proper skeletal setup (meaning it's still the same "nested" approach, with rigid body parts hinged to one another), and the rest pose is still the infamous "folded umbrella". A more intuitive rest pose may be implemented later.

At the time of writing, items (1) through (7) are done and ready for testing, whereas (8) and (9) are being finalized. The rebranding (0) is merely planned (subject to debate?). --geyser (talk) 18:55, 14 December 2021 (CET)

OniSplit version control - getting there...

The SVN version of OniSplit is currently labeled 0.9.99 (same as the source received from Neo), and at this point it is different from the non-synced 0.9.99.2 that I distributed through EdT.

• "0.9.99" has only two commits (44.1 kHz SNDD export - affecting only electric spark sounds - and other minor SNDD fixes and features);
• 0.9.99.2 has a few fixes to 0.9.99 (adjustment to XML parsing rules) and new functionality for Blender import/export of animations; also some new bugs, apparently.

To restore some order, I am going to merge my 0.9.99.2 fixes to SVN, then address the known bugs, and push the version number to 0.9.99.3 - and use SVN-synced development from then onwards.

I am not going to create a branch for my fixes/improvements, except maybe when it comes to adding something big, like C# FBX support. If you object to this, speak now. --geyser (talk) 00:09, 29 May 2020 (CEST)

List of bugfixes and improvements planned for 0.9.99.3 (feel free to add to the list if I forgot something)

• some running/walking anims end up upside-down after reimported into Oni
• -extract:dae -noanim -blender used on a TRBS, make sure the output .dae file has a TRBS prefix
• stairs_markers.dae not imported properly? (and more generally secondary/auxiliary .dae files not imported for levels?)
• TRAM export: support writing of sparse keyframe data (as in Oni), as opposed to writing rotations for every frame (interpolated)
• maybe allow the export of named TRCM (without material)
• TGA transparency bug? (not-yet-confirmed as a bug; supposedly happens only for SketchUp-made levels; see below)
• various SNDD glitches (inconsistent handling of ADPCM headers)

TGA Transparency bug

I extracted HQ_DOUBLED_GLASS as TGA format from vanilla Level8_Final with the OniSplit command -extract:tga. The file was opened in Photoshop and the alpha channel was visible. I then added that texture to Sketchup and applied it in a test level, it appeared transparent in Sketchup. I used the command -create:level to create a level in Oni. In the game Oni, the texture appeared opaque not transparent. When, the file TXMPHQ_DOUBLED_GLASS.oni was extracted from the test level as TGA, the alpha channel was gone or opaque.

Even if the following code was added to the textures.xml file, the texture still appeared opaque in game.

   <Texture Name="HQ_DOUBLED_GLASS">
       <GunkFlags>Transparent TwoSided</GunkFlags>
       <Format>RGBA</Format>
       <Image>images/HQ_DOUBLED_GLASS.tga</Image>
   </Texture>

However, if HQ_DOUBLED_GLASS.tga is converted to PNG format with alpha channel, and then the PNG file is used in Sketchup, then the texture will be transparent in game.

This occurs using OniSplit version 0.9.96.0 and previous versions, tested to version 0.9.90.0. (EDIT: Transparency loss is observed for all glass textures, not just HQ_DOUBLED_GLASS.)

EdT 15:03, 14 May 2020 (PDST)

When you described the issue to me in PM, I didn't notice/understand the key part about applying the texture in SketchUp and then using -create:level on SketchUp's output. I thought we were talking about a simple TXMP roundtrip through TGA. Can you confirm that the texture's alpha goes opaque (or disappears) if you use -extract:tga on TXMPHQ_DOUBLED_GLASS.oni, and then immediately reimport it either with -create (either from XML with extra flags or from TGA)? --geyser (talk) 11:44, 15 May 2020 (CEST)

Regarding the TXMP roundtrip, after extracting TXMPHQ_DOUBLED_GLASS.oni as TGA, I used the command -create:txmp -format:bgra32 -large -genmipmaps, the resulting file had the alpha channel. To confirm, I replaced the file TXMPHQ_DOUBLED_GLASS.oni from the -create:level with the one from -create:txmp in the test level and it was transparent. EdT 12:35, 15 May 2020 (PDST)

This looks like an issue with the SketchUp pipeline, rather than with OniSplit, since it is only -create:level that generates a non-transparent texture, and -create:txmp actually works fine. Can you: a) provide (in PM) the level data that you are using -create:level on (supposedly a .dae file, some texture files, and some XMLs)? b) provide an example of a level that imports with transparent glass as expected (supposedly a level that didn't come from SketchUp). Thanks. --geyser (talk) 16:46, 16 May 2020 (CEST)

Sound export bug

As discussed on Discord there's a sound export bug for SNDDzap*.oni

One might record the sounds in OBS and compare it with the exported ones to generate further hints on what went wrong. --paradox-01 (talk) 13:29, 21 July 2019 (CEST)

1. The zap sounds are randomized in an OSBD group, so recording them from the game in some identifiable way would require some extra work (custom OSBD). It's much easier to ask Mac folks for AIFF versions of those sounds. And actually, there are seven zap sounds in the PC demo as well.
2. OniSplit slaps a 22.05 kHz header on those files, although they're 44.1kHz and have a perfectly good 44.1kHz header in Oni. Not sure why they're getting a 22.05kHz header. Perhaps Neo got confused by the sloppy documentation of the WAV header, which until my edit gave the same hex listing for the three WAV header types. Either that, or he just assumes the sample rate to be 22.05kHz by default and doesn't update it from the actual file. Will check in the code. Actually yeah, he just does as if all the sounds were 22.05kHz.
3. Besides SNDDzap*.oni, there is one other 44.1kHz sound in PC Oni, SNDDap_hit_shld.aif; which suffers from the same export problem, although it's less noticeable.
4. Apparently 22.05 kHz stereo sounds are correctly exported as stereo, it's just 44.1 kHz mono files that get a wrong WAV header.
5. Will be fixing this in the latest "nightly" OniSplit (haven't yet decided on a version numbering and source control scheme).

   geyser (talk)

Page improvement

Try to click on OniSplit#CLI_tips

The section should be made to pop up, currently it doesn't. --Paradox-01

Yes, when a Divhide section is set to start hidden, it throws off the browser's ability to jump to the right spot. I've never been sure whether to have that section auto-show or auto-hide. Now that it's auto-showing, you'll see that the anchor works. --Iritscen (talk) 12:50, 12 March 2014 (CET)

Change Log

OniSplit version: OniSplit v0.9.90

What's new:

• Fixes bugs with Import/Export of M3GMs (obj files).
• Fixes euler angle issues with exported TRAMs.
• Fixes -noanim command to produce a standing pose.
• In the ONWC file, you should be able to reference the dae/obj file directly then you don't need the separate M3GM.
• When exporting AKEV, quads that have script ids assigned to them are exported to separate files. This is related to BSL commands such as env_show 540 0
• Adds option -anim-merge to export multiple animations to a single dae. Useful in dealing with walk/run or combo animations.

  onisplit -extract:xml <destination directory>  -anim-merge -anim-body:src\Path_To_\TRBSkonoko_body_high.oni Path_To\TRAMKONCOMcomb_k.oni  Path_To\TRAMKONCOMcomb_k_k.oni Path_To\TRAMKONCOMcomb_k_k_k.oni

• Adds a scene exporter

  OniSplit -extract:dae <destination directory> scene.xml

For example if the scene.xml file looks like this

  <Scene>
  <Node Name="camera">
       <Camera />
       <Animation>OBANCamout01.oni</Animation>
       <Animation>OBANCamout02.oni</Animation>
       <Animation>OBANCamout03.oni</Animation>
       <Animation>OBANCamout04.oni</Animation>
       <Animation>OBANCamout05.oni</Animation>
       <Animation>OBANCamout06.oni</Animation>
   </Node>
   <Node Name="motorcycle02">
       <Geometry>M3GMmotorcycle02.oni</Geometry>
       <Animation>OBANmotorcycle02.oni</Animation>
       <Animation>OBANmotorcycle02_stop.oni</Animation>
       <Node Name="hubs">
           <Geometry>M3GMhubs.oni</Geometry>
           <Animation>OBANhubs.oni</Animation>
           <Animation>OBANhubs_stop.oni</Animation>
       </Node>
       <Node Name="hubs_rear">
           <Geometry>M3GMhubs_rear.oni</Geometry>
           <Animation>OBANhubs_rear.oni</Animation>
           <Animation>OBANhubs_rear_stop.oni</Animation>
       </Node>
   </Node>
   </Scene>

When you load the dae in XSI you'll find the motorcycle and the lab level intro camera If you select the camera and press play you'll get to see the motorcycle into animation like you see it in game

OniSplit version: OniSplit v0.9.52

What's new:

• When a TRBS file is exported to xml the geometry is exported to separate .dae files, one .dae file for each LOD
• New -anim-body option. This allows a particular body (ONCC or TRBS) to be specified when exporting animations:

  onisplit -extract:xml out -anim-body:ONCCbarabus.oni TRAMsomething.oni

• New -recurse option for the xml exporter. Have fun :)

This extracts all linked files e.g. it starts with ONCC, goes to TRAC and then extracts all TRAMs too. (This works by replacing classes file links.)

Recursive extracted files can be re-converted recursively again.

  onisplit -extract:xml out -recurse ONCCbarabus.oni

• Changed light color in the environment importer to white (it used to be blueish)
• New -env-notxmp option. This prevents the automatic creation of TXMP files while importing the environment.
• Made -normals work when importing TRBS from xml + dae files.
• Fixed the Collada importer to work with 3DSMax exported files

OniSplit version: OniSplit v0.9.40

What's new:

• support for exporting/importing sound animations to/from xml files
• better Collada export for environment
• support for full color transparent textures (-format:bgra32 on the command line, ARGB8888 format in an xml file)
• different (hopefully better) xml export format for animations (this one is actually from 0.9.38 but since that wasn't mentioned here...)
• a more or less complete animation importer. This one deservers some notes:

-unlike other importers that produce .oni files this one produces and .xml file (similar to the one you get when exporting a TRAM)

when you do

 onisplit -create:tram target_dir animation.dae

in the target dir you'll get a TRAManimation.xml file.

You need to add some stuff to that file to make it actually work as an animation. In particular the animation type, from/to states and varient needs to be set.

-For all I know this works with animations exported from Oni and modified in Softimage. If you come up with a completly new animation it should work as long as the skeleton is similar to the one used in Oni.

-Note that the geometry that is present inside the Collada file is used to compute the "vertical extents" so it better be the same or close to the one the animation is intended for.

-The biggest problem are the attacks. While it's not difficult to add attacks to the xml file, computing the necessary "extents" is not going to be easy. I guess in the end I'll have to add some command to OniSplit to do it.

-Everything else that I forgot :)

Neo

New OniSplit version: OniSplit v0.9.37

What's new:

• support for transparency in the environment importer

New OniSplit version: OniSplit v0.9.35

What's new:

• conversion of recorded films (.dat binary files) to xml files that can be used to create FILM .oni files

   OniSplit film2xml out_dir film.dat

New OniSplit version: OniSplit v0.9.34

What's new:

• SNDD importer

-WAV files (.wav, mono/stereo, 22.05KHz/44.1KHz, uncompressed(PCM)/compressed(MS-ADPCM)) produce SNDD files that are compatible with Oni PC retail.

-AIFC files (.aif/.aifc/.afc, mono/stereo 22.05KHz, compressed(ima4)) produce SNDD files that are compatible with Oni Mac.

Example

 OniSplit -create out_dir test.aif
 OniSplit -create out_dir test.wav

• LOD support for creating TRBS files. This can be done by creating an xml file containing the following:

 <?xml version="1.0" encoding="utf-8"?>
 <Oni Version="0.9.29.0">
     <Instance id="0" type="TRBS">
         <Elements>
             <Link>barabus_body_1.dae</Link>
             <Link>barabus_body_2.dae</Link>
             <Link>barabus_body_3.dae</Link>
             <Link>barabus_body_4.dae</Link>
             <Link>barabus_body_5.dae</Link>
         </Elements>
     </Instance>
 </Oni>

and running the command (assuming the created xml file's name is barabus_body.xml):

 OniSplit -create out_dir barabus_body.xml

It's not strictly necessary to create 5 different geometries for each LOD. The following works just as well:

 <?xml version="1.0" encoding="utf-8"?>
 <Oni Version="0.9.29.0">
     <Instance id="0" type="TRBS">
         <Elements>
             <Link>barabus_body_1.dae</Link>
             <Link>barabus_body_2.dae</Link>
             <Link>barabus_body_2.dae</Link>
             <Link>barabus_body_2.dae</Link>
             <Link>barabus_body_3.dae</Link>
         </Elements>
     </Instance>
 </Oni>

• An xml file can contain "links" to other xml/obj/dae files. For example you can have the following line in an ONWC xml file:

 <Geometry>pistol.obj</Geometry>

Assuming the file pistol.obj exists in the same directory an M3GM .oni file will be automatically created from it.

Relative paths work just as well:

 <Geometry>geometry/pistol.obj</Geometry>

• The -create:subt, -create:txmp and -create:m3gm are sort of obsolete. They still work but now you can simply use '-create' (or just 'create'):

 OniSplit -create out_dir crate.dae
 OniSplit create out_dir -format:bgr555 -genmipmaps pic.tga
 OniSplit create out_dir subtitles.txt

• Work in progress: the AKEV importer now reads Collada materials so the resulting AKEV is textured.

Sample levels:

TestLevel1 -- This level should look like this in-game: Image 1 Image 2 Image 3

TestLevel2 -- This level should look like this in-game: Image 1

A zip file contains the minimum needed to get a new level running in Oni. To "compile" a level extract it to a folder and run the following commands:

 OniSplit -create out -genmipmaps -format:dxt1 *.xml
 OniSplit -import:nosep . Oni\GameDataFolder\level1_Final.dat

(Of course, you need to change the output .dat file path to match your Oni installation path)

Note1: The hexagon level needs to be scaled up to work properly. Use the envscale option for this:

 OniSplit -create out -genmipmaps -format:dxt1 -envscale:40 *.xml

Note2: I've updated the level files to contain 20 empty corpses to prevent crashes.

If OniSplit refuses to be used

delete file at AE/Tools/OniSplit.exe.config