Talk:OniSplit: Difference between revisions

From OniGalore
Jump to navigation Jump to search
m (→‎OniSplit 1.0 release notes: updated nightly (only slight changes so far))
(moved change log to new subpage of OniSplit)
Line 1: Line 1:
Talk page archives: [[/Archive1|#1]]
Talk page archives: [[/Archive1|#1]]
==OniSplit 1.0 release notes==
::Really? Yes, really. Get the nightly [http://geyser.oni2.net/OniSplit/1.0a2_2021.12.17/ HERE] (1.0a2, 2021/12/17)
:&nbsp;&nbsp;0.&nbsp;'''''Possible introduction of aliases''''' for the notoriously confusing '''-export''' and '''-import''' (easily mistaken for the '''-extract'''/'''-create''' functionality).<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Historical synonyms for '''-export''' and '''-import''' are "unpack" and "pack", but I am leaning towards new aliases, most probably '''-split''' and '''-link'''.<br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OniSplit itself may be rebranded as ''GameDataTool'', and a lightweight app (limited to the split/link functionality) may be dubbed ''(Oni)SpLink''.
#'''''The required .NET framework''''' is changed back to .NET 2.0 (a few past versions of OniSplit were built for .NET 4.0 for some reason, but it appears that 2.0 is sufficient).
#'''''A new packing format''''' called '''-import:onix''' was added, which is the same as '''-import:sep''' (PC demo-like) but suppresses template checksums and replaces them with a new versioning system.<br/>The produced .dat/.raw/.sep files are marked VR33 instead of VR31, only work with the OniX engine, and allow for new features in the game data format - provided that the OniX engine implements their support as well.
#The misguided implementation of '''''32-bit transparent textures''''' has been remastered, with a distinction between "rgba" and "bgra32" import formats: "rgba" corresponds to TXMP type 11 (directly supported by all Oni engines); "bgra32" is TXMP type 7, mistakenly used by previous versions of OniSplit (and Daodan/Mac patches) as a substitute for the overlooked type 11.<br/>Old instances of type 7 TXMPs are automatically detected and repaired (replaced with type 11) at the first opportunity.<br/>New TXMPs created with '''-format:bgra32''' are stored as legitimate type 7, which will only be handled properly by the new OniX engine (or by an amended Daodan DLL).
#'''''Sound conversion''''' is now (almost) fully featured, although with a strong bias towards the Windows platform (a.k.a. "PC") and MS ADPCM compression.<br/>In addition to the previously available stream copying, you can now '''-extract:wav''' from a Mac SNDD ('''-extract:aif''' from a PC SNDD may be added later).<br/>When using '''-extract:wav''', an additional option '''-pcm''' decompresses the waveform and produces a .wav file with raw PCM storage.<br/>Compressed ADPCM .wav-files (exported through '''-extract:wav''') are now standard-compliant, and the importing of a .wav preserves custom (AD)PCM settings fully as well, meaning that third-party WAV files can now be roundtripped in and out of Oni (PC retail) without any issues. (However, the validation of corrupt third-party WAVs has not been thoroughly implemented or tested.)<br/>In addition to the automatic creation of PC or Mac SNDDs through the '''-create''' command (depending on whether the input is a .wav or an .aif file), the '''-demo''' option can be used to generate a PC demo-like SNDD (short header, no custom sample rate or compression) both from a .wav and from an .aif file (useful when mass-converting sounds for VR33/OniX).<br/>By default all SNDDs made with '''-create''' are ADPCM-compressed SNDDs even if the input is an uncompressed .wav file. The '''-pcm''' option allows uncompressed/decompressed storage for PC SNDDs, but this is not recommended: PCM sounds take up 4 times as much space as ADPCM, and their playback is currently broken for the PC demo and OniX engines.<br/>Considering that, as of now, 18:55, 14 December 2021 (CET), all Oni engines play back SNDDs as 22.05 kHz waveforms (the PC retail engine only ''pretends'' to support custom sample rates, see [[OBD:SNDD#Known_engine_issues|HERE]]), it is now OniSplit's duty to report any waveforms that are ''not'' sampled at 22.05 kHz and to suggest the custom playback speed that should be used at [[OBD:OSBD/OSGr|OSGr]] level. 44.1 kHz sounds will automatically be downsampled to 22.05 kHz except when doing so would involve an additional round of decoding and reencoding; this potentially lossy operation must be explicitly requested with '''-forcestd'''.<br/>When creating short-header SNDDs for the PC demo/OniX (using the '''-demo''' tag), incoming MS ADPCM files are ultimately required to have a standard block size of 512 bytes per channel. Here too, a potentially lossy reencoding must be explicitly requested through '''-forcestd''', otherwise importing will fail with a report of the non-conformant block size.<br/>Uncompressed PCM sounds can be imported from .wav, with support for the following bit depths: 16-bit (CD-quality), 24-bit (overkill) and 32-bit (super-overkill). Bit depth reduction is straightforward, practically non-lossy and therefore automatic, with merely a warning printed after the conversion. 8-bit linear PCM input (unsigned) is also supported, but don't tell anyone.
#'''''[[OBD:SUBT|Subtitle]] files now have a custom XML format''''' as requested by Script10k. You go from SUBT*.oni to XML using '''-extract:xml''', and you import using '''-create''' (both the old .txt subtitles and the new XML format are accepted as input). Vanilla English SUBTs roundtrip exactly both through TXT and through XML. Non-Vanilla SUBTs and other language versions not tested. 
#The '''-fullname''' option is available for '''-extract:dae''' operations, '''''prepending the 4-character template tag to the filename''''' (TRBS, ONCC, AKEV, etc).
#'''Level creation works again.''' The <Import> feature of Physics.xml has been repaired (it is the feature that lets you import several OBOA entries and their OBANs from the same Collada file) and enhanced with a couple of options, like explicit script IDs for each animated object and custom prefixes/suffixes for animations. As a minor convenience, all _marker textures are now imported automatically.
#'''Interaction with Blender has been consolidated''' through the '''-blender''' option, available both when '''-extract'''ing and '''-create'''ing.<br/>The primary use of '''-blender''' is to control the transition between Oni's and Blender's axis conventions, both in terms of scene orientation (Y-up for Oni, Z-up for Blender) and rotation order (X-then-Y-then-Z for Oni, Z-then-Y-then-X for Blender).<br/>Another effect of '''-blender''', exclusive to '''-extract''', is to replace all instance of <technique_common> with <technique sid="common"> in the exported Collada files. When importing a Collada file through '''-create''', both <technique_common> and <technique sid="common"> are accepted.
#'''Characters/animations have been consolidated.''' Among minor improvements, quaternion math is more accurate (preventing "upside-down" pelvis animation and other artifacts), the identification of animation curves in Collada input is more robust, and OniSplit's internal animation buffer (for raw TRAM data) is no longer limited to 65535 bytes.<br/>A new '''-keepkeys''' option is available for TRAM operations: for '''-extract''' it preserves Oni's sparse animation data instead of keying every frame; for '''-create''' it bypasses the keyframe reduction pass and uses the same keys as in the Collada file.<br/>When '''-create'''ing a TRBS, non-standard skeleton hierarchy is still allowed (controlled by the node tree in the Collada file), but standard sibling order is enforced for "thigh" and "shoulder" bones (if detected).<br/>At this point there is still no proper skeletal setup (meaning it's still the same "nested" approach, with rigid body parts hinged to one another), and the rest pose is still the infamous "folded umbrella". A more intuitive rest pose may be implemented later.
:At the time of writing, items (1) through (7) are done and ready for testing, whereas (8) and (9) are being finalized. The rebranding (0) is merely planned (subject to debate?). --[[User:Geyser|geyser]] ([[User talk:Geyser|talk]]) 13:11, 17 December 2021 (CET)


==OniSplit version control - getting there...==
==OniSplit version control - getting there...==
Line 32: Line 19:


==TGA Transparency bug==
==TGA Transparency bug==
I extracted HQ_DOUBLED_GLASS as TGA format from vanilla Level8_Final with the OniSplit command -extract:tga. The file was opened in Photoshop and the alpha channel was visible.
I extracted HQ_DOUBLED_GLASS as TGA format from vanilla Level8_Final with the OniSplit command -extract:tga. The file was opened in Photoshop and the alpha channel was visible.
I then added that texture to Sketchup and applied it in a test level, it appeared transparent in Sketchup. I used the command -create:level to create a level in Oni.  In the game Oni, the texture appeared opaque not transparent.  When, the file TXMPHQ_DOUBLED_GLASS.oni was extracted from the test level as TGA, the alpha channel was gone or opaque.  
I then added that texture to Sketchup and applied it in a test level, it appeared transparent in Sketchup. I used the command -create:level to create a level in Oni.  In the game Oni, the texture appeared opaque not transparent.  When, the file TXMPHQ_DOUBLED_GLASS.oni was extracted from the test level as TGA, the alpha channel was gone or opaque.  
Line 75: Line 61:


:Yes, when a Divhide section is set to start hidden, it throws off the browser's ability to jump to the right spot. I've never been sure whether to have that section auto-show or auto-hide. Now that it's auto-showing, you'll see that the anchor works. --[[User:Iritscen|Iritscen]] ([[User talk:Iritscen|talk]]) 12:50, 12 March 2014 (CET)
:Yes, when a Divhide section is set to start hidden, it throws off the browser's ability to jump to the right spot. I've never been sure whether to have that section auto-show or auto-hide. Now that it's auto-showing, you'll see that the anchor works. --[[User:Iritscen|Iritscen]] ([[User talk:Iritscen|talk]]) 12:50, 12 March 2014 (CET)
==Change Log ==
OniSplit version: OniSplit v0.9.90
What's new:
:* Fixes bugs with Import/Export of M3GMs (obj files).
:* Fixes euler angle issues with exported TRAMs.
:* Fixes -noanim command to produce a standing pose.
:* In the ONWC file, you should be able to reference the dae/obj file directly then you don't need the separate M3GM.
:* When exporting AKEV, quads that have script ids assigned to them are exported to separate files.  This is related to BSL commands such as env_show 540 0
:* Adds option -anim-merge to export multiple animations to a single dae. Useful in dealing with walk/run or combo animations.
  onisplit -extract:xml <destination directory>  -anim-merge -anim-body:src\Path_To_\TRBSkonoko_body_high.oni Path_To\TRAMKONCOMcomb_k.oni  Path_To\TRAMKONCOMcomb_k_k.oni Path_To\TRAMKONCOMcomb_k_k_k.oni
:* Adds a scene exporter
  OniSplit -extract:dae <destination directory> scene.xml
For example if the scene.xml file looks like this
  <Scene>
  <Node Name="camera">
        <Camera />
        <Animation>OBANCamout01.oni</Animation>
        <Animation>OBANCamout02.oni</Animation>
        <Animation>OBANCamout03.oni</Animation>
        <Animation>OBANCamout04.oni</Animation>
        <Animation>OBANCamout05.oni</Animation>
        <Animation>OBANCamout06.oni</Animation>
    </Node>
    <Node Name="motorcycle02">
        <Geometry>M3GMmotorcycle02.oni</Geometry>
        <Animation>OBANmotorcycle02.oni</Animation>
        <Animation>OBANmotorcycle02_stop.oni</Animation>
        <Node Name="hubs">
            <Geometry>M3GMhubs.oni</Geometry>
            <Animation>OBANhubs.oni</Animation>
            <Animation>OBANhubs_stop.oni</Animation>
        </Node>
        <Node Name="hubs_rear">
            <Geometry>M3GMhubs_rear.oni</Geometry>
            <Animation>OBANhubs_rear.oni</Animation>
            <Animation>OBANhubs_rear_stop.oni</Animation>
        </Node>
    </Node>
    </Scene>
When you load the dae in XSI you'll find the motorcycle and the lab level intro camera
If you select the camera and press play you'll get to see the motorcycle into animation like you see it in game
OniSplit version: OniSplit v0.9.52
What's new:
:* When a TRBS file is exported to xml the geometry is exported to separate .dae files, one .dae file for each LOD
:* New -anim-body option. This allows a particular body (ONCC or TRBS) to be specified when exporting animations:
  onisplit -extract:xml out -anim-body:ONCCbarabus.oni TRAMsomething.oni
:* New -recurse option for the xml exporter. Have fun :)
:: This extracts all linked files e.g. it starts with ONCC, goes to TRAC and then extracts all TRAMs too. (This works by replacing classes file links.)
:: Recursive extracted files can be re-converted recursively again.
  onisplit -extract:xml out -recurse ONCCbarabus.oni
:* Changed light color in the environment importer to white (it used to be blueish)
:* New -env-notxmp option. This prevents the automatic creation of TXMP files while importing the environment.
:* Made -normals work when importing TRBS from xml + dae files.
:* Fixed the Collada importer to work with 3DSMax exported files
OniSplit version: OniSplit v0.9.40
What's new:
:* support for exporting/importing [[OBD:BINA/SABD|sound animations]] to/from xml files
:* better Collada export for environment
:* support for full color transparent textures (-format:bgra32 on the command line, ARGB8888 format in an xml file)
:* different (hopefully better) xml export format for animations (this one is actually from 0.9.38 but since that wasn't mentioned here...)
:* a more or less complete animation importer. This one deservers some notes:
::-unlike other importers that produce .oni files this one produces and .xml file (similar to the one you get when exporting a TRAM)
::when you do
  onisplit -create:tram target_dir animation.dae
::in the target dir you'll get a TRAManimation.xml file.
::You need to add some stuff to that file to make it actually work as an animation. In particular the animation type, from/to states and varient needs to be set.
::-For all I know this works with animations exported from Oni and modified in Softimage. If you come up with a completly new animation it should work as long as the skeleton is similar to the one used in Oni.
::-Note that the geometry that is present inside the Collada file is used to compute the "vertical extents" so it better be the same or close to the one the animation is intended for.
::-The biggest problem are the attacks. While it's not difficult to add attacks to the xml file, computing the necessary "extents" is not going to be easy. I guess in the end I'll have to add some command to OniSplit to do it.
::-Everything else that I forgot :)
[[User:Neo|Neo]]
New OniSplit version: [http://cid-639aa31296681bfe.skydrive.live.com/self.aspx/Oni/OniSplit/OniSplit|_v0.9.37.zip OniSplit v0.9.37]
What's new:
:* support for transparency in the environment importer
New OniSplit version: OniSplit v0.9.35
What's new:
:* conversion of recorded films (.dat binary files) to xml files that can be used to create FILM .oni files
    OniSplit film2xml out_dir film.dat
New OniSplit version: OniSplit v0.9.34
What's new:
:* SNDD importer
::-WAV files (.wav, mono/stereo, 22.05KHz/44.1KHz, uncompressed(PCM)/compressed(MS-ADPCM)) produce SNDD files that are compatible with Oni PC retail.
::-AIFC files (.aif/.aifc/.afc, mono/stereo 22.05KHz, compressed(ima4)) produce SNDD files that are compatible with Oni Mac.
::Example
 
  OniSplit -create out_dir test.aif
  OniSplit -create out_dir test.wav
:*LOD support for creating TRBS files. This can be done by creating an xml file containing the following:
  <?xml version="1.0" encoding="utf-8"?>
  <Oni Version="0.9.29.0">
      <Instance id="0" type="TRBS">
          <Elements>
              <Link>barabus_body_1.dae</Link>
              <Link>barabus_body_2.dae</Link>
              <Link>barabus_body_3.dae</Link>
              <Link>barabus_body_4.dae</Link>
              <Link>barabus_body_5.dae</Link>
          </Elements>
      </Instance>
  </Oni>
::and running the command (assuming the created xml file's name is barabus_body.xml):
  OniSplit -create out_dir barabus_body.xml
::It's not strictly necessary to create 5 different geometries for each LOD. The following works just as well:
  <?xml version="1.0" encoding="utf-8"?>
  <Oni Version="0.9.29.0">
      <Instance id="0" type="TRBS">
          <Elements>
              <Link>barabus_body_1.dae</Link>
              <Link>barabus_body_2.dae</Link>
              <Link>barabus_body_2.dae</Link>
              <Link>barabus_body_2.dae</Link>
              <Link>barabus_body_3.dae</Link>
          </Elements>
      </Instance>
  </Oni>
:*An xml file can contain "links" to other xml/obj/dae files. For example you can have the following line in an ONWC xml file:
  <Geometry>pistol.obj</Geometry>
::Assuming the file pistol.obj exists in the same directory an M3GM .oni file will be automatically created from it.
::Relative paths work just as well:
  <Geometry>geometry/pistol.obj</Geometry>
:*The -create:subt, -create:txmp and -create:m3gm are sort of obsolete. They still work but now you can simply use '-create' (or just 'create'):
  OniSplit -create out_dir crate.dae
  OniSplit create out_dir -format:bgr555 -genmipmaps pic.tga
  OniSplit create out_dir subtitles.txt
:*Work in progress: the AKEV importer now reads Collada materials so the resulting AKEV is textured.
:Sample levels:
:[http://cid-639aa31296681bfe.skydrive.live.com/self.aspx/Oni/noglass.zip TestLevel1] -- This level should look like this in-game: <u>[[:Image:NoGlass Import Test 1.jpg|Image 1]]</u> <u>[[:Image:NoGlass Import Test 2.jpg|Image 2]]</u> <u>[[:Image:NoGlass Import Test 3.jpg|Image 3]]</u>
:[http://cid-639aa31296681bfe.skydrive.live.com/self.aspx/Oni/hexagon.zip TestLevel2] -- This level should look like this in-game: <u>[http://edt.oni2.net/images/hexagon1.jpg Image 1]</u>
:A zip file contains the minimum needed to get a new level running in Oni. To "compile" a level extract it to a folder and run the following commands:
  OniSplit -create out -genmipmaps -format:dxt1 *.xml
  OniSplit -import:nosep . Oni\GameDataFolder\level1_Final.dat
:(Of course, you need to change the output .dat file path to match your Oni installation path)
:Note1: The hexagon level needs to be scaled up to work properly. Use the envscale option for this:
  OniSplit -create out -genmipmaps -format:dxt1 -envscale:40 *.xml
:Note2: I've updated the level files to contain 20 empty corpses to prevent crashes.


==If OniSplit refuses to be used==
==If OniSplit refuses to be used==
delete file at AE/Tools/OniSplit.exe.config
delete file at AE/Tools/OniSplit.exe.config --[[User:Paradox-01|Paradox-01]] 18:00, 28 June 2020

Revision as of 22:35, 18 December 2021

Talk page archives: #1

OniSplit version control - getting there...

The SVN version of OniSplit is currently labeled 0.9.99 (same as the source received from Neo), and at this point it is different from the non-synced 0.9.99.2 that I distributed through EdT.

  • "0.9.99" has only two commits (44.1 kHz SNDD export - affecting only electric spark sounds - and other minor SNDD fixes and features);
  • 0.9.99.2 has a few fixes to 0.9.99 (adjustment to XML parsing rules) and new functionality for Blender import/export of animations; also some new bugs, apparently.

To restore some order, I am going to merge my 0.9.99.2 fixes to SVN, then address the known bugs, and push the version number to 0.9.99.3 - and use SVN-synced development from then onwards.

I am not going to create a branch for my fixes/improvements, except maybe when it comes to adding something big, like C# FBX support. If you object to this, speak now. --geyser (talk) 00:09, 29 May 2020 (CEST)

List of bugfixes and improvements planned for 0.9.99.3 (feel free to add to the list if I forgot something)

  • some running/walking anims end up upside-down after reimported into Oni
  • -extract:dae -noanim -blender used on a TRBS, make sure the output .dae file has a TRBS prefix
  • stairs_markers.dae not imported properly? (and more generally secondary/auxiliary .dae files not imported for levels?)
  • TRAM export: support writing of sparse keyframe data (as in Oni), as opposed to writing rotations for every frame (interpolated)
  • maybe allow the export of named TRCM (without material)
  • TGA transparency bug? (not-yet-confirmed as a bug; supposedly happens only for SketchUp-made levels; see below)
  • various SNDD glitches (inconsistent handling of ADPCM headers)

TGA Transparency bug

I extracted HQ_DOUBLED_GLASS as TGA format from vanilla Level8_Final with the OniSplit command -extract:tga. The file was opened in Photoshop and the alpha channel was visible. I then added that texture to Sketchup and applied it in a test level, it appeared transparent in Sketchup. I used the command -create:level to create a level in Oni. In the game Oni, the texture appeared opaque not transparent. When, the file TXMPHQ_DOUBLED_GLASS.oni was extracted from the test level as TGA, the alpha channel was gone or opaque.

Even if the following code was added to the textures.xml file, the texture still appeared opaque in game. 

   <Texture Name="HQ_DOUBLED_GLASS">
       <GunkFlags>Transparent TwoSided</GunkFlags>
       <Format>RGBA</Format>
       <Image>images/HQ_DOUBLED_GLASS.tga</Image>
   </Texture>

However, if HQ_DOUBLED_GLASS.tga is converted to PNG format with alpha channel, and then the PNG file is used in Sketchup, then the texture will be transparent in game.

This occurs using OniSplit version 0.9.96.0 and previous versions, tested to version 0.9.90.0. (EDIT: Transparency loss is observed for all glass textures, not just HQ_DOUBLED_GLASS.)

EdT 15:03, 14 May 2020 (PDST)

When you described the issue to me in PM, I didn't notice/understand the key part about applying the texture in SketchUp and then using -create:level on SketchUp's output. I thought we were talking about a simple TXMP roundtrip through TGA. Can you confirm that the texture's alpha goes opaque (or disappears) if you use -extract:tga on TXMPHQ_DOUBLED_GLASS.oni, and then immediately reimport it either with -create (either from XML with extra flags or from TGA)? --geyser (talk) 11:44, 15 May 2020 (CEST)

Regarding the TXMP roundtrip, after extracting TXMPHQ_DOUBLED_GLASS.oni as TGA, I used the command -create:txmp -format:bgra32 -large -genmipmaps, the resulting file had the alpha channel. To confirm, I replaced the file TXMPHQ_DOUBLED_GLASS.oni from the -create:level with the one from -create:txmp in the test level and it was transparent. EdT 12:35, 15 May 2020 (PDST)

This looks like an issue with the SketchUp pipeline, rather than with OniSplit, since it is only -create:level that generates a non-transparent texture, and -create:txmp actually works fine. Can you: a) provide (in PM) the level data that you are using -create:level on (supposedly a .dae file, some texture files, and some XMLs)? b) provide an example of a level that imports with transparent glass as expected (supposedly a level that didn't come from SketchUp). Thanks. --geyser (talk) 16:46, 16 May 2020 (CEST)

Sound export bug

As discussed on Discord there's a sound export bug for SNDDzap*.oni

One might record the sounds in OBS and compare it with the exported ones to generate further hints on what went wrong. --paradox-01 (talk) 13:29, 21 July 2019 (CEST)

  1. The zap sounds are randomized in an OSBD group, so recording them from the game in some identifiable way would require some extra work (custom OSBD). It's much easier to ask Mac folks for AIFF versions of those sounds. And actually, there are seven zap sounds in the PC demo as well.
  2. OniSplit slaps a 22.05 kHz header on those files, although they're 44.1kHz and have a perfectly good 44.1kHz header in Oni. Not sure why they're getting a 22.05kHz header. Perhaps Neo got confused by the sloppy documentation of the WAV header, which until my edit gave the same hex listing for the three WAV header types. Either that, or he just assumes the sample rate to be 22.05kHz by default and doesn't update it from the actual file. Will check in the code. Actually yeah, he just does as if all the sounds were 22.05kHz.
  3. Besides SNDDzap*.oni, there is one other 44.1kHz sound in PC Oni, SNDDap_hit_shld.aif; which suffers from the same export problem, although it's less noticeable.
  4. Apparently 22.05 kHz stereo sounds are correctly exported as stereo, it's just 44.1 kHz mono files that get a wrong WAV header.
  5. Will be fixing this in the latest "nightly" OniSplit (haven't yet decided on a version numbering and source control scheme).
    geyser (talk)

Page improvement

Try to click on OniSplit#CLI_tips

The section should be made to pop up, currently it doesn't. --Paradox-01

Yes, when a Divhide section is set to start hidden, it throws off the browser's ability to jump to the right spot. I've never been sure whether to have that section auto-show or auto-hide. Now that it's auto-showing, you'll see that the anchor works. --Iritscen (talk) 12:50, 12 March 2014 (CET)

If OniSplit refuses to be used

delete file at AE/Tools/OniSplit.exe.config --Paradox-01 18:00, 28 June 2020

Retrieved from "https://wiki.oni2.net/w/index.php?title=Talk:OniSplit&oldid=34860"