Talk:Mod Tool/OniTools addon

From OniGalore
Jump to: navigation, search

Notes on usage

master xml file

How a healthy master file should look like:

<?xml version="1.0" encoding="utf-8"?>
<Oni>
 <Level SharedPath="../shared">
   <Environment>
     <Model>
       <Import Path="levelN_Final/alpha_helix_env.dae" />
       <Import Path="levelN_Final/alpha_helix_env_markers.dae" />
     </Model>
     <Rooms>
       <Import Path="levelN_Final/alpha_helix_bnv.dae" />
     </Rooms>
     <Textures Format="BGR" AlphaFormat="RGBA" MaxSize="512"></Textures>
   </Environment>
   <Sky>sunset</Sky>
   <Objects lsi="torch">
     <Import>levelN_Final/BINACJBOCharacter.xml</Import>
     <Import>levelN_Final/BINACJBOPowerup.xml</Import>
     <Import>levelN_Final/BINACJBOTurret.xml</Import>
     <Import>levelN_Final/BINACJBOConsole.xml</Import>
   </Objects>
   <Films></Films>
   <Cameras></Cameras>
 </Level>
</Oni>

GUI generated master files come with a "level specific item" (LSI) attribute in the <Objects> tag.

The tag is used by the GUI to copy the correct LSI files into the output folder where OniSplit generates all other level files for the "Final" archive or plugin.

Note that you can use only one type of LSI per level.


Dev notes for the remake

alpha helix

Mod Tool Oni level rebuilder logo.jpg

The original plan was to do all relevant stuff with Mod Tool scripts.

But having another program in between ease some shortcomings of MT such as its capability of creating GUIs/controls.

Currently working on: update #3


Update management

GUI

Starting with 14.01.2017 the GUI is distributed via AEI.

Shared vanilla classes

These zipped resources are about 2 MB and for now they will ship embedded inside the GUI. After being written to separate files the oni files get converted to dae and xml.

Shared additional classes

These zipped resources aren't yet available. They are going to be downloaded by the GUI.

There are supposed to be many files in that zip. Due to copyright concerns the files are encrypted and will be made available only if Oni retail resources are found on client computer.

XSI addon

Normally you have to install a *.xsiaddon file by hand but it seems that is just gets unzipped to another folder. So that can be automated.

Also, the scripts don't need much space so they will ship embedded inside the GUI.

Location of used *.xsiaddon files
C:\Users\UserName\Autodesk\Softimage_Mod_Tool_7.5\Addons\InstalledAddons


Location of unzipped addons
C:\Users\UserName\Autodesk\Softimage_Mod_Tool_7.5\Addons

Addon name folder
 |
 +-- Application
       |
       +-- Plugins
            |
            +-- *.vbs

We only need the GUI to figure out where the folders are.

The welcome screen triggers an automatic search. The user either confirms or changes the information.

Known issues

  • level projects cannot migrate from original release to update #1
Folder naming rule was changed from numbered levels to "levelN_Final".
This avoids the need to update all links in master file if the level number changes.
You would have to rename your local level and update the master file by hand.
  • self-closing PPG leaves a blank page Fixed in #2. DeleteObj "..." and PPG.Close need to be embedded within oPPGLayout.Logic
  • GUI pops up when you save the console from XSI. (Cause is unexpected hard to track down.)
  • XSI cannot brought to front by GUI when it is minimized. Fixed in #2
  • unstable updating of shared classes Fixed in #2 by renaming old folder. That way custom made changes don't get lost however still need to be moved by hand.
  • quick test function in xsi produces level archives of name "levelN_Final" Fixed in #2. N is now an actual number.


Detailed checklist


General checklist of addon and GUI

estimated time
XSI addon features (1y)
small GUI features and editors (1y)
BSL and XML editor (1y)
fine-tuning (v2), bug-fixes, refactoring (11mos.)
demo-level with prone-mode (1mo.)
wth ...


convert level dat to project

  1. create project by AKEV name [x]
  2. create master file [x]
  3. copy BSL folder []
  4. populate master file with object collection [x]
  5. populate master file with films [x]
  6. populate master file with cameras [] (not all OBAN are cams, have to examine BSL)
  7. get sky name [x]
  8. connect master with shared classes [x]
  9. transfer ONLV door locklights to particle collection [x]
  10. transfer animated objects to physics file [x]
  11. populate master file with "script" geometry [x] (those are named LevelName_script_scriptId.dae, static parts of animated objects)
In simplest case you can create and run your first level after two clicks. Konoko and a sunset sky is set by default and will be immediately displayed.


create level project from scratch

  1. create project folder [x]
  2. create bsl folder [x]
  3. create ONLD (for level number and savegame name) [x]
  4. create master file [x]
  5. set default sky for fast tests [x]
  6. set default player for fast tests [x]
  7. set default ground and PF for fast tests [x]
  8. set default texture quality [x]
  9. set path to classes folder in master file [x]
  10. add "quick test" function [x] (finalizes the level and starts Oni)


level finalization

  1. patch final ONLV physics with OFGA particles [] (Physics.xml is equivalent to OBOA except it imports *.dae instead of OFGA*.oni)
  2. check dependencies []
  3. check for deco (TXMB) []
  4. check for logic (BSL main func) []
  5. check for package information []
  6. convert project to package []


interaction

  1. xsi user cam movement is monitored by GUI []
  2. GUI plays sounds when cam is within range of sound spheres in xsi []
difficulty: medium


file management and editors

  1. detect Oni retail* version []
  2. provide normal shared classes collection for everyone [x]
  3. provide extended shared classes collection for retail Oni version []
  4. install file watcher for classes folder []
  5. creating classes in xsi auto-updates html object library []
  6. decal particle editor []
  7. xml editor []
  8. populate xml editor with auto-updating classes cache []
  9. BSL editor []
  10. populate BSL editor with auto-updating classes cache []
  11. load script functions of TV, CONS, DOOR, TRIG, TURR on demand []
difficulty: medium - high
* GUI searches for a few binary signatures that are only present in retail version. The signatures are hashed so that decompiling the GUI won't reveal the actual search strings. The downloaded file is then automatically decrypted.


convenient features

  1. if GUI is located in AE tools' folder (e.g. via pkg install) it will choose those AE resources by default [x]
  2. release GUI as AE package []
  3. improve project handling when you have more than one []
  4. option to transfer AISA content to CHAR
  5. setup XSI for Oni modding (non-transparent CHAR tex, frame rate) []
  6. get package numbers from Oni Mod Depot []
  7. integrate TRAM assistant []
  8. SNDD assistant []
  9. pages assistant []
  10. fly-in image assistant (for BSL) []
  11. file association xml []
  12. file association oni []
  13. file association bsl []
  14. file association dat []
  15. gunk flag recovery for special textures (ONLV's ONOA and AKEV's AGQG and AKEV's texture list must be read for this) [] (laborious)
  16. take OniSplit conversion into background worker and add a progress bar []
difficulty: high


addon

  1. read, write, edit CHAR [][][]
  2. read, write, edit CONS [x][x][x]
  3. read, write, edit DOOR [][][]
  4. easy option to create door frames [] (2d walls require a bit more work)
  5. read, write, edit ENV markers [][][]
  6. read, write, edit FLAG [][][]
  7. read, write, edit FURN [][] (original FURN is part of AKEV core geometry)
  8. read, write, edit NEUT [][][]
  9. read, write, edit PART [][][]
  10. read, write, edit PATR [][][]
  11. read, write, edit PWRU [][][]
  12. read, write, edit SNDG [][][]
  13. read, write, edit TRIG [][][]
  14. read, write, edit TRGV [][][]
  15. read, write, edit TURR [x][x][x]
  16. read, write, edit WEAP [][][]
  17. read, write, edit CONS class [][][] ?
  18. read, write, edit DOOR class [][][] ?
  19. read, write, edit decal class [][][] ?
  20. read, write, edit TURR class [][][] ?
  21. read, write, edit TRIG class [][][] ?
  22. read, write, edit WEAP class [][][] ?
  23. xsi vertex shading editor (with easy controls for light sources) []
  24. add new corpses []
  25. material-depending polygon deletion []
  26. create cameras from dae []


features that are unlikely to happen - so this is kind of a wishlist

  1. edit old corpses
  2. FILM creation within MT
  3. automatic sound creation
  4. full-featured particle editor
  5. CHAR rendering in Blender
  6. file context menu xml (multiple options)
  7. folder context menu (built dat)
  8. use only one Onisplit process to speed up conversions ?
  9. unlock savegame slot that is required for own tests but has never been used so far


It seems CRSA can be re-created in Mod Tool after all if you break the character's bone hierarchy before applying the rotation values.

This needs more testing but for now I got slightly optimistic that it works this time.





New code snippets

Latest MT addon attempt. Reducing complexity by dropping support for adding AE resources. Vanilla material only.

Download link to the "shared" collection should only provided if Oni retail installation is found on client machine.

Dragging and dropping items from Oni Object Library into MT works with Firefox but not Google Chrome. Have to look into this.

<a href="characters/ONCC/ONCCbarabus.xml">
	<div class="element-item Syndicate Barabas Daodan Boss Male" id="img" style="background-image: url(characters/ONCC/ONCCbarabus.png);">
		<p class="name">barabus</p>
	</div>
 </a>
Oni Object Library, character page

The relative path of the xml is transformed into an absolute path when dropped into MT.

If the link doesn't begin with file:/// MT will add it. I removed it from html file so xml can be clicked and viewed.

Scratching together...

x = done
- = not needed
- characters consoles doors env markers furniture ghost quads powerups triggers turrets weapons
shared (collection) x x x x x - x x x x
OniToDae + Html html - html x x html
preview imgs x x - x x x
DnD recognition x - x x x
layer support x x x x x
PPG interface x x - x -
save in project
PrjToXml

more ... later

flags, patrol path, tvs, crsa, sound, script objects (spawnable geometry)

Patrol paths are the trickiest. There's no easy way to show the connection from flag to another. It might be possible with multiple chains used for rigging but that's not very comfortable to deal with.

Idea: add ID property to each used flag. When a flag is clicked, a PPG opens with a button to show flag connections (one lose chain with many bones, old one is deleted). MT also opens/updates an application (have to look into writing command line apps first). I guess that's necessary to preview (and edit) the complete xml code as this would be to awkward to code directly in MT. The xml file is saved on change events. MT reads that xml when a patrol path flag is selected.

21.06.2016 Yay. Ugly PATR xml editor underway. "Full manual edits" tab already ready... took long enough. Figured out command line part. Next stop MT interaction.


Editing simultaneously multiple ghost quads heights

Onisplit-generated ghost quads vertex ids can differ sometimes. So before changing multiple ghosts in height, their lowest and highest points must be identified.

Display keywords

SetUserKeyword selection(0), "type=akev, id=1" 

for each n in GetUserKeyword(selection(0))
	logmessage n
next

SetValue "Camera.camvis.attrseluserkeywords", False
SetValue "Camera.camvis.attrseluserkeywords", True

xsi_char_save_settings.png


Display flag via ICE

Using a "Simulate Rigid Bodies" ICETree node for this is overkill. But it works and I fail to come up with alternatives, so...

abusing_ICE_for_displaying_a_flag.png

'### setup flag display by abusing ICE
CreatePrim "Cube", "MeshSurface"
ApplyOp "ICETree", selection(0), siNode, , , 0
AddICENode "$XSI_DSPRESETS\ICENodes\SimulateRigidBodiesNode.Preset", selection(0) & ".polymsh.ICETree"
ConnectICENodes selection(0) & ".polymsh.ICETree.port1", selection(0) & ".polymsh.ICETree.SimulateRigidBodies.simulate"
AddICENode "$XSI_DSPRESETS\ICENodes\IntegerNode.Preset", selection(0) & ".polymsh.ICETree"
ConnectICENodes selection(0) & ".polymsh.ICETree.SimulateRigidBodies.precision", selection(0) & ".polymsh.ICETree.IntegerNode.result"
DisplayPortValues selection(0) & ".polymsh.ICETree.SimulateRigidBodies.precision"
SetValue selection(0) & ".polymsh.ICETree.IntegerNode.value", 7
SaveKey selection(0) & ".polymsh.ICETree.IntegerNode.value", 1
'### check if ICETree op exist on this object
for each oOperator in selection(0).ActivePrimitive.ConstructionHistory
	if oOperator.Name = "ICETree" then
 		got_ICE = 1
  	end if
Next
if got_ICE then
  		logmessage "found ICETree"
else
  		logmessage "couldn't find ICETree"
end if


Dev notes for remake v2

Labor-intense improvements (aka skipped cuz of lazyness) will be probably tackled only after v1 (alpha helix) was completed.

beta helix


GUI

  • Add a "current tasks" screen.
    • use background worker threads
    • use a progress bar
  • Use separated form for txmp creation (avoid impression that GUI is only for that one file type)
  • Explore and fix issue that causes GUI to remain in front when called via command-line in XSI
  • Add html section for turret screens
  • Add html section for door decals
  • Migrate custom classes to new "shared" folder after an update
  • Unify use of internal update variables
    • v1 uses shared classes version right inside the source code
    • v1 uses xsi scripts version in the app's settings (a section provided by Visual Studio)

XSI

  • remove the deprecated exchange file tag <activeLevelNumber>


More wild ideas

FILM creation within MT

FILMs could be created with certain approximation.

Maybe we can use curves and add null objects to specific point that hold FILM properties like key presses.

Cameras would indicate the current perspective.

Exact values might require information about ONCC and starting TRAM.


Animated reference

Every step on the timeline triggers an event, namely siOnTimeChangeEvent.

It could be used to exchange an the image of an object.

This could be either good for an animated textures (BINA3RAPd*.oni) or for an animated reference for TRAM creation.

Strategy:

  • Making a folder named "Animated_Ref".
  • Drag and drop the folder into Mod Tool.
  • A grid becomes created on a new layer "Animated_Ref". Selection disabled.
  • Mod Tool gets then the first image and looks for the pixels.
  • Based on that pixels the grids becomes deformed to match the image's proportions.
  • Then this "Animated_Ref" grid should be ready for usage.


Similar can be done with the Rotoscope.

Something like that:

function siOnTimeChangeEvent_OnEvent( in_ctxt )
	' fixed code
	'SetValue "Views.ViewD.RightCamera.camdisp.rotoenable", True
	'SetValue "Views.ViewD.RightCamera.rotoscope.imagename", "Clips.Animated_Ref_N_jpg"

	' flexible code
	' find active viewport, set Rotoscope active and choose image
	'SetValue GetViewCamera(GetViewIndex) & ".camdisp.rotoenable", True
	'SetValue GetViewCamera(GetViewIndex) & ".rotoscope.imagename", "Clips.Animated_Ref_N_jpg"


	current_frame = cstr(in_ctxt.GetAttribute("Frame"))
	if GetValue(GetViewCamera(GetViewIndex) & ".camdisp.rotoenable") = true then
	   ' Animated_Ref_60.jpg ' . becomes _
	   SetValue GetViewCamera(GetViewIndex) & ".rotoscope.imagename", "Clips.Animated_Ref_" & current_frame & "_jpg"      
	end if

	siOnTimeChangeEvent_OnEvent = true
end function


New camera animations

last frame of OBANOutroCam02
camera_OBAN_examination_tn_zpsc8295474.png

There's already an excel macro available but now I want to build cam anims without excel.

The last frame's rotation of level1_Final file OBANOutroCam02 is: -0.0158410165 0.854542 0.0261086561 -0.518483639.

Those quaternions are sorted in X Y Z -W order, so we've to watch out a bit when setting the input for converting them to euler rotations in degrees.

If a newly created object (in this case a cube) shall serve as camera placeholder then the X output has to be multiplied by -1 and the Y output has to be reduced by -180. Those changes must repeated again when exporting the object's rotation to xml.

Update: Sept. 29, 2012

If camera is a real one then the additional X and Y changes aren't necessary. (Primitive > Camera > any should do)

Positions and rotations keyframes can be applied to the camera root object.


Adding multiple textures to level geometry [out-dated]

Material and textures are normally stored right under the object. But AKEV geometry can have more than one texture. Here comes polygon clusters in to play.

An example:

set oCube = Application.ActiveProject.ActiveScene.Root.AddGeometry("Cube","MeshSurface")
SelectObj "cube", , True
SelectGeometryComponents "cube.poly[LAST]"
CreateCluster

AddToCluster "cube.polymsh.cls.Polygon, cube.poly[2-4]"
RemoveFromCluster "cube.polymsh.cls.Polygon, cube.poly[3]"

The cube has now a polygon cluster. More can be added. Each cluster must get its own material before a texture can be applied to the desired polygons.

[...]


TRBS-fitting TRMA creation

With the following code ...

  • the objects get listed one after another (TRMA-ready)
  • the textures get listed for TRMA and TXMP creation
  • the texture sizes get checked to see if "-large" argument is necessary

One more idea is to add support for reflective textures.


Current status:

hexhound hierarchy

hexhound_hierarchy_tn_zpsd82230ce.png

if selection.count > 0 then
	' any part could be selected, let's find the root body part
	SelectNeighborObj selection(0), 4
	' get all members including the pelvis
	set bodyparts = selection(0).FindChildren( , , siMeshFamily)
	for each member in bodyparts
		logmessage "object name: " & member.name
		if not typename(member.Material.CurrentImageClip) = "Nothing" then
			logmessage "texture: " & member.Material.CurrentImageClip.source.filename.value
			'logmessage "material: " & member.Material.name
			'logmessage "shader: " & member.Material.shaders(0).name
			logmessage "X: " & member.Material.CurrentImageClip.source.Parameters("XRes").Value
			logmessage "Y: " & member.Material.CurrentImageClip.source.Parameters("YRes").Value
		end if
		logmessage "----------------------------------------------------------"
	next
	logmessage "counted body parts: " & bodyparts.count
else
	logmessage "no object was selected"
end if


Old dev notes

New door animations

Final door animation depends on:

  • BINACJBODOOR position and rotation
  • OBAN positions and rotations
  • ZAxisUp (Does this flag do something else?)


Theory of the (Oni to ModTool) import from scratch:

  • import BINACJBODOOR
  • get door classes
  • extract M3GM from DOOR, etc.
  • import M3GM
  • move center to lower bounding box Z
  • move object to Z = 0
  • import OBAN
  • apply rotations and positions to object
  • create null object
  • make null a parent of door object
  • rotate null in X = -90 (to make Z axis pointing up)
  • add BINACJBODOOR rotation and position to null object

The current version of onisplit (v0.9.68.0) imports the doors from AKEV without animations.

Those door have there centers at the lower bounding box Z, got corrected from their ZAxisUp, and have (BINACJBODOOR) rotation and position.

Writing an own combined import would not only take more time, also it wouldn't be so fast and efficient. So, I will go with the existing import but then do some changes on the doors (appling null object, custom properties, etc.)

The door objects can be identified by their naming. Each name contains the BINACJBODOOR ID, e.g. <DOOR Id="7294">.

There could be a loop scanning all objects for those numbers.


Hierarchy builder for characters

Based on old knowledge. See HERE for an easy manual way to change the meshes in hierarchies.


(old notes in the following)

hierarchy breaker beta
hierarchy_breaker_beta_tn.png

(Automatic rigging.) Maybe null objects could stand for the centers of body parts. Then one click and the real centers gets positioned. The rotations are known and the names would lead to the correct hierarchy.

In order to make edits there is also a need of destroying the hierarchy again while preserving the rotation and position of the meshes.


code pieces


hierarchy breaker


hierarchy builder


Keyframe affects hierarchy building

situation: "cube" and "cube1" were created

  • case 1:
    • apply 45° y rotation to cube
    • make cube1 a child of cube
    • cube1 will not inherit the 45°
  • case 2:
    • apply 45° y rotation to cube
    • make rotation keyframe for cube1
    • make cube1 a child of cube
    • cube1 inherit the 45°
  • case 3:
    • apply 45° y rotation to cube
    • make rotation keyframe for cube1
    • remove that keyframe again
    • make cube1 a child of cube
    • cube1 inherit the 45°
  • case 4:
    • apply 45° y rotation to cube
    • make rotation keyframe for cube1
    • remove that keyframe again
    • save both cube to a dae file
    • make new scene and load dae
    • make cube1 a child of cube
    • cube1 will not inherit the 45°

Case 1 might be a bug (case 4 is a variant of it) but it simplify the hierarchy creation.


While doing those test you can observe red letters next to the meshes in the Schematic window.

A means the presence of keyframes. Removing the keyframe doesn't remove the A. Object must be saved and reloaded.
D means that mesh center was changed by rotation or translation (center mode) or by clicking Transform > Freeze(...). D can be removed by clicking the normal Freeze button. Yea, whatever...
C can be seen next to camera objects, present when Camera_Interest is horizontal attached to it.

A

Canceled

Reimporting and recovering of original vertex shading on doors, consoles, etc.

Q: Can we reimport and recover such data?
A: XML-based AKEV editing isn't possible. But editing the binary file is possible.
However, OniSplit cannot easily reapply original vertex shading since the used M3GMs have different vertex ids.

Old notes from realization attempt:

  1. import a console into Mod Tool
  2. associate an ID (and everything else necessary to build the "BINACJBOconsole.xml" file later)
  3. apply vertex shading
  4. hit level export button to build the BINACJBOconsole and other xml and eventully all level files as onis
  5. now it would be time for the binary AKEV patching
    1. console id can be used to identify the polygons in AKEV's AGQG instance where the vertex shading is stored (<Colors>)
    2. now we have to hope that onisplit imported the mesh's polygons in the same order as Mod Tool read the polygons
    3. getting the vertex color from Mod Tool: there's already a code piece on MT's page ("[42] get vertex color")


I'm not fluent with binary stuff so I will dump some notes here:


Data table

Reading the AKEV's name table was more or less easy. This time we need to look up the data table.

Three things to take care about: reading hex numbers backwards (as usual in Oni), adding relative offsets to data table's offset (took a pretty while to find that one out*), subtracting 8 bytes for no other reason to get the correct offset.

* Regarding binary information I found THAT page very useful.


What AKEV data do we need? The structure is fixed, so actually only the size or the item count is needed to pin down all colors.

data table offset at 0x20
AGQG offset (A) at 0x94
AGQG size (B) at 0x9C
AGQG items at 0x20 + A + 0x1C - 8
AGQG start at 0x20 + A - 8
AGQG end at 0x20 + A + B - 8


Color data in AGQG

The byte array is like this: BF BF BF FF (191 191 191 255 = grey).

The format is BGRA whereby I couldn't see Alpha to have an effect on the color strength. Maybe it will be different with a texture (not tested yet).


AGQG reader

https://dl.dropboxusercontent.com/u/139715/OniGalore/temp/AKEV_AGQG_reader.txt

output sample:

' INFO : 000280
' INFO : data table offset: 640
' INFO : 0004A8
' INFO : AGQG table offset: 1192
' INFO : 0001E0
' INFO : AGQG table size: 480
' INFO : ------------------------------
' INFO : AGQG_start: 1824
' INFO : ------------------------------
' INFO : AGQG array size: 8
' INFO : ----------
' INFO : element: 1
' INFO : 00000000
' INFO : 01000000
' INFO : 02000000
' INFO : 03000000
' INFO : 00000000
' INFO : 00000000
' INFO : 00000000
' INFO : 00000000
' INFO : BFBFBFFF (color: 191 191 191 255)
' INFO : BFBFBFFF (color: 191 191 191 255)
' INFO : BFBFBFFF (color: 191 191 191 255)
' INFO : BFBFBFFF (color: 191 191 191 255)
' INFO : 00000804
' INFO : FFFFFFFF (object id: -1)
' INFO : ----------
' INFO : [...]


Complications

M3GM files use triangles while AGQG instances use quads (look, there are always 4 points each with its own 32-bit color slot).

That means Onisplit converts the M3GM triangles into quads on import.

So Mod Tool would need to exactly mimic OniSplit's way turning triangles into quads.

This information is based on following experience:

  • Console0 triangles = 94
  • After triangle-to-quad conversion.
Console0: 47 quads = 38 (main) + 6 (collision box) + 1 + 1 + 1 (light tex quads)
  • AGQG has 47 elements.


Failed on:

  • read the AKEV xml file to get the mesh's vertices/quads (slow and uncool)
  • extract the mesh as dae from AKEV and use that as vertex shading object (reference for M3GM) (faster but uncool because it still requires all consoles, doors, etc.)

AKEV data and extracted console mesh have different polygon IDs and point IDs. Identifying each point by position (taking tolerances into account) and then identifing the quads and triangles would take too long. So, it's still possible but at the moment it's a black hole consuming too much dev time... Darn.