[Question] XIEvent Type

[jamesbradleym]

Hello! First off I want to say thank you for the work you've done here in establishing a structure for the XIEvent Dats, It's been a tremendous (read: I could not have done this without you 😂) help! I've been tinkering with xi-tinkerer and made a fork that implements a round trip from .dat to .yml for events.

Now I'm at the point where I'm trying to determine how to make this useful, such as having the .yml include human-readable info, but to do so I'd need to know that the immid data is of a certain type.

For example, how might I know what immed_data 30 is below, or that 8033 is a string id? Why are index 0, 1, and 2 event_exec_nums the max value of unsigned 16-bit integer? Where should I be looking for event 271? What can I do with the event_data bytes? This is my first foray into decoding and investigating .dat files so I may be missing something obvious for how to decipher this but figured I'd ask first before going into fruitless rabbit holes.

Thanks again!

https://github.com/jamesbradleym/xi-tinkerer/tree/event

actor_number: 17764522
  tag_count: 4
  tag_offsets:
  - 0
  - 1
  - 2
  - 11
  event_exec_nums:
  - 65535
  - 65535
  - 65535
  - 271
  immed_count: 2
  immed_data:
  - 30
  - 8033
  event_data_size: 26
  event_data:
  - 0
  - 0
  - 94
  - 105
  - 100
  - 108
  - 48
  - 28
  - 0
  - 128
  - 0
  - 30
  - 240
  - 255
  - 255
  - 127
  - 111
  - 112
  - 29
  - 1
  - 128
  - 35
  - 32
  - 0
  - 33
  - 0
  - 255
  - 255

[atom0s]

Hello! First off I want to say thank you for the work you've done here in establishing a structure for the XIEvent Dats, It's been a tremendous (read: I could not have done this without you 😂) help! I've been tinkering with xi-tinkerer and made a fork that implements a round trip from .dat to .yml for events.

Hello, glad that my work in this repo can be of use to someone. :)

Now I'm at the point where I'm trying to determine how to make this useful, such as having the .yml include human-readable info, but to do so I'd need to know that the immid data is of a certain type.

The manner in which data is used by the event VM is dependent on the opcode(s) that are referencing it and what additional calls, if any, are used when handling said data. It is possible that a value is used in multiple ways through casting or lookups with other things and so on.

For example, how might I know what immed_data 30 is below, or that 8033 is a string id? Why are index 0, 1, and 2 event_exec_nums the max value of unsigned 16-bit integer? Where should I be looking for event 271? What can I do with the event_data bytes? This is my first foray into decoding and investigating .dat files so I may be missing something obvious for how to decipher this but figured I'd ask first before going into fruitless rabbit holes.

The way the data is used will be dependent on the opcode(s) that are using it at the given moment in the event. It is possible for data to be reused as well allowing it to have more than one type/purpose. Since you asked multiple questions here, I'll answer them individually.

Q. For example, how might I know what immed_data 30 is below, or that 8033 is a string id?

You can't determine what they are or how they are used simply by looking at them. You need to decompile the bytecode data back into readable opcode instructions to determine how they are being used and what they represent. But again, data can be reused by multiple things and not always have a single specific use. It can also be manipulated after being read to become some other value for another purpose as well.

For example, that 30 could represent a string index, a time delay, a flag, a value to be used in an equation, a value used as a compare, and so on. There are a ton of ways the data can be used.

Q. Why are index 0, 1, and 2 event_exec_nums the max value of unsigned 16-bit integer?

These are generally used as special condition values throughout the event VM and can be used for different purposes. In some cases, it can denote an empty event block to tell the event system to reset, or it can be dead/old event entries that are just left over and have had their ids removed. I don't think -1 (65535) has a specific purpose itself, but -2 (65534), for example, does and causes the event id loop checks to exit early if hit.

Q. Where should I be looking for event 271?

There is no specific event with the id '271'. Every zone can have an event with the id of 271. They are not unique ids across the whole game. If you are referring to the block you have dumped, based on the information you shared that would be for:

• Zone: Windurst Woods
• Entity: Two of Spades

Which does have the entry for event '271'. To determine which event block in the zone has the desired event id, you would have to parse the full file, loop the blocks and look at each EventIds array to find the id you are looking for.

Q. What can I do with the event_data bytes?

The EventData array holds the raw bytecode data for all of the events associated with that block.

You can use the information that is stored within the TagOffset table to determine where the bytecode of each event starts for that given block. To determine the size, you would need to make use of the EventDataSize value and the information you get from the TagOffset array. Each event in a given block will have an entry in the TagOffset table to determine where that given events' bytecode starts.

We can use the block you gave as an example. In that example this is how those arrays would connect to each other:

• Event Id: -1, TagOffset: 0
• Event Id: -1, TagOffset: 1
• Event Id: -1, TagOffset: 2
• Event Id: 271, TagOffset: 11

Next, you can use this information to determine the bytecode data and size of data for each event in the block based on the offsets and the EventDataSize value. You would walk the TagOffset array, starting from index 0, upward to determine the size of each events bytecode data. Using this approach we can see that:

• Event index 0 starts at data index 0.
  • Look ahead to the next events start index (1) to determine the length. [1 - 0 = 1]
• Event index 1 starts at data index 1.
  • Look ahead to the next events start index (2) to determine the length. [2 - 1 = 1]
• Event index 2 starts at data index 2.
  • Look ahead to the next events start index (11) to determine the length. [11 - 2 = 9]
• Event index 3 starts at data index 11.
  • Look ahead to the next events start index. This is the last event so no next index. Instead, use the EventDataSize to determine the length of data remaining which will be used for this events data.
  • EventDataSize for this block is 26. [26 - 11 = 15]

So with this, we can then determine that the bytecode data for each event in this block would be:

Event Index: 0 - Event Id: -1 - Data Length: 1

00

Event Index: 1 - Event Id: -1 - Data Length: 1

00

Event Index: 2 - Event Id: -1 - Data Length: 9

5E 69 64 6C 30 1C 00 80 00

Event Index: 3 - Event Id: 271 - Data Length: 15

1E F0 FF FF 7F 6F 70 1D 01 80 23 20 00 21 00

From here, you would need to then disassemble the bytecode back into its opcode form to determine how things are being used. Each of the opcodes pages contain information regarding their raw bytecode layout and sizes based on the various sub-conditions that can change how the data is used. I'll use the last event (271) as an example to show you how this would look.

We now know the bytecode for this event is: 1E F0 FF FF 7F 6F 70 1D 01 80 23 20 00 21 00

The first byte here is the start of the first instruction 0x1E so we can go to that page to see what that opcode means:
https://github.com/atom0s/XiEvents/blob/main/OpCodes/0x001E.md

Here we can see the length is always 5 bytes long. This opcode is used to look at a given entity to begin the talking animation.

So then we have:

1E F0 FF FF 7F 
6F 70 1D 01 80 23 20 00 21 00

The next opcode starts with 0x6F as we took the first 5 bytes for the first instruction. So we again check that page:
https://github.com/atom0s/XiEvents/blob/main/OpCodes/0x006F.md

This is a delay opcode telling the event to 'wait' an amount of time. This kind of delay uses a hardcoded delay of 16 and is only a single byte long. So then we have:

1E F0 FF FF 7F 
6F
70 1D 01 80 23 20 00 21 00

We repeat this for each starting byte of the opcode to fully disassemble what this event is doing. Once done, you should end up with:

1E F0 FF FF 7F - Look at entity and begin talking. [Entity = 7FFFFFF0 -> (Special Id) Local Player]
6F             - Wait [Hard coded 16.0f delay time.]
70             - TurnCancel [Stop the target entities movement to talk with the other entity.]
1D 01 80       - NpcTalk [Prints a string as if the NPC said it to chat.][String Index: 0x8001]
23             - MesWait [Waits for the player to progress the event dialog forward.]
20 00          - SetEventUcFlg [Sets the event user control flag to 0.]
21             - Finish [Marks the event as completed and tells the event VM it can cleanup.]
00             - Ends the event and cleans up the current event stack.

[atom0s]

I forgot to explain the string handling for this event as well with opcode 0x1D.

The event data you showed in your example is for Two of Spades within Windurst Woods. Using event 271 from that block, we know that it is Two of Spades printing its dialog to chat which is:
TwO<star>UN-AblE<star>To<star>Me-MoR-IZe...

That said, the way this works is through the opcode 0x1D. You can read how that opcode works here:
https://github.com/atom0s/XiEvents/blob/main/OpCodes/0x001D.md

This opcode uses its own data to determine which string index is used to be printed. This is done using the XiEvent::getworkofs helper function, which you can read about here:
https://github.com/atom0s/XiEvents/blob/main/Event%20VM%20Functions.md#xieventgetworkofs

In this case, the opcode is 1D 01 80 meaning the read index from the opcode is 0x8001. This value is another special-value used by the event VM. Inside of the getworkofs helper function, we can see that this means the value will make use of the ImidData array. The value 0x8000 will be masked off of this leaving the index into this array that is used. In this case, that means we would get:

index = (0x8001 & 0x7FFF) // index would be 1

That means that this would use the value 8033 as the string index. For Windurst Woods, you would then use the string DAT file for that zones event strings and dump the string at that index to be printed.

[jamesbradleym]

Wow, this is absolutely awesome. Thank you for taking the time to step through this! This gives me more than enough to work with - apologies for completely glazing over all the opCodes md files you laid out. 🙏🏼

[jamesbradleym]

So then, for Event Index 2 we would see:

5E 69 64 6C 30 1C 00 80 00

OpCodes	Action
5E 69 64 6C 30	Stop the event entity's current action and reset them back to an idle motion.
1C 00 80	Wait. Sets or updates (decreases) the current ReqStack[RunPos].WaitTime value. When first called, this opcode will check if ReqStack[RunPos].WaitTime is < 0.0. If true, then the WaitTime will be set from the value read from the opcode (e.g., 30 for immed_data[0]?).
00	Ends the event and cleans up the current event stack.

[atom0s]

I've converted your Issue into a Discussion since it is just questions about the repo. And yes, that looks correct for your last post/question.

[jamesbradleym]

Following up on this, I've noticed several opcodes have multiple size options, and the logic you've discerned for these opcodes indicates what position the pointer moves to. This logic seems to be a mix of actual game conditions as well as straight forward value checking from upcoming bytes from the opcode. So, would that mean without a client and active game it isn't possible to break down these events? Or am I thinking about these the wrong way?

0xD4, 0xA7, 0x75, 0x72, 0x66, and 0x5B are some examples of this. For instance 5B is looking at an input variable param3 to determine if it should be size 15 or 17 🤷‍♂️.

If determining the true size for these opcodes is (at least not yet) plausible, what would be the right way to deconstruct as much of the event data as possible? decode up to the point of uncertainty and then spit out the raw bytes from then on for the event series we're currently in? (Event Series being the events within an event block for an actor, aka 271 above)

[atom0s]

Hello again. Everything can be determined offline without running the client. There are some conditional opcodes that will wait for certain conditions before progressing through other paths, but their conditions are all deterministic so the flow of instructions is able to be determined while statically disassembling.

The event VM is written in an old school async manner. When there is an active event on the client, the client will call the main VM executor every frame. The current ExecPointer will determine which opcode is being executed. This means that it is possible for the same opcode to keep being called across multiple frames until an expected condition is met to allow it to step forward. This is how waits and other conditional things can happen without freezing the client.

For the opcodes you asked about:

0x5B and 0x66 share the same main handler but have a sub-call first to set the parameters.

• 0x5B calls the main handler with the params set as: 0, 1, 0
• 0x66 calls the main handler with the params set as: 1, 1, 0 (The docs page is wrong sorry about that!)

0x72, 0x75, 0xA7 and 0xD4 all share the same manner of handling different cases/conditions. (Along with a lot of other opcodes as well.) They use additional bytes from the instruction to determine the sub-kind of that opcode that is being used. This allows a single opcode to have multiple sub-purposes and can change how the execution happens.

For example, with opcode 0xD4, you will see at the start of the handler, there is:

switch (this->EventData[this->ExecPointer + 1])

This is reading the next byte from the instruction after the initial opcode 0xD4 byte to determine the sub-kind and how to handle the rest of the data of the instruction. From that handler and how the switch is being performed, it is then possible to determine that the valid sub-kind instructions for 0xD4 would be:

• D4 00
• D4 01 ?? ?? ?? ?? ?? ??
• D4 02
• D4 03 ?? ?? ?? ??
• D4 04 ?? ?? ?? ?? ?? ?? ?? ?? ?? ??
• D4 05 ?? ?? ?? ?? ?? ?? ?? ?? ?? ??

Keeping in mind that some of these sub-kinds can also reset the ExecPointer back to the start of the 0xD4 opcode to be reprocessed as needed.

[jamesbradleym]

Understood, thanks for the clarification on these!

I've run through my opcode handling and think I'm on the right track for a majority of these opcodes, but I'm hitting some unexpected results.. sorry for being a nuisance 😅

For instance, ROM/21/50.dat, Blocks[11].TagOffset[21] (Relative Offset: 2919) (Total offset: 0:3D83 | 15747):

The OpCode series is:

• 42
• 46 01
• 69 01 08 80
• 5C 00 8C 80 5C 01
• 8C 80 ...

But from my understanding of 0x08C, 80 would not resolve in the switch/case to give us a proper size to work with?

OpCode 42 has size 1
OpCode 46 can be 2 or 4 - with 01 it seems to resolve to 2
OpCode 69 has size 4
OpCode 5C can be 4 or 6 - with 00 it resolves to 6

Then we get to 8C but 80 is not a valid case, the default simply breaks after setting the retflag to 1 to continue - this is where I'm stuck and bail out to the raw bytecode for this event.

So in this case, what's the right course of action? Either I'm not picking the correct size for 0x0046/0x005C or I'm misunderstanding the loop for 0x008C to get an appropriate size..

[atom0s]

Hello again. Sorry about the confusion, you are not seeing things wrong. There were some incorrect bits of information on some of the opcode pages that I have now corrected. I've done a few additional 'soft passes' over every page to do some additional reviews and cleanup to things as well as updated some other function/variable names and so on. There were a few pages that had the incorrect opcode sizes listed or some incorrect information in general (usually from a bad/stale copy-paste of another page being used as the start of a given opcodes own page).

You can see the list of changes/commits I've made in the last few days to correct those bits of information and update the other things here:
https://github.com/atom0s/XiEvents/commits/main/?since=2024-12-27&until=2024-12-29

In the example you gave above, opcode 0x5C had the incorrect size information. There was a typo causing the first set of cases in the switch to be incremented too far ahead. (It should have been 4 instead of 6 specifically.)

[jamesbradleym]

No need to apologize, I'd be absolutely nowhere without all of this incredible work you've done!

Amazing, thank you so much for running through those! With those updates I'm only seeing a couple of inconsistencies left which is wonderful. One interesting one is an overflow of the conquest NPC menu event. Again in ROM/21/50.dat, Blocks[190].EventData[13643] - Total offset 6:2C5F.

This is one massive event series which appears to end with EventData[13643] = 0x0017 which would be size 7. Of course we're at the end of our eventblock on this byte, so this seems to be a dead end. Is it a valid circumstance to come to a dead byte? I would've assumed it would be something like 0x00 or 0xFF as a filler.

Running up to this eventcode I've got:

0x80 0x0C 0x17 0x0D 0x17
0x0E 0x17 0x0F 0x17 0x10
0x17 0x11 0x17 0x12 0x17 0x13 0x17
0x14 0x17 0x15 0x17 0x16
0x17 0x17 0x17 0x18 0x17 0x19 0x17
0x1A 0x17 0x1B

This is the tail end of 2777 cleanly parsed OpCodes for Harara W.W in Windurst Woods!

[jamesbradleym]

Bumping this for visibility but no urgency. Wish I had a better understanding to investigate myself 😅

[jamesbradleym]

Bumping again 😊