Mixin 0.8.4

Sometimes new Mixin releases present an opportunity to discuss different aspects of Mixin which are not covered in detail in the docs, or limitations which exist are not well understood by mixin authors, leading them to experience behaviour they aren't anticipating. The topic of the 0.8 series has definitely bounced around the topic of Locals, in particular CallbackInjector's unique ability to capture locals, so I will discuss some of the changes to Locals later in these notes. But first, the headlines:

Features

Minecraft 1.17, ModLauncher 9, Java 16, Oh My!

Minecraft 1.17 represents a significant jump in technology, adopting as it does Java 16, a big jump from the previous Java 8. Those in the modding community who remember the jump to Java 8 from Java 6 will anticipate something of what that means. Mixin has been for some time rooted in that Java 8 land because the primary consumers are using Java 8 and and (in some cases) Java 6, so supporting newer language features hasn't been a high priority.

With the release of Mixin 0.8.4, the core functionality of Mixin now supports up to Java 13 (cheating a little since nothing of note changed between Java 11 and 13), the missing piece of the puzzle being nesting. Mixins applied to inner classes will now be properly added to the existing nest.

The internal version of ASM has been bumped to 9.x to track ModLauncher, which now provides access to class version literals up to Java 18, which have been duly contributed to the CompatibilityLevel enum, along with associated language feature flags. Records and sealed classes are now on the radar for support, but will not be immediately available as things are ahead of them in the feature queue.

Mixin itself is now built as as Java module, though classes in the final jar will still be compatible back as far as Java 8 without modification, Java 6 users will still need to compile from source.

Fixes and Improvements

Besides the internal restructuring to support Java 16 and ModLauncher 9, this release is primarily aimed at addressing some regressions introduced in version 0.8.3. One of my primary goals with Mixin is to make a library with stable, predictable behaviour - at least where this can be reasonably asserted - and some regressions in 0.8.3 I felt breached this threshold of acceptable change.

Version 0.8.4 does still bring some changes to the table, but the scope - and type - of these changes are now much more manageable with respect to 0.8.2.

Mixin 0.8.4 therefore represents the "what 0.8.3 should have been" release, and offers the promised improvements of the previous release, with fewer gotchas. Check out the release notes for 0.8.3 if you haven't already, as they cover the major changes and bugfixes since 0.8.2.

Local Variables in Mixins

This brings us on to the bonus topic of today's release notes: Locals.

Support for interacting with local variables has a long and storied past and has, during its tenure, overcome such obstactles as differing output from ASM in production vs. development. Yielding different results across different versions and platforms for ostensibly the same code. Failing to expose locals which seem intuitively in-scope at the capture point, and exposing others which shouldn't be. Accidentally capturing locals added by Mixin itself. The list goes on. I can say that the 0.8 series of Mixin definitely represents a high point in the general functionality of local capture.

Those who have been on board the Mixin train for a long time have observed the growing pains of locals and treat it with the air of an unexploded bomb which might detonate at any moment, and they are justified in doing so. However I realise that I have been negligent in assuming that this feeling prevails and therefore failing to provide the appropriate caveats and warnings around the use of locals. This is especially true recently since the state of Locals is objectively quite good, and belies the dangers of change over time that still lurk beneath the surface.

So why, if it's in such a good place, does it need to change at all?

I have two main goals for Locals in the long run:

• Reveal locals that are intuitively in scope from the mixin author's perspective
• Provide stable results even across different platforms and versions, for code which appears to change. Eg. reduce sensitivity to binary-level changes
• Do all this with reasonable efficiency, eg. avoid going full-blown analytical decompiler, make an algorithm the works predictably even if it's not perfect

The reason for the first should be obvious, authors of mixins should have a reasonable expectation of being able to capture variables which appear to be in-scope based on reading the decompiled source. This reduces development effort and makes for a better experience.

The second goal is the reason that things are still changing: reducing binary sensitivity is a tricky thing to achieve given that it conflicts with the third goal somewhat, this is where tuning the algorithm comes into play.

The algorithm used by Locals does a relatively simple scan of target methods from top to bottom, weaving knowledge gained from the LVT (either explicit or computed), the stack map frames, and LOAD and STORE opcodes in the method itself together to create a not unreasonable view of the available local variables at any point in the method. However, the weight given to different information sources is where the tunables come in, as the entire algorithm is essentially a hueristic which tries to blend sometimes conflicting accounts of what is available and where.

If you're wondering why on earth different aspects of the method structure would disagree, and why there is no "universal truth" that can be easily reverse engineered from simply reading the bytecode, it's important to understand that no part of the runtime actually needs to know the information we're after, and the different mechanims in the bytecode (the instructions, the LVT and the stack map frames) have entirely different goals. The LVT is the closest thing we have to truth, and it's not even required at runtime since it's purely used for debugging. In fact in older versions of the game the LVT was stripped, and even to this day it is obfuscated.

Where am I going with this?

As I said above, Locals is in quite a good place right now, but tweaks and changes may still occur. This means that using Locals in a defensive manner is still - for the time being - important. The end goal is to achieve a level of repeatability between versions, platforms and binary changes that can provide a sense that captured locals will change when the code changes, in other words converging with the general contract of injectors as a whole.

Mixin 0.8.3 and 0.8.4 represent changes to the locals algorithm which make it less leaky¹, however regressions reported after the release of 0.8.3 hinted that the leakiness was a useful characteristic in some cases, and justifiably so. Mixin 0.8.4 doesn't turn the leakiness all the way back up to pre-0.8.3 levels but now uses a more intelligent algorithm to decide when to bring a variable back into scope which was previously axed, a kind of intelligent leak if you will.

Does this mean all injectors written with 0.8.2 will now work in 0.8.4? No, though fewer should break. Does it provide a good step towards more stable results in future versions of Mixin? I hope so.

¹ Leaky in this case is one of the trade-offs when processing the stack map frames in target code. A stack map frame in the code may indicate that some variables have gone out of scope. However the stack map frames may just indicate this because the method doesn't use them any more, the variable values themselves still exist in the method frame, but the stack map "chops" the variables because the code afterward doesn't use them any more. In our algorithm we can choose to "leak" those variables (not chop them) but this can lead to some unintuitive results where - for example - a variable declared inside a loop seems to be still available after the loop, and this becomes very binary-sensitive if the same variable slot is reused later on for a different variable.

Why Take This Approach?

In short, because stability of results allows Mixin to do more things with locals. I have, to date, been reluctant to expand local capture and interaction with locals (eg. via ModifyVariable). This is because the unstable nature of locals means that interacting with the frame as a whole, or only a single variable, at least provides some safeguards in that if the frame doesn't match what the code is expecting then it breaks quickly and predictably.

The driving force behind these changes is that a huge overhaul of injectors, "Injectors three point oh" if you will, is roadmapped for Mixin 0.10, this will include the ability to interact with local variables at a more granular level (eg. capturing and even modifying specific locals in injectors) and extending locals functionality to all injectors, not just limiting it to capturing the whole frame in @Inject. Achieving more predictable results when determining available locals will make these improvements much more feasible.

What's Next

The next version of Mixin will be 0.8.5, which is specifically aimed at eliminating some of the pain points with selecting lambda methods to target, and limitations of some existing injectors when multiple mixins want to target the same instruction. As always, watch this space.

Join the discussion in #mixin on the Sponge Discord.