Mods / ShaderPatchesLib

Tags:
Library Shaders
Author:
Coriaender
Side:
Client
Created:
May 18th at 5:32 AM
Last modified:
May 26th at 12:04 AM
Downloads:
201
Latest release (for Vintage Story 1.22.2, potentially outdated):
ShaderPatchesLib-v0.0.2_v1.22.2.zip  1-click install

ShaderPatchesLib

Patch existing shader code without replacing the file.

What it does

Adds a new asset category, shaderpatches, and loads them during setup. Whenever a shader program is loaded it modifies the code based on the patches.

For players:

If the mod is installed like usual it will detect and apply patches automatically, no extra steps needed.

For modders:

The mod loads all assets/shaderpatches the AssetManager finds, which are parsed as JSON. An overview of the available syntax is provided below, see the full API in the included .md file or here.

Shaderpatches assets will be applied automatically, but additional patching control is provided with an API object. With the dll included in a project, the API can be fetched from the mod system: IShaderPatchesAPI spapi = (CApi.ModLoader.GetModSystem("ShaderPatchesLib.ShaderPatchesModSystem") as ShaderPatchesModSystem).GetShaderPatchesAPI();

As a bonus the mod adds any mod assets/shaderincludes to the ShaderRegistry. The game loads these assets from mods but doesn't register them, by adding them the compiling pipeline is able to find and insert them when handling includes.

Writing Patches
The mod expects assets/shaderpatches to be JSON files that are an array of PatchAssetSet blocks. The name or number of these files is not considered, but my convention is having one file per target shader or shaderinclude.

PatchAssetSet blocks from any file that have the same value for "group" are treated as a unit. The main use of this is that if any patch from a group has an error, the group is not applied. If no value for "group" is provided, it is set to "default".

Note that patch groups and sets are seperate with regards to enabling/disabling, meaning enabling a group will not enable the sets it owns nor vice versa. This is true for both API functions and the "startenabled" field.

PatchGroupSettings

If a patchgroups.json file is included in assets/shaderpatches, the mod will parse it for group metadata. Currently the only setting is "startenabled", which defaults to true if not assigned or if no patchgroups.json is present.

PatchAssetSet
These say which shaders the block should modify, some metadata for using the runtime API, and a list of ShaderPatch blocks. Target shaders are provided as an array, or a "shaderinclude" value can be provided for the patch to target any shaders which use that include.

ShaderPatch
Individual patches consist of a regex pattern to match against, lines of code in a string[], and optionally prefixes to add to the IShader. The patcher will call Regex.Replace, replacing the pattern with the patch.

Patches can make use of additional functionality with INSERT statements that are replaced before patching:

  • INSERT_REINSERTMATCH
    • This will be replaced with the text matched by the pattern. This is always checked for, but not required.
  • INSERT_PRAGMAONCE
    • This will be replaced by the lines provided in the "pragmaonce" field's array, skipping any lines already present in the shader code.
    • Helpful for things like #extension statements.
  • INSERT_FETCHREPLACE
    • Provide an array of regex patterns to be matched against the shader code, and this statement will be replaced with the first match group.
    • Allows for referencing the shader being patched, for example some shaders use different variable names for the same thing, e.g. worldPos/fWorldPos.
    • Provided in the "fetchreplace" field as an array, with each corresponding index being appended to the INSERT, e.g. INSERT_FETCHREPLACE0, INSERT_FETCHREPLACE1.

For more thorough examples of use, I have a branch of CoriaenderShaders with all patching implemented using ShaderPatchesLib that can be found here.

PatchGroupSettings type

Namespace

ShaderPatchesLib

Summary

Group metadata, defined in patchgroups.json.

Example
{
    "watereffects": {
        "startenabled": false
    },
    "cloudshadows": {
        "startenabled": false
    }
}
Remarks

Groups are unique to a domain. Run time enable/disable is done via the API. If a patchgroups.json is not found, all groups use default settings.

StartEnabled property

Summary

bool Whether the group should be enabled when first loaded. Default true.

PatchAssetSet type

Namespace

ShaderPatchesLib

Summary

Structure of input data. JSON file should be an array of these.

Example
"group": "watereffects",
"targetshaders": [
    "chunkliquid.vsh"
],
"prefixes": [
    "CS_WATEREFFECTS_ENABLED"
],
"patches": [
    {
        (see ShaderPatch for patch object structure)
    }
]

Group property

Summary

string Optional group name for associating patch sets. Not shared across mod domains. Defaults to "default".

Example

"group": "cloudshadows"

Remarks

Group metadata is optionally declared in a patchGroups.json file.

Patches property

Summary

ShaderPatch[] An array of ShaderPatch entries.

Example
"patches": [ 
    { 
        (see ShaderPatch for patch object structure)
    } 
]

Prefixes property

Summary

string[] Added to the program's prefix string as a #define statement.

Example

"prefixes": ["CS_WATEREFFECT_ENABLED", "CS_GBUFFERBINDING 6"]

Remarks

If the define includes a value, this should be a single string that includes an initial value.

StartEnabled property

Summary

bool Whether these patches should start enabled. Default true.

Example

"startenabled": false

Remarks

Manual enabling or disabling is done through the API. To disable a group, include a patchgroups.json.

TargetInclude property

Summary

string Optional. A shaderinclude file. If given, all shaders that #include this will be added as target shaders.

Example

"targetinclude": "colorutil.ash"

TargetShaders property

Summary

string[] The shader files the patches should be applied to.

Example

"targetshaders": ["chunkopaque.fsh", "chunkliquid.fsh", "chunktopsoil.fsh"]

Remarks

Can be ommitted if a targetinclude is provided.

ShaderPatch type

Namespace

ShaderPatchesLib

Summary

Structure of individual patches.

Example
"name": "fshadowapply",
"type": "VertexShader",  
"pattern": "[\\r\\n\\s](frag)?[Nn]ormal = (unpack[Nn]ormal[v]?\\((render)?[Ff]lags(In)?\\)|normalv\\.xyz);[\\r\\n]",  
"patch": [  
    "INSERT_REINSERTMATCH",  
    " ",  
    "// Normal offset shadow biasing, based on https://github.com/godotengine/godot/issues/17260",  
    "float cosLightAngle = dot(sunDir, INSERT_FETCHREPLACE0);",  
    "normalOffsetPrecursor = clamp(1 - cosLightAngle, 0, 1);"  
],  
"fetchreplace": [  
    "[\\r\\n\\s](fragNormal|normal) = "  
],  
"regexoptions": [  
    "ExplicitCapture"  
]

FetchReplace property

Summary

string[] Optional insert flag. An array of regex strings where the first match group will be used to replace the insert flag for that array index, e.g. INSERT_FETCHREPLACE0.

Example

To replace 'INSERT_FETCHREPLACE0' with 'worldPos' or 'fWorldPos': "fetchreplace": ["in vec\\d (worldPos|fWorldPos);[\\r\\n]]"

Name property

Summary

string Identifier used in a watermark. Unique to domain+group.

Example

"name": "addextension"

Patch property

Summary

string[] The code to insert. An array of strings, where each string is a line of code.

Example
"patch": [
    "#ifdef CS_WATEREFFECTS_ENABLED", 
    "outputCSWaterEffectsInput();", 
    "#endif", 
    " ", 
    "INSERT_REINSERTMATCH"
]

Pattern property

Summary

string Regex string of the shader text to replace. Should be fully escaped. INSERT_REINSERTMATCH will be replaced in the patch text with the matched text, if present.

Example

"pattern": "^#version\\s[\\d]+\\s[\\w]+[\\r\\n\\s]+"

PragmaOnce property

Summary

string[] Optional insert flag. An array of strings where each string is a line of code that will replace the text INSERT_PRAGMAONCE only if it is not already in the shader code.

Example

"pragmaonce": ["#extension GL_ARB_explicit_uniform_location : enable"]

RegexOptions property

Summary

string[] Optional. An array of RegexOptions flag names to pass to Regex.Replace when applying patches.

Example

"regexoptions": ["Multiline", "ExplicitCapture"]

Mod Version Mod IdentifierFor Game version Downloads Released Changelog Download 1-click mod install*
0.0.2 shaderpatcheslib 127 May 26th at 12:00 AM ShaderPatchesLib-v0.0.2_v1.22.2.zip 1-click install

Canonize groups

  • To allow for better control when patching fails, groups now internally manage their owned patches.
  • If any patch in a group fails to apply, the entire group will not be applied and will be disabled for later patching.
  • Group metadata is provided by an optional 'patchgroups.json' file.
    • Currently The only option is whether the group starts enabled, which defaults to true.
  • API enabling/disabling is independent for groups and for the patches the group owns.
    • To clarify this, the API distinguishes between patch 'groups' and 'sets'.
    • Patch sets correspond to the individual data blocks defined in the json, meaning they are primarily distinguished by the shaders they target.
    • 'EnablePatchesForShader' is replaced with 'EnablePatchSetsForShader', and *ForGroup and *ForProgram functions are added.

Error reporting

  • Patching no longer throws exceptions.
    • On error, a patch group will set it's state to Failed and no longer be applied during patching.
    • Errors are instead signalled to the ShaderPatchingError event.
    • GetFailedGroups(domain) and CheckGroupFailed(domain, group) functions have been added to the API for checking the state of patch groups.
0.0.1 shaderpatcheslib 51 May 19th at 9:42 PM ShaderPatchesLib-v0.0.1_v1.22.2.zip 1-click install

Changed

  • Require file extensions on target shaders. Remove "type" from ShaderPatch.
  • Add "targetinclude" to PatchAssetData.
    • If provided target shaders will be populated with shaders that #include targetinclude.
0.0.0 shaderpatcheslib 23 May 18th at 5:32 AM Empty ShaderPatchesLib-v0.0.0_v1.22.2.zip 1-click install

3 Comments (oldest first | newest first) (threaded | flat)

Cheeki, May 21st at 12:07 PM

pretty good. one thing i would add is a "match all and patch otherwise do nothing" behavior, based on the content of the shader. so all patches in a group only gets applied if they all match. I have something like this for darkvision, whose shader patching system was inspired by the one you use in the shader mod. though the one I wrote is much more self contained than this.

Coriaender , May 22nd at 3:35 AM
@Cheeki: pretty good. one thing i would add is a "match all and patch otherwise do nothing" behavior, based on the content of the shader. so all patches in a group only gets applied if they all match. I have something like this for darkvision, whose shader patchin

Thanks, that's a smart idea, right now it just gives up because if my patches didn't work then my mod would crash anyway, but it makes sense here to have what you described as the default behavior and then leave it to the mods to decide how to handle failures.

BraniyaKz, May 18th at 7:29 AM

Yasss