Skip to main content

Schema by Melty Player

NuGet / site data​

Nuget GitHub last commit GitHub Repo stars

Details​

Info​

info

Name: Schema

Library for converting classes to and from binary. Provides a C# Roslyn generator that automatically implements conversion logic for simple classes.

Author: Melty Player

NuGet: https://www.nuget.org/packages/Schema/

You can find more details at https://github.com/MeltyPlayer/Schema

Source: https://github.com/MeltyPlayer/Schema

Author​

note

Melty Player Alt text

Original Readme​

note

Schema

GitHub Nuget Nuget Unit tests Coverage Status

Overview​

Library for serializing C# types to/from binary. Provides a Roslyn generator that automatically implements read/write logic.

Warning: The design of this library is still in flux, so anticipate making changes when upgrading to future versions.

Credits​

Usage​

Implementing binary schema classes​

To write a binary schema class, you must first have it implement the IBinarySerializable or IBinaryDeserializable interfaces (or IBinaryConvertible if you need both).

Then, based on how complicated your schema class is, you can either choose to automatically or manually implement Read()/Write() methods.

Automatically​

For most schema classes, you should be able to use the automatic code generator.

All you have to do is annotate the schema class with the [BinarySchema] attribute and mark it as partial; this will flag to the generator that it should implement read/write methods for this class. It will then look into all fields/properties in the schema class, and attempt to implement read/write logic in the same order that the fields/properties appear.

Any nested schema classes will be automatically read/written as expected.

Some types require additional attributes in order to clarify any ambiguity. For example, booleans require a [IntegerFormat(SchemaIntegerType.###)] attribute to know what type of integer to read, which it will then compare to 0.

Any readonly primitives will treated as assertions, which is useful for validating things like magic text or padding.

Manually​

For complicated schema classes, such as ones that use decompression logic or pointers, you'll need to implement the read/write logic manually.

Specifically, you'll need to implement both a Read(IBinaryReader br) and Write(IBinaryWriter bw) method. The SchemaBinaryReader and SchemaBinaryWriter classes provide many helpful methods for reading/writing a number of different primitive formats, including basic ones such as byte/int/float, but also more complex/unique ones such as Half (two-byte float) and un16 (unsigned normalized 16-bit float).

Similar to the automatic process, you can nest schema classes and manually read/write them by calling their Read()/Write() methods. This can allow you to automatically generate subsections, so only the most complex logic needs to be manually written.

How to use a binary schema class​

To convert a given schema class to or from binary, simply instantiate an SchemaBinaryReader or SchemaBinaryWriter and pass it into the corresponding Read() or Write() methods in the schema class.

Supported Attributes​

The following attributes are currently supported in this library when automatically generating code. Some attributes are only used at read or write time—these are prefixed with an R or W respectively.

Warning: These names are not final, so they may change in future versions.

Align​

Specifies how a field or property's offset (relative to the start of the stream) should be aligned when reading/writing. If misaligned, the SchemaBinaryReader/SchemaBinaryWriter will automatically insert the remaining bytes of padding. For example, [Align(4)] would force a field/property's starting offset to be a multiple of 4 (0, 4, 8, 12, 16, etc.).

[Align(4)]
public int alignedField;

[Align(4)]
public int AlignedProperty \{ get; set; }

Endianness​

Forces a type, field, or property to be read/written with a given endianness (big-endian or little-endian). Tracked via a stack within the SchemaBinaryReader/SchemaBinaryWriter. If unspecified, will use whatever endianness was last specified in the stack (or the system endianness by default).

[BinarySchema]
[Endianness(Endianness.BigEndian)]
public partial class BigEndianType : IBinaryConvertible {
  ...
  
  [Endianness(Endianness.LittleEndian)]
  public int LittleEndianProperty \{ get; set; }
  
  ...
}

IfBoolean/RIfBoolean​

Marks that a nullable field or property will only be read/written if some other boolean field or property is true.

[IntegerFormat(SchemaIntegerType.BYTE)]
public bool HasValue \{ get; set; }

[RIfBoolean(nameof(this.HasValue))]
public int? Value \{ get; set; }

IChildOf<TParent>​

This pseudo-attribute marks a type as a "child" of some "parent" type—that it is contained as one of the members of the "parent type"—and passes the parent down to the child so it can be referenced in Schema logic.

Used by having the child type implement the IChildOf<TParent> interface, where TParent stores the child type in a field/property or as a member of a sequence (array/list):

[BinarySchema]
public partial class ChildType : IBinaryConvertible, IChildOf<ParentType> {
  public ParentType Parent \{ get; set; }
  
  ...
}

Below is a simple example where a boolean from the parent is used to decide when to read a value in the child:

[BinarySchema]
public partial class ParentType : IBinaryConvertible {
  [IntegerFormat(SchemaIntegerType.BYTE)]
  public bool ChildHasSomeField \{ get; set; }

  public ChildType Child \{ get; \} = new();
}

[BinarySchema]
public partial class ChildType : IBinaryConvertible, IChildOf<ParentType> {
  // This is automatically skipped while reading/writing.
  public ParentType Parent \{ get; set; }

  [Skip]
  private bool HasSomeField => Parent.ChildHasSomeField;

  [RIfBoolean(nameof(HasSomeField))]
  public int? SomeField \{ get; set; }
}

Skip​

Designates that a field or property should be skipped while reading/writing.

Note: IChildOf<TParent>.Parent is automatically skipped.

[Skip]
public int skippedField;

[Skip]
public int SkippedProperty \{ get; set; }

This can be used to encapsulate logic within properties, such as in the following examples:

  1. Value conversion
[StringLengthSource(4)]
public string Magic \{ get; set; }

[Skip]
public MagicType Type => this.Magic switch {
  "IMGE" => MagicType.IMAGE,
  "SOND" => MagicType.SOUND,
  "TEXT" => MagicType.TEXT,
};
  1. "Switch" cases
[NullTerminatedString]
public string Magic \{ get; set; }

[Skip]
public ISection? Section => this.imageSection_ ?? this.soundSection_ ?? this.textSection_;

[Skip]
private bool IsImage_ => this.Magic == "IMAGE";
[Skip]
private bool IsSound_ => this.Magic == "SOUND";
[Skip]
private bool IsText_ => this.Magic == "TEXT";

[RIfBoolean(nameof(this.IsImage))]
private ImageSection? imageSection_ \{ get; set; }

[RIfBoolean(nameof(this.IsSound_))]
private SoundSection? soundSection_ \{ get; set; }

[RIfBoolean(nameof(this.IsText_))]
private TextSection? textSection_ \{ get; set; }

Numbers/Enums​

NumberFormat​

TODO

IntegerFormat​

TODO

Strings​

Note: At the moment, only ASCII is fully supported.

StringLengthSource/RStringLengthSource​

Designates the length of a string field or property via one of three cases.

Note: Any trailing null terminators will be ignored at read time.

  1. Constant length

If a constant is passed into StringLengthSource, that many characters will be read/written.

[StringLengthSource(8)]
public string Text \{ get; set; }
  1. Preceding value

If a SchemaIntegerType is passed into StringLengthSource, an integer of that type will first be read and used as the length of the string, or the length of the string will first be written before writing the string itself.

[StringLengthSource(SchemaIntegerType.BYTE)]
public string TextWithByteLength \{ get; set; }
  1. Another field or property

If the name of another field or property is passed into RStringLengthSource, that other value will be used as the length of the string when reading.

public byte TextLength \{ get; set; }

[RStringLengthSource(nameof(this.TextLength))]
public string Text \{ get; set; }
NullTerminatedString​

Designates that a string field or property will be read until a null terminator is reached, and written with a null terminator affixed to the end.

[NullTerminatedString]
public string Text \{ get; set; }

Sequences​

Note: "Sequence" is the term used within Schema to refer to an array/list of elements.

SequenceLengthSource/RSequenceLengthSource​

TODO

RSequenceUntilEndOfStreamAttribute​

TODO

Pointers/Memory​

TODO

WPointerTo​

TODO

WSizeOfMemberInBytes​

TODO

About​

note

Generate binary serialization code

How to use​

Example (source csproj, source files)​

This is the CSharp Project that references Schema

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>
	<PropertyGroup>
		<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
		<CompilerGeneratedFilesOutputPath>$(BaseIntermediateOutputPath)\GX</CompilerGeneratedFilesOutputPath>
	</PropertyGroup>
	<ItemGroup>
	  <PackageReference Include="CommunityToolkit.HighPerformance" Version="8.3.0" />
	  <PackageReference Include="schema" Version="0.6.13" />
	</ItemGroup>
	
</Project>

Generated Files​

Those are taken from $(BaseIntermediateOutputPath)\GX

// <auto-generated/>
#pragma warning disable CS8669

using System;
using global::Nino.Core;
using System.Buffers;
using System.Collections.Generic;
using System.Collections.Concurrent;
using System.Runtime.InteropServices;
using System.Runtime.CompilerServices;

namespace Serializer.NinoGen
{
    public static partial class Deserializer
    {
#region System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, object?>> - Generated by transformer TrivialEnumerableUsingAdd
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void Deserialize(ReadOnlySpan<byte> data, out System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, object?>> value) 
        {
            var reader = new Reader(data);
            Deserialize(out value, ref reader);
        }

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void Deserialize(out System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, object?>> value, ref Reader reader)
        {
        #if WEAK_VERSION_TOLERANCE
            if (reader.Eof)
            {
                value = default;
                return;
            }
        #endif
        
            if (!reader.ReadCollectionHeader(out var length))
            {
                value = default;
                return;
            }
        
        #if WEAK_VERSION_TOLERANCE
            Reader eleReader;
        #endif
        
            var lst = new System.Collections.Generic.List<System.Collections.Generic.KeyValuePair<string, object?>>();
            for (int i = 0; i < length; i++)
            {
        #if WEAK_VERSION_TOLERANCE
                eleReader = reader.Slice();
                NinoDeserializer.Deserialize(out System.Collections.Generic.KeyValuePair<string, object?> item, ref eleReader);
        #else
                NinoDeserializer.Deserialize(out System.Collections.Generic.KeyValuePair<string, object?> item, ref reader);
        #endif
                lst.Add(item);
            }
        
            value = lst;
        }
        
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void DeserializeRef(ref System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, object?>> value, ref Reader reader) => Deserialize(out value, ref reader);
        
#endregion

#region System.Collections.Generic.KeyValuePair<string, object?> - Generated by transformer KeyValuePair
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void Deserialize(ReadOnlySpan<byte> data, out System.Collections.Generic.KeyValuePair<string, object?> value) 
        {
            var reader = new Reader(data);
            Deserialize(out value, ref reader);
        }

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void Deserialize(out System.Collections.Generic.KeyValuePair<string, object?> value, ref Reader reader)
        {
        #if WEAK_VERSION_TOLERANCE
            if (reader.Eof)
            {
                value = default;
                return;
            }
        #endif
            NinoDeserializer.Deserialize(out string k, ref reader);
            object v = NinoDeserializer.DeserializeBoxed(ref reader, null);
            value = new System.Collections.Generic.KeyValuePair<string, object?>(k, v);
        }
        
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void DeserializeRef(ref System.Collections.Generic.KeyValuePair<string, object?> value, ref Reader reader)
            => Deserialize(out value, ref reader);
        
#endregion

#region byte[] - Generated by transformer Array
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void Deserialize(ReadOnlySpan<byte> data, out byte[] value) 
        {
            var reader = new Reader(data);
            Deserialize(out value, ref reader);
        }

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void Deserialize(out byte[] value, ref Reader reader)
        {
        #if WEAK_VERSION_TOLERANCE
            if (reader.Eof)
            {
                value = default;
                return;
            }
        #endif
            reader.Read(out value);
        }
        
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void DeserializeRef(ref byte[] value, ref Reader reader)
        {
        #if WEAK_VERSION_TOLERANCE
            if (reader.Eof)
            {
                value = default;
                return;
            }
        #endif
            reader.ReadRef(ref value);
        }
        
#endregion

#region int[] - Generated by transformer Array
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void Deserialize(ReadOnlySpan<byte> data, out int[] value) 
        {
            var reader = new Reader(data);
            Deserialize(out value, ref reader);
        }

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void Deserialize(out int[] value, ref Reader reader)
        {
        #if WEAK_VERSION_TOLERANCE
            if (reader.Eof)
            {
                value = default;
                return;
            }
        #endif
            reader.Read(out value);
        }
        
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void DeserializeRef(ref int[] value, ref Reader reader)
        {
        #if WEAK_VERSION_TOLERANCE
            if (reader.Eof)
            {
                value = default;
                return;
            }
        #endif
            reader.ReadRef(ref value);
        }
        
#endregion

#region long? - Generated by transformer Nullable
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void Deserialize(ReadOnlySpan<byte> data, out long? value) 
        {
            var reader = new Reader(data);
            Deserialize(out value, ref reader);
        }

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void Deserialize(out long? value, ref Reader reader)
        {
        #if WEAK_VERSION_TOLERANCE
            if (reader.Eof)
            {
                value = default;
                return;
            }
        #endif
            reader.Read(out bool hasValue);
            if (!hasValue)
            {
                value = default;
                return;
            }
        
            reader.UnsafeRead(out long ret);
            value = ret;
        }
        
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static void DeserializeRef(ref long? value, ref Reader reader) => Deserialize(out value, ref reader);
        
#endregion

    }
}

Useful​

Download Example (.NET C#)​

tip

Download Example project Schema

Share Schema​

https://ignatandrei.github.io/RSCG_Examples/v2/docs/Schema

Category "Serializer" has the following generators:​

1 GenPack

2 jsonConverterSourceGenerator

3 JsonPolymorphicGenerator

4 mvvmgen

5 Nino

6 ProtobufSourceGenerator

7 Schema

8 StackXML

9 System.Text.Json

See category​

Serializer