Skip to main content

Schema by Melty Player

NuGet / site data

Nuget GitHub last commit GitHub Repo stars

Details

Info

info

Name: Schema

Library for converting classes to and from binary. Provides a C# Roslyn generator that automatically implements conversion logic for simple classes.

Author: Melty Player

NuGet: https://www.nuget.org/packages/Schema/

You can find more details at https://github.com/MeltyPlayer/Schema

Source: https://github.com/MeltyPlayer/Schema

Author

note

Melty Player Alt text

Original Readme

note

Schema

GitHub Nuget Nuget Unit tests Coverage Status

Overview

Library for serializing C# types to/from binary. Provides a Roslyn generator that automatically implements read/write logic.

Warning: The design of this library is still in flux, so anticipate making changes when upgrading to future versions.

Credits

Usage

Implementing binary schema classes

To write a binary schema class, you must first have it implement the IBinarySerializable or IBinaryDeserializable interfaces (or IBinaryConvertible if you need both).

Then, based on how complicated your schema class is, you can either choose to automatically or manually implement Read()/Write() methods.

Automatically

For most schema classes, you should be able to use the automatic code generator.

All you have to do is annotate the schema class with the [BinarySchema] attribute and mark it as partial; this will flag to the generator that it should implement read/write methods for this class. It will then look into all fields/properties in the schema class, and attempt to implement read/write logic in the same order that the fields/properties appear.

Any nested schema classes will be automatically read/written as expected.

Some types require additional attributes in order to clarify any ambiguity. For example, booleans require a [IntegerFormat(SchemaIntegerType.###)] attribute to know what type of integer to read, which it will then compare to 0.

Any readonly primitives will treated as assertions, which is useful for validating things like magic text or padding.

Manually

For complicated schema classes, such as ones that use decompression logic or pointers, you'll need to implement the read/write logic manually.

Specifically, you'll need to implement both a Read(IBinaryReader br) and Write(IBinaryWriter bw) method. The SchemaBinaryReader and SchemaBinaryWriter classes provide many helpful methods for reading/writing a number of different primitive formats, including basic ones such as byte/int/float, but also more complex/unique ones such as Half (two-byte float) and un16 (unsigned normalized 16-bit float).

Similar to the automatic process, you can nest schema classes and manually read/write them by calling their Read()/Write() methods. This can allow you to automatically generate subsections, so only the most complex logic needs to be manually written.

How to use a binary schema class

To convert a given schema class to or from binary, simply instantiate an SchemaBinaryReader or SchemaBinaryWriter and pass it into the corresponding Read() or Write() methods in the schema class.

Supported Attributes

The following attributes are currently supported in this library when automatically generating code. Some attributes are only used at read or write time—these are prefixed with an R or W respectively.

Warning: These names are not final, so they may change in future versions.

Align

Specifies how a field or property's offset (relative to the start of the stream) should be aligned when reading/writing. If misaligned, the SchemaBinaryReader/SchemaBinaryWriter will automatically insert the remaining bytes of padding. For example, [Align(4)] would force a field/property's starting offset to be a multiple of 4 (0, 4, 8, 12, 16, etc.).

[Align(4)]
public int alignedField;

[Align(4)]
public int AlignedProperty \{ get; set; }

Endianness

Forces a type, field, or property to be read/written with a given endianness (big-endian or little-endian). Tracked via a stack within the SchemaBinaryReader/SchemaBinaryWriter. If unspecified, will use whatever endianness was last specified in the stack (or the system endianness by default).

[BinarySchema]
[Endianness(Endianness.BigEndian)]
public partial class BigEndianType : IBinaryConvertible {
...

[Endianness(Endianness.LittleEndian)]
public int LittleEndianProperty \{ get; set; }

...
}

IfBoolean/RIfBoolean

Marks that a nullable field or property will only be read/written if some other boolean field or property is true.

[IntegerFormat(SchemaIntegerType.BYTE)]
public bool HasValue \{ get; set; }

[RIfBoolean(nameof(this.HasValue))]
public int? Value \{ get; set; }

IChildOf<TParent>

This pseudo-attribute marks a type as a "child" of some "parent" type—that it is contained as one of the members of the "parent type"—and passes the parent down to the child so it can be referenced in Schema logic.

Used by having the child type implement the IChildOf<TParent> interface, where TParent stores the child type in a field/property or as a member of a sequence (array/list):

[BinarySchema]
public partial class ChildType : IBinaryConvertible, IChildOf<ParentType> {
public ParentType Parent \{ get; set; }

...
}

Below is a simple example where a boolean from the parent is used to decide when to read a value in the child:

[BinarySchema]
public partial class ParentType : IBinaryConvertible {
[IntegerFormat(SchemaIntegerType.BYTE)]
public bool ChildHasSomeField \{ get; set; }

public ChildType Child \{ get; \} = new();
}

[BinarySchema]
public partial class ChildType : IBinaryConvertible, IChildOf<ParentType> {
// This is automatically skipped while reading/writing.
public ParentType Parent \{ get; set; }

[Skip]
private bool HasSomeField => Parent.ChildHasSomeField;

[RIfBoolean(nameof(HasSomeField))]
public int? SomeField \{ get; set; }
}

Skip

Designates that a field or property should be skipped while reading/writing.

Note: IChildOf<TParent>.Parent is automatically skipped.

[Skip]
public int skippedField;

[Skip]
public int SkippedProperty \{ get; set; }

This can be used to encapsulate logic within properties, such as in the following examples:

  1. Value conversion
[StringLengthSource(4)]
public string Magic \{ get; set; }

[Skip]
public MagicType Type => this.Magic switch {
"IMGE" => MagicType.IMAGE,
"SOND" => MagicType.SOUND,
"TEXT" => MagicType.TEXT,
};
  1. "Switch" cases
[NullTerminatedString]
public string Magic \{ get; set; }

[Skip]
public ISection? Section => this.imageSection_ ?? this.soundSection_ ?? this.textSection_;

[Skip]
private bool IsImage_ => this.Magic == "IMAGE";
[Skip]
private bool IsSound_ => this.Magic == "SOUND";
[Skip]
private bool IsText_ => this.Magic == "TEXT";

[RIfBoolean(nameof(this.IsImage))]
private ImageSection? imageSection_ \{ get; set; }

[RIfBoolean(nameof(this.IsSound_))]
private SoundSection? soundSection_ \{ get; set; }

[RIfBoolean(nameof(this.IsText_))]
private TextSection? textSection_ \{ get; set; }

Numbers/Enums

NumberFormat

TODO

IntegerFormat

TODO

Strings

Note: At the moment, only ASCII is fully supported.

StringLengthSource/RStringLengthSource

Designates the length of a string field or property via one of three cases.

Note: Any trailing null terminators will be ignored at read time.

  1. Constant length

If a constant is passed into StringLengthSource, that many characters will be read/written.

[StringLengthSource(8)]
public string Text \{ get; set; }
  1. Preceding value

If a SchemaIntegerType is passed into StringLengthSource, an integer of that type will first be read and used as the length of the string, or the length of the string will first be written before writing the string itself.

[StringLengthSource(SchemaIntegerType.BYTE)]
public string TextWithByteLength \{ get; set; }
  1. Another field or property

If the name of another field or property is passed into RStringLengthSource, that other value will be used as the length of the string when reading.

public byte TextLength \{ get; set; }

[RStringLengthSource(nameof(this.TextLength))]
public string Text \{ get; set; }
NullTerminatedString

Designates that a string field or property will be read until a null terminator is reached, and written with a null terminator affixed to the end.

[NullTerminatedString]
public string Text \{ get; set; }

Sequences

Note: "Sequence" is the term used within Schema to refer to an array/list of elements.

SequenceLengthSource/RSequenceLengthSource

TODO

RSequenceUntilEndOfStreamAttribute

TODO

Pointers/Memory

TODO

WPointerTo

TODO

WSizeOfMemberInBytes

TODO

About

note

Generate binary serialization code

How to use

Example (source csproj, source files)

This is the CSharp Project that references Schema

<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net9.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<PropertyGroup>
<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
<CompilerGeneratedFilesOutputPath>$(BaseIntermediateOutputPath)\GX</CompilerGeneratedFilesOutputPath>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="CommunityToolkit.HighPerformance" Version="8.3.0" />
<PackageReference Include="schema" Version="0.6.13" />
</ItemGroup>

</Project>

Generated Files

Those are taken from $(BaseIntermediateOutputPath)\GX

// <auto-generated/>
#pragma warning disable CS8669

using System;
using global::Nino.Core;
using System.Buffers;
using System.Collections.Generic;
using System.Collections.Concurrent;
using System.Runtime.InteropServices;
using System.Runtime.CompilerServices;

namespace Serializer.NinoGen
{
public static partial class Deserializer
{
#region System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, object?>> - Generated by transformer TrivialEnumerableUsingAdd
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void Deserialize(ReadOnlySpan<byte> data, out System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, object?>> value)
{
var reader = new Reader(data);
Deserialize(out value, ref reader);
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void Deserialize(out System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, object?>> value, ref Reader reader)
{
#if WEAK_VERSION_TOLERANCE
if (reader.Eof)
{
value = default;
return;
}
#endif

if (!reader.ReadCollectionHeader(out var length))
{
value = default;
return;
}

#if WEAK_VERSION_TOLERANCE
Reader eleReader;
#endif

var lst = new System.Collections.Generic.List<System.Collections.Generic.KeyValuePair<string, object?>>();
for (int i = 0; i < length; i++)
{
#if WEAK_VERSION_TOLERANCE
eleReader = reader.Slice();
NinoDeserializer.Deserialize(out System.Collections.Generic.KeyValuePair<string, object?> item, ref eleReader);
#else
NinoDeserializer.Deserialize(out System.Collections.Generic.KeyValuePair<string, object?> item, ref reader);
#endif
lst.Add(item);
}

value = lst;
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void DeserializeRef(ref System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, object?>> value, ref Reader reader) => Deserialize(out value, ref reader);

#endregion

#region System.Collections.Generic.KeyValuePair<string, object?> - Generated by transformer KeyValuePair
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void Deserialize(ReadOnlySpan<byte> data, out System.Collections.Generic.KeyValuePair<string, object?> value)
{
var reader = new Reader(data);
Deserialize(out value, ref reader);
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void Deserialize(out System.Collections.Generic.KeyValuePair<string, object?> value, ref Reader reader)
{
#if WEAK_VERSION_TOLERANCE
if (reader.Eof)
{
value = default;
return;
}
#endif
NinoDeserializer.Deserialize(out string k, ref reader);
object v = NinoDeserializer.DeserializeBoxed(ref reader, null);
value = new System.Collections.Generic.KeyValuePair<string, object?>(k, v);
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void DeserializeRef(ref System.Collections.Generic.KeyValuePair<string, object?> value, ref Reader reader)
=> Deserialize(out value, ref reader);

#endregion

#region byte[] - Generated by transformer Array
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void Deserialize(ReadOnlySpan<byte> data, out byte[] value)
{
var reader = new Reader(data);
Deserialize(out value, ref reader);
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void Deserialize(out byte[] value, ref Reader reader)
{
#if WEAK_VERSION_TOLERANCE
if (reader.Eof)
{
value = default;
return;
}
#endif
reader.Read(out value);
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void DeserializeRef(ref byte[] value, ref Reader reader)
{
#if WEAK_VERSION_TOLERANCE
if (reader.Eof)
{
value = default;
return;
}
#endif
reader.ReadRef(ref value);
}

#endregion

#region int[] - Generated by transformer Array
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void Deserialize(ReadOnlySpan<byte> data, out int[] value)
{
var reader = new Reader(data);
Deserialize(out value, ref reader);
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void Deserialize(out int[] value, ref Reader reader)
{
#if WEAK_VERSION_TOLERANCE
if (reader.Eof)
{
value = default;
return;
}
#endif
reader.Read(out value);
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void DeserializeRef(ref int[] value, ref Reader reader)
{
#if WEAK_VERSION_TOLERANCE
if (reader.Eof)
{
value = default;
return;
}
#endif
reader.ReadRef(ref value);
}

#endregion

#region long? - Generated by transformer Nullable
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void Deserialize(ReadOnlySpan<byte> data, out long? value)
{
var reader = new Reader(data);
Deserialize(out value, ref reader);
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void Deserialize(out long? value, ref Reader reader)
{
#if WEAK_VERSION_TOLERANCE
if (reader.Eof)
{
value = default;
return;
}
#endif
reader.Read(out bool hasValue);
if (!hasValue)
{
value = default;
return;
}

reader.UnsafeRead(out long ret);
value = ret;
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void DeserializeRef(ref long? value, ref Reader reader) => Deserialize(out value, ref reader);

#endregion

}
}

Useful

Download Example (.NET C#)

Share Schema

https://ignatandrei.github.io/RSCG_Examples/v2/docs/Schema

Category "Serializer" has the following generators:

1 GenPack

2 jsonConverterSourceGenerator

3 JsonPolymorphicGenerator

4 Nino

5 ProtobufSourceGenerator

6 Schema

7 StackXML

8 System.Text.Json

See category

Serializer