C#百萬物件序列化深度剖析：如何在網路傳輸中實現速度與體積的完美平衡

目錄

1. 本文背景
2. 構建測試資料
3. 方案比較
   • Json序列化
   • 自訂二進位序列化
   • BinaryWriter\BinaryReader
   • ProtoBuf
   • MessagePack
   • 方案分析
   • 基準測試
4. 總結

1. 本文背景

大家好，我是沙漠盡頭的狼。

在網路通訊中，資料序列化是將物件狀態轉換為可儲存或可傳輸形式的過程，這對於TCP網路傳輸尤為關鍵。在專案中，當需要處理幾十萬筆資料的傳輸時，傳統的Json序列化方式由於其冗餘的欄位名稱和字串格式，導致了二進位封包體積龐大，且序列化與反序列化的效率低下。為了解決這些問題，我考慮採用更高效的序列化方法，以減少封包大小並提升處理速度。本文將探討自訂二進位序列化、BinaryWriter/BinaryReader、MessagePack和ProtoBuf等4種序列化方法，並透過比較它們的效能，為大家提供我目前認為的最佳實踐指南。

2. 構建測試資料

建立C#控制台程式，新增Organization和Member兩個類別，類別中包含基本的資料型別和List<T>，其他陣列、字典可以自行擴充：

public class Organization
{
    public int Id { get; set; }

    public string[]? Tags { get; set; }

    public List<Member>? Members { get; set; }
}

public class Member
{
    public int Id { get; set; }

    public string? Name { get; set; }

    public string? Description { get; set; }

    public string? Address { get; set; }

    public double Value { get; set; }

    public long UpdateTime { get; set; }
}

建立測試資料100萬筆，為了和標題遙相呼應：

public class BenchmarkTest
{
    /// <summary>
    /// 測試資料量
    /// </summary>
    private const int DataCount = 1000000;

    private static readonly Random RandomShared = new(DateTime.Now.Millisecond);

    /// <summary>
    /// 測試資料
    /// </summary>
    private static readonly Organization TestData = new()
    {
        Id = 1,
        Tags = Enumerable.Range(0, 5).Select(index => $"測試標籤{index}").ToArray(),
        Members = Enumerable.Range(0, DataCount).Select(index => new Member()
        {
            Id = index,
            Name = $"測試名字{index}",
            Description = $"測試描述{RandomShared.Next(1, int.MaxValue)}",
            Address = $"測試地址{RandomShared.Next(1, int.MaxValue)}",
            Value = RandomShared.Next(1, int.MaxValue),
            UpdateTime = ((DateTimeOffset)DateTime.Now).ToUnixTimeMilliseconds()
        }).ToList()
    };
}

3. 方案比較

首先建立序列化介面ISerializeHelper，各種序列化提供者需要分別實作：

public interface ISerializeHelper
{
    byte[] Serialize(Organization data);

    Organization? Deserialize(byte[] buffer);
}

再建立BenchmarkTest類別，新增RunSerialize方法用於執行序列化提供者，在此方法中依序呼叫提供者的序列化和反序列方法，統一對測試方法進行耗時和組包大小列印輸出統計：

public class BenchmarkTest
{
    // ...省略前面的程式碼
    private static void RunSerialize(ISerializeHelper helper)
    {
        Stopwatch sw = Stopwatch.StartNew();

        var buffer = helper.Serialize(TestData);

        sw.Stop();
        Log($"{helper.GetType().Name} Serialize {sw.ElapsedMilliseconds}ms {buffer.Length}byte");

        sw.Restart();

        var data = helper.Deserialize(buffer);

        sw.Stop();

        Log($"{helper.GetType().Name} Deserialize {sw.ElapsedMilliseconds}ms {data?.Members?.Count}項");
    }

    private static void Log(string log)
    {
        Console.WriteLine($"{DateTime.Now:yyyy-MM-dd HH:mm:ss fff}: {log}");
    }
}

3.1. Json序列化

首先測試Json序列化，安裝System.Text.Json套件：

<PackageReference Include="System.Text.Json" Version="8.0.0" />

建立JsonSerializeHelper提供者，並實作ISerializeHelper介面：

using ByteTest.Core.Models;
using System.Text.Json;

namespace ByteTest.Core.Helpers;

public class JsonSerializeHelper : ISerializeHelper
{
    public byte[] Serialize(Organization data)
    {
        return JsonSerializer.SerializeToUtf8Bytes(data);
    }

    public Organization? Deserialize(byte[] buffer)
    {
        var data = JsonSerializer.Deserialize<Organization>(buffer);
        return data;
    }
}

JsonSerializer.SerializeToUtf8Bytes 方法會直接產生位元組資料，而不是先產生字串然後再轉換為位元組，這對於一些需要將 JSON 資料寫入檔案或網路流的場景非常有用，因為這些場景通常需要位元組資料而不是字串。此外，由於避免了不必要的字串分配，它還可以提高效能並減少記憶體壓力。

再在BenchmarkTest類別新增測試方法Test：

public static void Test()
{
    RunSerialize(new JsonByteSerializeHelper());
}

在Program中呼叫Test()方法：

BenchmarkTest.Test();

程式輸出如下：

2023-12-10 22:28:24 880: JsonByteSerializeHelper Serialize 2813ms 196227181byte
2023-12-10 22:28:26 858: JsonByteSerializeHelper Deserialize 1964ms 1000000項

Json序列化100萬筆資料，需要2.8s，資料封包187.14MB，真大、真慢。

3.2. 自訂二進位序列化

接下來測試下自訂的二進位序列化，這是我原來常用的方式，目前看也是囉嗦，首先定義資料封包欄位規範：

新增CustomSerializeHelper實作介面ISerializeHelper：

using ByteTest.Core.Models;

namespace ByteTest.Core.Helpers;

public class CustomSerializeHelper : ISerializeHelper
{
    public byte[] Serialize(Organization data)
    {
        // 1、計算Id
        var idBuffer = BitConverter.GetBytes(data.Id);

        // 2、計算Tag陣列
        var tagBuffer = GetBytes(data.Tags);

        // 3、計算Members
        var membersBuffer = GetBytes(data.Members);

        return GetBytes(new[] { idBuffer, tagBuffer, membersBuffer });
    }

    public Organization? Deserialize(byte[] buffer)
    {
        var data = new Organization();
        var index = 0;

        data.Id = BitConverter.ToInt32(buffer, index);
        index += sizeof(int);

        data.Tags = GetTags(buffer, ref index);
        data.Members = GetMembers(buffer, ref index);
        return data;
    }

    /// <summary>
    /// 取得字串列表byte[]
    /// </summary>
    /// <param name="data"></param>
    /// <returns></returns>
    private byte[] GetBytes(string[]? data)
    {
        var dataCount = data?.Length ?? 0;
        var dataCountBuffer = BitConverter.GetBytes(dataCount);

        if (dataCount <= 0)
        {
            return dataCountBuffer;
        }

        var dataValueBuffers = data!.Select(item => ByteHelper.GetBytes(item)!).ToArray();
        var dataValueBuffer = GetBytes(dataValueBuffers);
        return GetBytes(new[] { dataCountBuffer, dataValueBuffer });
    }

    /// <summary>
    /// 取得成員列表byte[]
    /// </summary>
    /// <param name="data"></param>
    /// <returns></returns>
    private byte[] GetBytes(List<Member>? data)
    {
        var dataCount = data?.Count ?? 0;
        var dataCountBuffer = BitConverter.GetBytes(dataCount);

        if (dataCount <= 0)
        {
            return dataCountBuffer;
        }

        var dataValueBuffers = data!.Select(item =>
        {
            var idBuffer = BitConverter.GetBytes(item.Id);
            var nameBuffer = ByteHelper.GetBytes(item.Name);
            var descriptionBuffer = ByteHelper.GetBytes(item.Description);
            var addressBuffer = ByteHelper.GetBytes(item.Address);
            var valueBuffer = BitConverter.GetBytes(item.Value);
            var updateTimeBuffer = BitConverter.GetBytes(item.UpdateTime);

            var buffer = GetBytes(new byte[][]
                { idBuffer, nameBuffer, descriptionBuffer, addressBuffer, valueBuffer, updateTimeBuffer });
            return buffer;
        }).ToArray();
        var dataValueBuffer = GetBytes(dataValueBuffers);
        return GetBytes(new[] { dataCountBuffer, dataValueBuffer });
    }

    private byte[] GetBytes(byte[][] data)
    {
        var dataBufferLen = data.Sum(itemBuffer => itemBuffer.Length);
        var dataBuffer = new byte[dataBufferLen];
        var dataIndex = 0;
        for (var i = 0; i < data.Length; i++)
        {
            var itemBuffer = data[i];
            Array.Copy(itemBuffer, 0, dataBuffer, dataIndex, itemBuffer.Length);

            dataIndex += itemBuffer.Length;
        }

        return dataBuffer;
    }

    private string[]? GetTags(byte[] buffer, ref int index)
    {
        var count = BitConverter.ToInt32(buffer, index);
        index += sizeof(int);

        if (count <= 0)
        {
            return default;
        }

        var data = new string[count];
        for (var i = 0; i < count; i++)
        {
            data[i] = GetString(buffer, ref index);
        }

        return data;
    }

    private List<Member>? GetMembers(byte[] buffer, ref int index)
    {
        var count = BitConverter.ToInt32(buffer, index);
        index += sizeof(int);

        if (count <= 0)
        {
            return default;
        }

        var data = new List<Member>();
        for (var i = 0; i < count; i++)
        {
            var people = new Member();

            people.Id = BitConverter.ToInt32(buffer, index);
            index += sizeof(int);

            people.Name = GetString(buffer, ref index);

            people.Description = GetString(buffer, ref index);

            people.Address = GetString(buffer, ref index);

            people.Value = BitConverter.ToDouble(buffer, index);
            index += sizeof(double);

            people.UpdateTime = BitConverter.ToInt64(buffer, index);
            index += sizeof(long);

            data.Add(people);
        }

        return data;
    }

    private string GetString(byte[] buffer, ref int index)
    {
        var count = BitConverter.ToInt32(buffer, index);
        index += sizeof(int);

        if (count <= 0)
        {
            return string.Empty;
        }

        var data = ByteHelper.DefaultEncoding.GetString(buffer, index, count);
        index += count;
        return data;
    }
}

程式碼不少，看個大概就好，主要用BitConverter.GetBytes和BitConverter.ToXXX取得或設定基本資料型別的byte[]資料，對於多byte[]的複製，如果資料較小，用Array.Copy，如果資料較大建議用Buffer.BlockCopy。

程式碼中使用到輔助類別ByteHelper，用於計算字串byte[]：

public static class ByteHelper
{
    public static Encoding DefaultEncoding = Encoding.UTF8;

    /// <summary>
    /// 取得字串二進位資料：字串二進位資料=4個位元組int表示字串實際值長度+n個位元組表示字串實際值
    /// </summary>
    /// <param name="str"></param>
    /// <returns></returns>
    public static byte[] GetBytes(string? str)
    {
        if (string.IsNullOrEmpty(str))
        {
            return BitConverter.GetBytes(0);
        }

        var strValueBuffer = DefaultEncoding.GetBytes(str);
        var strValueLen = strValueBuffer.Length;
        var strValueLenBuffer = BitConverter.GetBytes(strValueLen);

        var strBufferLen = sizeof(int) + strValueLen;
        var strBuffer = new byte[strBufferLen];

        var index = 0;
        Array.Copy(strValueLenBuffer, 0, strBuffer, index, sizeof(int));
        index += sizeof(int);

        Array.Copy(strValueBuffer, 0, strBuffer, index, strValueLen);

        return strBuffer;
    }
}

我們修改Test方法，加入CustomSerializeHelper測試：

public static void Test()
{
    var serializeHelpers = new List<ISerializeHelper>
    {
        new JsonSerializeHelper(),
        new CustomSerializeHelper()
    };

    serializeHelpers.ForEach(RunSerialize);
}

程式輸出如下：

2023-12-10 22:45:14 701: JsonSerializeHelper Serialize 2774ms 196225588byte
2023-12-10 22:45:16 613: JsonSerializeHelper Deserialize 1898ms 1000000項
2023-12-10 22:45:17 414: CustomSerializeHelper Serialize 801ms 92854209byte
2023-12-10 22:45:18 072: CustomSerializeHelper Deserialize 657ms 1000000項

能看出最佳化了不少，自訂二進位序列化減少1億個位元組（100MB左右），序列化（組包）快2s左右，反序列化（解包）快1s多。

3.3. BinaryWriter\BinaryReader

BinaryWriter 和 BinaryReader 類別是用於以二進位格式寫入和讀取資料的類別。它們分別提供了一系列的方法來寫入和讀取各種基本資料型別（如int, float, double, string等）的二進位表示。這些類別通常與檔案串流（FileStream）一起使用，但也可以與其他型別的串流（如MemoryStream）配合使用。

自訂的方式全手工操作，需要自己進行位元組陣列的複製，各種轉換，有點原始，使用BinaryWriter\BinaryReader進行序列化操作應該二進位序列化的標準用法了。

建立BinarySerializeHelper類別，並實作ISerializeHelper介面：

public class BinarySerializeHelper : ISerializeHelper
{
    public byte[] Serialize(Organization data)
    {
        using var stream = new MemoryStream();
        using var writer = new BinaryWriter(stream, ByteHelper.DefaultEncoding);
        writer.Write(data.Id);
        Write(writer, data.Tags);
        Write(writer, data.Members);

        return stream.ToArray();
    }

    public Organization Deserialize(byte[] buffer)
    {
        var data = new Organization();
        using var stream = new MemoryStream(buffer);
        using var reader = new BinaryReader(stream, ByteHelper.DefaultEncoding);
        data.Id = reader.ReadInt32();
        data.Tags = ReadStringList(reader);
        data.Members = ReadPeopleList(reader);

        return data;
    }

    private static void Write(BinaryWriter writer, string[]? data)
    {
        var count = data?.Length ?? 0;
        writer.Write(count);
        if (count <= 0)
        {
            return;
        }

        foreach (var item in data!)
        {
            writer.Write(item);
        }
    }

    private static void Write(BinaryWriter writer, List<Member>? data)
    {
        var count = data?.Count ?? 0;
        writer.Write(count);
        if (count > 0)
        {
            foreach (var item in data)
            {
                writer.Write(item.Id);
                writer.Write(item.Name ?? string.Empty);
                writer.Write(item.Description ?? string.Empty);
                writer.Write(item.Address ?? string.Empty);
                writer.Write(item.Value);
                writer.Write(item.UpdateTime);
            }
        }
    }

    private static string[]? ReadStringList(BinaryReader reader)
    {
        var count = reader.ReadInt32();
        if (count <= 0)
        {
            return default;
        }

        var values = new string[count];
        for (int i = 0; i < count; i++)
        {
            values[i] = reader.ReadString();
        }

        return values;
    }

    private static List<Member>? ReadPeopleList(BinaryReader reader)
    {
        var count = reader.ReadInt32();
        if (count <= 0)
        {
            return default;
        }

        var values = new List<Member>();
        for (int i = 0; i < count; i++)
        {
            var item = new Member();
            item.Id = reader.ReadInt32();
            item.Name = reader.ReadString();
            item.Description = reader.ReadString();
            item.Address = reader.ReadString();
            item.Value = reader.ReadDouble();
            item.UpdateTime = reader.ReadInt64();

            values.Add(item);
        }

        return values;
    }
}

程式碼也不少，列表的序列化和反序列在上面自訂二進位序列化時就應該封裝成方法，透過反射實現通用列表的序列化和反序列化，這一小節也是，不想再折騰了，我們在BenchmarkTest類別的Test方法內加上BinarySerializeHelper，再執行程式：

2023-12-10 22:52:56 986: JsonSerializeHelper Serialize 2715ms 196225584byte
2023-12-10 22:52:58 910: JsonSerializeHelper Deserialize 1910ms 1000000項
2023-12-10 22:52:59 730: CustomSerializeHelper Serialize 819ms 92853722byte
2023-12-10 22:53:00 389: CustomSerializeHelper Deserialize 659ms 1000000項
2023-12-10 22:53:00 660: BinarySerializeHelper Serialize 269ms 83853707byte
2023-12-10 22:53:01 466: BinarySerializeHelper Deserialize 806ms 1000000項

BinaryWriter\BinaryReader序列化封包又比自訂二進位序列化方式小8.5MB左右，序列化速度也快0.5s，反序列化稍微慢點，不錯喲。

3.4. ProtoBuf

大家聽過或使用過Google 的 Protocol Buffers吧？

本小節介紹使用protobuf-net函式庫，這是一個在 .NET 環境中使用的函式庫，它提供了對 Google 的 Protocol Buffers 資料序列化格式的支援。Protocol Buffers 是一種輕量級、高效的結構化資料序列化機制，通常用於跨服務或應用程式的通訊，以及資料儲存。

安裝套件protobuf-net：

<PackageReference Include="protobuf-net" Version="3.2.30" />

給測試的類別新增類別序列化特性[ProtoContract]和屬性序列化特性[ProtoMember(序列化順序)]：

[ProtoContract]
public class Organization
{
    [ProtoMember(1)] public int Id { get; set; }

    [ProtoMember(2)] public string[]? Tags { get; set; }

    [ProtoMember(3)] public List<Member>? Members { get; set; }
}

[ProtoContract]
public class Member
{
    [ProtoMember(1)] public int Id { get; set; }

    [ProtoMember(2)] public string? Name { get; set; }

    [ProtoMember(3)] public string? Description { get; set; }

    [ProtoMember(4)] public string? Address { get; set; }

    [ProtoMember(5)] public double Value { get; set; }

    [ProtoMember(6)] public long UpdateTime { get; set; }
}

新增ProtoBufSerializeHelper類別，並實作ISerializeHelper介面：

using ByteTest.Core.Models;
using ProtoBuf;

namespace ByteTest.Core.Helpers;

public class ProtoBufSerializeHelper : ISerializeHelper
{
    public byte[] Serialize(Organization data)
    {
        using var stream = new MemoryStream();
        Serializer.Serialize(stream, data);
        return stream.ToArray();
    }

    public Organization? Deserialize(byte[] buffer)
    {
        using var stream = new MemoryStream(buffer);
        return Serializer.Deserialize<Organization>(stream);
    }
}

還有一歩，給上面的Test方法加上ProtoBufSerializeHelper序列化方式，程式輸出如下：

2023-12-10 23:01:17 478: JsonSerializeHelper Serialize 2767ms 196225803byte
2023-12-10 23:01:19 556: JsonSerializeHelper Deserialize 2064ms 1000000項
2023-12-10 23:01:20 350: CustomSerializeHelper Serialize 793ms 92853782byte
2023-12-10 23:01:21 012: CustomSerializeHelper Deserialize 662ms 1000000項
2023-12-10 23:01:21 271: BinarySerializeHelper Serialize 258ms 83853767byte
2023-12-10 23:01:22 086: BinarySerializeHelper Deserialize 815ms 1000000項
2023-12-10 23:01:22 629: ProtoBufSerializeHelper Serialize 542ms 88837248byte
2023-12-10 23:01:23 688: ProtoBufSerializeHelper Deserialize 1058ms 1000000項

咦，ProtoBuf比BinaryWriter的序列化封包還大，並且還慢，難道我用的不對？可能還需要新增壓縮演算法吧，後面再研究了，我們繼續看最後一個MessagePack，有使用問題歡迎指出。

3.5. MessagePack

介紹最後一種序列化套件MessagePack，這是一種高效能的二進位序列化格式，它允許資料在不同的系統之間進行快速且緊湊的傳輸。它類似於JSON，但是更小、更快、更節省空間。

需要安裝套件MessagePack套件：

<PackageReference Include="MessagePack" Version="2.6.100-alpha" />

新增類別MessagePackSerializeHelper，並實作介面ISerializeHelper：

using ByteTest.Core.Models;
using MessagePack;

namespace ByteTest.Core.Helpers;

public class MessagePackSerializeHelper : ISerializeHelper
{
    // 這種方式需要在類別和欄位上新增特性，稍顯麻煩，但新增壓縮選項後，組包體積、組包和解包速度更快
    //readonly MessagePackSerializerOptions _options = MessagePackSerializerOptions.Standard.WithCompression(MessagePackCompression.Lz4BlockArray);

    // 這種方式不需要給傳輸物件新增特性，也可新增壓縮選項
    readonly MessagePackSerializerOptions _options =
        MessagePack.Resolvers.ContractlessStandardResolver.Options.WithCompression(MessagePackCompression
            .Lz4BlockArray);

    public byte[] Serialize(Organization data)
    {
        return MessagePackSerializer.Serialize(data, _options);
    }

    public Organization? Deserialize(byte[] buffer)
    {
        return MessagePackSerializer.Deserialize<Organization>(buffer, _options);
    }
}

看上面的註解程式碼，提供的選項可能更優，壓縮體積更小，後面我們新增測試。

最後修改Test方法：

   public static void Test(List<ISerializeHelper>? moreHelpers = null)
    {
        var serializeHelpers = new List<ISerializeHelper>
        {
            new JsonSerializeHelper(),
            new CustomSerializeHelper(),
            new BinarySerializeHelper(),
            new ProtoBufSerializeHelper(),
            new MessagePackSerializeHelper(),
        };
        if (moreHelpers?.Count() > 0)
        {
            serializeHelpers.AddRange(moreHelpers);
        }

        serializeHelpers.ForEach(RunSerialize);
    }

執行程式輸出：

2023-12-11 21:34:47 782: JsonSerializeHelper Serialize 2456ms 196225500byte
2023-12-11 21:34:51 215: JsonSerializeHelper Deserialize 3430ms 1000000項
2023-12-11 21:34:52 186: CustomSerializeHelper Serialize 970ms 92853911byte
2023-12-11 21:34:52 711: CustomSerializeHelper Deserialize 526ms 1000000項
2023-12-11 21:34:53 734: BinarySerializeHelper Serialize 1022ms 83853896byte
2023-12-11 21:34:54 354: BinarySerializeHelper Deserialize 620ms 1000000項
2023-12-11 21:34:55 170: ProtoBufSerializeHelper Serialize 815ms 88837377byte
2023-12-11 21:34:56 205: ProtoBufSerializeHelper Deserialize 1035ms 1000000項
2023-12-11 21:34:57 123: MessagePackSerializeHelper Serialize 917ms 43583878byte
2023-12-11 21:34:58 527: MessagePackSerializeHelper Deserialize 1403ms 1000000項

再切換上面的註解選項，程式碼修改如下：

using ByteTest.Core.Models;
using MessagePack;

namespace ByteTest.Core.Helpers;

public class MessagePackSerializeHelper : ISerializeHelper
{
    // 這種方式需要在類別和欄位上新增特性，稍顯麻煩，但新增壓縮選項後，組包體積、組包和解包速度更快
    readonly MessagePackSerializerOptions _options = MessagePackSerializerOptions.Standard.WithCompression(MessagePackCompression.Lz4BlockArray);

    // 這種方式不需要給傳輸物件新增特性，也可新增壓縮選項
    //readonly MessagePackSerializerOptions _options =
    //    MessagePack.Resolvers.ContractlessStandardResolver.Options.WithCompression(MessagePackCompression
    //        .Lz4BlockArray);

    public byte[] Serialize(Organization data)
    {
        return MessagePackSerializer.Serialize(data, _options);
    }

    public Organization? Deserialize(byte[] buffer)
    {
        return MessagePackSerializer.Deserialize<Organization>(buffer, _options);
    }
}

並給傳輸類別新增特性[MessagePackObject]，需要序列化的屬性新增特性[Key(序列化索引)]：

using MessagePack;
using ProtoBuf;

namespace ByteTest.Core.Models;

[ProtoContract]
[MessagePackObject]
public class Organization
{
    [ProtoMember(1)] [Key(0)] public int Id { get; set; }

    [ProtoMember(2)] [Key(1)] public string[]? Tags { get; set; }

    [ProtoMember(3)] [Key(2)] public List<Member>? Members { get; set; }
}

[ProtoContract]
[MessagePackObject]
public class Member
{
    [ProtoMember(1)] [Key(0)] public int Id { get; set; }

    [ProtoMember(2)] [Key(1)] public string? Name { get; set; }

    [ProtoMember(3)] [Key(2)] public string? Description { get; set; }

    [ProtoMember(4)] [Key(3)] public string? Address { get; set; }

    [ProtoMember(5)] [Key(4)] public double Value { get; set; }

    [ProtoMember(6)] [Key(5)] public long UpdateTime { get; set; }
}

程式輸出：

2023-12-11 21:49:34 153: JsonSerializeHelper Serialize 2383ms 196226140byte
2023-12-11 21:49:37 736: JsonSerializeHelper Deserialize 3581ms 1000000項
2023-12-11 21:49:38 720: CustomSerializeHelper Serialize 983ms 92854251byte
2023-12-11 21:49:39 250: CustomSerializeHelper Deserialize 530ms 1000000項
2023-12-11 21:49:40 273: BinarySerializeHelper Serialize 1023ms 83854236byte
2023-12-11 21:49:40 907: BinarySerializeHelper Deserialize 632ms 1000000項
2023-12-11 21:49:41 660: ProtoBufSerializeHelper Serialize 754ms 88837717byte
2023-12-11 21:49:42 676: ProtoBufSerializeHelper Deserialize 1014ms 1000000項
2023-12-11 21:49:43 357: MessagePackSerializeHelper Serialize 681ms 38706475byte
2023-12-11 21:49:44 344: MessagePackSerializeHelper Deserialize 986ms 1000000項

這裡再貼出上一個選項MessagePack的輸出：

2023-12-11 21:34:57 123: MessagePackSerializeHelper Serialize 917ms 43583878byte
2023-12-11 21:34:58 527: MessagePackSerializeHelper Deserialize 1403ms 1000000項

看出還是第一個選項序列化封包體積和速度更優秀。

方案分析

100萬筆測試資料，5種序列化方法測試統計資料列出表格如下：

透過上表分析，序列化後，MessagePack的封包最小，為36.91MB，Json最大達到187.13MB，另三種在80MB左右；如果考慮序列化效率MessagePack最好，反序列化效率竟然是我的自訂二進位方式最快？我們去掉Json方式，再執行一次：

2023-12-11 21:55:47 813: CustomSerializeHelper Serialize 1263ms 92854890byte
2023-12-11 21:55:48 804: CustomSerializeHelper Deserialize 989ms 1000000項
2023-12-11 21:55:49 215: BinarySerializeHelper Serialize 410ms 83854875byte
2023-12-11 21:55:50 081: BinarySerializeHelper Deserialize 866ms 1000000項
2023-12-11 21:55:50 726: ProtoBufSerializeHelper Serialize 644ms 88838356byte
2023-12-11 21:55:51 725: ProtoBufSerializeHelper Deserialize 999ms 1000000項
2023-12-11 21:55:52 426: MessagePackSerializeHelper Serialize 701ms 38701799byte
2023-12-11 21:55:53 427: MessagePackSerializeHelper Deserialize 999ms 1000000項

組包大小不變，序列化使用BinaryWriter最快，反序列也是BinaryReader，測試資料不可靠呀，我們使用基準測試。

基準測試

安裝BenchmarkDotNet套件用於基準測試：

<PackageReference Include="BenchmarkDotNet" Version="0.13.11" />

修改BenchmarkTest類別：

using BenchmarkDotNet.Attributes;
using ByteTest.Core.Helpers;
using ByteTest.Core.Models;
using MessagePack;
using System.Diagnostics;

namespace ByteTest.Core.Test;

[MemoryDiagnoser, RankColumn]
public class BenchmarkTest
{
    // 省略測試資料程式碼

    //[Benchmark]
    //public void JsonByteSerialize()
    //{
    //    RunSerialize(new JsonSerializeHelper());
    //}

    [Benchmark]
    public void CustomSerialize()
    {
        RunSerialize(new CustomSerializeHelper());
    }

    [Benchmark]
    public void BinarySerialize()
    {
        RunSerialize(new BinarySerializeHelper());
    }

    [Benchmark]
    public void ProtoBufPackSerialize()
    {
        RunSerialize(new ProtoBufSerializeHelper());
    }

    [Benchmark]
    public void MessagePackSerialize()
    {
        RunSerialize(new MessagePackSerializeHelper());
    }
	
    // 省略統計相關程式碼
}

修改Program.cs：

using BenchmarkDotNet.Running;
using ByteTest.Core.Test;

// 執行基準測試
BenchmarkRunner.Run<BenchmarkTest>();

// 普通測試
//BenchmarkTest.Test();

測試結果如下：

大致看下，體積肯定MessagePack佔優秀，網路傳輸中分片更少，意味著網路來回時間花費少；組包（封包）和解包（拆包）原生的BinaryWriter和BinaryReader更優。

4. 總結

總的來說，資料封包大小需要根據網路環境和裝置能力來合理設定，以確保高效的資料傳輸。同時，高效的組包和解包處理能力對於維持網路傳輸效能也是至關重要的，前者可考慮MessagePack進行壓縮，後者考慮原生BinaryWriter和BinaryReader。

你有更好的方式推薦嗎？歡迎留言討論，您也可以給本文測試程式碼提PR哦，連結地址是ByteTest

參考

• msgpack/msgpack: MessagePack is an extremely efficient object serialization library. It's like JSON, but very fast and small. (github.com)

• MessagePack簡介及使用 - 簡書 (jianshu.com)