In actual development, an application may be composed of multiple applications, so the data interaction between the various components of the application becomes the key. How can we conduct data interaction quickly and efficiently? If it is a cross-server process interaction, remote procedure call technologies (RPC) such as Remoting, WCF, and GRPC can be used. This method will be transmitted through the network card, which will cause certain performance consumption such as data conversion and network transmission. If it is data interaction between processes on the same server and this remote procedure call technology is also adopted, it is not the optimal solution. So how can we bypass the network for cross-process data interaction? The answer is "shared memory". Today, we will use a simple small example to briefly describe the application of how to interact with data between processes through shared memory. It is only for learning and sharing. If there are any shortcomings, please correct them.
What is shared memory?
In the operating system, the system allocates an independent piece of memory for each process to run programs and store data. The memories between various processes are independent and do not interfere with each other to ensure stable and orderly operation of the program. Although this process memory protection mechanism largely ensures data security and program stability, it also forms insurmountable barriers between processes that need to interact. Fortunately, the operating system also takes into account this situation, which is shared memory. Shared Memory is an inter-process communication (IPC) mechanism that allows multiple processes to share the same piece of physical memory, thereby improving data exchange efficiency. Compared with other IPC methods (such as pipes, message queues, etc.), shared memory has the advantages of fast speed and low overhead because data is stored directly in memory without the need to copy data through the kernel.
Shared memory in. NET
On the. NET platform, shared memory is implemented through MemoryMappedFile. Memory mapping files allow you to reserve an address space and then map that physical storage to this memory space for operations. Physical storage is file management, while memory-mapped files are ** operating system-level memory management. ** The main knowledge points involved in memory-mapped file technology are as follows:
- Create shared memory: There are two ways to create memory-mapped files. One is to create them directly, which is achieved through the MemoryMappedFile.CreateNew method; the other is to create them through existing files, which is achieved by calling MemoryMappedFile.CreateFromFile.
- Shared memory accessor: In. NET, shared memory is accessed through MemoryMappedViewAccessor, which is created through the CreateViewAccessor of the instance object of MemoryMappedFile.
- Read and write method: Data is stored in the form of Byte arrays in shared memory. You can read and write byte arrays through ReadArray and WriteArray of the accessor.
Memory mapped file creation
There are two ways to create memory-mapped files:
The first is direct creation, which is created using the static method CreateNew provided by MemoryMappedFile, as follows:
The second type is created through the static method CreateFromFile provided by MemoryMappedFile. It is created using existing files or file streams, as follows:
Open an existing memory-mapped file, which is implemented using the static method OpenExist provided by MemoryMappedFile, as follows:
Create or open a memory-mapped file, which is achieved through the static method CreateOrOpen provided by MemoryMappedFile, as follows:
Memory-mapped file accessor
Memory-mapped file accessor, mainly used to operate shared memory, is implemented through the CreateViewAccessor of the MemoryMappedFile instance object, as follows:
Memory mapped file resource release
MemoryMappedFile implements the IDisposable interface and simply calls its Dispose method directly.
Shared memory application steps
In this example, there are mainly two WinForm executable programs, which read/write to shared memory respectively. During running, the two programs run simultaneously, as follows:
It mainly introduces the fixed-format Struct data read-write shared memory and the invariable length data read-write shared memory.
Memory mapped file object creation
This example creates a memory mapping file when writing shared memory, and opens the memory mapping file when reading shared memory, as shown below:
/// <summary>
/// 创建或打开共享内存
/// </summary>
public void CreateOrOpenSharedMemory()
{
this.memoryMapped = MemoryMappedFile.CreateOrOpen(this.MapName, this.capacity, MemoryMappedFileAccess.ReadWriteExecute, MemoryMappedFileOptions.None, HandleInheritability.Inheritable);
this.memoryAccessor = this.memoryMapped.CreateViewAccessor();
}
/// <summary>
/// 从文件创建共享内存
/// </summary>
public void CreateFromFileShareMemory()
{
this.memoryMapped = MemoryMappedFile.CreateFromFile(new FileStream(@"", FileMode.Create), this.MapName, this.capacity, MemoryMappedFileAccess.ReadWriteExecute, HandleInheritability.Inheritable, true);
this.memoryAccessor = this.memoryMapped.CreateViewAccessor();
}
/// <summary>
/// 打开已存在的共享内存
/// </summary>
public void OpenShareMemory()
{
this.memoryMapped = MemoryMappedFile.OpenExisting(this.MapName);
this.memoryAccessor = this.memoryMapped.CreateViewAccessor();
}
indefinite length byte read and write
Read by byte array with variable length. Taking the picture opened by the user as an example, the picture path and picture content are exchanged through shared memory in the form of a Byte array. Their storage format in shared memory is as shown in the following figure:
Solid model ImageData
First, create a solid model ImageData. Its main function is to convert the image object Bitmap and Byte array, as follows:
using System;
using System.Collections.Generic;
using System.Drawing;
using System.Drawing.Imaging;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Okcoder.ShareMemory.Common
{
public class ImageData
{
public string ImageFullPath { get; set; }
public Bitmap ImageContent { get; set; }
/// <summary>
/// 从对象转换成数组
/// </summary>
/// <returns></returns>
public byte[] ImageToBytes()
{
var byteFullPath = Encoding.UTF8.GetBytes(this.ImageFullPath);
MemoryStream stream = new MemoryStream();
int lenFullPath = byteFullPath.Length;
byte[] byteFullPathLen = BitConverter.GetBytes(lenFullPath);
ImageContent.Save(stream, ImageContent.RawFormat);
var byteImageContent = stream.ToArray();
int lenImageContent = byteImageContent.Length;
byte[] byteImageContentLen = BitConverter.GetBytes(lenImageContent);
byte[] total = new byte[4 + lenFullPath + 4 + lenImageContent];
byteFullPathLen.CopyTo(total, 0);
byteFullPath.CopyTo(total, 4);
byteImageContentLen.CopyTo(total, 4 + lenFullPath);
byteImageContent.CopyTo(total, 4 + lenFullPath + 4);
stream.Close();
stream.Dispose();
return total;
}
/// <summary>
/// 从对象转换成对象
/// </summary>
/// <param name="bytes"></param>
public void BytesToImage(byte[] bytes)
{
int lenFullPathLen = BitConverter.ToInt32(bytes, 0);
var byteFullPath = new byte[lenFullPathLen];
bytes.Skip(4).Take(lenFullPathLen).ToArray().CopyTo(byteFullPath, 0);
this.ImageFullPath = Encoding.UTF8.GetString(byteFullPath);
int lenImageContent = BitConverter.ToInt32(bytes, 4 + lenFullPathLen);
var byteImageContent = new byte[lenImageContent];
bytes.Skip(4 +lenFullPathLen+4).Take(lenImageContent).ToArray().CopyTo(byteImageContent,0);
MemoryStream stream = new MemoryStream(byteImageContent);
this.ImageContent = (Bitmap)Image.FromStream(stream);
stream.Close();
stream.Dispose();
}
}
}
Shared memory byte read and write
Byte read/write to shared memory of variable length is as follows:
/// <summary>
/// 读取字节
/// </summary>
/// <returns></returns>
public byte[] ReadMemoryWithBytes()
{
byte[] bytes = new byte[this.capacity];
this.memoryAccessor.ReadArray<byte>(0, bytes, 0, bytes.Length);
return bytes;
}
/// <summary>
/// 写入字节
/// </summary>
/// <param name="bytes"></param>
public void WriteMemoryWithBytes(byte[] bytes)
{
this.memoryAccessor.WriteArray<byte>(0, bytes, 0, bytes.Length);
}
Capacity is the default capacity of the memory-mapped file, and the size is 10M.
Write shared memory call
First, on the Okcoder.ShareMemory.Writer project page, the user selects an image and displays it on the UI page, as follows:
private void btnBrowser_Click(object sender, EventArgs e)
{
OpenFileDialog openFileDialog = new OpenFileDialog();
openFileDialog.Title = "请选择图片";
openFileDialog.Filter = "PNG图片|*.png|JPG图片|*.jpg";
if (openFileDialog.ShowDialog() == DialogResult.OK)
{
string fileName = openFileDialog.FileName;
this.pictureBox1.Image = Bitmap.FromFile(fileName);
this.pictureBox1.SizeMode = PictureBoxSizeMode.StretchImage;
this.txtImagePath.Text = fileName;
}
}
Then click the "Shook" button to encapsulate the ImageData object, convert it to a Byte array, and then write it to shared memory, as shown below:
private void btnWriteMemory_Click(object sender, EventArgs e)
{
//定义ImageData并转换成字节数组
ImageData imageData = new ImageData();
imageData.ImageFullPath = this.txtImagePath.Text;
imageData.ImageContent = (Bitmap)this.pictureBox1.Image;
byte[] bytes = imageData.ImageToBytes();
//定义共享内存帮助对象并打开共享内存
ShareMemoryHelper helper = new ShareMemoryHelper();
helper.CreateOrOpenSharedMemory();
//以字节方式写入共享内存
helper.WriteMemoryWithBytes(bytes);
}
read shared memory call
On the Okcoder.ShareMemory.Reader project page, the user clicks the "Swap" button, reads the shared memory, converts the internal ImageData, and then displays it on the UI page, as follows:
private void btnRead_Click(object sender, EventArgs e)
{
//定义共享内存帮助对象并打开共享内存
ShareMemoryHelper helper = new ShareMemoryHelper();
helper.OpenShareMemory();
//以字节方式读取
byte[] bytes = helper.ReadMemoryWithBytes();
ImageData imageData = new ImageData();
//转换字节到ImageData对象
imageData.BytesToImage(bytes);
//页面赋值
this.txtImagePath.Text = imageData.ImageFullPath;
this.pictureBox1.Image = imageData.ImageContent;
this.pictureBox1.SizeMode = PictureBoxSizeMode.StretchImage;
}
In this way, with two swipes, the picture selected by the user of one program can be transferred to another application through shared memory. God is not magical, no surprise!
Fixed length content reading and writing
In practical applications, shared memory supports structures of value types, the bottom of the data bed of Byte arrays of reference types, and the ordering of attributes in the structure, which is its order in memory.
Defining a solid model
First, define the structure TestData. In order to allow the structure to be converted between pointers, the structure must be declared as serializable through the StructLayout feature, as follows:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;
namespace Okcoder.ShareMemory.Common
{
[StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Ansi)]
public struct TestData
{
/// <summary>
/// Id，长度为4个字节
/// </summary>
public int Id;
/// <summary>
/// 年龄长度为4个字节
/// </summary>
public int Age;
/// <summary>
/// 分数 10个元素
/// </summary>
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 0x0A)]
public int[] Scores;
public TestData()
{
this.Id = 0;
this.Age = 0;
this.Scores = new int[10];
}
public override string ToString()
{
return $"Id={this.Id},Age={this.Age},Scores={string.Join(",",this.Scores)}";
}
}
}
Write structure type
First, use Marshal to apply for memory space, and use Marshal's StructureToPtr method to store the structure in the requested memory, with the pointer pointing to the starting position. Then use Marshal's Copy method to copy the memory pointed to by the pointer to a Byte array, and then write to the shared memory. Finally, use Marshal's FreeHGlobal to release the requested pointer, as follows:
/// <summary>
/// 写入结构体
/// </summary>
/// <param name="data"></param>
public void WriteMemoryWithStruct(TestData data)
{
//获取结构体的长度
int len = Marshal.SizeOf(typeof(TestData));
byte[] bytes = new byte[len];
//申请内存并获取指针
IntPtr p = Marshal.AllocHGlobal(len);
//结构体写入内存
Marshal.StructureToPtr(data, p, false);
//将内存复制到数组
Marshal.Copy(p, bytes, 0, len);
//写入数组共享到内存
this.memoryAccessor.WriteArray<byte>(0, bytes, 0, bytes.Length);
//释放内存
Marshal.FreeHGlobal(p);
//释放指针
p = IntPtr.Zero;
}
Read structure type
First, use Marshal's AllocHGlobal method to apply for memory space, read out the byte array of shared memory and copy it into the requested memory, and point to the starting position. Then use Marshal's Point-to-Structure method PtrToStructure to convert it into a structure object. Finally, use Marshal's FreeHGlobal to release the requested pointer, as follows:
/// <summary>
/// 读取结构体
/// </summary>
/// <returns></returns>
public TestData ReadMemoryWithStruct()
{
//获取结构体类型的长度
int len = Marshal.SizeOf(typeof(TestData));
byte[] bytes = new byte[len];
//申请内存
IntPtr p = Marshal.AllocHGlobal(len);
//从共享内存读取数据
this.memoryAccessor.ReadArray<byte>(0, bytes, 0, bytes.Length);
//将字节数组复制到指针
Marshal.Copy(bytes, 0, p, len);
//指针转换到结构体
TestData data = (TestData)Marshal.PtrToStructure(p, typeof(TestData));
//释放内存
Marshal.FreeHGlobal(p);
//释放指针
p = IntPtr.Zero;
return data;
}
Write shared memory call
On the Okcoder.ShareMemory.Writer project page, the user clicks the Struct button, then encapsulates the TestData instance object, and then writes it to shared memory, as follows:
private void btnWriteStruct_Click(object sender, EventArgs e)
{
//定义TestData并赋值
TestData testData = new TestData();
testData.Id = 100;
testData.Age = 20;
for (int i = 0; i < 10; i++)
{
testData.Scores[i] = i + 60;
}
//定义共享内存帮助对象并打开共享内存
ShareMemoryHelper helper = new ShareMemoryHelper();
helper.CreateOrOpenSharedMemory();
//以结构体的形式写入共享内存
helper.WriteMemoryWithStruct(testData);
}
read shared memory call
On the Okcoder.ShareMemory.Reader project page, the user clicks the Struct button to retrieve the structure from shared memory and display it in a pop-up box, as follows:
private void btnReadStruct_Click(object sender, EventArgs e)
{
//定义共享内存帮助对象并打开共享内存
ShareMemoryHelper helper = new ShareMemoryHelper();
helper.OpenShareMemory();
//读取结构体
TestData testData = helper.ReadMemoryWithStruct();
MessageBox.Show(testData.ToString());
}
example shows
The first is that pictures of variable length exchange data between processes, as follows:
Fixed length structure types exchange data between processes, as follows:
The above is the entire content of "Recommend a Solution for High-Speed Data Exchange between Processes", which aims to attract valuable ideas, learn together, and make progress together!
