C# 玩转MongoDB(三)

前面两篇文章,已经讲解了C#对MongoDB的基本操作以及小文件的读写存储,那么对于大型(>=16M)文件呢?具体又该如何操作呢,本文主要以一个简单的小例子,简述C#如何通过GridFS进行MongoDB的大文件的操作,仅供学习分享使用,如有不足之处,还请指正。

什么是GridFS?

在实现GridFS方式前我先讲讲它的原理,为什么可以存大文件。驱动首先会在当前数据库创建两个集合:”fs.files”和”fs.chunks”集合,前者记录了文件名,文件创建时间,文件类型等基本信息;后者分块存储了文件的二进制数据(并支持加密这些二进制数据)。分块的意思是把文件按照指定大小分割,然后存入多个文档中。”fs.files”怎么知道它对应的文件二进制数据在哪些块呢?那是因为在”fs.chunks”中有个”files_id”键,它对应”fs.files”的”_id”。”fs.chunks”还有一个键(int型)”n”,它表明这些块的先后顺序。这两个集合名中的”fs”也是可以通过参数自定义的。

GridFS存储原理

一个文件存储在两个集合中,一个用于存储元数据(文件名称,类型,大小等内容,可便于索引),一个用于存储真实二进制数据(分块存储),如下所示:

C# 玩转MongoDB(三)

GridFS安装

如果需要存储大型文件,则需要安装GridFS插件,如下所示:

项目–右键–管理Nuget程序包–打卡Nuget包管理器–浏览搜索MongoDB.Driver.GridFS–安装。如下所示:

C# 玩转MongoDB(三)

示例截图

首先是文件的查询,如下所示:

C# 玩转MongoDB(三)

文件的新增

C# 玩转MongoDB(三)

核心代码

本示例主要是在MongoDB中进行文件的操作,所以之前的MongoHelper已不再适用,本例新增了文件专用帮助类MongoFileHelper,如下所示:

using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Driver.GridFS;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace DemoMongo.Common
{

    public class MongoFileHelper
    {

        private string connStr = "mongodb://127.0.0.1:27017";//服务器网址

        private string dbName = "hexdb";//数据库名称

        private IMongoClient client;//连接客户端

        private IMongoDatabase db;//连接数据库

        private string collName;//集合名称

        public MongoFileHelper()
        {

        }

        public MongoFileHelper(string connStr, string dbName, string collName)
        {
            this.connStr = connStr;
            this.dbName = dbName;
            this.collName = collName;
            this.Init();
        }

        /// <summary>
        /// 初始化连接客户端
        /// </summary>
        private void Init()
        {
            if (client == null)
            {
                client = new MongoClient(this.connStr);
            }
            if (db == null)
            {
                db = client.GetDatabase(this.dbName);
            }
        }

        /// <summary>
        /// 通过字节方式上传
        /// </summary>
        /// <param name="filePath"></param>
        public void UploadFile(string filePath)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            byte[] source = File.ReadAllBytes(filePath);
            string fileName = Path.GetFileName(filePath);
            var options = new GridFSUploadOptions
            {
                ChunkSizeBytes = 64512, // 63KB
                Metadata = new BsonDocument
                {
                    { "resolution", "1080P" },
                    { "copyrighted", true }
                }
            };
            var id = bucket.UploadFromBytes(fileName, source);
            //返回的ID,表示文件的唯一ID


        }

        /// <summary>
        /// 通过Stream方式上传
        /// </summary>
        /// <param name="filePath"></param>
        public void UploadFile2(string filePath)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            var stream = new FileStream(filePath, FileMode.Open);

            string fileName = Path.GetFileName(filePath);
            var options = new GridFSUploadOptions
            {
                ChunkSizeBytes = 64512, // 63KB
                Metadata = new BsonDocument
                {
                    { "resolution", "1080P" },
                    { "copyrighted", true }
                }
            };
            var id = bucket.UploadFromStream(fileName, stream);
            //返回的ID,表示文件的唯一ID


        }

        /// <summary>
        /// 通过字节写入到流
        /// </summary>
        /// <param name="filePath"></param>
        public void UploadFile3(string filePath)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            byte[] source = File.ReadAllBytes(filePath);
            string fileName = Path.GetFileName(filePath);
            var options = new GridFSUploadOptions
            {
                ChunkSizeBytes = 64512, // 63KB
                Metadata = new BsonDocument
                {
                    { "resolution", "1080P" },
                    { "copyrighted", true }
                }
            };
            using (var stream = bucket.OpenUploadStream(fileName, options))
            {
                var id = stream.Id;
                stream.Write(source, 0, source.Length);
                stream.Close();
            }
        }

        /// <summary>
        /// 下载文件
        /// </summary>
        /// <param name="id"></param>
        public void DownloadFile(ObjectId id,string filePath)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            byte[] source = bucket.DownloadAsBytes(id);
            //返回的字节内容
            //var bytes = await bucket.DownloadAsBytesAsync(id);
            using (Stream stream = new FileStream(filePath, FileMode.OpenOrCreate)) {
                stream.Write(source, 0, source.Length);
            }
        }

        public void DownloadFile2(ObjectId id)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            Stream destination = null;
            bucket.DownloadToStream(id, destination);
            //返回的字节内容
            //await bucket.DownloadToStreamAsync(id, destination);

        }

        public void DownloadFile3(ObjectId id)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            Stream destination = null;
            using (var stream = bucket.OpenDownloadStream(id))
            {
                // read from stream until end of file is reached
                stream.Close();
            }
        }

        public void DownloadFile4(string fileName)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            var bytes = bucket.DownloadAsBytesByName(fileName);

            // or

            Stream destination = null;
            bucket.DownloadToStreamByName(fileName, destination);

            // or

            using (var stream = bucket.OpenDownloadStreamByName(fileName))
            {
                // read from stream until end of file is reached
                stream.Close();
            }
        }

        public List<MongoFile> FindFiles()
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            var filter = Builders<GridFSFileInfo>.Filter.And(
                //Builders<GridFSFileInfo>.Filter.Eq(x => x.Filename, string.Empty),
                Builders<GridFSFileInfo>.Filter.Gte(x => x.UploadDateTime, new DateTime(2015, 1, 1, 0, 0, 0, DateTimeKind.Utc)),
                Builders<GridFSFileInfo>.Filter.Lt(x => x.UploadDateTime, new DateTime(2022, 2, 1, 0, 0, 0, DateTimeKind.Utc)));
            var sort = Builders<GridFSFileInfo>.Sort.Descending(x => x.UploadDateTime);
            var options = new GridFSFindOptions
            {
                //Limit = 1,
                Sort = sort
            };
            List<MongoFile> lstFiles = new List<MongoFile>();
            using (var cursor = bucket.Find(filter, options))
            {
                var fileInfos = cursor.ToList();
                foreach (var fileInfo in fileInfos) {
                    MongoFile f = new MongoFile()
                    {
                         Id=fileInfo.Id,
                         name = fileInfo.Filename,
                         suffix = Path.GetExtension(fileInfo.Filename),
                        size = int.Parse(fileInfo.Length.ToString())
                    };
                    lstFiles.Add(f);
                }
            }
            return lstFiles;
        }

        public List<MongoFile> FindFileByName(string fileName)
        {
            IGridFSBucket bucket = new GridFSBucket(db);
            var filter = Builders<GridFSFileInfo>.Filter.And(
                Builders<GridFSFileInfo>.Filter.Eq(x => x.Filename, fileName),
                Builders<GridFSFileInfo>.Filter.Gte(x => x.UploadDateTime, new DateTime(2015, 1, 1, 0, 0, 0, DateTimeKind.Utc)),
                Builders<GridFSFileInfo>.Filter.Lt(x => x.UploadDateTime, new DateTime(2015, 2, 1, 0, 0, 0, DateTimeKind.Utc)));
            var sort = Builders<GridFSFileInfo>.Sort.Descending(x => x.UploadDateTime);
            var options = new GridFSFindOptions
            {
                //Limit = 1,
                Sort = sort
            };
            List<MongoFile> lstFiles = new List<MongoFile>();
            using (var cursor = bucket.Find(filter, options))
            {
                var fileInfos = cursor.ToList();
                foreach (var fileInfo in fileInfos)
                {
                    MongoFile f = new MongoFile()
                    {
                        Id = fileInfo.Id,
                        name = fileInfo.Filename,
                        suffix = Path.GetExtension(fileInfo.Filename),
                        size = int.Parse(fileInfo.Length.ToString())
                    };
                    lstFiles.Add(f);
                }
            }
            return lstFiles;
        }
    }
}

然后操作时,调用帮助类即可,如下所示:

查询调用

private void btnQuery_Click(object sender, EventArgs e)
        {
            string name = this.txtName.Text.Trim();
            List<MongoFile> fileInfos = new List<MongoFile>();
            if (string.IsNullOrEmpty(name))
            {
                fileInfos = helper.FindFiles();
            }
            else {
                fileInfos = helper.FindFileByName(name);
            }

            this.dgView.AutoGenerateColumns = false;
            this.bsView.DataSource = fileInfos;
            this.dgView.DataSource = this.bsView;
        }

下载调用

private void dgView_CellContentClick(object sender, DataGridViewCellEventArgs e)
        {
            if (e.ColumnIndex == 3) {
                //第3个是下载按钮
                SaveFileDialog sfd = new SaveFileDialog();

                var file = (MongoFile)(this.dgView.Rows[e.RowIndex].DataBoundItem);
                sfd.FileName = file.name;
                sfd.Title = "请保存文件";
                if (DialogResult.OK == sfd.ShowDialog())
                {
                    helper.DownloadFile(file.Id,sfd.FileName);
                    MessageBox.Show("保存成功");

                }
            }
        }

保存调用

private void btnSave_Click(object sender, EventArgs e)
        {
            string filePath = this.txtPath.Text;
            if (!string.IsNullOrEmpty(filePath))
            {
                this.helper.UploadFile(filePath);
                MessageBox.Show("保存成功");
            }
            else {
                MessageBox.Show("请先选择文件");
            }

        }

MongoDB查询

当通过GridFS方式保存文件成功后,会在GridFS Buckets下生成fs对象,且在集合下生成两个集合【fs.files,fs.chunks】,用于存储文件,如下所示:

C# 玩转MongoDB(三)

 通过查询fs.files集合,可以查找上传文件的列表,如下所示:

C# 玩转MongoDB(三)

通过查询fs.chunks集合,可以查询文件的内容(二进制数据),如下所示:

C# 玩转MongoDB(三)

注意:如果文件太大,在fs.chunks集合中,进行分片存储,n表示存储的顺序。

 以上就是C#操作MongoDB大文件存储的相关内容,旨在抛砖引玉,共同进步。

备注

点绛唇·感兴

【朝代】宋代 【作者】王禹偁【chēng】雨恨云愁,江南依旧称佳丽。水村渔市,一缕孤烟细。
天际征鸿,遥认行如缀。平生事,此时凝睇,谁会凭栏意。(栏 通:阑)

C# 玩转MongoDB(三)
C# 玩转MongoDB(三)
作者:Alan.hsiang
出处:http://www.cnblogs.com/hsiang/
本文版权归作者和博客园共有,写文不易,支持原创,欢迎转载【点赞】,转载请保留此段声明,且在文章页面明显位置给出原文连接,谢谢。
关注个人公众号,定时同步更新技术及职场文章

原文出处:cnblogs【Alan.hsiang】

原文链接:https://www.cnblogs.com/hsiang/p/15391520.html

本文观点不代表Dotnet9立场,转载请联系原作者。

发表评论

登录后才能评论