Repository: happyfish100/fastdfs
Stars: 9227
README.md
FastDFS is an open source high performance distributed file system. Its major
functions include: file storing, file syncing and file accessing (file uploading
and file downloading), and it can resolve the high capacity and load balancing
problem. FastDFS should meet the requirement of the website whose service based
on files such as photo sharing site and video sharing site.
FastDFS Features
* grouped storage servers, simple and flexible
* peer-to-peer structure, no single point of failure
* The file ID is generated by FastDFS and serves as a credential for file access
FastDFS does not require the traditional name server or meta server
* support large, medium, and small files well; supports the merged storage for small files
and can store a massive number of small files
* the storage server supports multiple disks and supports data recovery for the single disk
* provide a nginx extension module that can seamlessly integrate with nginx
* support IPv6, NAT network and cross data center or hybrid cloud deployment
* support read-write separation and cross data center disaster backup
* provide truncate, append and modify APIs for appender files
* supports multi-threaded file upload and download, support resume from a broken point
Architecture Interpretation
FastDFS has two roles: tracker and storage. The tracker takes charge of
scheduling and load balancing for file access. The storage store files and it's
function is file management including: file storing, file syncing, providing file
access interface. It also manage the meta data which are attributes representing
as key value pair of the file. For example: width=1024, the key is "width" and
the value is "1024".
The tracker and storage contain one or more servers. The servers in the tracker
or storage cluster can be added to or removed from the cluster by any time without
affecting the online services. The servers in the tracker cluster are peer to peer.
The storage servers organizing by the file group to obtain high capacity.
The storage system contains one or more groups whose files are independent among
these groups. The capacity of the whole storage system equals to the sum of all
groups' capacity. A file group contains one or more storage servers whose files
are same among these servers. The servers in a file group backup each other,
and all these servers are load balancing. When adding a storage server to a
group, files already existing in this group are replicated to this new server
automatically, and when this replication done, system will switch this server
online to providing storage services.
When the whole storage capacity is insufficient, you can add one or more
groups to expand the storage capacity. To do this, you need to add one or
more storage servers.
The identification of a file is composed of two parts: the group name and
the file name.
Client test code use client library please refer to the directory: client/test.
Client SDK
* C Language: client/ subdir under FastDFS source code, header files default installation path is /usr/include/fastdfs/
* PHP extension: php_client/ subdir under FastDFS source code
* Java SDK: https://github.com/happyfish100/fastdfs-client-java
* Go SDK: https://github.com/qifengzhang007/fastdfs_client_go
For FastDFS installation and configuration documentation, please refer to the Wiki;
For more FastDFS related articles, please subscribe the Wechat/Weixin public account (Chinese Language): fastdfs
See also
FastDFS is a lightweight object storage solution. If you need a general distributed
file system for databases, K8s and virtual machines (such as KVM), you can learn about
FastCFS which achieves strong data consistency
and high performance.
We provide technical support service and customized development. Welcome to use WeChat or email for discuss.
email: 384681(at)qq(dot)com