LICENSE-MIT Documentation

Official TypeScript developer resources and documentation for ipfs/LICENSE-MIT. Verified technical context for AI.

0 stars TypeScript TypeScript 1 file · ~8,668 tokens
RAW Doc

1. Project Overview & Quickstart (ipfs/LICENSE-MIT)

LICENSE-MIT

Open-source repository ipfs/LICENSE-MIT

Repository Details

Note: High-volume repository documentation is actively indexed and synchronized by YakaAI.

2. Official Technical Reference & Guides (ipfs/docs)

File: README.md

THIS REPOSITORY IS BEING DEPRECATED

Heads up! On May 27th, this repository will be deprecated and replaced with ipfs/ipfs-docs. This repository will not be receiving any updates. It is here for historical purposes.



This repository organizes overall documentation issues across the IPFS project. It's also home to the codebase for the legacy IPFS documentation site (available at docs.ipfs.io). The code for the current (beta) version of IPFS documentation is available at github.com/ipfs/ipfs-docs-v2, and all work on IPFS docs features OR content should be done there. Once we fully deprecate the legacy site, we'll move the beta codebase into this repo.

Join our monthly sync

Every month the team gets together to discuss plans for the coming four-ish weeks. This meeting usually happens on the last Tuesday of each month. Feel free to join us using 614386160 as the Zoom meeting ID! Here are the details for the next meeting:

Date Time Zoom ID Link Notes
Tuesday 26th May 2020 UTC: 16:00 - 16:30EST: 11:00 - 11:30PST 08:00-08:30Unix: 1590508800 92681067428 protocol.zoom.us/j/92681067428 Meeting notes

If you can't make the meeting, don't worry. We post a recording of each meeting in our YouTube playlist. Notes from current and past calls can be found here.

Get involved

We would love ❤️ your help to improve existing items or make new ones even better! We also have bounties available! Here are some ideas to get you started:

  1. Join us on our weekly call! See the top of this readme for full details on how and when.

  2. Claim an issue labeled "help wanted" in either this docs repo or the IPFS website repo! All our issues are T-shirt sized and roughly difficulty-graded (both using labels), so have a look for something that interests you in the time you have available, and dive on in! Leave your thoughts and questions in issue comments, and we'll get back to you as soon as possible.

  3. Create (or improve) an example, tutorial or concept guide! At present, we have a variety of how-tos and concept guides available at docs-beta.ipfs.io. However, every new item we can offer makes it easier for the world's community to use and build on IPFS. If you'd like to write a new example/tutorial or concept guide, take a look in our open issues for items with the Category: Content label and see if any already-identified needs appeal to you. Note that some issues may have bounties! If you'd like to write something entirely new, feel free — PRs are welcome! Or, if you'd just like to improve existing docs content, feel free to fork it and add your suggestions.

  4. Help us improve how we present reference and API documentation. In Q2 2020, we're investigating the best way to improve how we present reference materials like API documentation and command-line dictionaries. Got thoughts? Please discuss them!

If you're able to contribute to any of the categories above, we thank you in advance -- your work directly improves the future of IPFS for the global community! Before posting a PR with your changes, please be sure to check our documentation style guide and the overall IPFS contributor guidelines so we can ensure that docs are clear, consistent, and simple to read. Finally, please make sure to follow the IPFS code of conduct.

Issues

If you find something wrong within this repository, please raise an issue here.

IPFS Docs beta

Issues regarding the new IPFS Docs beta website should be raised in this repository's issue tracker. The Issues tab has been disabled in the ipfs/ipfs-docs-v2 repository to stop issues being posted there. The content of ipfs/ipfs-docs-v2 will be merged with the content of this repository in quarter 2 of 2020. We could completely delete the ipfs/docs repository and replace it with ipfs/ipfs-docs-v2. However, we still want to be able to view the git commit history and issues previously raised within this repository.

Project organization

Objectives for 2020 Q1

Here's a summary of this our objectives for the first three months of 2020:

  1. ✅ Launch the beta IPFS documentation site on the IPFS network.
  2. Improve documentation by creating new and/or enhancing existing content based on issues in this repo.
  3. Resolving or addressing any new GitHub issues created in this repo this quarter.
  4. Setting up and enabling a community of writers and content creators.

For more information regarding these objectives, check out the IPFS-wide Objectives and Key Results (OKRs) tracking document.

Suggestions

Do you have suggestions on future improvements to IPFS docs? You can vote on what you think the IPFS docs site should contain, along with features that you'd like to see over at canny.io/docs-features.

IPFS docs core members

Bounties

You can earn the undying love of the IPFS community and make some money by closing an issue with the bounty tag! Submissions must be production-ready and meet all the specifications listed on the issue page. Check out the current list of open bounties →

Legacy code details

IMPORTANT! As noted above, this repo is home to the LEGACY docs site; if you want to make changes to IPFS docs, please visit github.com/ipfs/ipfs-docs-v2 and use the codebase there. However, if for some reason you need to run the legacy code locally, instructions are below.

One-time setup

  1. Install AEgir

    sh
    npm install aegir

  2. Download dependencies and IPFS libraries (e.g., go-ipfs, js-ipfs) and generate their documentation:

    sh
    make install

This installs dependencies and generates source files from other projects (e.g., API documentation, theme resources from the ipfs-css, etc). When dependencies or external packages (like go-ipfs) have new releases, you should regenerate files based on them by running make resources.

Build and run

  1. In the root directory, run make dev.
  2. Load localhost:1313 in your web browser.
  3. Edit and add things!

To create a production build, run make build instead. You'll find the final static site in the public directory.

License

All software code is copyright (c) Protocol Labs, Inc. under the MIT license. Other written documentation and content is copyright (c) Protocol Labs, Inc. under the Creative Commons Attribution-Share-Alike License. See LICENSE file for details.

File: content/community/contribute/_index.md

title: "Get Involved" headless: true opened: true beta_equivalent: project/contribute

File: content/community/contribute/how-to-help.md

title: "Ways to contribute"
menu:
community:
parent: contribute
weight: 1

beta_equivalent: project/contribute

So you want to contribute to IPFS and the ecosystem? Here is a quick listing of things we need help with and how you can get started. Even if what you want to do is not listed here, we probably accept contributions for it! If you're unsure, please open a issue.

Code

IPFS and its sister-projects are big, with lots of code written in multiple languages. We always need help writing and maintaining code, but it can be daunting to just jump in. We use the label Help Wanted on features or bugfixes that people can help out with. They are an excellent place for you to start contributing code.

The biggest and most active repositories we have today are:

If you want to start contributing to the core of IPFS, those repositories are a great place start. But the Help Wanted label exists in all of our repositories across the Github organizations:

Documentation

Again, IPFS is a huge project and undertaking. With lots of code comes the need for lots of good documentation! However, we need a lot more help to write the awesome docs the project needs. If writing technical documentation is your area, we’d absolutely love your help!

The best place to get started is by looking through the Github issues within this repository.

Support

IPFS already has lots of users and curious people experimenting and using IPFS in their applications. These users sometimes get stuck or have questions that need answering. If you’ve contributed something with code or documentation, chances are that you can probably help a lot of people with their questions.

The IPFS forum is the best place to go if you're looking for support.

Testing

We’re continuously improving IPFS every day, but mistakes can happen and we could release something that doesn’t work as well as it should — or simply doesn't work at all! If you like to dig into edge-cases or write testing scenarios, wrangling our testing infrastructure could be the job for you.

We work on CI tools and plan larger scale tests at ipfs/kubernetes-ipfs.

Design

We have many design needs, but only a small team of visual and UX designers who divide their time between IPFS-related projects and other work here. That means there are a number of issues which could use your design contributions. As home to IPFS Desktop and Companion, the ipfs-gui repo is a good place to look for opportunities to help move our design work forward. Simply filter by the label that best fits your skillset, such as design-visual or design-ux.

Issues / Triaging

With lots of code come lots of Github Issues. We need YOU to help with organizing all of this in some manner. We don’t yet have any proper resources for getting started with this. Get in touch if you can contribute a sense of extreme organization!

We currently don't have a single place for this. If you'd like to start it, please let us know.

Community

If interacting with people is your favorite thing to do in this world, IPFS and
co. are always happy to help you organize events and/or workshops to teach IPFS.

We have a repository for organizing community events and would love your help to have meetups in more locations or make the existing ones more regular.

Applications

IPFS is designed for others to build applications around it! Building applications and services using IPFS is an excellent way to find use cases where IPFS doesn’t yet do a perfect job or uncover bugs and inefficiencies.

Get started by looking at our awesome-ipfs list. Build anything you think is missing! If you're unsure about something, please create an issue to get help or feedback on your specific problem/idea.

Protocol Design

IPFS is ultimately about building better protocols, and we always welcome ideas and feedback on how to improve those protocols. Post feedback, issues, and proposals in the specs projects in our Github organizations or in ipfs/notes.

Research

Finally, we see Protocol Labs as a research lab, where YOUR ideas can become technologies that have a real impact on the world. If you're interested in contributing to our research, please reach out to mailto:[email protected] or [email protected] for more information. Include what your interests are so we can make sure you get to work on something fun and valuable.

File: content/community/_index.md

title: "Community" headless: true opened: true beta_equivalent: community

The IPFS community is as decentralized and distributed as IPFS itself. Our community members are globally located, active both online and in person, and use IPFS in countless different ways. You can learn about meetings, events, apps people are building, and more with the resources listed here.

Discussion/Support Forum

Join the conversation on the IPFS Forums for everything from technical questions, project brainstorming, talk on IPFS use cases and more.

Research Discussion

The IPFS Collaborative Notebook for Research repo tracks discussions on topics related to IPFS research, ideas and proposals for new systems/features, and other items that haven't yet been tracked into a specific project.

Events Calendar

Find all of the open community video calls for the IPFS project in the IPFS Community events calendar.

Community Tools

Join the discussion and documentation on IPFS community practices in the community tools repo. If you want to start or get involved with a community chapter to help expand IPFS awareness in your local geographical or online community, here's where to look.

Project Management

Keep abreast of IPFS team planning, management and coordination issues in the IPFS team management repo.

Blog

Explore the latest news, events and other happenings in the IPFS multiverse on the official IPFS Blog.

IPFS YouTube channel

The IPFS YouTube channel is home to a wealth of information about the IPFS project — everything from IPFS Camp course content to working group meeting archives — so you can explore playlists and subscribe to ones that interest and inform you.

Twitter

Get your IPFS news in tweet-sized bites. Follow these accounts for the latest:

Community projects on Awesome IPFS

Get inspired by Awesome IPFS, a catalog of community-built projects using IPFS for everything from image sharing to decentralized archiving. Have a project of your own to add to Awesome IPFS? Make a PR in the repo!

Public archives on IPFS

Curious about what public datasets are available on IPFS for you to play with? There's a section on Awesome IPFS dedicated to just that.

Forum FAQ Archive

We're in the midst of migrating frequently-asked questions from the IPFS forum archives into our primary FAQ, but in the meantime, you can find them in the forums.

File: content/community/irc.md

title: "IRC" beta_equivalent: community/irc

There are several IRC channels on Freenode dedicated to IPFS, and it’s a great place for live chat:

For longer-lived discussions and for support, please use the discussion forums at https://discuss.ipfs.io instead of IRC! It’s easy for complex discussions to get lost in a sea of new messages on IRC.

If you’re new to IRC, there are lots of mobile and desktop clients for working with it. Here are a few to get you started:

File: content/guides/concepts/_index.md

title: "Concepts"
menu:
guides:
identifier: concepts
weight: 2

beta_equivalent: essentials

File: content/guides/concepts/cid.md

title: "Content Identifiers (CIDs)"
menu:
guides:
parent: concepts

beta_equivalent: concepts/content-addressing

A content identifier, or CID, is a label used to point to material in IPFS. It doesn't indicate where the content is stored, but it forms a kind of address based on the content itself. CIDs are short, regardless of the size of their underlying content.

CIDs are based on the content’s [cryptographic hash]({{< relref "./hashes.md" >}}). That means:

  • Any difference in content should produce a different CID and
  • The same piece of content added to two different IPFS nodes using the same settings should produce exactly the same CID.

CID formats

CIDs can take a few different forms with different encoding bases or CID versions. Many of the existing IPFS tools still generate v0 CIDs, although the files ([MFS]({{< relref "./mfs.md" >}})) and object operations now use CIDv1 by default.

Version 0

When IPFS was first designed, we used base 58-encoded multihashes as the content identifiers (This is simpler, but much less flexible than newer CIDs). CIDv0 is still used by default for many IPFS operations, so you should generally try to support v0.

If a CID is 46 characters starting with "Qm", it's a CIDv0 (for more details, check the decoding algorithm in the CID specification).

Version 1

CID v1 contains some leading identifiers that clarify exactly which representation is used, along with the content-hash itself. These include:

  • A multibase prefix, specifying the encoding used for the remainder of the CID
  • A CID version identifier, which indicates which version of CID this is
  • A multicodec identifier, indicating the format of the target content — it helps people and software to know how to interpret that content after the content is fetched

These leading identifiers also provide forward-compatibility, supporting different formats to be used in future versions of CID.

You can use the first few bytes of the CID to interpret the remainder of the content address and know how to decode the content after it's fetched from IPFS. For more details, check out the CID specification. It includes a decoding algorithm and links to existing software implementations for decoding CIDs.

Explore CIDs for yourself

Want to pull apart a specific CID's multibase, multicodec, or multihash info? You can use the CID Inspector or the CID Info panel in IPLD Explorer (both links launch using a sample CID) for an interactive breakdown of differently-formatted CIDs. Also learn about the future of CIDs in IPFS.

File: content/guides/concepts/dht.md

title: "Distributed Hash Tables (DHT)"

menu:
guides:
parent: concepts

beta_equivalent: concepts/dht

DHTs

What is a DHT?

Distributed Hash Tables (DHTs) are distributed key-value stores where keys are cryptographic hashes.

DHTs are distributed. Each "peer" (or "node") is responsible for a subset of the DHT.
When a peer receives a request, it either answers it, or the request is passed to another peer until a peer that can answer it is found.
Depending on the implementation, a request not answered by the first node contacted can be:

  • forwarded from peer to peer, with the last peer contacting the requesting peer
  • forwarded from peer to peer, with the answer forwarded following the same path
  • answered with the contact information of a node that has better chances to be able to answer. IPFS uses this strategy.

DHTs' decentralization provides advantages compared to a traditional key-value store, including:

  • scalability, since a request for a hash of length n takes at most log2(n) steps to resolve.
  • fault tolerance via redundancy, so that lookups are possible even if peers unexpectedly leave or join the DHT. Additionally, requests can be addressed to any peer if another peer is slow or unavailable.
  • load balancing, since requests are made to different nodes and no unique peers process all the requests.

How does the IPFS DHT work?

Peer IDs

Each peer has a peerID, which is a hash with the same length n as the DHT keys.

Buckets

A subset of the DHT maintained by a peer is called a 'bucket'.
A bucket maps to hashes with the same prefix as the peerID, up to m bits. There are 2^m buckets. Each bucket maps for 2^(n-m) hashes.

For example, if m=16 and we use hexadecimal encoding (four bits per displayed character), the peer with peerID 'ABCDEF12345' maintains mapping for hashes starting with 'ABCD'.
Some hashes falling into this bucket would be ABCD38E56, ABCD09CBA or ABCD17ABB, just as examples.

The size of a bucket is related to the size of the prefix. The longer the prefix, the fewer hashes each peer has to manage, and the more peers are needed.
Several peers can be in charge of the same bucket if they have the same prefix.

In most DHTs, including IPFS's Kademlia implementation, the size of the buckets (and the size of the prefix), are dynamic.

Peer lists

Peers also keep a connection to other peers in order to forward requests if the requested hash is not in their own bucket.

If hashes are of length n, a peer will keep n-1 lists of peers:

  • the first list contains peers whose IDs have a different first bit.
  • the second list contains peers whose IDs have first bits identical to its own, but a different second bit
  • ...
  • the m-th list contains peers whose IDs have their first m-1 bits identical, but a different m-th bit
  • ...

The higher m is, the harder it is to find peers that have the same ID up to m bits. The lists of "closest" peers typically remains empty.
"Close" here is defined as the XOR distance, so the longer the prefix they share, the closer they are.
Lists also have a maximum of entries (k) — otherwise the first lists would contain half the network, then a fourth of the network, and so on.

Using the DHT

When a peer receives a lookup request, it will either answer with a value if it falls into its own bucket, or answer with the contacting information (IP+port, peerID, etc.) of a closer peer. The requesting peer can then send its request to this closer peer. The process goes on until a peer is able to answer it.
A request for a hash of length n will take at maximum log2(n) steps, or even log2m(n).

Keys and hashes

In IPFS's Kademlia DHT, keys are SHA256 hashes.
PeerIDs are those of libp2p, the networking library used by IPFS.

We use the DHT to look up two types of objects, both represented by SHA256 hashes:

  • Content IDs of the data added to IPFS. A lookup of this value will give the peerIDs of the peers having this immutable content.
  • IPNS records. A lookup will give the last Content ID associated with this IPNS address, enabling the routing of mutable content.

Consequently, IPFS's DHT is one of the ways to achieve mutable and immutable content routing. It's currently the only one implemented.

You can learn more in the libp2p Kademlia DHT specification.

Usage

Adding an entry

Adding a blob of data to IPFS is the equivalent of advertising that you have it. Since DHT is the only content routing implemented, you can just use:
ipfs add myData
IPFS will automatically chunk your data and add a mapping on the DHT between the Content ID and your peerID. Note that there can be other peerIDs already mapped to that value, so you will be added to the list. Also note that if the provided data is bigger than 124KB, it will be chunked into "blocks", and both those blocks and the overall data will be mapped.

You can publish an IPNS record using ipfs.name.publish.

File: content/guides/concepts/dnslink.md

title: "DNSLink"
menu:
guides:
parent: concepts

NOTE: The info below is a quick guide to DNSLink. For a more complete guide, including tutorials, usage examples and FAQs, check out dnslink.io.

DNSLink uses DNS TXT records to map a domain name (like ipfs.io) to an IPFS address. Because you can edit your DNS records, you can use them to always point to the latest version of an object in IPFS (remember that an IPFS object’s address changes if you modify the object). Since DNSLink uses DNS records, you can assign names/paths/(sub)domains/whatever that are easy to type, read, and remember.

A DNSLink address looks like an IPNS address, but it uses a domain name in place of a hashed public key:

text
/ipns/myexampledomain.org

Just like normal IPFS addresses, they can include links to other files — or other types of resources that IPFS supports, like directories and symlinks:

text
/ipns/myexampledomain.org/media/

Publishing using a subdomain

While you can publish the TXT record to the exact domain if you so wish, it can be more advantageous to publish DNSLink records using a special subdomain called _dnslink. This enables you to improve the security of an automated setup, or delegate control over your DNSLink records to a third party without giving away full control over the original DNS zone.

For example, docs.ipfs.io does not have a TXT record, but the page still loads
because a TXT record exists for _dnslink.docs.ipfs.io. If you look up the DNS records for _dnslink.docs.ipfs.io, you'll see its DNSLink entry:

sh
$ dig +noall +answer TXT _dnslink.docs.ipfs.io
_dnslink.docs.ipfs.io.  34  IN  TXT "dnslink=/ipfs/QmVMxjouRQCA2QykL5Rc77DvjfaX6m8NL6RyHXRTaZ9iya"

When an IPFS client or node attempts to resolve an address, it looks for a TXT record that is prefixed with dnslink=. The rest can be an /ipfs/ link (as in the example below), or /ipns/, or even a link to another DNSLink.

text
dnslink=/ipfs/<CID for your content here>

For example, let's go back to when we looked up the DNS records for _dnslink.docs.ipfs.io and saw its DNSLink entry:

sh
$ dig +noall +answer TXT _dnslink.docs.ipfs.io
_dnslink.docs.ipfs.io.  34  IN  TXT "dnslink=/ipfs/QmVMxjouRQCA2QykL5Rc77DvjfaX6m8NL6RyHXRTaZ9iya"

Based on that, this address:

text
/ipns/docs.ipfs.io/introduction/

Will get you this block:

text
/ipfs/QmVMxjouRQCA2QykL5Rc77DvjfaX6m8NL6RyHXRTaZ9iya/introduction/

File: content/guides/concepts/hashes.md

title: "Hashes"
menu:
guides:
parent: concepts

beta_equivalent: concepts/hashing

Hashes are functions that take some arbitrary input and return a fixed-length value. The particular value depends on the given hash algorithm in use, such as SHA-1 (used by Git), SHA-256, or BLAKE2, but a given hash algorithm always returns the same value for a given input. Have a look at the full list of hash functions for more.

As an example, the input:

text
Hello world

would be represented by SHA-1 as:

text
0x7B502C3A1F48C8609AE212CDFB639DEE39673F5E

However, the exact same input generates the following output using SHA-256:

text
0x64EC88CA00B268E5BA1A35678A1B5316D212F4F366B2477232534A8AECA37F3C

Notice that the second hash is longer than the first one. This is because SHA-1 creates a 160 bit hash, while SHA-256 creates a 256 bit hash. Also, the prepended 0x is just an indicator that tells us that the following hash is represented as a base 16 (or hexadecimal) number.

Hashes can be represented in different bases (base2, base16, base32, etc.). In fact, IPFS makes use of that as part of its [Content Identifiers]({{< relref "cid.md" >}}) and supports mulitiple base representations at the same time, using the Multibase protocol.

For example, the SHA-256 hash of "Hello World" from above can be represented as base 32 as:

text
mtwirsqawjuoloq2gvtyug2tc3jbf5htm2zeo4rsknfiv3fdp46a

Characteristics of cryptographic hashes

Cryptographic hashes come with a couple of very important characteristics:

  • deterministic - the same input message always returns exactly the same output hash
  • uncorrelated - a small change in the message should generate a completely different hash
  • unique - it's infeasible to generate the same hash from two different messages
  • one-way - it's infeasible to guess or calculate the input message from its hash

It turns out these features also mean we can use a cryptographic hash to identify any piece of data: the hash is unique to the data we calculated it from and it’s not too long (a hash is a fixed length, so the SHA-256 hash of a 1 Gigabyte video file is still only 32 bytes), so sending it around the network doesn't take up a lot of resources.

That's critical for a distributed system like IPFS, where we want to be able to store and retrieve data from many places. A computer running IPFS can ask all the peers it's connected to whether they have a file with a particular hash and, if one of them does, they send back the whole file. Without a short, unique identifier like a cryptographic hash, that wouldn't be possible. This technique is called “content addressing” — because the content itself is used to form an address, rather than information about the computer and disk location it's stored at.

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →