Skip to main content

Getting Started

Overview​

DefraDB is a peer-to-peer(P2P) Edge Database with a NoSQL document store interface. DefraDB's data model is the core data storage system for Source Ecosystem. It is:

  • backed by MerkleCRDTs for multi write-master architecture
  • built using IPFS technologies (IPLD, LibP2P) and features Semantic web3 properties.

See the Technical Overview here.

Early Access​

DefraDB is currently an Early Access Alpha program (not ready for production deployments). For support with your use-case and deployment, please contact Source by email.

Installation​

To install a DefraDB node, you can either:

  • download the pre-compiled binaries available from the releases page, or

    go install github.com/sourcenetwork/defradb/cli/defradb

  • compile it yourself if you have a local Go Toolchain installed

    git clone git@github.com:sourcenetwork/defradb-early-access.git
    mv defradb-early-access defradb # Rename the folder
    cd defradb/cli/defradb
    go install

Before You Begin​

Before installing DefraDB, review the following pre-requisites.

  • Install DefraDB Command Line Interface (defradb cli) locally or in an accessible remote node.
  • Install GraphQL client (e.g. GraphiQL). See download link.

SetUp​

Set up a DefraDB node with this command in the terminal:

  • defradb start

This action prompts the following items:

  • Starts a node with default settings (running at http://localhost:9181)
  • Creates a configuration file at $HOME/.defra/config.yaml, which is the user home directory

DefraDB supports two storage engines:

  • BadgerDB (used by default, provides disk-backed persistent storage)
  • In-memory store

To choose a storage type you can

  • use --store option with the start command, or
  • edit the local config file
note

If you are using BadgerDB, and you get an error message:

  • Failed to initiate database:Map log file. Path=.defradb/data/000000.vlog. Error=exec format error

It means that the terminal client does not support Mmap'ed files. This is common with older version of Ubuntu on Windows, Windows SubSystem for Linux (WSL). Unfortunately, BadgerDB uses Mmap to interact with the filesystem, so you will need to use a terminal client which supports it.

note

Once your local environment is successfully setup, you can test your connection with:

  • defradb client ping for a Success! response.

Once you've confirmed your node is running correctly, if you're using the GraphiQL client to interact with the database, then make sure you set the GraphQL Endpoint to http://localhost:9181/graphql and the Method to GET.

Adding a Schema Type​

To add a new schema type to the database, you can write the schema to a local file using the GraphQL SDL format, and submit that to the database like so:

  • Add this to the users.gql file
type user 
{
    name: String 
    age: Int 
    verified: Boolean 
    points: Float
}
  • Run the following command:
defradb client schema add -f users.gql

These steps register the type, build a dedicated collection and

This will register the type, build a dedicated collection, and generate the typed GraphQL endpoints for querying and mutation.

You can find more examples of schema type definitions in the cli/defradb/examples folder.

Query Documentation​

Once your local node is populated with data, we can query that data.

query {
  user {
    _key
    age
    name
    points
  }
}

This will query all the users, and return the fields _key, age, name, points. GraphQL queries only ever return the exact fields you request, there's no * selector like with SQL.

We can further filter our results by adding a filter argument to the query.

query {
  user(filter: {points: {_ge: 50}}) {
    _key
    age
    name
    points
  }
}

This will only return user documents which have a value for the points field Greater Than or Equal to ( _ge ) 50.

To see all the available query options, types, and functions please see the Query Documentation.

Interact with Document Commits​

Internally, DefraDB uses MerkleCRDTs to store data. MerkleCRDTs convert all mutations and updates a document has into a graph of changes; similar to Git. Moreover, the graph is a MerkleDAG, which means all nodes are content identifiable with CIDs, and each node, references its parents CIDs.

To get the most recent commit in the MerkleDAG for a with a docKey of bae-91171025-ed21-50e3-b0dc-e31bccdfa1ab, we can submit the following query:

query {
  latestCommits(dockey: "bae-91171025-ed21-50e3-b0dc-e31bccdfa1ab") {
    cid
    delta
    height
    links {
      cid
      name
    }
  }
}

This will return a structure similar to the following, which contains the update payload that caused this new commit (delta), and any sub graph commits this references.

{
  "data": [
    {
      "cid": "QmPqCtcCPNHoWkHLFvG4aKqDkLLzhVDAVEDSzEs38GHxoo",
      "delta": "pGNhZ2UYH2RuYW1lY0JvYmZwb2ludHMYWmh2ZXJpZmllZPU=",
      "height": 1,
      "links": [
        {
          "cid": "QmSom35RYVzYTE7nGsudvomv1pi9ffjEfSFsPZgQRM92v1",
          "name": "age"
        },
        {
          "cid": "QmYJrCcfMmfFp4JcbChLfMLCv8TSHjGwRVHUBgPazWxPga",
          "name": "name"
        },
        {
          "cid": "QmXLuVB5CCGqWcdQitingkfRxoVRLKh2jNcnX4UbYnW6Mk",
          "name": "points"
        },
        {
          "cid": "QmNRQwWjTBTDfAFUHkG8yuKmtbprYQtGs4jYxGJ5fCfXtn",
          "name": "verified"
        }
      ]
    }
  ]
}

Additionally, we can get all commits in a document MerkleDAG with allCommits, and lastly, we can get a specific commit, identified by a cid with the commit query, like so:

query {
  commit(cid: "QmPqCtcCPNHoWkHLFvG4aKqDkLLzhVDAVEDSzEs38GHxoo") {
    cid
    delta
    height
    links {
      cid
      name
    }
  }
}

Here, you can see we use the CID from the previous query to further explore the related nodes in the MerkleDAG.

This only scratches the surface of the DefraDB Query Language, see below for the entire language specification.

Query Documentation​

Access the official DefraDB Query Language Docs [add new link]

CLI Documentation​

You can find generated documentation for the shipped CLI interface here

Next Steps​

The current early access release has much of the digial signatute, and identity work removed, until the cryptographic elements can be finalized.

The following will ship with the next release:

  • schema type mutation/migration
  • data syncronization between nodes
  • grouping and aggregation on the query language
  • additional CRDT type(s)

We will release a project board outlining the planned, and completed features of our roadmap.

Licensing​

Current DefraDB code is released under a combination of two licenses, the Business Source License (BSL) and the DefraDB Community License (DCL).

When contributing to a DefraDB feature, you can find the relevant license in the comments at the top of each file.

Further Reading​

to be updated

Getting Started
Edit this page