How to Store and Retrieve User Data

Data storage is a critical function in the Web3 stack. Without it, you can’t create a user profile, track their progress using an application, or execute a whole host of other important functions.

In this guide, we will break down how to store both public and private data off-chain while letting users retain complete control over their data.

Off-chain is an important detail here. By keeping the data off-chain, we can ensure that apps can provide users with high availability and high performance by quickly storing and retrieving that data. Blockspace is limited, and if we stored all data on-chain there would be a number of negative consequences, including:

1. That data inherently becomes publicly accessible by default (even if the content is encrypted).
2. The cost of data storage will drastically increase as the app scales and takes up larger amounts of blockspace.
3. Applications couldn’t store data in real time. Data writing speed would be dependent upon block speed.

Instead, our approach involves managed or self-hosted off-chain storage, which offers various use-cases for storing and even sharing data. Let’s get into it.

Saving Data For a User

In order to save or retrieve data for a user, you first have to authenticate them. Please see our guide on user authentication for more information on this process. Once you have authenticated a user, you can now store data for them.

To do that, we use Gaia. Gaia serves as a key-value store, in which data is stored and retrieved as files. Those files are kept in different Gaia hubs, or containers, that are owned by, or managed for, users.

The default hub for users that we will cover in this guide is a hub run by Hiro at https://gaia.blockstack.org/. 

This hub supports all file types, including text, image, video or binary. It’s also worth noting that files must be under 25 megabytes in size on the default Hiro-managed hub. If a data instance is larger, we recommend breaking it into several files, saving them individually, and recomposing them on retrieval.

Files are saved as strings that represent stringified JSON objects and contain a variety of properties for a particular model.

To save a file, you first need to instantiate a storage object using the userSession object for an authenticated user. Then proceed to call its putFile method with relevant parameters:

import { AppConfig, UserSession } from "@stacks/connect";
import { Storage } from "@stacks/storage";

const appConfig = new AppConfig(["store_write", "publish_data"]);
const userSession = new UserSession({ appConfig });
const storage = new Storage({ userSession });

const fileName = "car.json";

const fileData = {
  color: "blue",
  electric: true,
  purchaseDate: "2019-04-03",
};

const options = {
  encrypt: true,
};

const fileUrl = await storage.putFile(fileName, JSON.stringify(fileData), options)

As you can see in the snippet above, in this instance, we are storing data about a car purchase, and we are able to define the fileName, as well as fileData, such as the color, whether the car is electric, and the data of purchase. With each storage instance, you can define and include whatever data you want to.

Importantly, the options parameter object contains an encrypt property. When this property is set to true, the data will be encrypted with the user’s app private key before being saved to their Gaia hub. If encrypt is left undefined, all data is encrypted as a default.

The fileName can also include a prefixed path for better hierarchy. For example, instead of storing many color files we could use a fileName of “colors/blue.json”, where “colors” would be treated similarly to a folder on computers.

In the user authentication guide, we walk through two levels of permissions: store_write and publish_data. Both permissions allow applications to save privately encrypted data, but the user must have granted the publish_data permission during authentication for an app to save unencrypted data, which would allow the app to utilize the user’s data at any time without the private key of the user. This feature could be used for social media and content sharing. For example, a blogging platform on Stacks would require unencrypted read-access to articles, so it can share them with readers. However, unless the user grants explicit permission, all stored data is private.

The putFile method returns the public URL where the file can be retrieved from the user’s Gaia hub, and is used in the previous snippet to get the fileUrl value.

Retrieving Data For a User

To retrieve data you’ve previously saved for a user within an app, call the getFile method available from the storage object:

import { AppConfig, UserSession } from "@stacks/connect";
import { Storage } from "@stacks/storage";

const appConfig = new AppConfig(["store_write", "publish_data"]);
const userSession = new UserSession({ appConfig });
const storage = new Storage({ userSession });

const fileName = "car.json";

const options = {
  decrypt: true,
};

const fileData = await storage.getFile(fileName, options)

In this case, we are calling the same file as the previous example “car.json” and since that file was originally encrypted, you will see that the decrypt property in the options object equals true. The decrypt property defaults to true if left undefined.

Note that encrypted files need decrypt set to true so the app knows to decrypt the data with the user’s app private key before the data is made available in the callback (shown here as fileData, a JavaScript object of the original data stored).

Deleting Data for a User

Users maintain control of their data and can opt to delete it at any time. If a user requests a deletion, you can call the deleteFile method on storage to remove data found at a particular file path.

import { AppConfig, UserSession } from "@stacks/connect";
import { Storage } from "@stacks/storage";

const appConfig = new AppConfig(["store_write", "publish_data"]);
const userSession = new UserSession({ appConfig });
const storage = new Storage({ userSession });

let fileName = "car.json";

await storage.deleteFile(fileName);

By making this call, you can delete a user’s data in one go. This is an irreversible process, and once done, users cannot restore their deleted data.

Conclusion

That’s all there is to it! With this guide, you can now handle data storage for your application.

Need help to kickstart your Web3 development journey? Download our free guide to developing smart contracts.