New Image Caching Brings Better Performance to the Token Metadata API
We’ve added a new caching layer for token image data in the Token Metadata API. This change brings better performance, better security, and even thumbnail-sized images.
The Token Metadata API provides data on tokens in the Stacks ecosystem, including token attributes and their corresponding images (perhaps a token logo in the case of a fungible token or an image in the case of an NFT).
In order to get this data, we first pull the metadata from the smart contract that issues the token, and we then store that data in a Postgres database and serve that data via public endpoints.
In a JSON format, the data for a NFT looks something like this:
{
"version": 1,
"name": "Belle's Witch 97",
"collection": "Belle's Witches",
"description": "The Witches come with all sorts of accessories, hair styles, eyes, lips, clothes and elements with different rarities! There are 3 races: fairy, gnome and human. Which Witch will cast a spell on you?\n\nThis project was the first NFT created by the digital artist Isabelle Parra and was based on a self-portrait",
"image": "ipfs://QmUUf7WggwHSQ6gGEPpSordi9yyN6hSexSwhbowxRMnWFo/97.png",
"edition": 97,
"attributes": [
{
"trait_type": "Background",
"value": "Pink"
},
{
"trait_type": "Race",
"value": "Gnome Blue Skin"
},
{
"trait_type": "Clothing",
"value": "Purple Cape"
},
{
"trait_type": "Hair",
"value": "Blue Straight Hair"
},
{
"trait_type": "Eyes",
"value": "Yellow Cat Eyes"
},
{
"trait_type": "Lips",
"value": "Pink Lips"
},
{
"trait_type": "Accessories",
"value": "Black Glasses"
},
{
"trait_type": "Elemental",
"value": "Poison Elemental"
}
]
}
As you can see in the code above, you can see the token’s name, the collection it comes from, and a link to the image itself. That link is a problem.
The Problem With Gateways
NFTs usually store their data in IPFS, and in order to access that data, most users go through an ipfs.io gateway (which we use to access the "image" link in the snippet above).
In other words, when users request a token image from our API, we return the link to the image, not the image itself, and users must then fetch the token image via that linked gateway.
Using this gateway comes with a few problems. First of all, it is heavily rate limited. Second, this gateway may not return the image at all: perhaps the gateway (or you) are not near a node that contains the image. Third, it returns only one image file size, which can often be larger than needed, particularly if you’re only looking to use a thumbnail.
All of these issues can translate to delays. By going through this gateway, you may have to wait several seconds for the image to show up, whether by being throttled from the gateway’s rate limit, the size of the image, or something else.
We know this pain firsthand, since we use the Token Metadata API to fetch data for the Stacks Explorer. That’s why we decided to fix it.
A New Image Caching System
We built an internal image caching system to store that token data ourselves. That system begins with this script.
Now, every time we find new images when processing (or re-processing) tokens, we take the images from the original location. For example, in this snippet, you can see how we fetch an image from the IPFS gateway:
Then we process that image file using Node.js. Often images are stored as SVGs in IPFS, a file format that has security issues. So in this step, we transform image files into PNGs, and we also resize images, adding a new thumbnail-sized version to our database.
After that is complete, we upload these images to our own Google Cloud Storage bucket that hosts all of these images, which you can see in this snippet:
Those images, both the full-sized and thumbnail-sized versions, then get added to a <code-rich-text>token metadata api<code-rich-text> folder in our GCS bucket. For clients using the Token Metadata API, you will now see a new JSON response from our endpoints:
{
"token_uri": "ipfs://QmUpfBNUnVUzwhbahvRTrSPrQhFnBv1VVwe9t6csCPCF53/97.json",
"metadata": {
"sip": 16,
"name": "Belle's Witch 97",
"description": "The Witches come with all sorts of accessories, hair styles, eyes, lips, clothes and elements with different rarities! There are 3 races: fairy, gnome and human. Which Witch will cast a spell on you?\n\nThis project was the first NFT created by the digital artist Isabelle Parra and was based on a self-portrait",
"image": "ipfs://QmUUf7WggwHSQ6gGEPpSordi9yyN6hSexSwhbowxRMnWFo/97.png",
"cached_image": "https://assets.hiro.so/api/mainnet/token-metadata-api/SPJW1XE278YMCEYMXB8ZFGJMH8ZVAAEDP2S2PJYG.belles-witches/97.png",
"cached_thumbnail_image": "https://assets.hiro.so/api/mainnet/token-metadata-api/SPJW1XE278YMCEYMXB8ZFGJMH8ZVAAEDP2S2PJYG.belles-witches/97-thumb.png",
"attributes": [
{
"trait_type": "Background",
"display_type": "",
"value": "Pink"
},
{
"trait_type": "Race",
"display_type": "",
"value": "Gnome Blue Skin"
},
{
"trait_type": "Clothing",
"display_type": "",
"value": "Purple Cape"
},
{
"trait_type": "Hair",
"display_type": "",
"value": "Blue Straight Hair"
},
{
"trait_type": "Eyes",
"display_type": "",
"value": "Yellow Cat Eyes"
},
{
"trait_type": "Lips",
"display_type": "",
"value": "Pink Lips"
},
{
"trait_type": "Accessories",
"display_type": "",
"value": "Black Glasses"
},
{
"trait_type": "Elemental",
"display_type": "",
"value": "Poison Elemental"
}
]
}
}
As you can see in this snippet, both the full-sized and thumbnail-sized images are hosted at the assets.hiro.so URL alongside a link to the IPFS gateway.
This is great for clients like the Stacks Explorer. These clients rely on high availability services, and they can use the thumbnail-sized images to have pages load much faster.
Better Performance for the Token Metadata API
We have been running this processing for all fungible tokens, and now all fungible tokens use this new GCS gateway hosted by Hiro. We are currently rolling out support for all NFT collections, and soon all NFTs on Stacks will also live in our GCS bucket.
Try out the new endpoint responses yourself and get started in our documentation.
Thanks for subscribing.
Oops! Something went wrong while submitting the form.
Share this article
Copy link
Hiro news & product updates straight to your inbox
Only relevant communications. We promise we won’t spam.