Overview

Key-Value Store is a robust data storage solution that enables you to manage extensive lists of datapoints, such as hundreds (or millions) of wallet addresses, and key value sets. With Key-Value Store, you can update your lists and key-value sets on demand, manage and use them at any time via REST API or seamlessly within your Streams filter.

Why Key-Value Store?

When you are dealing with large volumes of data, efficient storage and quick retrieval are critical. Key-Value Store seamlessly integrates with Streams, enabling you to easily evaluate your lists and key-value sets against streaming real-time and historical data with ease, manage your lists and sets, and track datapoints block-over-block with high efficiency.

This solution is particularly useful for use cases that require tracking, updating and evaluating large datasets in real-time, such as:

• Wallet and portfolio applications that want to track the activity of a large, dynamic list of wallet addresses
• Tracking and checking token holdings
• Applications that track and alert on changes in values over time

Key-Value Store Features

• Scalable Data Management - Manage, access, and evaluate large lists of data, such as millions of addresses or other data points, with ease.
• Real-Time Updates - Update your data on demand and access it instantly.
• API Access - Interact with your lists and key value sets via REST API.
• Streams Integration - Use Key-Value Store seamlessly within Streams filters for enhanced data processing capabilities.
• High Performance - Key-Value Store features ultra-fast access time for efficient data lookup and updates.

Example Use Cases

• Wallet Address Management: Manage lists of wallet addresses to evaluate against streaming data.
• Historical Data Tracking: Track values over time for various metrics.
• Data Analysis: Calculate deltas and other statistics over large datasets.
• Build Your Own API: Create lists and sets using Key-Value Store with Streams filters, and create API endpoints with Functions to interact with the data.

By leveraging the Key-Value Store, you can enhance and customize Streams, enabling you to create more sophisticated, efficient, and responsive streaming datasets based on your specifications.

Feature Availability

Key-Value Store Beta is now available to all users on the Growth plan and higher.

Access

Access Key-Value Store within Streams in the QuickNode Developer Portal, within Functions and via Key-Value Store REST API.

Costs & Limits

Key-Value Store beta is available without limits to Growth, Business, Business+, and Enterprise users at no additional cost during beta. Please keep in mind that while Key-Value Store is in Beta these limits are subject to change.

Using Key-Value Store with Streams filters

Note

Key-Value Store items are case-sensitive. This is particularly important to remember when working with blockchain addresses, as they must be entered exactly as they appear onchain.

Key-Value Store can be accessed and managed seamlessly within a Streams filter. You can create new lists and key-value sets, add and remove items from your lists, update key-value sets, and evaluate your Key-Value Store data against streaming blockchain data.

Use lists when you want to associate many values to one key, for instance you could manage and evaluate a large list of wallet addresses by adding them as items to a list called addresses.

Use key value sets when you want to store datapoints for easy retrieval and updating, for instance you could calculate the average gas price in a block and store the value with they key gasPrice, and then retrieve it in the next block to calculate the delta.

Available Key-Value Store Functions inside your Streams Filter

Lists

• qnUpsertList: Creates or updates a new list in Key-Value Store.
• qnGetList: Retrieves all items from a specific list in Key-Value Store.
• qnGetAllLists: Retrieves all lists in Key-Value Store.
• qnAddListItem: Adds an item to a specific list in Key-Value Store.
• qnRemoveListItem: Removes an item from a specific list in Key-Value Store.
• qnContainsListItem: Checks if an item exists in a specific list in Key-Value Store.
• qnDeleteList: Deletes a specific list in Key-Value Store.

Example Filter Code for Lists

The filter code below can be used within Streams to demonstrate ways that Key-Value Store lists can be used within your Streams filter. You can deploy this example with one click here.

function main() {
    // List of results for each operation
    let results = {};

    try {
        // Create a new list
        results.createList = qnUpsertList("list_docs_example", {
            add_items: ["item1", "item2"],
        });

        // Update a list
        results.qnUpsertList = qnUpsertList("list_docs_example", {
            add_items: ["item3"],
            remove_items: ["item1"],
        });

        // Get a list
        results.qnGetList = qnGetList("list_docs_example");

        // Get all lists
        results.qnGetAllLists = qnGetAllLists();

        // Add item to the list
        results.qnAddListItem4 = qnAddListItem("list_docs_example", "item4");
        results.qnAddListItem5 = qnAddListItem("list_docs_example", "item5");

        // Remove item from the list
        results.qnRemoveListItem4 = qnRemoveListItem("list_docs_example", "item4");

        // Get a list after changes
        results.qnGetListAfterChanges = qnGetList("list_docs_example");

        // Check if an item is in the list
        results.qnContainsListItem2 = qnContainsListItem("list_docs_example", "item2");
        results.qnContainsListItem1 = qnContainsListItem("list_docs_example", "item1");

        // Delete the list
        results.qnDeleteList = qnDeleteList("list_docs_example");
    } catch (error) {
        results.error = error.message;
    }

    return results;
}

Example return

{
    "createList": "OK",
    "qnAddListItem4": "OK",
    "qnAddListItem5": "OK",
    "qnContainsListItem1": false,
    "qnContainsListItem2": true,
    "qnDeleteList": "OK",
    "qnGetAllLists": [
        "list_docs_example"
    ],
    "qnGetList": [
        "item2",
        "item3"
    ],
    "qnGetListAfterChanges": [
        "item2",
        "item3",
        "item5"
    ],
    "qnRemoveListItem4": "OK",
    "qnUpsertList": "OK"
}

Sets

• qnAddSet: Creates a key-value set in Key-Value Store.
• qnBulkSets: Creates and removes bulk key-value sets in Key-Value Store.
• qnDeleteSet: Deletes a key-value set from Key-Value Store.
• qnGetSet: Retrieves the value of a specific key from Key-Value Store.
• qnListAllSets: Lists all keys for sets in Key-Value Store.

Example Filter Code for Key-Value Store Sets

The filter code below can be used within Streams to demonstrate ways that Key-Value Store sets can be used within your Streams filter. You can deploy this example with one click here.

function main() {
    // List of results for each operation
    let results = {};

    try {
        // Create a set
        results.qnAddSet = qnAddSet("set_docs_example", "value1");

        // Get a set
        results.qnGetSet = qnGetSet("set_docs_example");

        // Get all sets
        results.qnListAllSets = qnListAllSets();

        // Bulk add/remove sets
        results.qnBulkSets = qnBulkSets({
            add_sets: {
                "set_docs_example2": "value1",
                "set_docs_example3": "value2"
            },
            delete_sets: ["set_docs_example1"]
        });

        // Get all sets after bulk
        results.qnListAllSets2 = qnListAllSets();

        // Delete key values
        results.qnDeleteSet2 = qnDeleteSet("set_docs_example2");
        results.qnDeleteSet3 = qnDeleteSet("set_docs_example3");

    } catch (error) {
        results.error = error.message;
    }

    return results;
}

Example Return

{
    "qnAddSet": "OK",
    "qnBulkSets": "OK",
    "qnDeleteSet2": "OK",
    "qnDeleteSet3": "OK",
    "qnGetSet": "value1",
    "qnListAllSets": [
        "kv_docs_example",
        "keyval1"
    ],
    "qnListAllSets2": [
        "kv_docs_example3",
        "kv_docs_example2",
        "kv_docs_example",
        "keyval1"
    ]
}

Using Key-Value Store with Functions

Key-Value Store can be accessed and managed seamlessly within a function. You can create new lists and key-value sets, add and remove items from your lists, update key-value sets, retrieve set value, and check if data matches an item on your list. To learn more about functions, please visit the Functions documentation

Example function code for Key-Value Store

The function code below demonstrates how Key-Value Store lists and sets can be used within your function.

async function main(params) {
    const results = {};
    
    try {
        // Test qnGetAllLists
        results.qnGetAllLists = await qnLib.qnGetAllLists();

        // Test qnDeleteList
        results.qnDeleteList = await qnLib.qnDeleteList("testList");

        // Test qnAddListItem
        results.qnAddListItem = await qnLib.qnAddListItem("testList", "testItem");
        
        // Test qnContainsListItem
        results.qnContainsListItem_before_remove = await qnLib.qnContainsListItem("testList", "testItem");

        // Test qnRemoveListItem
        results.qnRemoveListItem = await qnLib.qnRemoveListItem("testList", "testItem");

        // Test qnContainsListItem
        results.qnContainsListItem_after_remove = await qnLib.qnContainsListItem("testList", "testItem");

        // Test qnGetList
        results.qnGetList = await qnLib.qnGetList("testList");

        // Test qnUpsertList
        results.qnUpsertList = await qnLib.qnUpsertList("testList", { add_items: ["item1"], remove_items: ["item2"] });

        // Test qnDeleteSet
        results.qnDeleteSet = await qnLib.qnDeleteSet("testSet");

        // Test qnAddSet
        results.qnAddSet = await qnLib.qnAddSet("testSet", "testValue");

        // Test qnGetSet
        results.qnGetSet = await qnLib.qnGetSet("testSet", params);

        // Test qnListAllSets
        results.qnListAllSets = await qnLib.qnListAllSets();

        // Test qnBulkSets
        results.qnBulkSets = await qnLib.qnBulkSets({ add_sets: { key1: "value1" }, delete_sets: ["key2"] });

    } catch (error) {
        console.error("Error during method tests:", error);
    }
    
    // Return the results
    return {
        message: "Method test results",
        results: results
    };
}

Note

The function main must be async if you are accessing Key-Value Store within your function.

We ❤️ Feedback

❓ We want to hear from you! Please take a few minutes to fill out our Key-Value Store feedback form and let us know what you currently think about it. This helps us further improve the documentation.