Skip to main content

Monitor Solana Programs with Yellowstone gRPC Geyser Plugin

Updated on
Nov 15, 2024
Solana
TypeScript
Geyser
Marketplace
Video

13 min read

Overview​

In this guide, we'll learn about Solana Geyser plugins and explore how to use Yellowstone gRPC, a powerful Geyser plugin for Solana, to monitor real-time on-chain activity. Specifically, we'll create a TypeScript application that tracks new token mints from the Pump.fun program on Solana's mainnet. This project will demonstrate how to leverage Geyser's low-latency data access capabilities to build responsive and efficient monitoring tools.

Prefer a visual format? Follow along the video to learn how to monitor Solana program data using Yellowstone gRPC Add-on in 9 minutes.
Subscribe to our YouTube channel for more videos!

What You Will Do​

  • Learn about Geyser and Yellowstone gRPC
  • Write a script using Yellowstone to monitor new Pump.fun mints on Solana
  • Run and test your application

Example output:

Yellowstone Output

What You Will Need​

What is Geyser?​

Geyser is a plugin system for Solana validators that provides low-latency access to blockchain data without overloading validators with intensive RPC requests (e.g., getProgramAccounts). Instead of querying the validator directly, Geyser plugins stream real-time information about accounts, transactions, slots, and blocks to external data stores like relational databases, NoSQL databases, or streaming platforms like Kafka. This approach significantly reduces the load on validators while improving data access efficiency.

The key advantage of Geyser plugins is their ability to scale with high-volume Solana applications. By routing data queries to external stores, developers can implement optimized access patterns like caching and indexing, which is particularly valuable for applications requiring frequent access to large datasets or historical information. This separation allows validators to focus on their primary role of processing transactions while ensuring developers have the comprehensive, real-time data access they need.

What is Yellowstone Dragon's Mouth?​

Yellowstone Dragon's Mouth (referred to as "Yellowstone") is an open-source gRPC interface built on Solana's Geyser plugin system. It leverages gRPC, Google's high-performance framework that combines Protocol Buffers for serialization with HTTP/2 for transport, enabling fast and type-safe communication between distributed systems.

Yellowstone provides real-time streaming of:

  • Account updates
  • Transactions
  • Entries
  • Block notifications
  • Slot notifications

Compared to traditional WebSocket implementations, Yellowstone's gRPC interface offers lower latency and higher stability. It also includes unary operations for quick, one-time data retrievals. The combination of gRPC's efficiency and type safety makes Yellowstone particularly well-suited for cloud-based services and database updates. QuickNode supports Yellowstone through our Yellowstone gRPC Marketplace Add-on.

Let's see Yellowstone in action by writing a script to monitor new Pump.fun mints on Solana.

Create a New Project​

Let's start by setting up a new TypeScript project:

  1. Create a new directory for your project and navigate into it:

    mkdir pump-fun-monitor && cd pump-fun-monitor

  2. Initialize a new Node.js project:

    npm init -y

  3. Install the required dependencies:

    npm install @triton-one/yellowstone-grpc @solana/web3.js bs58 @types/node

  4. Create a tsconfig.json file in the root of your project:

    npx tsc --init

  5. Open the tsconfig.json file and ensure it includes the following options:

    {
      "compilerOptions": {
        "target": "es2020",
        "module": "commonjs",
        "strict": true,
        "esModuleInterop": true,
        "outDir": "./dist",
        "rootDir": ""
      }
    }

  6. Create a new file index.ts and add the following command:

    echo > index.ts

Now you're ready to start writing your script!

Write the Script​

Let's create our script to monitor Pump.fun mints using Yellowstone. We'll break this down into several steps:

Step 1: Import Dependencies and Define Constants​

Create a new file index.ts and start with the following code:

import Client, {
    CommitmentLevel,
    SubscribeRequest,
    SubscribeUpdate,
    SubscribeUpdateTransaction,
} from "@triton-one/yellowstone-grpc";
import { Message, CompiledInstruction } from "@triton-one/yellowstone-grpc/dist/grpc/solana-storage";
import { ClientDuplexStream } from '@grpc/grpc-js';
import { PublicKey } from '@solana/web3.js';
import bs58 from 'bs58';

// Constants
const ENDPOINT = "https://example.solana-mainnet.quiknode.pro:10000";
const TOKEN = "TOKEN_ID";
const PUMP_FUN_PROGRAM_ID = '6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P';
const PUMP_FUN_CREATE_IX_DISCRIMINATOR = Buffer.from([24, 30, 200, 40, 5, 28, 7, 119]);
const COMMITMENT = CommitmentLevel.CONFIRMED;

// Configuration
const FILTER_CONFIG = {
    programIds: [PUMP_FUN_PROGRAM_ID],
    instructionDiscriminators: [PUMP_FUN_CREATE_IX_DISCRIMINATOR]
};

const ACCOUNTS_TO_INCLUDE = [{
    name: "mint",
    index: 0
}];

This section imports necessary dependencies and defines constants we'll use throughout the script.

  • Make sure to replace ENDPOINT and TOKEN with your QuickNode endpoint and token.
  • We are getting the create instruction discriminator for the Pump.fun program from the unofficial SDK, here.
  • We are defining an array of accounts to monitor; in this case, we are just monitoring the mint account of the newly minted token. Feel free to include additional accounts you want to monitor (you can get their index from the IDL as well).

Step 2: Define Types and Interfaces​

Let's create an interface for the output data that we will generate. Add the following type definition:

// Type definitions
interface FormattedTransactionData {
    signature: string;
    slot: string;
    [accountName: string]: string;
}

This interface defines the structure of our formatted transaction data. We are using a varialbe length array of acccountName: string to store the account data for each account we choose to monitor through the ACCOUNTS_TO_INCLUDE constant. In our case, it will just be mint: string (the address of the newly minted token).

Step 3: Implement the Main Function​

Add the main function to your script:

// Main function
async function main(): Promise<void> {
    const client = new Client(ENDPOINT, TOKEN, {});
    const stream = await client.subscribe();
    const request = createSubscribeRequest();

    try {
        await sendSubscribeRequest(stream, request);
        console.log('Geyser connection established - watching new Pump.fun mints. \n');
        await handleStreamEvents(stream);
    } catch (error) {
        console.error('Error in subscription process:', error);
        stream.end();
    }
}

Let's walk through these functions:

  • This function initializes the Yellowstone gRPC client using the @triton-one/yellowstone-grpc library.
  • We create a stream using the subscribe method of the client, which returns a Stream object.
  • We then use a series of helper functions (which we will define in the next step) to handle the stream events and process the incoming data:
    • We create a subscription request using a helper function, createSubscribeRequest, which will define the accounts we want to monitor, the slots we want to monitor, and the transactions we want to monitor.
    • We then send the subscription request to the stream using the sendSubscribeRequest function.
    • We call handleStreamEvents to process the incoming data and log the results.

Great. Now, let's add those helper functions.

Step 4: Implement Helper Functions​

First, let's define our createSubscribeRequest function. This will simply return a constant SubscribeRequest object that we will use to configure our subscription. Add the following code:

// Helper functions
function createSubscribeRequest(): SubscribeRequest {
    return {
        accounts: {},
        slots: {},
        transactions: {
            pumpFun: {
                accountInclude: FILTER_CONFIG.programIds,
                accountExclude: [],
                accountRequired: []
            }
        },
        transactionsStatus: {},
        entry: {},
        blocks: {},
        blocksMeta: {},
        commitment: COMMITMENT,
        accountsDataSlice: [],
        ping: undefined,
    };
}

All this does is specify that we will be looking for transaction data for transactions that include our defined programIds (in this case, the Pump.fun program we specified in our constants). Structurally, this object is where you can customize the data you would like to receive. For more information on the available options, check out the documentation.

Next, create your sendSubscribeRequest function. This will take the subscription request and write it to the stream. Add the following code:

function sendSubscribeRequest(
    stream: ClientDuplexStream<SubscribeRequest, SubscribeUpdate>,
    request: SubscribeRequest
): Promise<void> {
    return new Promise<void>((resolve, reject) => {
        stream.write(request, (err: Error | null) => {
            if (err) {
                reject(err);
            } else {
                resolve();
            }
        });
    });
}

Finally, once we have our stream established, we need our function to handle the data that is being streamed. Let's create a simple function that utilizes the on method of the stream to listen for various events. Add the following code:

function handleStreamEvents(stream: ClientDuplexStream<SubscribeRequest, SubscribeUpdate>): Promise<void> {
    return new Promise<void>((resolve, reject) => {
        stream.on('data', handleData);
        stream.on("error", (error: Error) => {
            console.error('Stream error:', error);
            reject(error);
            stream.end();
        });
        stream.on("end", () => {
            console.log('Stream ended');
            resolve();
        });
        stream.on("close", () => {
            console.log('Stream closed');
            resolve();
        });
    });
}

We are simply specifying how the stream should handle data, error, end, and close events. We will use these events to process the incoming data and log the results. Not that when we receive data, we are calling, handleData. We will need to define a few functions to handle the data processing--let's do that now.

Step 5: Implement Data Processing Functions​

Add the following data processing functions. We will describe them in detail below. 

function handleData(data: SubscribeUpdate): void {
    if (!isSubscribeUpdateTransaction(data) || !data.filters.includes('pumpFun')) {
        return;
    }

    const transaction = data.transaction?.transaction;
    const message = transaction?.transaction?.message;

    if (!transaction || !message) {
        return;
    }

    const matchingInstruction = message.instructions.find(matchesInstructionDiscriminator);
    if (!matchingInstruction) {
        return;
    }

    const formattedSignature = convertSignature(transaction.signature);
    const formattedData = formatData(message, formattedSignature.base58, data.transaction.slot);

    if (formattedData) {
        console.log("======================================💊 New Pump.fun Mint Detected!======================================");
        console.table(formattedData);
        console.log("\n");
    }
}

function isSubscribeUpdateTransaction(data: SubscribeUpdate): data is SubscribeUpdate & { transaction: SubscribeUpdateTransaction } {
    return (
        'transaction' in data &&
        typeof data.transaction === 'object' &&
        data.transaction !== null &&
        'slot' in data.transaction &&
        'transaction' in data.transaction
    );
}

function convertSignature(signature: Uint8Array): { base58: string } {
    return { base58: bs58.encode(Buffer.from(signature)) };
}

function formatData(message: Message, signature: string, slot: string): FormattedTransactionData | undefined {
    const matchingInstruction = message.instructions.find(matchesInstructionDiscriminator);

    if (!matchingInstruction) {
        return undefined;
    }

    const accountKeys = message.accountKeys;
    const includedAccounts = ACCOUNTS_TO_INCLUDE.reduce<Record<string, string>>((acc, { name, index }) => {
        const accountIndex = matchingInstruction.accounts[index];
        const publicKey = accountKeys[accountIndex];
        acc[name] = new PublicKey(publicKey).toBase58();
        return acc;
    }, {});

    return {
        signature,
        slot,
        ...includedAccounts
    };
}

function matchesInstructionDiscriminator(ix: CompiledInstruction): boolean {
    return ix?.data && FILTER_CONFIG.instructionDiscriminators.some(discriminator =>
        Buffer.from(discriminator).equals(ix.data.slice(0, 8))
    );
}

These functions handle processing the incoming data, filtering for relevant transactions, and formatting the output. Let's look at each of these functions in detail:

  • handleData: This function is called for each incoming data chunk. It first checks if the instruction is the expected format and includes our pumpFun filter. We then filter incoming results to only include transactions that have an instruction that matches our filter criteria. Finally, we format the output to include the transaction signature and the instruction data. Then we log the results.
  • convertSignature: This helper function converts the transaction signature to a base58 string using the bs58 library.
  • formatData: This helper function formats the output to include the transaction signature and the specified account(s) from the transaction
  • matchesInstructionDiscriminator: This helper function checks if the instruction data matches any of the instruction discriminators in our filter configuration.

Step 6: Run the Main Function​

Finally, add this line at the end of your script to run the main function:

main().catch((err) => {
    console.error('Unhandled error in main:', err);
    process.exit(1);
});

This will execute our script and handle any unhandled errors. With these steps, you've created a complete script to monitor Pump.fun mints using Yellowstone. Let's give it a try!

Run Your Code​

Now that we've written our script let's run it and see it in action. In your terminal, run the following command:

ts-node index.ts

If everything is set up correctly, you should see output similar to this:

Geyser connection established - watching new Pump.fun mints

======================================💊 New Pump.fun Mint Detected!======================================
┌───────────┬────────────────────────────────────────────────────────────────────────┐
│ (index)   │ Values                                                                 │
├───────────┼──────────────────────────────────────────────────────────────────────--┤
│ signature │ '4vzEaCkQnKym4TdDv67JF9VYMbvoMRwWU5E6TMZPSAbHJh4tXhsbcU8dkaFey1kFn...' │
│ slot      │ '291788725'                                                            │
│ mint      │ 'AWcvL1GSNX8VDLm1nFWzB9u2o4guAmXM341imLaHpump'                         │
└───────────┴────────────────────────────────────────────────────────────────────────┘

The script will continue running, monitoring for new Pump.fun mints in real-time. Each time a new mint is detected, it will display the transaction signature, slot number, and the newly minted token address.

Great job!

Wrap Up​

In this guide, we've explored how to use Yellowstone, a powerful Geyser plugin, to monitor Solana programs in real-time. We focused on tracking new token mints from the Pump.fun program, but the principles we've covered can be applied to monitor any Solana program or account updates.

As you continue to build on Solana, consider how you can leverage Geyser plugins like Yellowstone to create more responsive and efficient applications. Whether you're building a trading bot, an analytics dashboard, or a complex DeFi application, real-time data access can give you a significant edge.

Resources​

Let's Connect!​

We'd love to hear how you are using Yellowstone. Send us your experience, questions, or feedback via Twitter or Discord.

We ❤️ Feedback!

Let us know if you have any feedback or requests for new topics. We'd love to hear from you.

Written by

Aaron Milano

Last updated on
Share this guide