Blob storage reference
API reference for GenSX Cloud blob storage components.
Installation
npm install @gensx/storage
BlobProvider
Provides blob storage capabilities to its child components.
Import
import { BlobProvider } from "@gensx/storage";
Props
Prop | Type | Default | Description |
---|---|---|---|
kind | "filesystem" | "cloud" | Auto-detected | Storage backend to use. Defaults filesystem when running locally and cloud when deployed to the serverless runtime. |
rootDir | string | .gensx/blobs | Root directory for filesystem storage |
defaultPrefix | string | undefined | Optional prefix for all blob keys |
project | string | Auto-detected | Project to use for cloud storage. If you don’t set this, it’ll first check your GENSX_PROJECT environment variable, then look for the project name in your local gensx.yaml file. |
environment | string | Auto-detected | Environment to use for cloud storage. If you don’t set this, it’ll first check your GENSX_ENV environment variable, then use whatever environment you’ve selected in the CLI with gensx env select . |
Example
import { BlobProvider } from "@gensx/storage";
const Workflow = gensx.Component("Workflow", ({ input }) => (
<BlobProvider kind="cloud">
<YourComponent input={input} />
</BlobProvider>
));
useBlob
Hook that provides access to blob storage for a specific key.
Import
import { useBlob } from "@gensx/storage";
Signature
function useBlob<T = unknown>(key: string): Blob<T>;
Parameters
Parameter | Type | Description |
---|---|---|
key | string | The unique key for the blob |
T | Generic type | Type of the JSON data (optional) |
Returns
Returns a blob object with methods to interact with blob storage.
Example
const blob = useBlob<UserProfile>("users/123.json");
const profile = await blob.getJSON();
Blob methods
The blob object returned by useBlob
provides these methods:
JSON operations
// Get JSON data
const data = await blob.getJSON(); // Returns null if not found
// Save JSON data
await blob.putJSON(data, options); // Returns { etag: string }
String operations
// Get string content
const text = await blob.getString(); // Returns null if not found
// Save string content
await blob.putString("Hello world", options); // Returns { etag: string }
Binary operations
// Get binary data with metadata
const result = await blob.getRaw(); // Returns null if not found
// Returns { content, contentType, etag, lastModified, size, metadata }
// Save binary data
await blob.putRaw(buffer, options); // Returns { etag: string }
Stream operations
// Get data as a stream
const stream = await blob.getStream();
// Save data from a stream
await blob.putStream(readableStream, options); // Returns { etag: string }
Metadata operations
// Check if blob exists
const exists = await blob.exists(); // Returns boolean
// Delete blob
await blob.delete();
// Get metadata
const metadata = await blob.getMetadata(); // Returns null if not found
// Update metadata
await blob.updateMetadata({
key1: "value1",
key2: "value2",
});
Options object
Many methods accept an options object with these properties:
{
contentType?: string, // MIME type of the content
etag?: string, // For optimistic concurrency control
metadata?: { // Custom metadata key-value pairs
[key: string]: string
}
}
useBlobStorage
Hook that provides direct access to the blob storage instance, allowing you to perform blob operations across multiple keys.
Import
import { useBlobStorage } from "@gensx/storage";
Signature
function useBlobStorage(): BlobStorage;
Example
const blobStorage = useBlobStorage();
The blob storage object provides these methods:
getBlob
Get a blob object for a specific key.
const blobStorage = useBlobStorage();
const userBlob = blobStorage.getBlob<UserProfile>("users/123.json");
listBlobs
List blobs.
const blobStorage = useBlobStorage();
const { blobs, nextCursor } = await blobStorage.listBlobs({
prefix: "users",
});
console.log(
"User blobs:",
blobs.map((blob) => blob.key),
); // ["users/123.json", "users/456.json"]
The method accepts an options object with these properties:
Option | Type | Description |
---|---|---|
prefix | string | Optional prefix to filter blob keys by |
limit | number | Maximum number of results to return per page |
cursor | string | Cursor for pagination from previous response |
Returns an object with:
blobs
: Array of blob objects containingkey
,lastModified
, andsize
propertiesnextCursor
: Cursor for the next page, or undefined if no more results
Example of a blob object:
{
key: "users/123.json",
lastModified: "2024-01-01T00:00:00Z",
size: 1024
}
blobExists
Check if a blob exists.
const blobStorage = useBlobStorage();
const exists = await blobStorage.blobExists("users/123.json");
if (exists) {
console.log("User profile exists");
}
deleteBlob
Delete a blob.
const blobStorage = useBlobStorage();
const { deleted } = await blobStorage.deleteBlob("temp/file.json");
if (deleted) {
console.log("Temporary file deleted");
}
BlobClient
The BlobClient
class provides a way to interact with GenSX blob storage outside of the GenSX workflow context, such as from regular Node.js applications or server endpoints.
Import
import { BlobClient } from "@gensx/storage";
Constructor
constructor(props?: BlobProviderProps)
Parameters
Parameter | Type | Default | Description |
---|---|---|---|
props | BlobProviderProps | {} | Optional configuration properties |
Example
// Default client (uses filesystem locally, cloud in production)
const blobClient = new BlobClient();
// Explicitly use filesystem storage
const localClient = new BlobClient({
kind: "filesystem",
rootDir: "./my-data",
});
// Explicitly use cloud storage with a prefix
const cloudClient = new BlobClient({
kind: "cloud",
defaultPrefix: "app-data/",
});
Methods
getBlob
Get a blob instance for a specific key.
getBlob<T = unknown>(key: string): Blob<T>
Example
const userBlob = blobClient.getBlob<UserProfile>("users/123.json");
const profile = await userBlob.getJSON();
// Update the profile
profile.lastLogin = new Date().toISOString();
await userBlob.putJSON(profile);
listBlobs
List all blobs.
async listBlobs(options?: { prefix?: string; limit?: number; cursor?: string }): Promise<{
blobs: Array<{ key: string; lastModified: string; size: number }>;
nextCursor?: string;
}>
Example
const { blobs, nextCursor } = await blobClient.listBlobs({
prefix: "chats",
});
console.log(
"Chat histories:",
blobs.map((blob) => blob.key),
); // ["chats/123.json", "chats/456.json"]
blobExists
Check if a blob exists.
async blobExists(key: string): Promise<boolean>
Example
if (await blobClient.blobExists("settings.json")) {
console.log("Settings file exists");
} else {
console.log("Need to create settings file");
}
deleteBlob
Delete a blob.
async deleteBlob(key: string): Promise<DeleteBlobResult>
Example
const { deleted } = await blobClient.deleteBlob("temp/cache.json");
if (deleted) {
console.log("Cache file was deleted");
}
Usage in applications
The BlobClient is particularly useful when you need to access blob storage from:
- Express.js or Next.js API routes
- Background jobs or workers
- Custom scripts or tools
- Any Node.js application outside the GenSX workflow context
// Example: Using BlobClient in an Express handler
import express from "express";
import { BlobClient } from "@gensx/storage";
const app = express();
const blobClient = new BlobClient();
// Save user data endpoint
app.post("/api/users/:userId", async (req, res) => {
try {
const { userId } = req.params;
const userBlob = blobClient.getBlob(`users/${userId}.json`);
// Get existing profile or create new one
const existingProfile = (await userBlob.getJSON()) || {};
// Merge with updated data
const updatedProfile = {
...existingProfile,
...req.body,
updatedAt: new Date().toISOString(),
};
// Save the updated profile
await userBlob.putJSON(updatedProfile);
res.json({ success: true });
} catch (error) {
console.error("Error saving user data:", error);
res.status(500).json({ error: "Failed to save user data" });
}
});
app.listen(3000, () => {
console.log("Server running on port 3000");
});