This tutorial demonstrates how to build an end-to-end streaming 3D reconstruction pipeline using LingBot-Map. We'll cover configuration, GPU-aware parameter tuning, model setup, inference, and exporting point clouds and other artifacts. By the end, you'll have a complete workflow ready for your own scenes.

Overview

LingBot-Map is a state-of-the-art framework for real-time 3D scene reconstruction from monocular video or image sequences. It leverages streaming attention and long-range trajectory memory to produce accurate camera poses and dense point clouds, even on limited hardware. This tutorial walks through a complete pipeline—from raw input to exportable 3D models—with special attention to GPU memory management for 2026-era hardware.

Configuration

First, we define a configuration dictionary that controls every stage of the pipeline. This includes input settings, model parameters, inference options, and output preferences.

CFG = {
"scene":        "courthouse",
"image_folder": None,
"video_path":   None,
"fps":          10,
"max_frames":   None,
"stride":       None,
"checkpoint":   "lingbot-map.pt",
"image_size":   518,
"patch_size":   14,
"use_sdpa":     True,
"mode":         "streaming",
"numscaleframes":      None,
"keyframe_interval":     None,
"kvcachesliding_window": 64,
"cameranumiterations": None,
"offloadtocpu":        True,
"window_size":           128,
"overlap_keyframes":     8,
"conf_percentile":       55.0,
"pixel_stride":          2,
"maxplotpoints":       60000,
"export_ply":            True,
"export_glb":            False,
"launch_viser":          False,
"run_ablation":          False,
"seed":                  0,
}

Key settings:

• Scene: Identifier for your dataset, used for output naming.
• Input: Provide either imagefolder (sequence of frames) or videopath (video file). Leave the other as None.
• FPS & max_frames: Control frame sampling rate and total number of frames processed.
• Checkpoint: Path to the pretrained model weights.
• Model parameters: imagesize, patchsize, and use_sdpa (scaled dot-product attention) affect efficiency and accuracy.
• Streaming settings: keyframeinterval, kvcacheslidingwindow, and window_size control memory usage during long sequences.
• Output: Toggle PLY/GLB exports and 3D visualization via Viser.

Setup and Dependencies

Next, we set up the workspace and clone the repository. We'll also install dependencies and download the pretrained checkpoint.

import os, sys, subprocess, glob, json, time, math, shutil, textwrap
os.environ.setdefault("PYTORCHCUDAALLOCCONF", "expandablesegments:True")

WORK = "/content"
REPO = f"{WORK}/lingbot-map"
OUT  = f"{WORK}/lingbot_out"
os.makedirs(OUT, exist_ok=True)

def sh(cmd, check=True):
"""Run a shell command and print output on failure."""
p = subprocess.run(cmd, shell=True, capture_output=True, text=True)
if p.returncode != 0 and check:
print(p.stdout[-3000:])
print(p.stderr[-3000:])
raise RuntimeError(f"Command failed: {cmd}")
return p

cd $WORK
git clone https://github.com/robbyant/lingbot-map.git
cd lingbot-map
pip install -r requirements.txt
wget -O lingbot-map.pt https://huggingface.co/robbyant/lingbot-map/resolve/main/lingbot-map.pt

GPU-Aware Configuration Tuning

To get the best performance, we dynamically adjust inference parameters based on the GPU's VRAM. This is crucial in 2026 as GPU memory sizes vary widely—from 8 GB laptops to 80 GB datacenter cards. We'll write a function that probes VRAM and sets frame limits, iteration counts, and caching parameters accordingly.

def tuneparametersforgpu(cfg, vramgb):
"""Adjust cfg based on available VRAM."""
if vram_gb >= 24:
cfg["maxframes"] = cfg.get("maxframes") or 300
cfg["cameranumiterations"] = 200
cfg["numscaleframes"] = 8
cfg["keyframe_interval"] = 10
elif vram_gb >= 12:
cfg["maxframes"] = cfg.get("maxframes") or 200
cfg["cameranumiterations"] = 150
cfg["numscaleframes"] = 5
cfg["keyframe_interval"] = 8
else:
Low VRAM: reduce load
cfg["maxframes"] = cfg.get("maxframes") or 100
cfg["cameranumiterations"] = 100
cfg["numscaleframes"] = 3
cfg["keyframe_interval"] = 5
cfg["offloadtocpu"] = True
cfg["kvcachesliding_window"] = 32  # smaller cache
return cfg

Detect VRAM (in GB)
try:
import torch
if torch.cuda.is_available():
vramgb = torch.cuda.getdeviceproperties(0).totalmemory / 1e9
else:
vram_gb = 0
except:
vram_gb = 0

print(f"Detected VRAM: {vram_gb:.1f} GB")
CFG = tuneparametersforgpu(CFG, vramgb)

Input Preprocessing

We now load frames from either an image folder or a video file, apply center cropping and resizing, and prepare them for the model.

import cv2
import numpy as np

def load_frames(cfg):
frames = []
if cfg["image_folder"]:
paths = sorted(glob.glob(os.path.join(cfg["imagefolder"], "*.jpg")))[:cfg["maxframes"]]
for p in paths:
img = cv2.imread(p)
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
frames.append(img)
elif cfg["video_path"]:
cap = cv2.VideoCapture(cfg["video_path"])
count = 0
while True:
ret, frame = cap.read()
if not ret:
break
if count % int(cfg["fps"] // 10) == 0 and len(frames) < cfg["max_frames"]:
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
frames.append(frame)
count += 1
cap.release()
Preprocess: resize to image_size with padding
processed = []
for frame in frames:
h, w = frame.shape[:2]
scale = cfg["image_size"] / min(h, w)
newh, neww = int(h  scale), int(w  scale)
resized = cv2.resize(frame, (neww, newh))
Center crop to image_size
top = (newh - cfg["imagesize"]) // 2
left = (neww - cfg["imagesize"]) // 2
cropped = resized[top:top+cfg["imagesize"], left:left+cfg["imagesize"]]
processed.append(cropped)
return np.array(processed)

frames = load_frames(CFG)
print(f"Loaded {len(frames)} frames")

Model Construction

We instantiate the GCTStream model with streaming attention and trajectory memory. The model also loads the pretrained checkpoint.

from lingbot_map.models import GCTStream

model = GCTStream(
imagesize=CFG["imagesize"],
patchsize=CFG["patchsize"],
usesdpa=CFG["usesdpa"],
mode=CFG["mode"],
numscaleframes=CFG["numscaleframes"],
keyframeinterval=CFG["keyframeinterval"],
kvcacheslidingwindow=CFG["kvcacheslidingwindow"],
cameranumiterations=CFG["cameranumiterations"],
offloadtocpu=CFG["offloadtocpu"],
windowsize=CFG["windowsize"],
overlapkeyframes=CFG["overlapkeyframes"],
)

checkpoint = torch.load(CFG["checkpoint"], map_location="cpu")
model.loadstatedict(checkpoint["modelstatedict"])
model.cuda().eval()

Inference and Point Cloud Generation

We run mixed-precision inference to predict camera poses and depth maps, then convert depth to world-coordinate point clouds.

from lingbotmap.utils.geometry import depthtopointcloud, posefrom_parameters

with torch.nograd(), torch.autocast(devicetype="cuda", dtype=torch.float16):
for i in range(0, len(frames), CFG["pixel_stride"]):
Prepare batch of frames
batch = torch.tensor(frames[i:i+CFG["pixel_stride"]], device="cuda").float() / 255.0
Model forward pass
predictions = model(batch)
Extract camera parameters and depth from predictions
cameraparams = predictions["cameraparams"]
depth_map = predictions["depth"]
Compute world coordinates
pts = depthtopointcloud(depthmap, cameraparams, CFG["conf_percentile"])
Accumulate points
all_points.append(pts.cpu().numpy())

Exporting Results

Finally, we export the 3D reconstruction to PLY (point cloud) and optionally GLB (mesh). We also validate the geometry and optionally visualize in Viser.

from lingbotmap.utils.export import exportply, export_glb

Combine all points
allpoints = np.concatenate(allpoints, axis=0)

Downsample for visualization/export (optional)
if len(allpoints) > CFG["maxplot_points"]:
indices = np.random.choice(len(allpoints), CFG["maxplot_points"], replace=False)
allpoints = allpoints[indices]

if CFG["export_ply"]:
exportply(allpoints, os.path.join(OUT, "scene.ply"))
print("Exported PLY.")

if CFG["export_glb"]:
exportglb(allpoints, os.path.join(OUT, "scene.glb"))
print("Exported GLB.")

Conclusion

You now have a complete GPU-aware inference pipeline with LingBot-Map. The framework adapts to your hardware, streams long sequences efficiently, and exports point clouds ready for downstream tasks. This tutorial aligns with 2026 best practices, including mixed-precision computation and dynamic VRAM tuning, ensuring your reconstruction tasks run smoothly across a wide range of hardware.

via MarkTechPost