2026-10-19 11:49

Introduction: What is EdenLive? EdenLive is VAST's real-time generative renderer, a milestone achievement of its World Model project, Project Eden

What is EdenLive?

EdenLive is VAST’s real-time generative renderer and a key milestone in its World Model initiative, Project Eden. It leverages Proxy to simplify reference visuals or video streams as control signals, combining the target first frame with text prompts to continuously generate photorealistic or stylized outputs in real time. On a single RTX 5090, it achieves over 720P resolution at 24 FPS with end-to-end latency under 100ms. The core design of EdenLive decouples world state maintenance from visual rendering, enabling interactive generation without requiring complete asset pipelines. Fully open-sourced, EdenLive is applicable across gaming, animation, and embodied intelligence data generation.

Core Features of EdenLive

• Real-Time Generative Rendering: Uses Proxy-simplified visuals or reference videos as control signals to continuously generate high-fidelity or stylized frames, exceeding 24 FPS at 720P.
• Arbitrary Style Switching: With only a target first frame and text prompt, the same scene can be transformed into anime, oil painting, photorealistic, or other visual styles.
• Flexible Reference Input: Supports diverse sources such as 2D animations, 3D scenes, and real-world video as structural and motion references, independent of specific rendering pipelines.
• Proxy-to-Fine Generation: Directly generates high-quality final frames from coarse scene layouts, eliminating the need for pre-built full art assets.
• Game Interaction Integration: Decoupled from upstream interaction systems, preserving UI and control mechanics while enabling full-stack state management + real-time rendering.
• Open Ecosystem Support: Open-sources model weights, inference code, and technical reports; includes Game Prototype Turbo for one-click generation of playable Proxy games from natural language input.
• Embodied Intelligence Data Synthesis: Modifies visual appearance while preserving original action trajectories and scene structures, providing diverse training data for robotics.

Technical Principles of EdenLive

• State-Rendering Decoupling Architecture: EdenLive separates world state maintenance from visual generation—game rules, character positions, collision detection are explicitly tracked and updated by upstream interaction systems. The video model no longer performs implicit state inference, preventing error accumulation over time and focusing solely on rendering fidelity and real-time performance.
• Proxy-to-Fine Generation Paradigm: The model uses Proxy-simplified reference inputs (scene layout, camera motion, coarse dynamics) as conditioning, without requiring geometric precision in the source, enabling high-fidelity output generation.
• Multimodal Conditional Control: Generation is driven by three input types: reference video stream provides structure and motion, target first frame defines overall visual style and appearance, and text prompts add stylistic details—collectively enabling flexible control over arbitrary references and arbitrary styles.
• Real-Time Streaming Generation: The model employs streaming inference instead of offline batch generation, continuously receiving dynamic reference inputs and outputting rendered frames in real time. Achieves over 24 FPS with sub-100ms end-to-end latency on a single RTX 5090 GPU, 5B parameters, 720P resolution, supporting minute-long continuous generation.

Follow WeChat and reply "open-source" to join the AI Open-Source Projects Community

How to Use EdenLive

• Acquire Code: Clone the open-source repository from GitHub at https://github.com/VAST-AI-Research/EdenLive.
• Download Weights: Fetch model weights from Hugging Face at https://huggingface.co/VAST-AI/EdenLive matching your desired resolution.
• Prepare Input One: Reference Visuals: Provide Proxy-simplified scenes or reference video streams, sourced from custom 2D/3D environments, game engine outputs, or screen recordings.
• Prepare Input Two: Target First Frame: Specify a single image defining the overall visual style and aesthetic.
• Prepare Input Three: Text Prompts: Write descriptive text specifying style and fine-grained details.
• Run Inference: Invoke the inference script—the EdenLive model will render the reference visuals into a styled, high-fidelity output or video stream in real time.
• Integrate with Interaction System: Continuously feed dynamic reference visuals from upstream engines into the model to achieve closed-loop rendering in real-time interactive scenarios like gaming.
• Experience End-to-End Demo: Use the open-sourced Game Prototype Turbo to generate a playable Proxy game via natural language, then connect it to EdenLive for real-time rendering—fully walking through the workflow.

Core Advantages of EdenLive

• High Real-Time Performance: Achieves continuous minute-long generation at 720P and over 24 FPS on a single RTX 5090, with end-to-end latency in the sub-100ms range.
• Stable Decoupled Architecture: World state is explicitly maintained by the interaction system; the video model focuses purely on rendering, avoiding error accumulation from implicit state inference.
• Low Entry Barrier: Proxy references require no precise geometry or specific rendering pipeline—coarse scenes suffice for production-grade output.
• Unrestricted Style Transfer: Same reference can generate photorealistic, anime, oil painting, or any other style—no need to re-create assets per style.
• Universal Reference Compatibility: Supports 2D animation, 3D scenes, real video, and more—all unified under a single generative rendering framework.

Project Links for EdenLive

• Official Website: https://vast-eden.ai/?p=edenlive
• GitHub Repository: https://github.com/VAST-AI-Research/EdenLive
• Hugging Face Model Hub: https://huggingface.co/VAST-AI/EdenLive
• Technical Paper: https://assets.vast-eden.ai/assets/edenlive/EdenLive-tech-report.pdf

Competitive Comparison with Similar Projects

Application Scenarios for EdenLive

• AI Game Development: Generate polished 3D visuals in real time via EdenLive, enabling indie developers to produce high-quality games at low cost.
• Pre-Production in Film & Animation: Directors sketch compositions, camera movements, and character paths using simple visuals, then instantly generate photorealistic, cartoon, or oil-painting versions to finalize artistic direction quickly.
• Real-Time Stylized Live Streaming/Video: Use live human video as reference to render performances into animated or other stylized visuals in real time—ideal for virtual influencers and short-form content creation.
• Embodied Intelligence Training Data Generation: Alter visual appearance while preserving robot motion trajectories and scene topology, rapidly generating diverse simulation datasets for robotic training.
• Interactive Education & Virtual Experiences: Render simplified 3D educational or exhibition scenes into high-quality visuals in real time, delivering immersive, interactive experiences accessible via browser or lightweight devices.

Source: AI Tools Kit