
FLUX.3 Image Klein in Diffusers: One Model for Generation and Layout Control

Black Forest Labs‘ FLUX.3 Image Klein is now available in Diffusers from day one, with the two open checkpoints hosted in the FLUX.3 Image Klein collection. The same model covers text-to-image, multi-reference image-to-image, and a new layout-control mode: you write bounding boxes into the prompt to specify where each part of the image should go, or which parts of a reference image to move. The distilled checkpoint produces a 1024×1024 image in 16 steps and about 2 seconds on an H100, while NVFP4 and FP8 weights can be loaded via from_single_file for smaller GPUs. The release also includes DreamBooth LoRA training scripts for text-to-image and image editing, and ships as a Modular Diffusers pipeline so blocks can be replaced or added.
Getting started is intentionally short: install the latest diffusers and transformers (needed for the Qwen3-VL text encoder), load ModularPipeline.from_pretrained(‘black-forest-labs/flux-3-image-klein’), and call the pipeline with a prompt. For text-to-image, the official guidance recommends full scene descriptions covering subject, setting, light, composition, and style. Image-to-image accepts one or more reference images; references are addressed in the prompt as , , … in the order passed. Layout control appends a JSON list of regions, each with id, bbox, and desc, where boxes are [y_min, x_min, y_max, x_max] on a normalized 0-1000 scale and each region’s desc is drawn inside its bbox. For editing an existing image, regions can carry src_bbox and tgt_bbox to move content from one place in the reference to another in the output; in the knight example, only the knight moves while backdrop, cliff, needle, dragon, and fire are pinned.
The base checkpoint runs 50 steps with classifier-free guidance and is the recommended base for fine-tuning. The transformer is offered as single-file bf16, fp8r (rowwise FP8), and nvfp4 checkpoints; loading with from_single_file lets you avoid downloading the bf16 transformer. Quantization differences matter: fp8r fast kernels require compute capability 8.9+ (Ada, Hopper, Blackwell) and only reduce memory elsewhere, while nvfp4 targets Blackwell with matching flashinfer-python and flashinfer-cubin versions. Memory measurements list 13.4 GiB peak after load for nvfp4 on an RTX 6000, 14.35 GiB for fp8r on an H100, and 18.80 GiB for bf16 on an H100; after inference these rise to 15.3, 16.27, and 20.72 GiB. Distilled end-to-end timings on one H100 80GB are 1.93 s for text-to-image, 3.87 s for image-to-image with one 1024×1024 reference, and 4.10 s for the layout-editing example; a reference roughly doubles time because its tokens join the same sequence. For lower memory, the article suggests quantized transformers, running the text encoder once and freeing it, or automatic CPU offloading, which was not benchmarked.
The architecture section describes FLUX.3 as a latent-space flow model. A VAE with the same architecture as Flux.2 compresses images 8x in each direction into 32 channels, then every 2×2 patch is packed into one token with 128 channels, so a 1024×1024 image becomes a 64×64 grid of 4,096 tokens. Text encoding uses Qwen3-VL: the model reads hidden states from eight of its 36 layers (layers 4, 8, …, 32), stacks them into a 20,480-dimensional vector per token, and projects to width 3072. The transformer runs in two stages: five per-mode blocks refine the text, generated-image, and reference streams independently, then 28 joint blocks concatenate [text | image | reference] and let all tokens attend to each other. FLUX.2’s double-stream blocks let text and image attend from the start, while FLUX.3 keeps streams fully separate before merging. Each block runs attention and a SwiGLU MLP in parallel on a modulated norm; queries and keys are RMS-normalized and get RoPE. Modulation depends on token type: generated-image tokens use the current timestep, text and reference tokens use timestep 0, and the distilled checkpoint also receives the destination timestep of each jump. RoPE uses four position axes (t, h, w, l); image and reference tokens share the same (h, w) coordinates, which gives src_bbox/tgt_bbox editing a precise spatial meaning. The denoising loop is flow matching with 50 steps and a time shift of 6.93, so 44 of the 50 steps run above sigma = 0.5.
FLUX.3 is implemented as a Modular Diffusers pipeline. Workflows can be listed and inspected without loading weights; for example, text2image and image_conditioned. Blocks can be removed—the post shows popping out the text encoder, encoding once, and generating many images from cached embeddings—or inserted, as with a custom preview block that records x0 guesses at each step to visualize how the image forms. The post closes with links to the model collection, Diffusers docs, DreamBooth LoRA scripts, and the Black Forest Labs prompting guide.


