Member of Technical Staff - Pretraining

BlackForestLabs, the innovative force behind groundbreaking technologies like Latent Diffusion, Stable Diffusion, and FLUX, is seeking a talented Member of Technical Staff to join their remote team. These foundational models have revolutionised visual content creation, empowering millions of creators, developers, and businesses globally. FLUX, in particular, stands as one of the most advanced generative systems available today. While headquartered in Freiburg, Germany, the company boasts a rapidly expanding international presence and is committed to maintaining its core values of research excellence, open science, and fostering human creativity.

About the Role

This is a critical role focused on the large-scale pretraining of multimodal foundation models. You will be instrumental in shaping the future of image, video, and audio generation by leading significant experimental undertakings. This involves delving deep into architectural design, devising novel objective functions, and formulating effective scaling strategies for these complex models. The position offers a unique opportunity to contribute across the entire technology stack, from optimising low-level GPU and system performance to designing high-level model architectures and writing robust research code. You will have the autonomy to lead focused research projects independently while also driving larger, cross-team initiatives that push the boundaries of generative AI.

Your responsibilities will encompass a broad spectrum of advanced AI research and development. A significant portion of your work will involve leading and executing large-scale pretraining experiments for multimodal foundation models. This includes exploring and implementing new ideas related to model architectures, optimising training algorithms, and refining training methodologies. You will be expected to make significant architectural decisions, influencing aspects such as attention mechanisms, modulation schemes, loss formulations, and tokenisation strategies.

A key element of this role is contributing to the full development lifecycle, from the foundational systems level up to the abstract model design. This means you'll be comfortable diving into low-level GPU and systems optimisation to ensure maximum efficiency and performance. Simultaneously, you will be engaged in the design and implementation of research code, translating theoretical concepts into practical, functioning models. Furthermore, you will contribute to the high-level conceptualisation and design of these advanced models. You will also be responsible for the evaluation of these models, assessing their performance and identifying areas for improvement.

What They're Looking For

BlackForestLabs is searching for individuals with a proven track record in the field of foundation models. Ideally, you will have led or co-owned the pretraining process for a foundation model – be it in image, video, LLM, or multimodal domains – that has seen a production release or significant public unveiling. Your experience should demonstrate a strong capability to make crucial architectural decisions that directly impact model performance and direction. This includes a deep understanding of and experience with attention patterns, modulation schemes, loss formulations, and tokenisation strategies.

A strong command of large-scale distributed training is essential. You should be proficient with techniques like Fully Sharded Data Parallel (FSDP), Tensor Parallelism (TP), and Pipeline Parallelism (PP). Experience running multi-node training jobs with 500+ GPUs is highly desirable. The ability to effectively debug issues that arise at scale, such as loss spikes, NaNs, throughput regressions, and silent correctness issues, is paramount. Furthermore, a keen intuition for architecture and objective design is sought, evidenced by past instances where your insights into attention patterns, modulation schemes, or loss formulations have demonstrably improved real-world models.

A history of delivering impactful results is key, whether through publications in top-tier venues like NeurIPS, ICML, ICLR, CVPR, ICCV, or ECCV, coupled with demonstrable production impact, or through clear production wins at a leading AI research lab. Proficiency in Python and PyTorch is a prerequisite, and you should be comfortable reading and modifying low-level training code. Familiarity with visual generative models is also a must.

In South Africa, roles involving advanced AI research and development, particularly in areas like generative models and large-scale pretraining, are often found within established tech companies, specialised AI research labs, and large enterprise IT departments. While specific remuneration varies widely based on experience, seniority, and the specific demands of the project, positions requiring deep expertise in distributed systems and cutting-edge machine learning can command competitive salaries. Career progression often involves moving into more senior research scientist roles, technical lead positions, or specialised engineering paths focused on scaling and optimisation.

• You've led or co-owned pretraining for a foundation model (image, video, LLM, or multimodal) that shipped to production or a major release
• Own architectural calls that move the model: attention patterns, modulation schemes, loss formulations, tokenization strategies
• Deep experience with large-scale distributed training: FSDP/TP/PP, multi-node runs at 500+ GPUs, debugging loss spikes, NaNs, throughput regressions, and silent correctness issues at scale
• Strong intuition for architecture and objective design — you've made calls on attention patterns, modulation schemes, or loss formulations that moved a real model
• Track record of shipping: top-venue publications (NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV) paired with production impact, or unambiguous production wins at a frontier lab
• Deep Python and PyTorch proficiency; comfortable reading and modifying low-level training code
• Familiarity with visual generative models is a must