xAI· Model· Palo Alto, CA; Seattle, WA
Member of Technical Staff - Imagine Model
Comp$180,000 – $440,000
Classified Tasks (28)
Automate 11%Augment 79%Human-Only 11%
Automate (3)
Fully handled by AI agents
Filter datasets to remove low-quality or irrelevant multimodal examples
operational
Augment image, video, and audio data using deterministic and synthetic augmentation techniques
technical
Produce captions and other metadata for visual and audio assets to enhance supervision
operational
Augment (22)
AI assists, human decides
Design and implement systems for controllable and long-horizon visual synthesis
technical
Design and implement agentic planning and RL training pipelines for multimodal systems
technical
Develop and prototype world simulation models that integrate visual and audio modalities
technical
Curate multimodal datasets through dataset selection, organization, and versioning
operational
Annotate visual and audio data to improve dataset quality and downstream model performance
operational
Synthetically generate training data to expand coverage for visual and audio tasks
technical
Conduct in-depth data studies and analyses to identify dataset gaps and bias issues
analytical
Design evaluation frameworks, metrics, and benchmarks for image, video, and audio quality and coherence
analytical
Develop and implement automatic evals and human-in-the-loop evaluation processes for multimodal outputs
technical
Design and train reward models tailored to image/video/audio tasks and coherence
technical
Research and implement efficient algorithms to improve model performance for multimodal tasks
technical
Optimize models and inference pipelines for real-time visual (and audiovisual) inference
technical
Implement model distillation workflows to produce smaller, faster multimodal models
technical
Build and operate scalable inference serving systems for visual and audiovisual content
operational
Develop scalable data collection pipelines for large-scale image and video datasets
operational
Build scalable data processing pipelines to preprocess, transform, and ingest multimodal datasets
technical
Design and implement model architectures for high-fidelity image, video, and audiovisual generation and understanding
technical
Pretrain and fine-tune multimodal models across pretraining and post-training phases
technical
Debug model failure modes through systematic experiments, analysis, and iterative fixes
analytical
Integrate multimodal AI solutions into production systems and end-user products
operational
Rapidly iterate on models and product integrations based on user feedback and evaluation results
operational
Implement tooling and infrastructure to monitor model performance and data quality in production
operational
Human-Only (3)
Requires human judgment
Create engineering roadmaps and agendas to advance multimodal capabilities in image, video, and audio domains
leadership
Drive execution of engineering agendas focused on image/video generation, editing, and understanding
leadership
Collaborate cross-functionally with product and engineering teams to prioritize and deliver features
communication
Job description
ABOUT xAI xAI’s mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. Our team is small, highly motivated, and focused on engineering excellence. This organization is for individuals who appreciate challenging themselves and thrive on curiosity. We operate with a flat organizational structure. All employees are expected to be hands-on and to contribute directly to the company’s mission. Leadership is given to those who show initiative and consistently deliver excellence. Work ethic and strong prioritization skills are important. All employees are expected to have strong communication skills. They should be able to concisely and accurately share knowledge with their teammates. ABOUT THE ROLE: As a multimodal engineer on the Imagine Model Team, you will develop cutting-edge AI experiences beyond text, with a strong focus on enabling high-fidelity understanding and generation across image and video modalities, while also incorporating audio where it enhances visual content (e.g., synchronized audio for video). Responsibilities span data curation, modeling, training, inference serving, and product integration, covering both pretraining and post-training phases. You will collaborate closely with product teams to push model frontiers and deliver exceptional end-to-end user experiences. RESPONSIBILITIES: Create and drive engineering agendas to advance multimodal capabilities, with emphasis on image and video generation, editing, understanding, controllable/long-horizon synthesis, agentic planning, RL training, and world simulation (including audio integration for richer video experiences). Improve data quality through annotation, filtering, augmentation, synthetic generation, captioning, and in-depth data studies, particularly for visual and audio data. Design evaluation frameworks, metrics, benchmarks, evals, and reward models tailored to image/video/audio quality and coherence. Implement efficient algorithms for state-of-the-art model performance, including real-time inference, distillation, and scalable serving for visual content. Develop scalable data collection and processing pipelines for multimodal (primarily image/video-focused) datasets. Collaborate cross-functionally to integrate AI solutions into production and rapidly iterate based on user feedback. BASIC QUALIFICATIONS: Track record in leading studies that significantly improve neural network capabilities and performance through better data or modeling. Experience in data-driven experiment designs, systematic analysis, and iterative model debugging. Experience developing or working with large-scale distributed machine learning systems. Ability to deliver optimal end-to-end user experiences. Hands-on contributor with initiative, excellence, strong work ethic, prioritization skills, and excellent communication. PREFERRED SKILLS AND EXPERIENCE: Experience in SFT, RL, evals, human/synthetic data collection, or agentic systems. Proficiency in Python, JAX/XLA, PyTorch, Rust/C++, Spark, Ray, and related large-scale frameworks. Domain expertise in multimodal applications such as graphics engines, rendering techniques, image/video understanding and generation, world models, real-time simulation, or controllable/long-horizon visual content creation (audio/speech processing or music/audio generation experience is a plus where it supports video)