xAI· Model· Palo Alto, CA
Member of Technical Staff - Multimodal Understanding
Comp$180,000 – $440,000
Classified Tasks (11)
Automate 0%Augment 64%Human-Only 36%
Augment (7)
AI assists, human decides
Design, build, and optimize large-scale distributed systems for multimodal pre-training, post-training, inference, data processing, and tokenization at web/petabyte scale
technical
Develop high-throughput pipelines for data acquisition, preprocessing, filtering, generation, decoding, loading, crawling, visualization, and management of images, videos, audio, and text
technical
Drive data quality by curating human and synthetic datasets, implementing filtering techniques, conducting data analyses, and building scalable pipelines to support trillion-parameter models
analytical
Create evaluation frameworks, internal benchmarks, reward models, and metrics that capture real-world usage, failure modes, interactive dynamics, and human-AI synergy
analytical
Build research tooling, user-facing interfaces, prototypes, demos, and full-stack applications to enable rapid iteration based on feedback
technical
Work across the training stack (pre-training → supervised fine-tuning/RL/post-training) to enable reasoning, tool calling, agentic behaviors, orchestration, and seamless real-time interactions
technical
Train and optimize tokenizers for multimodal data
technical
Human-Only (4)
Requires human judgment
Advance multimodal model capabilities including spatial-temporal compression, cross-modal alignment, world modeling, reasoning, emergent abilities, audio/image/video understanding and generation, real-time video processing, and noisy data handling
creative
Innovate on algorithms, modeling approaches, hardware/software/algorithm co-design, and scaling paradigms to improve state-of-the-art performance
creative
Collaborate cross-functionally with pre-training, post-training, reasoning, data, applied, and product teams to deliver frontier multimodal capabilities
communication
Contribute to building models that perceive (see, hear), reason about, and interact with the world in real time
technical
Job description
ABOUT xAI xAI’s mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. Our team is small, highly motivated, and focused on engineering excellence. This organization is for individuals who appreciate challenging themselves and thrive on curiosity. We operate with a flat organizational structure. All employees are expected to be hands-on and to contribute directly to the company’s mission. Leadership is given to those who show initiative and consistently deliver excellence. Work ethic and strong prioritization skills are important. All employees are expected to have strong communication skills. They should be able to concisely and accurately share knowledge with their teammates. ABOUT THE ROLE: You will join the multimodal team to push toward superhuman multimodal intelligence. Advance understanding and generation across modalities—image, video, audio, and text—spanning the full stack: data curation/acquisition, tokenizer training, large-scale pre-training, post-training/alignment, infrastructure/scaling, evaluation, tooling/demos, and end-to-end product experiences. Collaborate cross-functionally with pre-training, post-training, reasoning, data, applied, and product teams to deliver frontier capabilities in multimodal reasoning, world modeling, tool use, agentic behaviors, and interactive human-AI collaboration. Contribute to building models that can see, hear, reason about, and interact with the world in real time at unprecedented levels. RESPONSIBILITIES: Design, build, and optimize large-scale distributed systems for multimodal pre-training, post-training, inference, data processing, and tokenization at web/petabyte scale. Develop high-throughput pipelines for data acquisition, preprocessing, filtering, generation, decoding, loading, crawling, visualization, and management (images, videos, audio + text). Advance multimodal capabilities including spatial-temporal compression, cross-modal alignment, world modeling, reasoning, emergent abilities, audio/image/video understanding & generation, real-time video processing, and noisy data handling. Drive data quality and studies: curation (human/synthetic), filtering techniques, analysis, and scalable pipelines to support trillion-parameter models. Create evaluation frameworks, internal benchmarks, reward models, and metrics that capture real-world usage, failure modes, interactive dynamics, and human-AI synergy. Innovate on algorithms, modeling approaches, hardware/software/algorithm co-design, and scaling paradigms for state-of-the-art performance. Build research tooling, user-friendly interfaces, prototypes/demos, full-stack applications, and enable rapid iteration based on feedback. Work across the stack (pre-training → SFT/RL/post-training) to enable reasoning, tool calling, agentic behaviors, orchestration, and seamless real-time interactions. BASIC QUALIFICATIONS: Hands-on experience with multimodal pre-training, post-training, or fine-tuning (vision, audio, video, or cross-modal). Expert-level proficiency in Python (core language), with strong experience in at least one of: JAX / PyTorch / XLA. Proven track record building or optimizing large-scale distributed ML systems (training/inference optimization, GPU utilization, multi-GPU/TPU setups, hardware co-design). Deep experience designing and running data pipelines at s