Data Engineer — Training Data

About the Role

OpenAI is hiring a Data Engineer to build and maintain the data pipelines that power our model training. You will develop systems for data collection, cleaning, deduplication, and quality assessment at massive scale.

Requirements

• 5+ years of data engineering experience
• Strong skills in Python, SQL, and distributed data processing
• Experience with Apache Spark, Beam, or similar frameworks
• Knowledge of data quality and governance
• Experience with petabyte-scale data systems

Nice to Have

• Experience with training data for language models
• Background in web crawling and text extraction
• Familiarity with data deduplication techniques
• Experience with data labeling and annotation platforms

Benefits

Skills

Related Jobs