Data Quality Engineer iTunes BI
Santa Clara Valley, CA
Join the Global Business Intelligence team at Apple. We process and integrate large volumes of business data from 100s of enterprise systems to provide reporting, analytics, insight, data science solutions for our business and external partners.
We have an immediate opening for a Data Quality Engineer in our Internet Services team. We process iTunes app, music, movie and book, iCloud and search ad platform data. We have an extensive data pipeline with batch and real-time processing in various technologies to support our solutions, analysts and data scientists. Our technologies include traditional relational and MPP systems like Oracle, Vertica and Teradata and BigData technologies like Hadoop Hive, Spark, Druid. Data quality and integrity across these systems is critical for our applications. You will be responsible to define data pipeline quality standards, evaluate and engineer tools to monitor data accuracy. Application teams will use your tools and provide feedback of their effectiveness. The role includes working with application teams and data quality product managers to understand data quality requirements.
Hands on experience with data quality tools across data pipeline processing in relational and BigData systems
Development experience to build data quality tools
Technology knowledge of Hadoop HDFS, Hive, Spark, Vertica
Successful track record of building and deploying data quality tools across a data pipeline systems
Great understanding of accurate data definition, type of data quality checks, data variances, predictive data patterns, duplicate data detection
Knowledge of different Data Science models and its implementation details to detect anomaly in a data pipeline. Also train the model to prevent false positives
Proactively initiates, develops, and maintains effective working relationships with team members
Strong written, verbal, and presentation skills
Demonstrated experience of collaboration in fast paced environment with minimal guidance. Proactive, results-oriented attitude
Ability to generate options and create a positive working experience in fairly agile business situations
The position offers the opportunity to create data quality monitoring systems and for deployment of them by our application teams. You will have overall ownership and responsibility of the data quality architecture and the tools. You will directly contribute to the improvement of the data quality in our applications.
The data quality tools need to monitor quality of data across various layers of the data pipeline. The tools need to process large amounts of data with millions to billions of data rows in a day. The tools need to be efficient to detect bad data in a short time without major delays in the data pipeline. In addition, the tool should also use appropriate predictive models for anomaly detection in the data pipeline. Application teams will provide data quality requirements for the data quality monitoring tools.
You should have extensive experience with large data pipeline processing in BigData systems. Ideally you used and developed data quality tools in the Business Intelligence data warehousing area. Strong understanding of the BI industry, products, and vendors is important.
BS Degree in Computer Science, IT or equivalent, 8+ years of experience in Data Warehouse/Business Intelligence space
Apple revolutionized personal technology with the introduction of the Macintosh in 1984. Today, Apple leads the world in innovation with…