Data Engineer II
Hyderabad, Telangana · On-site · 3+ yrs · Mid · Full-time
- sql
- python
- java
- scala
- nodejs
- hadoop
- hive
- spark
- emr
- redshift
Data Engineer II at Amazon ShipTech Analytics building ETL pipelines and data infrastructure for transportation business.
Apply on Amazon's site Ask for a referral Interview prep
Interview prep: questions this posting points to
- Walk me through how you would design an ETL pipeline on AWS Glue and Redshift that consolidates finance data from 11 countries into a single unified dataset tracking 500+ operational and cost metrics at daily granularity. How would you structure it so that a new country can be onboarded without rewriting the pipeline?
- It's the morning before a weekly business review and the finance team's cost-performance dashboard is showing numbers that don't reconcile between finance, stores, and operations datasets. Walk me through how you'd triage this.
- These teams are actively migrating legacy systems to AWS and consolidating data from 11 countries and 100+ finance users. Tell me about a time you had to manage a complex data migration while keeping the old system running. What went wrong, and how did you handle it?
Read the full job description
The Amazon ShipTech Analytics organization is looking for a smart and ambitious individual to support the development of analytical tools and data processing solutions for the Amazon transportation business. This role will work within a multi-functional BI and data engineering team with a focus on AMXL (Amazon Extra Large) and other rapidly growing delivery programs where Amazon is innovating to find new ways to deliver to our customers. The ideal candidate thrives working with large volumes of data, enjoys the challenge of highly complex technical contexts, and is passionate about data and analytics. The candidate is an expert within data modeling, ETL design and cloud/big-data technologies and passionately partners with the business to identify strategic opportunities where improvements in data infrastructure creates large-scale business impact.
The candidate should be a self-starter; comfortable with ambiguity, able to think big, and enjoy working in a fast-paced and global team. It’s a big ask, and we’re excited to talk to those up to the challenge! Key job responsibilities · Build end-to-end data pipelines to ingest and transform data from different types of data sources and systems; from traditional ETL pipelines to event data streams · Utilize data from disparate data sources to build meaningful datasets for analytics, reporting and ML use cases · Evaluate and implement various big-data technologies and solutions (e.g. Redshift, Hive/EMR, Spark, SNS, SQS, Kinesis) to optimize processing of extremely large datasets · Understand and analyze business processes, logical data models and relational database implementations · Write high performing and optimized SQL queries · Execute research projects, and generate practical results and recommendations · Design and implement automated data processing solutions and data quality controls
Basic qualifications
- Bachelor's degree - 3+ years of data engineering experience - 3+ years of SQL experience - Experience with data modeling, warehousing and building ETL pipelines - Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS - Experience with big data technologies such as: Hadoop, Hive, Spark, EMR - Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases)
Preferred qualifications
- Master's degree in computer science, engineering, analytics, mathematics, statistics, IT or equivalent - 5+ years of data engineering experience - Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.
Members get roles like this as soon as we find them on the company's careers page, and paid plans email the ones that match their resume.
Get new roles first — free