Data Platforms Engineer
Job Description
We are seeking a "Data Platforms Engineer" for our client, a top service provider renowned for trading and refurbishing used technology, such as cellphones and laptops. This company is dedicated to assisting manufacturers with sustainable practices and plays a significant role in reducing new tech expenses on a global scale.
In this role, you'll design and build the data pipelines and platforms that turn raw data from multiple sources into reliable, usable information. You'll work mainly in AWS, building and maintaining ETL pipelines, data lakes and warehouses, and a fit-for-purpose reporting layer the business and its partners can depend on. You'll also drive data quality and help make data accessible across the organisation.
What you'll do
Build and maintain ETL pipelines in AWS, feeding data from core transactional systems into a reliable reporting layer
Design, build and maintain large-scale data solutions using AWS analytics services (EMR, Glue, Redshift, Kinesis, Lambda, DynamoDB)
Build and maintain data lakes, warehouses and data marts
Improve data quality across core reporting systems
Apply data modelling to design models across multiple subject areas
Deliver automated, reliable and secure data services
Set up monitoring and alerting for data environments
Create and maintain clear documentation
What we're looking for
Experience designing and deploying production data pipelines in AWS, from ingestion to consumption, within a big data architecture
Real-time data experience with Kafka or Apache Spark
Strong SQL, plus scripting and extracting data via APIs
Hands-on with large-scale databases and storage (Redshift, Hive, Parquet, Hadoop)
Data modelling and building data marts
Data cleansing, wrangling, visualisation and reporting
Experience delivering automated, reliable and secure services, with monitoring and alerting
Comfortable working in Agile teams (Scrum/Kanban) across time zones, using Jira and Confluence
Nice to have
Programming in SQL plus one of Python, R or Scala
AWS certifications
Dashboard visualisation
Open source contributions
Experience in a DevOps culture
What you'll do
- Build and maintain ETL pipelines in AWS, feeding data from core transactional systems into a reliable reporting layer
- Design, build and maintain large-scale data solutions using AWS analytics services (EMR, Glue, Redshift, Kinesis, Lambda, DynamoDB)
- Build and maintain data lakes, warehouses and data marts
- Improve data quality across core reporting systems
- Apply data modelling to design models across multiple subject areas
- Deliver automated, reliable and secure data services
- Set up monitoring and alerting for data environments
- Create and maintain clear documentation
What we're looking for
- Experience designing and deploying production data pipelines in AWS, from ingestion to consumption, within a big data architecture
- Real-time data experience with Kafka or Apache Spark
- Strong SQL, plus scripting and extracting data via APIs
- Hands-on with large-scale databases and storage (Redshift, Hive, Parquet, Hadoop)
- Data modelling and building data marts
- Data cleansing, wrangling, visualisation and reporting
- Experience delivering automated, reliable and secure services, with monitoring and alerting
- Comfortable working in Agile teams (Scrum/Kanban) across time zones, using Jira and Confluence
Nice to have
- Programming in SQL plus one of Python, R or Scala
- AWS certifications
- Dashboard visualisation
- Open source contributions
- Experience in a DevOps culture