iSmartRecruit

Job Description

Data Engineer (AWS)

Job Description

A leading Fintech company is looking for a Data Engineer (AWS) work on its payment system to deliver the best payment experience for customers.

Main Responsibilities :

Design, build and maintain ETL/ELT data pipeline to process and ingest data from multiple sources.
Work with structured and semi/un-structured data.
Utilizing your skills in engineering best practices to solve complex data problems.
Advising on specific technologies and methodologies for utilising cloud resources to efficiently ingest and process data quickly.
Collaborating with teams to ensure that prospective data architecture plans maximize the value of data across the organization.
Document all data integration processes , workflows and technical & system specifications.
Ensure compliance with data governance policies , industry standards and regulatory requirements.

Tech Stack :

Python, Shell, Go, SQL, PL/SQL
Aurora, MySQL, TiDB, Redis, DynamoDB
AWS
GitHub, Terraform, Jenkins, Ansible, Flyway
Victoria Metrics, Grafana, Prometheus, Newrelic
Elasticsearch , Opensearch , Athena , Glue , EMR, Redshift, Lambda

Qualifications

More than 3 years of AWS experience
4+ years experience as a Data Engineer or in a similar role
Strong problem-solving skills and ability to troubleshoot complex data issues
Hands-on experience with Infra as Code (IaC - terraform)
Excellent communication and collaboration skills
Ability to work in a fast-paced, dynamic environment and manage multiple tasks simultaneously

Preferred Qualifications

Experience designing and building solutions utilising various Cloud services such as EC2 , S3 , EMR , Kinesis , Glue , Athena , Lambda , Redshift , etc.,
Design and implement data integration workflows and Ensuring data quality and integrity across systems.
Validate and cleanse structured and semi/un-structured data to maintain high data quality.
Monitor and optimize the performance of Elasticsearch / Opensearch clusters and Kafka clusters
. Experience in at least one primary language (e.g. Scala, Python, Java) and SQL (any variant).
Stay update on cloud security best practices and apply them to the organization's environments.

Nice to Have :

Design and implement Elastic search / Open search clusters to meet the organization's search and analytics requirements.
Experience in operating and managing search and analytics engines like Elastic search , Open search , Logstash and Vector.
Design, implement and manage Kafka-based data pipelines and messaging solutions to support critical business operations and enable real-time data processing.
Experience in distributed event streaming platform and data processing framework.
Familiarity with Jenkins, Ansible, Docker, Kubernetes.
Experience in operating distributed systems.

Skills

Ansible, Stash, Redis, Mysql, Ec2, Distributed Systems, Java, Elastic Search, Elasticsearch, Jenkins, Pl/sql, Scala, Compliance, Problem-solving Skill, Python, Workflow, Methodologies, Excel, Sql, Data Integration, Docker, Cloud Services

About Company

We are a subsidiary of a leading Japanese company Talisman Corporation that has been providing recruitment services for the last 10 years. It's a Staffing and Recruiting firm.

We are a team of global talent hunters and executive search experts. We excel at matching the best talent in the digital space with the most exciting companies in various industries across the APAC Region.

With our expertise and experience in finding highly skilled executives, we help you build a successful team for the results you seek.

Apply for this Job