Job Description

Our US client is on the lookout for a talented Senior Data Engineer (Python) to join their growing team in Sri Lanka.

Responsibilities:

• Build and tune a high performance data platform focusing on data normalization, transformation, and aggregation.
• Transform large datasets using open-source technologies (e.g. Polars).
• Optimize intermediate and final data storage formats (e.g. Parquet) and systems (Blob, different FS, etc).
• Orchestrate data pipelines using frameworks like Temporal.io (similar to Airflow).
• Build, test and deploy solutions on cloud-native environments.
• Integrate with various clouds, Saas provider, APIs, and storage systems.

Requirements:

• Demonstrated expertise in building and maintaining enterprise-scale data platforms.
• 3+ years of experience in Python and cloud-native software development.
• 3+ years of experience with transforming large datasets using frameworks like Pandas and Polars.
• Expertise in performance tuning distributed aggregations, transformations and storage techniques.
• Experience in orchestrating data pipelines using frameworks like Temporal.io, Airflow, etc.
• Experience integrating with various clouds, Saas provider APIs, and storage and data systems.
• Experience with big data technologies such as Hadoop, Hive, Spark, EMR.
• Experience in general backend development including REST APIs.
• Familiarirty with DevOps practices - Kubernetes, Docker, CI/CD, logging / metrics, cloud-native development, operations, etc.
• AI/ML experience with multivariate forecasting and anomaly detection based on factors from disparate data sources (added advantage).
• Experience with GenAI, including training and deploying commercial and open-source models (added advantage).
• Familiarity with FinOps and financial data platforms (added advantage).
• Excellent spoken and written English skills to work together with the onsite team in the US.

Interested candidates can apply online or email your CV to [email protected]