Target profile
You will love this job if:
You are pioneering and innovative and want to be part of the cutting-edge and disruptive crypto-currency world
You are eager to learn new knowledge in both financial and technical fields
You thrive in a non-hierarchical organization with a casual working environment
You enjoy solving complex distributed systems challenges and optimizing streaming data pipelines
You value comprehensive documentation and collaborative problem-solving
Team / Role
As a Data Engineer you will:
Build real-time data pipelines using Apache Flink (PyFlink) to process high-volume logs through EC2 → Vector → MSK → PyFlink → S3 → ClickHouse architecture
Design stream processing systems with watermark strategies, window operations, exactly-once semantics, and state management for critical data
Deploy and manage AWS infrastructure including Managed Flink, S3 data lakes, MSK, IAM, and CloudWatch monitoring with optimized partitioning strategies
Optimize performance and troubleshoot SQL queries (Flink/ClickHouse), production issues, data skew, and build Grafana dashboards for pipeline monitoring
Implement data quality frameworks with schema evolution, validation strategies, and translate business requirements into scalable data solutions
Manage DevOps processes including JAR dependencies, Docker containerization, Kubernetes deployments, and comprehensive documentation
Required Skillset
University degree in Computer Science, Software Engineering or related disciplines
Apache Flink/PyFlink with watermarks, state management, and window operations
Python (pandas, polars, boto3) and SQL (Flink SQL, ClickHouse)
Kafka/AWS MSK and message streaming concepts
AWS services: Managed Flink, S3, MSK, CloudWatch, IAM
Docker, Kubernetes, and containerization fundamentals
Grafana monitoring and production troubleshooting experience
Nice to Have:
Airflow, Terraform, Maven/Gradle
ClickHouse table engines and optimization
CI/CD pipelines and testing frameworks
We encourage applicants to read through our Privacy Notice for Applicants before submitting your applications - https://pulsar.com/careers/