Senior (Big Data) Software Engineer | Intempt Technologies
As a technical leader of the platform team at Intempt, you will lead a team of engineers to create enterprise-grade big data pipelines that serve as a foundational layer to power the customer segmentation, analytics, and journeying systems for all of Intempt’s data pipeline and processing. You will constantly seek to improve the platform - availability, performance, data quality, usability, testability, and developer productivity - of not just our data pipelines but for the entire company through frameworks, tools, and best practices.
Through the above challenges, you will be encouraged to lead by example, envision and drive large cross-organizational technical projects, build a culture of technical excellence within the team, experiment with the latest technologies, and explore their limits.
- Model Intempt’s product features and architect data flows into a set of microservices;
- Actively improve the level of craftsmanship at Intempt by developing best practices and defining best strategies. Make it easy for others to create enterprise-grade data pipelines through open source tooling and frameworks;
- Create a culture of strong ownership of data flows via producer/consumer data contracts around availability, freshness, data quality, supportability, and schema changes;
- Deliver impact by driving innovation while building and shipping software at scale;
- Provide architectural guidance and mentorship to up-level the engineering organization;
- Design products/services/tools and code that can be used by others while upholding the operational impact of all decisions;
- Identify problems and opportunities and lead teams to architect, design, implement and operationalize systems;
- Passion for mentoring and developing other engineers is a must;
- Able to communicate effectively with engineers and senior executives;
- Lead or helped to drive data governance through frameworks, processes, and tooling.
- BA/BS or Masters Degree in Computer Science or related technical discipline, or equivalent practical experience;
- Strong understanding of Object-Oriented Analysis and Design, Algorithms, and Data Structures;
- 5+ years in building and maintaining ETL pipelines using Bigdata technologies like Spark Streaming, Spark SQL, MapReduce, Kafka, and Hive/Impala;
- Professional experience defining, developing, and maintaining scalable web service API;
- Expert skills working with SQL queries, including performance tuning, utilizing indexes/partitions, and materialized views to improve query performance;
- Exceptional in database design & development, writing optimized SQL friendly data structures;
- 2+ years of experience in an architect or leadership position;
- Experience with large-scale distributed and multi-tenant systems.
- 5+ years experience in software design, development, and data related solutions/architecture;
- 5+ years of programming experience in an object-oriented programming language such as Java, C/C++ (our microservices are written in Java with some Python and Scala);
- 5+ years of experience working with Big Data systems (Flink, Spark, Kafka, Cassandra, Clickhouse) at least multi-PB scale;
- Experience with big data technologies such as Spark and Flink;
- Experience with machine learning and AI;
- Experience in low latency, high-performance, real-time data processing;
- Build the next-generation libraries, APIs, and data pipelines;
- Develop data engineering components and applications and entities to empower self-serve for big data products;
- Lead complex and technically challenging projects from concept to completion.