ABOUT US: At Nexus Cognitive, we are at the forefront of technological innovation, driving transformative solutions in the world of data and analytics. Our mission is to enable your business to harness the power of big data, transforming complex data sets into actionable insights that disrupt industries and deliver a competitive advantage for our clients. We are a trusted advisor that handles the heavy lifting of big data management and governance, so our clients can focus on making informed, strategic decisions that drive growth and innovation. Nexus Cognitive firmly grasps the transformative power of data and artificial intelligence.  We have a spirited team of employees across the U.S. dedicated to redefining the boundaries of technology and cognitive solutions.  

THE OPPORTUNITY: As we continue our journey, we are building a team of sharp, intelligent innovators who can move fast with great agility and passion. We are seeking an exceptional Principal Platform Engineer to lead client engagements and architect modern lakehouse solutions using open source technologies. This client-facing role combines hands-on technical expertise with project leadership to deliver enterprise-scale data platforms built on Apache Spark, Trino, Apache Iceberg, and cloud-native orchestration tools.

WHAT YOU’LL DO:   

Client Project Leadership & Delivery (35%)

- Lead technical delivery of lakehouse implementations for enterprise clients using open source stack

- Architect and implement production-grade data platforms leveraging Apache Spark, Trino, and Apache Iceberg

- Design and deploy orchestration solutions using Apache Airflow on Kubernetes

- Implement security and governance using Apache Ranger and related open source tools

- Serve as primary technical point of contact for client stakeholders

- Conduct architecture reviews and provide technical guidance throughout project lifecycle

- Manage technical scope, timelines, and deliverables for multiple concurrent client engagements

- Build trusted advisor relationships with client data platform teams

Technical Architecture & Platform Engineering (40%)

- Design scalable lakehouse architectures using Apache Iceberg table format

- Implement high-performance query engines with Trino and optimize for diverse workloads

- Deploy and manage Kyuubi for enhanced Spark SQL gateway capabilities

- Build robust data pipelines with Apache Spark for both batch and streaming use cases

- Orchestrate complex workflows using Airflow with Kubernetes executors

- Implement fine-grained access control and data governance with Apache Ranger

- Establish CI/CD pipelines and infrastructure as code for platform components

Team Mentorship & Knowledge Transfer (15%)

- Mentor junior engineers on open source data platform technologies

- Conduct technical workshops on Spark optimization, Trino query tuning, and Iceberg best practices

- Create reusable frameworks and accelerators for common lakehouse patterns

- Lead code reviews focusing on performance, reliability, and maintainability

- Develop technical documentation and runbooks for platform operations

- Transfer knowledge to client teams ensuring successful platform adoption

Open Source Contribution & Community Engagement (10%)

- Contribute bug fixes and features to core projects (Spark, Trino, Iceberg, Airflow, Kyuubi, Ranger)

- Participate in community discussions and technical forums

- Share learnings through blog posts and conference talks

- Evaluate emerging open source technologies for client solutions

- Build relationships within the open source data engineering community

Technical Expertise

- 8+ years of data platform engineering experience with 5+ years in client-facing roles

- Production experience with Apache Spark (performance tuning, custom applications, structured streaming)

- Hands-on expertise with Trino/Presto for interactive analytics at scale

- Deep knowledge of Apache Iceberg including table maintenance, partitioning strategies, and time travel

- Proficiency with Kubernetes for deploying and managing data platform components

- Experience with Apache Airflow for complex workflow orchestration

- Implementation experience with Apache Ranger for security and governance

- Strong programming skills in Python, Scala/Java, and SQL

Platform & Infrastructure Skills

- Cloud platform expertise (AWS, Azure, GCP) with focus on data services

- Infrastructure as code using Terraform, Helm charts for Kubernetes deployments

- Container orchestration and microservices architecture

- Performance optimization for large-scale data processing

- Data security, encryption, and compliance implementation

Client Engagement & Leadership

- Proven track record leading technical delivery for enterprise clients

- Experience presenting technical solutions to C-level executives

- Ability to translate business requirements into technical architectures

- Strong project management skills with Agile delivery experience

- Excellent written and verbal communication skills

- Experience managing distributed teams and remote client engagements

Preferred Qualifications

- Contributions to Apache Spark, Trino, Iceberg, or related projects

- Experience with Kyuubi for multi-tenant Spark deployments

- Knowledge of additional open source tools (Delta Lake, Hudi, Flink, Kafka)

- Certifications in cloud platforms or Kubernetes (CKA/CKAD)

- Experience building data mesh or federated query architectures

- Background in financial services, healthcare, or retail data platforms

What We Offer

- Lead cutting-edge lakehouse implementations for Fortune 500 clients

- Work with the latest open source data technologies

- Competitive compensation with performance-based bonuses

- Professional development budget for training and conferences

- Flexible work arrangements with travel up to 25%

- Collaborative environment with deep technical expertise

- Opportunity to shape industry best practices

Impact & Growth

As a Principal Platform Engineer, you'll directly impact how large enterprises modernize their data infrastructure using open source technologies. You'll have the opportunity to solve complex technical challenges, lead transformational projects, and establish yourself as a thought leader in the modern data platform space while working with cutting-edge open source tools at scale.