Lead Databricks Engineer/Architect

Job Description

As a Lead Data Engineer- Databricks Architect, you will make an impact by leading the end-to-end modernization of a large-scale enterprise data warehouse environment from Oracle Exadata to Azure Databricks Lakehouse. You will be a valued member of the Data Engineering team and work collaboratively with client stakeholders, Cognizant leadership, offshore delivery teams, and cross-functional data professionals to deliver a successful cloud transformation initiative.

In this role, you will:

• Lead the technical architecture and delivery of Oracle Exadata-to-Databricks migrations, ensuring data integrity, performance, scalability, and alignment with business requirements.
• Design and oversee Azure Databricks Lakehouse solutions, including Bronze, Silver, and Gold data layers, Delta Lake architecture, ADLS Gen2 integration, and enterprise orchestration frameworks.
• Drive the modernization of ETL workloads by converting Informatica PowerCenter processes into PySpark and Spark SQL solutions while validating and optimizing AI-assisted code generation outputs.
• Establish and govern enterprise data quality, reconciliation, security, lineage, and compliance controls through Unity Catalog and standardized validation frameworks.
• Serve as the primary technical advisor and escalation point for the client, leading issue resolution, production readiness activities, performance optimization efforts, and successful production cutovers.

Work model

We believe hybrid work is the way forward as we strive to provide flexibility wherever possible. Based on this role's business requirements, this is a hybrid position requiring a minimum of three days per week in a client office in New York, NY. Regardless of your working arrangement, we are here to support a healthy work-life balance through our various wellbeing programs.

The working arrangements for this role are accurate as of the date of posting. This may change based on the project you're engaged in, as well as business and client requirements. Rest assured; we will always be clear about role expectations.

What you need to have to be considered

• 15+ years of experience in enterprise data engineering, data architecture, or large-scale data platform modernization initiatives.
• Proven experience leading Oracle data warehouse migrations to Azure Databricks Lakehouse environments, including Oracle Exadata, Delta Lake, Databricks SQL, and cloud-native data architectures.
• Strong hands-on expertise with Oracle databases, PL/SQL, schema design, performance tuning, and enterprise-scale data migration strategies.
• Advanced experience developing and optimizing PySpark and Spark SQL solutions for complex data transformation and processing workloads.
• Demonstrated success leading client-facing technical engagements, managing escalations, and driving issue resolution throughout all phases of project delivery.
• Experience designing and implementing data governance, security, lineage, and access control frameworks using Unity Catalog or similar enterprise governance platforms.
• Ability to lead and coordinate distributed teams across onshore and offshore locations while collaborating effectively with client stakeholders and delivery leadership.
• Willingness to work onsite in New York, NY, a minimum of three days per week throughout the engagement.

These will help you stand out

• Experience with Databricks Lakebridge and automated migration acceleration frameworks.
• Experience with Lakeflow Connect, data ingestion modernization, and large-scale historical and incremental data migration initiatives.
• Knowledge of Informatica PowerCenter architecture, workflow migration, and ETL modernization strategies.
• Experience supporting healthcare, payer, provider, clinical, or operational healthcare data environments.
• Familiarity with AI-assisted development tools, automated code conversion frameworks, and human-in-the-loop engineering practices.
• Experience leading enterprise testing, parallel run activities, production cutovers, and legacy platform decommissioning programs.

We're excited to meet people who share our mission and can make an impact in a variety of ways. Don't hesitate to apply, even if you only meet the minimum requirements listed. Think about your transferable experiences and unique skills that make you stand out as someone who can bring new and exciting things to this role.

Please note, this role is not able to offer visa transfer or sponsorship now or in the future

Salary and Other Compensation:

Applications will be accepted until October 25, 2026

The annual salary for this position is between $ 155,000 - $ 175,000 depending on experience and other qualifications of the successful candidate.

This position is also eligible for Cognizant's discretionary annual incentive program, based on performance and subject to the terms of Cognizant's applicable plans.

Benefits: Cognizant offers the following benefits for this position, subject to applicable eligibility requirements:

• Medical/Dental/Vision/Life Insurance
• Paid holidays plus Paid Time Off
• 401(k) plan and contributions
• Long-term/Short-term Disability
• Paid Parental Leave
• Employee Stock Purchase Plan

Disclaimer: The salary, other compensation, and benefits information is accurate as of the date of this posting. Cognizant reserves the right to modify this information at any time, subject to applicable law.