Senior data engineer

Job Description

A leading healthcare payor is actively seeking to hire a Senior Azure Data Engineer with deep Databricks expertise to help shape the future of data innovation. This is your chance to join a growing Enterprise Data Office where your work will directly impact patient outcomes, operational efficiency, and data-driven decision-making. Key responsibilities include:

• Design, develop and implement end-to-end data solutions using Azure Databricks.

• Convert legacy SQL workflows into modern Python-based pipelines.

• Maintain and optimize high-volume data pipelines in production.

• Write, test, and tune PySpark and SQL scripts for performance and scalability.

• Ensure data quality and integrity through robust validation and cleansing processes.

• Collaborate cross-functionally, bringing critical thinking and clear communication to a fast-paced, agile team.

PLEASE NOTE: We are looking for someone who values in person collaboration and is willing to come onsite twice a month in Newtown Square, PA 19073.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to [email protected] learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:

Skills and Requirements

7+ years of experience as a Data Engineer working in a heavy Azure environment 5+ years of experience building pipelines with Databricks

Expertise with Python for scripting, automation, data cleaning and transformation

5+ years using PySpark to implement ETL pipelines 2+ years of recent experience working in a healthcare environment 3+ years of experience working with different data types (JSON, HL7, XML, CSV, etc.)