Azure Databricks Data Engineer (Remote)
No visa sponsorship is able to be provided
Talento has partnered with a Marketing organization in South Florida on a search for an Azure Data Engineer to play a critical role in building and maintaining reliable and scalable data pipelines. You will leverage your expertise in Python, SQL, and cloud technologies to extract, transform, and load data efficiently, enabling seamless data access and analysis for their users.
- Design, develop, and implement data pipelines utilizing the Databricks platform on Microsoft Azure.
- Utilize technologies like Spark, Delta Lake, and Unity Catalog for efficient data processing and governance.
- Implement the Medallion Architecture pattern for robust and modular data pipelines.
- Implement CI/CD pipelines for automated testing and deployment of data pipelines.
- Configure and manage Azure Data Lake Storage for secure and scalable data storage.
- Collaborate with cross-functional teams, including data analysts, data scientists, and software engineers.
- Monitor and optimize data pipelines for performance and reliability.
- Validate data quality and integrity using Great Expectations.
- Apply agile methodologies (Scrum and/or Kanban) for efficient project management and collaboration.
- Stay up-to-date with the latest advancements in Databricks, Azure, and big data technologies.
- 3+ years of experience as a Data Engineer or similar role.
- Fluent in Python and SQL.
- Experience with Spark and distributed data processing frameworks.
- Familiarity with the Medallion Architecture pattern.
- Knowledge of Unity Catalog and data governance principles.
- Experience with CI/CD pipelines and tools.
- Familiarity with Great Expectations for data validation.
- Familiarity with agile methodologies (Scrum and/or Kanban).
- Experience with cloud technologies, preferably Azure Data Lake Storage.
- Strong problem-solving and analytical skills.
- Excellent communication and teamwork skills.
Talento Human Capital Management is an equal opportunity employer. People are at the center of what we do! Our organization continues to thrive through our ongoing commitment to building an inclusive and diverse workforce from different backgrounds, perspectives, and skills is how.
People + Passion + Perseverance = Progress.
Talento Human Capital Management provides talent and organizational solutions enabling businesses to evolve beyond tactical human resources management. Our footprint spans across the Americas, Europe and Asia.