Sr. AI & Data Engineer

MAKE STRATEGY A REALITY | ACCELERATE YOUR GROWTH | CHOOSE YOUR PATH

As the world's leading change and transformation consultancy, we're helping businesses move from strategy to reality by taking a pragmatic and practical approach to build solutions that last.

We're seeking a Sr. AI & Data Engineer in Quito to help us take vision to value and create lasting impact.

SUMMARY

We are seeking a highly skilled Senior Data Engineer to join our growing data and analytics team. This role will be instrumental in designing, building, and maintaining scalable cloud-based data solutions on Databricks, enabling advanced analytics, reporting, and data-driven decision-making across the organization. 

The ideal candidate will bring strong expertise in Databricks, Python, and PySpark, along with a deep understanding of modern data architectures, ETL/ELT frameworks, and big data processing. This person will collaborate closely with cross-functional teams to deliver reliable, efficient, and secure data pipelines and platforms. 

YOU WILL:
 

• Design, build, and optimize scalable data solutions including data pipelines and data integration workflows in Databricks. 

• Develop and maintain robust ETL/ELT processes using Python and PySpark to ingest, transform, and load large volumes of structured and unstructured data. 

• Build and manage Lakehouse, and data warehouse solutions on Databricks, ensuring high performance, scalability, and reliability. 

• Work with large-scale distributed data processing frameworks to support batch and near real-time data processing needs. 

• Collaborate with data architects, analysts, and business stakeholders to gather requirements and translate them into scalable technical solutions. 

• Optimize data models, partitioning strategies, and query performance for analytics and reporting use cases. 

• Implement data quality checks, monitoring, and alerting processes to ensure data integrity and operational excellence. 

• Troubleshoot and resolve issues across data pipelines, cloud infrastructure, and processing jobs. 

• Contribute to best practices in data engineering, including code quality, documentation, testing, and governance. 

• Partner with DevOps and engineering teams to support CI/CD pipelines, infrastructure automation, and deployment of data solutions. 

• Ensure data solutions follow security, compliance, and access control standards across AWS environments. 

• Support data modernization and cloud transformation initiatives by identifying opportunities to improve existing architectures and workflows. 

IDEALLY, WE'D LIKE:

• 5+ years of experience in data engineering, building and maintaining enterprise data pipelines and cloud-based data platforms. 

• Experience with Databricks, Delta Lake, or other modern big data platforms. 

• Advanced proficiency in Python for data engineering and backend data processing. 

• Strong experience with PySpark for large-scale data transformation and distributed processing. 

• Solid understanding of data lake, data warehouse, and modern data architecture principles. 

• Experience designing and optimizing ETL/ELT pipelines for performance, scalability, and maintainability. 

• Strong SQL skills and experience working with large datasets in cloud environments. 

• Familiarity with orchestration and scheduling tools such as Apache Airflow or AWS-native orchestration solutions. 

• Knowledge of data modeling, partitioning, file formats (e.g., Parquet, ORC, JSON, CSV), and query optimization techniques. 

• Understanding of cloud security, IAM roles, data encryption, and access management best practices. 

• Strong problem-solving, troubleshooting, and performance tuning skills. 

• Excellent communication skills and ability to collaborate effectively with technical and non-technical stakeholders. 

• Strong hands-on experience with cloud services, such as S3, AWS Glue, Lambda, Redshift, EMR, Athena, and related services. 

• Experience working in Agile or cross-functional product teams. 

Applicants must be authorized to work in Ecuador, without the need for visa sponsorship by North Highland. Work visa sponsorship will not be provided, either now or in the future, for this position.

North Highland is an equal opportunity employer, and we adhere to all applicable laws and regulations to ensure a fair and equitable workplace. All qualified applicants will receive fair and impartial consideration without regard to race, color, sex, gender identity, religion, national origin, age, sexual orientation, disability, veteran status, or any other characteristic protected by law. We handle all information in accordance local privacy standards and maintain strict confidentiality.