Data Engineer

The Data Engineer is responsible for preparing data for analytical and operational purposes. This role involves designing and implementing data pipelines to consolidate information from diverse source systems, ensuring that data is available, accessible, and optimized for analysis. The ideal candidate will possess strong skills in SQL, and NoSQL (CosmosDB), and have a deep understanding of Azure services, including Azure Data Factory and JSON.

Responsibilities:

1. Data Integration: Develop and maintain data pipelines to consolidate data from various source systems into a unified and accessible format.
2. Data Transformation: Transform, clean, and enrich data to ensure its quality, consistency, and relevance for analytical and operational needs.
3. Database Management: Manage and optimize databases, including NoSQL databases like CosmosDB, to support efficient data storage and retrieval.
4. Azure Expertise: Utilize Azure services, such as Azure Data Factory, to facilitate data movement, transformation, and integration.
5. SQL Mastery: Proficiently write SQL queries to extract, manipulate, and analyze data as needed for business requirements.
6. JSON Handling: Work with JSON data structures for data modeling and exchange within the data ecosystem.
7. Data Security: Implement data security measures to protect sensitive information and ensure compliance with data privacy regulations.
8. Documentation: Maintain comprehensive documentation of data pipelines, transformations, and data-related processes for reference and auditing.

Qualifications::

• Strong proficiency in SQL and experience with NoSQL databases, particularly CosmosDB.
• In-depth knowledge of Azure services, with hands-on experience in Azure Data Factory.
• Proficiency in working with JSON data structures.
• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration skills to work effectively with cross-functional teams.
• Familiarity with data modeling and data warehousing concepts.