at Invenergy LLC in Chicago, Illinois, United States
Job Description
Invenergy drives innovation in energy. Powered by decades of entrepreneurial experience and unparalleled execution, we solve the energy challenges facing our customers and communities. We provide power generation and storage solutions at scale around the world to create a cleaner energy future. We develop. We build. We own. We operate. We are Invenergy.This position will be open for application for at least 3 calendar days from the posting date. This position will remain open for application based on business need, which may be before or after the 3-day posting window.
Job Description
Position Overview
The Data Engineer should be an experienced hands-on Data Engineer that will be responsible for our enterprise data assets, work with our various business organizations to develop timely and useful data pipelines and data lake storage and provide expertise and best practice designs and ways of working for business groups reuse. Working with the Data Architect and business groups the Lead Data Engineer will create pipelines, databases, and data lake storage for critical data to be used enterprise wide by internal business groups and external vendors.
Responsibilities
- Actively contribute to the design, development, and optimization of data pipelines, ensuring high-performance and scalability.
- Utilize expertise in Azure Cloud technologies to build and maintain robust data solutions.
- Design, develop, and maintain data pipelines and data storage, including the enterprise data lake house.
- Support maintenance and governance of the existing data pipelines, databases, and data lake storage.
- Collaborate with cross-functional teams to design, develop, and implement scalable and efficient data solutions.
- Work with IT data architects, business analysts, and cross-functional teams on their data needs and provide solutions based on requirements.
- Create, manage, and optimize data stores, including SQL Server, Azure SQL Server, and Azure Synapse.
- Implement and maintain data lake stores, including Databricks DeltaLake and Azure OneLake.
- Ensure proper security of all data from databases to data lake storage.
- Mentor and guide team members and business units on best practices, design patterns, and effective ways of working in the data engineering domain.
- Provide technical leadership and support to ensure the successful execution of data engineering projects.
- Support multiple business groups to understand and gather data requirements.
- Collaborate with stakeholders to translate business needs into technical specifications and data solutions.
- Utilize strong development skills in SQL and a data manipulation language such as Python to design and implement data solutions.
- Develop and optimize data pipelines for data processing, transformation, and loading.
- Increase user adoption and utilization of common data across the organization.
- Stay informed of new trends and best practices in data processing and storage and determine if there are opportunities for the company to improve.
- Build and maintain our data catalog to track enterprise data assets
- Work with the technical and functional teams to identify data anomalies and ensure data quality.
- Define and document all data requirements.
- Create, execute, and maintain data quality checks.
- Create alarms and alerts for automatic tracking and notification of data pipeline or processing issues.
- Perform and Review change control, management of change control execution, and documentation
- Work with 3rd parties to provide additional breadth to IT delivery capabilities for data processing and architecture.
Minimum Qualifications
- 5+ years of experience in data engineering, including ETL processing, relational database tools, data warehousing, database architecture, SQL Server, SSIS, Synapse, & Databricks with a focus on hands-on work.
- Bachelors Degree in Computer Science or related field.
- Proven hands-on experience with Azure Cloud technologies such as ADF, Synapse, SQL Server,
- Proven hands-on experience with Databricks, including workspace provisioning and setup, as well as Unity Catalog for establishing and securing managed, unmanaged, and federated catalogs.
- Expert in SQL and a data manipulation language, preferably Python.
- Demonstrated experience with data lake stores, SQL Server, Azure SQL Server, and Azure Synapse.
- Proven ability to mentor and guide other team members and business analysts in best practices.
- Understanding of data warehousing concepts and relational data models
- Experience with commercial ETL platforms and in-depth knowledge of data transformation paradigms.
- Excellent written and verbal communication skills and strong cognitive ability
- Demonstrated ability to organize information, manage tasks and projects to support business needs
- Ability to flexibly accept dynamically changing priorities
- Eligible to work in the United States without the need for employer visa sponsorship now or in the future.
Preferred Qualifications
- Familiarity with Snowflake and exposure to other relevant data technologies such as Azure Fabric.
#LI-ONSITE
Salary Range
$116,000 - $141,000
Benefits
Eligible for medical, dental, vision, 401(k), bonus, paid time off, etc.
Invenergy LLC is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to, among other things, race, color, religion, sex, sexual orientation, gender identity, national origin, age, status as a protected veteran, or disability. To view full details and how to apply, please login or create a Job Seeker account
