Senior Data Engineer - Cognite Data Fusion Job

Job description:

Experience required: 5+ years

Job Title: Deployment Engineer 

Job Summary 

We are seeking a highly motivated and technically strong Deployment Engineer to design, deploy, and manage end-to-end industrial data pipelines and digital solutions built on Cognite Data Fusion (CDF). The successful candidate will be responsible for data onboarding, transformation, contextualization, quality validation, and data model management across a wide range of enterprise and industrial data sources. 

This role requires a strong background in data engineering, industrial data analysis, cloud-based deployments, and software delivery practices. The engineer will work closely with client stakeholders, data owners, domain experts, software development teams, and solution architects to ensure high-quality data foundations that enable successful deployment of client digital solutions such as OptiFlow and OptiSite. 

Success in this role requires excellent analytical skills, strong communication capabilities, a passion for learning new technologies, and the ability to rapidly adapt to the continuously evolving digital technologies by Client & Cognite. 

Key Responsibilities 

Data Integration & Pipeline Deployment 

• Design, configure, and deploy end-to-end data ingestion pipelines into Cognite Data Fusion (CDF). 

• Integrate diverse data sources including: 

• CSV and flat files 

• SQL and relational databases 

• PI Systems 

• SCADA systems 

• SAP systems 

• Historians and operational databases 

• Unstructured documents and files 

• Other industrial and enterprise data sources 

• Ensure reliable, scalable, and secure data ingestion architectures. 

• Troubleshoot ingestion issues and optimize pipeline performance. 

Data Transformation & Contextualization 

• Transform raw source data into consumable business and engineering datasets. 

• Perform data contextualization and establish relationships between assets, events, time series, documents, and engineering objects. 

• Develop mapping strategies that align source systems with enterprise data models. 

• Configure and maintain metadata structures and contextualization frameworks within CDF. 

Data Quality & Engineering Data Analysis 

• Assess source data quality, completeness, accuracy, consistency, and readiness for deployment. 

• Perform detailed data profiling and validation activities. 

• Analyze sample datasets to identify data gaps, anomalies, inconsistencies, and missing relationships. 

• Establish data quality checks and validation procedures throughout the deployment lifecycle. 

• Work directly with client data owners to resolve data issues and improve data reliability. 

• Serve as a subject matter expert for engineering and operational data structures. 

CDF Data Modeling & Governance 

• Design, implement, and maintain data models within Cognite Data Fusion. 

• Define entities, relationships, metadata structures, and governance approaches. 

• Support lifecycle management of data models as business requirements evolve. 

• Ensure data structures align with downstream application and analytics requirements. 

OptiFlow & OptiSite Deployment 

• Configure and deploy cleint digital solutions including OptiFlow and OptiSite. 

• Configure backend workflows, business logic, and application integrations. 

• Support UI configuration and solution customization based on customer requirements. 

• Validate end-to-end functionality from source systems through application consumption. 

• Participate in solution testing, commissioning, and production deployment activities. 

Stakeholder Collaboration 

• Engage with multiple client teams responsible for different data sources and operational systems. 

• Gather technical requirements and source system information from stakeholders. 

• Facilitate workshops, data discovery sessions, and deployment planning activities. 

• Communicate data findings, risks, and recommendations to technical and non-technical audiences. 

• Collaborate closely with software developers, architects, project managers, and customer teams. 

Software Delivery & Cloud Operations 

• Work closely with software development teams that consume CDF data for application development. 

• Understand software development lifecycle (SDLC) methodologies and deployment practices. 

• Ensure data structures support current and future application requirements. 

• Support cloud-based deployment and operational activities. 

• Contribute to continuous improvement of deployment methodologies and standards. 

Continuous Learning & Innovation 

• Stay current with new capabilities, updates, and best practices of OptiFlow, OptiSite, & Cognite Data Fusion. 

• Rapidly learn and adopt emerging platform features and deployment approaches. 

• Evaluate new technologies and tools that improve deployment efficiency and data quality. 

• Contribute to internal knowledge sharing and capability development initiatives. 

Required Qualifications 

• Bachelor's degree in computer science, Information Technology, Data Engineering, Engineering, or a related technical discipline. 

• 5+ years of experience in data engineering, deployment engineering, industrial digitalization, or related fields. 

• Experience implementing enterprise-scale data integration and data management solutions. 

• Experience working with cloud-based platforms and deployments. 

Required Technical Skills 

Data Engineering 

• Strong understanding of data integration, ETL/ELT, and data management practices. 

• Hands-on experience working with structured and unstructured data. 

• Expertise in data profiling, cleansing, validation, and quality assessment. 

• Experience with enterprise and industrial data architectures. 

Databases & SQL 

• Advanced SQL scripting and query optimization skills. 

• Experience with relational databases and data warehousing concepts. 

• Ability to investigate and troubleshoot complex data issues. 

Python & Automation 

• Intermediate to advanced Python scripting skills. 

• Experience developing automation scripts for data validation, transformation, and deployment tasks. 

• Familiarity with APIs and integration frameworks. 

Cognite Data Fusion (Preferred) 

• Experience with Cognite Data Fusion (CDF) or similar industrial data platforms. 

• Knowledge of contextualization, data modeling, and industrial data management solutions. 

Industrial Systems 

Experience with one or more of the following: 

• PI Systems (OSIsoft/AVEVA PI) 

• SCADA systems 

• SAP 

• Manufacturing Execution Systems (MES) 

• Industrial Historians 

• Engineering Information Management Systems 

• Industrial IoT platforms 

Cloud Technologies 

• Experience with cloud environments such as Azure, AWS, or Google Cloud Platform. 

• Understanding of cloud deployment, security, networking, and operational concepts. 

Software Development 

• Understanding of software development lifecycle (SDLC). 

• Familiarity with Agile delivery methodologies. 

• Experience collaborating with software engineering and product development teams. 

• Working knowledge of Git and GitHub, including repository management, branching strategies, commits, pull requests, code reviews, and issue tracking. 

Documentation & Knowledge Management 

• Experience creating and maintaining technical documentation, deployment guides, runbooks, and operational procedures. 

• Ability to document data flows, system configurations, troubleshooting steps, and deployment activities. 

• Experience working with documentation and collaboration platforms such as Confluence, SharePoint, or similar knowledge management tools. 

Key Competencies 

• Strong analytical and problem-solving skills. 

• Exceptional attention to detail and data quality. 

• Excellent stakeholder management and communication skills. 

• Ability to work across multiple teams and organizational boundaries. 

• Strong customer-facing and consulting capabilities. 

• Ability to learn quickly and adapt to evolving technologies. 

• Self-driven with a high degree of ownership and accountability. 

• Ability to manage competing priorities in dynamic project environments. 

Preferred Qualifications 

• Experience deploying industrial digitalization solutions. 

• Experience with client digital products such as OptiFlow and OptiSite. 

• Experience in energy, manufacturing, process industries, or industrial operations. 

• Knowledge of master data management and data governance practices. 

• Familiarity with DevOps and CI/CD deployment practices. 

• Understanding of asset hierarchies, engineering data models, and operational data workflows. 

Success Measures 

The successful candidate will: 

• Deliver reliable and scalable CDF data pipelines. 

• Ensure high standards of data completeness, correctness, and contextualization. 

• Successfully deploy OptiFlow and OptiSite solutions for customers. 

• Build and maintain robust CDF data models supporting application development. 

• Establish trusted relationships with client stakeholders and data owners. 

• Enable software development teams through high-quality, well-structured data foundations. 

• Continuously adopt and leverage new capabilities within the CDF ecosystem to maximize customer value.