Senior Data Engineer

Job Description

The DevOps Engineer supports software and data engineering with DevOps tools, culture, and process changes to enable rapid and stable delivery of applications and technologies. Identifies, prototypes, engineers, and deploys emerging software engineering and DevOps tools into existing baseline. Operates and supports a suite of enterprise DevOps services. Establishes, manages, automates, and enhances DevOps pipelines. Defines and promotes enterprise software engineering and DevOps standards, practices, and behaviors. Supports project teams as they integrate into the DevOps environment. May work in hybrid environments (multi-cloud, multi-domain, private/public). Writes and maintains infrastructure and configuration files, DevOps tool configuration scripts, and application run-time configuration scripts. Facilitates technical exchange meetings to identify, collect, and establish coordinated requirements with key stakeholders.

Key Responsibilities

• Work with Chief Engineer and engineers to incorporate DevOps best practices and automation to improve program performance while reducing risk
• Develops Infrastructure as Code to control all aspects of software environments and version control releases
• Containerizes Java and Python code optimized for deployment in Kubernetes environment, ensuring high availability and parallel processing
• Leverages AWS best practices to reduce compute costs while incorporating autoscaling and scheduling of resources according to program demand
• Collaborate with development teams as part of software development lifecycle to design software while considering the production environment
• Automate and version control deployments while ensuring a controlled deployment process that enables zero down time
• Incorporates security scanning and other best practices to ensure new deployments comply with industry best practices
• Works with the ISSE implementing security tools in the build pipeline
• Document work in alignment with Agile Development practices
• Works well as part of a fast-paced team with a desire to innovate and improve a multi-faceted organization

Required Skills

Minimum of 4-7 years demonstrated experience with:

• Kubernetes and Helm Charts
• Docker and/or Podman Containerization best practices
• AWS Services (EKS, Lambdas, EMR, ECR, VPC, etc.)
• Terraform Experience

Additional Required Skills:

• Demonstrated experience working with ElasticSearch
• Demonstrated experience using AWS to include S3, EBS, and EC2
• Demonstrated experience in platform and application automated deployment
• Demonstrated experience in cloud deployment, DevOps, containerization, web service and API management, and computing optimization
• Demonstrated experience in cloud database, storage, virtualization, and network configurations
• Demonstrated experience implementing DevSecOps CI/CD pipelines to support a multi-tenant environment
• Demonstrated experience incorporating automated software quality checks, security scanning, infrastructure-as-code, and other modern DevSecOps tools and practices
• Demonstrated experience working within cloud computing environments, Linux and Unix-based operating systems, and using ElasticSearch and Kibana
• Demonstrated experience using requirements tracking programs such as JIRA or Confluence
• Demonstrated experience maintaining, upgrading, troubleshooting, and managing software, hardware and networks
• Demonstrated experience installing, configuring, testing, and maintaining operating systems, application software, and system management tools
• Demonstrated experience monitoring system performance and troubleshooting issues
• Demonstrated experience developing in Java
• Demonstrated experience with Scripting Languages such as SSH, Python, JavaScript
• Demonstrated experience using advanced query functions in relational databases
• Demonstrated experience using Apache Tomcat Application Servlets
• Demonstrated experience maintaining data across various systems and databases
• Demonstrated experience aggregating data across structured data files (JSON, XML, CSV, YAML, and XLSX) and handling and processing dirty data and unstructured data
• Demonstrated experience performing data analysis and data clustering
• Demonstrated experience using Git as a source code repository
• Demonstrated experience collecting and processing data, particular data presented in machine learning (translation skills not required)
• Demonstrated experience developing and documenting scripts on Linux OS
• Demonstrated experience using Javascripts & jquery to include but not limited to: D3, Vue
• Demonstrated experience in app development, deployment, and maintenance

Desired Skills

• Terraform and Packer with AWS (AMIs, Environment, Configuration Automation, etc.)
• Database ORM Tools (Flyway, etc.)
• CI/CD Pipeline Automation (Git, Jenkins, Automated Testing, etc.)
• Cloud environment and infrastructure engineering
• Implementing DevOps within a Secure Environment
• Demonstrated experience working to accredit systems for the Sponsor's cloud environment according to the Sponsor's security requirements
• Demonstrated experience in running DataCTRL ETL tool
• Demonstrated experience using NiFi to ETL data
• Demonstrated experience in Data Management