Description for Data Migration Engineer:
To support the Information System Division and the enterprise by providing comprehensive data engineering solutions in translating Navy Federal's business vision and strategies into effective IT and business capabilities through the design, implementation, and integration of IT systems in the big data domain. The Data Engineer will be responsible for migrating on-prem applications & data as per Navy Federal standards with a specific focus on Azure as well as Hadoop Data solutions.
Required (Individual Role):
- Extensive experience in migrating on-prem Hadoop applications & data to Azure ADLS. Extensive experience in engineering and designing data management solutions using Python, Scala, Spark, Azure Data Factory and Azure Databricks for a minimum of three (3) years. Experience with Precisely ecosystem tools and technologies with particular emphasis in using Precisely Connect ETL & Connect for Big Data is desirable.
- Proficient in data migration pipeline process, data ingestion pipeline process, exception handling and metadata management from multiple source formats such as Mainframe, relational sources, flat files, etc. to Azure ADLS as well as Hadoop HDFS.
- Extensive experience in managing data in ADLS as well as HDFS
- Extensive experience using Azure Data Factory, Azure Databricks notebooks for Python, Scala & Spark, Azure DevOps.
- Experience with Azure Data Lake (ADLS) Governance and Data Pipeline Management
- Experience with Cloudera Enterprise Data Hub (EDH) Governance and Data Pipeline Management
- Advanced degree in MIS, computer science, statistics, marketing, management, finance or related field
- Prior experience in the Financial Industries and large banks
- Experience in reading and analyzing complex VSAM Copybooks is desirable
- Experience in developing Precisely Connect ETL & Precisely Connect for Big Data jobs for Mainframe VSAM files, and DB2 Mainframe systems is desirable
- Experience with Azure HDInsight is desirable.