Senior Data Engineer

TEKmights   Columbia, MD   Full-time     Information Services / Technology (IT)
Posted on June 25, 2024

TEKmights has a job opening through HQ in Columbia, MD. 

Job location: multiple undetermined worksites in U.S. (relocation may be req’d, must be willing to relocate). 

   Job Description

-Design and implement data pipelines using Spark, Scala, and Python to ingest, process and transform large volumes of data from various sources for projects. 

-Use Hive and Impala to query and manipulate data stored in data lakes or cloud-based storage systems. 

-Create and maintain ETL workflows using Oozie, a workflow scheduler, to orchestrate data processing tasks and ensure data integration and synchronization. 

 -Write SQL queries and leverage Impala to perform ad-hoc data analysis and generate reports on large datasets. 

 -Design and implement data processing workflows using AWS Glue and AWS Lambda Functions. 

 -Develop and maintain data pipelines using AWS EMR (Elastic MapReduce) for large-scale data processing. 

 -Utilize AWS CDK (Cloud Development Kit) to automate the creation and management of cloud resources for data processing applications. 

 -Collaborate with business analysts to ensure data accessibility and availability in AWS S3 for data analytics and reporting purposes. 

 -Document code, data pipelines, and data processes for knowledge sharing and future reference.

 

 Master in Comp Sci or Info Sys + 1yr exp

 

Please visit http://www.tekmights.com/careers.html for detailed position opening. Send resumes to vikas@tekmights.com.


TEKmights

Columbia , MD