Sr. Software EngineerScepter Technologies Inc
Sr. Software Engineer
MS or Foreign equivalent in CS, Engineering (any) or any related 1 year experience in Design, develop of Hadoop Big Data & Proc data using Map Reduce, Hive, Pig, Scoop, Oozie, & Apache Spark and Kafka. Design utility jobs to move data into Amazon Redshift from Hortonworks. Integrate Apache Storm with Kafka to perform web analytics. Upload click stream data from Kafka to Hdfs, Hbase and Hive by integrating with Storm. Config Flume to transport web server logs into HDFS. Use Kite logging module to upload webserver logs into HDFS. Ingesting huge amount of XML files into Hadoop by utilizing DOM Parsers with in Map Reduce; Processing large data sets utilizing Hadoop cluster. Automate data loading in to Hadoop using Oozie. Design & Implement Java Map/Reduce programs. Migrate map reduce jobs into Spark Jobs & Use Spark SQL, Data frames API to load structured data into Spark Clusters; Ability manage multiple priorities and provide guidance.
1 year experience as System/Program Analyst /SW is acceptable.
Job Locations: Columbia MD or Oak Brook, IL or any other unanticipated client locations in US.