Senior Big Data Engineer - Spark

Location: Lviv, Ukraine
Date Posted: 06-27-2017
Sr. Big Data Engineer - Spark
 
As a core engineering team member, this development position requires the creation of a Spark ecosystem that is built to handle data at the scale equal to today’s largest Internet properties. The successful candidate will be someone who enjoys and has experience building very large-scale business critical data systems with the challenge of low latency access for real time application.  You must be passionate about Big Data and be able to contemplate new and valuable uses of the data.
 
You must be able to work in a team environment, be a problem solver that is able to communicate and prioritize in a fast moving challenging, stimulating, and rewarding environment.
 
Responsibilities
 
- Significant contributor to Spark architecture and infrastructure
- Contribute to the ideation of new uses of data and new data centric services
- Create CI automated tests
- Create Spark Apps for specific real-time reporting requirements
- Contribute to adding Spark to DevOps Framewaork
- Dig deep into any performance, scalability, capacity or reliability problem in order to resolve the issues
 
Qualifications
 
- Proven experience building and scaling out a Spark based ecosystem for a large-scale on-line Service
- Previous experience with Kafka
- 8+ years of professional software engineering experience preferably in online service oriented businesses
- 2+ years of experience writing Spark RDD jobs in Java, Scala specifically to perform aggregation calculations for both real-time streaming and batch delivery
- Previous experience using NoSQL databases such as Cassandra, Mongo, Riak, Couchbase, etc.
- Experience writing software in Continuous Integration and automated deployment environment
- Prior analytics experience in Global Content Delivery is a plus
 
About VidScale,  Inc.
VidScale is the first distributed edge compute platform that enables services such as content delivery to deployed on service provider access networks.  Our goal is to enable the next generation of applications such as VR/AR, gaming, IoT, and big data to be deployed in a tiered compute infrastructure that enables massive scale with ultra-low latency. Our first service, a massively saleable Content Delivery service is already in production with our OEM partners.  Successful candidates will have the opportunity to work with the largest names in the business.
 
or
this job portal is powered by CATS