Big Data Systems Engineer

Location: Cambridge, Boston
Date Posted: 05-17-2017
Big Data Systems Engineer

Vidscale is growing! The Vidscale operations team is responsible for infrastructure/technical planning, implementation and support activities for the production content delivery systems and portal.  We are currently looking to grow the team by bringing in the right individuals to help augment an already talented group of individuals. The specific responsibilities include operational support activities such as resolving software and hardware related problems, administering the different technology layers of our platform, managing monitoring and alerting functions, performing capacity planning and conducting version management. Responsibilities also include installing and supporting system hardware and software, performing system upgrades, and evaluating and installing patches and software updates. 
• Provide technical leadership and ownership of the production analytics flume-based pipeline
• Provide technical leadership and ownership of the production Hadoop/Spark infrastructure 
• Provide technical leadership and ownership of issues across multiple disciplines and technologies.
• Support / Create readers/writers to message bus (Kafka, MapR Streams, etc.)
• Capacity planning, process improvement, maintenance, upgrade and end-of-life planning, roadmap development for the analytics pipeline.
• Dig deep into any performance, scalability, capacity or reliability problem in order to resolve the issues
• Identify automation opportunities and implement scripted solutions.
• Identify technical innovation and process improvement opportunities.
• Utilize standard tools and methodology to develop system and support performance metrics.
• Perform crisis management during high-severity operational incidents
• Resolves or facilitates resolution of complex problems for assigned program.
• Has a thorough and comprehensive mastery of supported platforms/products and environments.
• Focuses the majority of time on complex engineering, architectural and implementation tasks. 
• Project assignments are large in scope and are highly complex.
• Perform daily production support activities.
• Participate in team on-call rotations.
• Perform patching and code deployments across all environments.
• Lead projects associated to the enhancement, upgrade/patching, or implementation of new or existing technology solutions.
• Coordinate implementation and support efforts between the Operation team and the development teams.
• Conduct performance tuning and troubleshooting.
• Provides escalation support to Tier 1 and 2 engineers. 
To be successful, individuals must possess a combination of technical and leadership skills. They will require a solid knowledge of infrastructure, architecture, applications development, delivery and support. A strong aptitude in understanding large systems as a whole including networks and server operations is a must.
• 6+ years working with the Linux Operating System in an enterprise environment 
• Proven experience building and scaling out a Hadoop based ecosystem for a large-scale on-line Service
• 3+ year's experience in Hadoop administration 
• 3+ years of experience writing Map/Reduce jobs preferably running under Spark in Java or Scala
• 3+ years of experience using NoSQL databases such as HBase or Cassandra
• 1+ years of experience with DevOps frameworks such as SaltStack, Chef, Puppet, Ansible, or similar technology stack experience 
- Experience writing software in Continuous Integration and automated deployment environment
- Prior analytics experience in Global Content Delivery is a plus
- Prior experience in MapR platform is a plus
• Ability to debug and troubleshoot network, performance and software
• Knowledge of configuration and tuning of OS and Application related parameters 
• Ability to understand and focus on project requirements 
• Proficient with High Availability and Load Balancing 
• Shell scripting and automation skills
• Ability to work with other technical leaders within the organization.
• Ability to handle multiple tasks, set priorities, schedule and meet deadlines in a past paced work environment.
• Demonstrate effective time management and organizational skills in prioritizing and executing assigned tasks.
• Excellent oral and written communications skills.
• Must be able to work independently as well as work close in team environment when needed.
• Strong conflict resolution skills.
• Ability to assume technical leadership role as needed within area of technical expertise.
• Experience working in a large complex networking environment is required.
• Ability to manage projects independently from inception to completion.
• Analytical thinking and problem solving from inception to completion.
• Commitment to quality and high level of integrity.
About VidScale Inc:
Our team has an average of 17 years in the media delivery space. We are made up of several former CTOs and engineering heads from notable companies in this industry. As veterans in CDN, many with more than 15 years of experience, we are very well versed in the industry.
VidScale is the first end-to-end Content Delivery solution in the OEM market that combines Content Delivery from Origin to Edge, Just-in-time-Packaging, Request Routing, and Analytics. Our solution allows content providers and distributors to virtually deploy and manage networks anywhere in the world. Our vision is driven by global availability, scalability, return on investment, performance and security. Since 2011, VidScale has been assisting their clients to deliver content at a scale that is consistent and seamless in order to enhance the viewing experience of our end-users.
Today, VidScale is currently assisting one of the world’s largest infrastructure providers- with revenue upwards of $30 billion. We have been brought in to power their managed CDN solutions for their operator partners and together are building out a global network.
this job portal is powered by CATS