Lead Data Engineer, Big Data - Data Platform

Company: Visa
Location: Palo Alto
Posted on: January 15, 2021

Job Description:

Company Description Common Purpose, Uncommon Opportunity. Everyone at Visa works with one goal in mind - making sure that Visa is the best way to pay and be paid, for everyone everywhere. This is our global vision and the common purpose that unites the entire Visa team. As a global payment technology company, tech is at the heart of what we do: Our VisaNet network processes over 20,000 transactions per second for people and businesses around the world, enabling them to use digital currency instead of cash and checks. We are also global advocates for financial inclusion, working with partners around the world to help those who lack access to financial services join the global economy. Visa's sponsorships, including the Olympics and FIFA----- - World Cup, celebrate teamwork, diversity, and excellence throughout the world. If you have a passion to make a difference in the lives of people around the world, Visa offers an uncommon opportunity to build a strong, thriving career. Visa is fueled by our team of talented employees who continuously raise the bar on delivering the convenience and security of digital currency to people all over the world. Join our team and find out how Visa is everywhere you want to be. Job Description Drive Big Data Hadoop platform vision and strategy for an entire team of Data Engineers Build and expand on Visa Data Platform's Hadoop Open source Big data Clusters within an agile collaboration environment Work on big projects across multiple product teams, and with a wide range of stakeholders Imagine working in a startup in the Apache Hadoop Open source Big data platforms domain tasked with processing the plethora of payment data that comes to Visa on an Hourly basis Observe the thriving open source projects in the descriptive and predictive analytics big data space and bring it into Visa to further enrich customer offerings Must love coding - prepare to spend 80% of the time on hands-on development with development teams Deliver end-to-end applications and frameworks ranging from system programming to micro-services to simple front-end applications Be responsible for automated deployment of the product through CICD-based deployments into Kubernetes, Virtual Machines and ability to do small scale implementation fast implementation for localization requirements Security-first mindset to ensure that any software built will satisfy Visa's stringent security standards Ability to own complete project or a subject area delivery while leading team members in a scrum setting Mentor/manage team members, develop procedures and best practices standards Qualifications Basic Qualifications: 10 years of work experience with a Bachelor's Degree or at least 8 years of work experience with an Advanced Degree (e.g. Masters/MBA/JD/MD) or at least 3 years of work experience with a PhD Preferred Qualifications: 12-15 years of work experience with a Bachelor's Degree or 8-10 years of experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) or 6 years of work experience with a PhD Active development experience with Java, Python, Spark, Hive, Kafka, Kstreams, NoSql Databases, while having core fundamentals experience of Apache Hadoop components Experience in building Apache Open source projects, contributed to open source projects and deploying into containers platform Hand-on experience in optimizing existing Big Data application written in Spark, Hive. Optimizing Can navigate a Linux terminal with ease Relational database and SQL development experience required Understanding best practices for Big Data (in Hadoop), data warehousing, consumer analytics, knowledge management and key understanding of streaming and other NoSql databases Hands-on experience in developing and managing development of Data Integration applications for large corporations with experience in both batch and online systems. Have experience delivering a localization big data implementation is a big plus In-depth knowledge of the software development life cycle required Strong communication skills with ability to present complex ideas and document in a clear and concise way Quick learner; self-starter, detailed and thorough Proactive sense of urgency and "can do" attitude Additional Information Travel Requirements: This position requires the incumbent to travel for work 5% of the time. Mental/Physical Requirements: This position will be performed in an office setting. The position will require the incumbent to sit and stand at a desk, communicate in person and by telephone, frequently operate standard office equipment, such as telephones and computers, and reach with hands and arms. Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

Keywords: Visa, Palo Alto , Lead Data Engineer, Big Data - Data Platform, Other , Palo Alto, California