Data Engineer | Sydney
|Attachments:||No File Attached|
|Application Close Date:||30-May-2018|
About the Team
Our Insight and Data team helps our clients make a better business decision-making by transforming an ocean of data into streams of insight. Our clients are among Australia's top performing companies and they choose to partner with Capgemini for a very good reason - our exceptional people.
About the Role
- Design and develop Big Data solutions in Cloudera/Hortonworks platform using Big data technologies like - Spark, Hive, Pig, Scoop,Apache Nifi, Kakfa, etc
- Build reusable code, libraries, patterns and consumable frameworks for data sourcing, cleansing, validations, staging, transform, conforming and extraction
- Develop and maintain standards for development, source code management, continuous integration and automation: build, test, and deploy on Big data ecosystem
- Develop and maintain standards for administration and operation Big data jobs including the scheduling, running, monitoring, logging, management of errors, recovery from failures, and validation of outputs
- Collaborate with business analysts to understand business requirements/use cases and translate them to detailed technical specifications
- Collaborate with project managers for project/sprint planning by estimating technical tasks and deliverables
- Contribute and provide support to all Big Data initiatives
Desired Skills and Experience
- Minimum 5 years of experience with 2+ years of design and hands-on development in Big data systems using Spark, Pig, Hive, MapReduce, SparkSQL, Apache Nifi, Kakfa etc
- Minimum 2 years of hands-on experience in Scala or Python or Java programming for building Spark application - using RDDs, Dataframe APIs
- Minimum of 3 years of hands-on experience with Unix/Linux shell scripting
- Experience in various Data Ingestions technologies - Scoop, Flume, Apache NiFi, Kakfa etc
- Knowledge of NoSQL databases like Hbase, Mongo etc
- Knowledge of Data warehouse and analytical technologies in the industry including MPP databases, Data Warehouse design, BI reporting and Dashboard development.
- Hadoop or Spark Developer Certification (Nice to Have)
- Experience in working in a fast-paced Agile environment
- Strong problem solving and troubleshooting skills
“Our seven values are at the heart of everything we do - Honesty, Boldness, Trust, Team Spirit, Freedom, Fun and Modesty.”
With more than 180,000 people in over 40 countries, Capgemini is one of the world's foremost providers of consulting, technology and outsourcing services. The Group reported 2015 global revenues of EUR 11.9 billion. Together with its clients, Capgemini creates and delivers business, technology and digital solutions that fit their needs, enabling them to achieve innovation and competitiveness. A deeply multicultural organization, Capgemini has developed its own way of working, the Collaborative Business ExperienceTM, and draws on Rightshore®, its worldwide delivery model.
Want to know more?
To learn more about Capgemini and find out about what makes our people unique, visit www.capgemini.com.au or http://www.backelite.com.au/