Data Engineer | Melbourne
|Attachments:||No File Attached|
|Application Close Date:||26-Mar-2018|
About the Role
- Develop Big Data processing pipelines in Hortonworks platform using Spark (with Scala) and Hive for creation of metrics and dimensions to support BI needs
- Build reusable code, libraries, patterns and consumable frameworks for data sourcing, staging, transform, conforming and extraction
- Ensure adherence to standards in areas of code quality, logging, exception management, operability and performance
- Unit testing and deployment into SIT and Production
- Collaborate with Business Analyst to clarify business requirements
- Collaborate with Technical lead and Architects to adherence to development standards
- Collaborate with project managers for project/sprint planning by estimating technical tasks and deliverables
- Contribute and provide support to all Big Data initiatives
Desired Skills and Experience
- Minimum 5 years of experience with 2+ years of design and hands-on development in Big data systems using Spark, Pig, Hive, MapReduce, SparkSQL, Apache Nifi, Kakfa etc
- Minimum 2 years of hands-on experience in Scala programming for building Spark application - using RDDs, Dataframe APIs
- Minimum of 2 years of hands-on experience with Unix/Linux shell scripting
- Experience in various Data Ingestions technologies - Apache NiFi, Kakfa, Scoop etc
- Knowledge of NoSQL databases like Hbase, Mongo, Marklogic etc
- Knowledge of Data warehouse and analytical technologies in the industry including MPP databases, Data Warehouse design, BI reporting and Dashboard development.
- Hadoop or Spark Developer Certification (Nice to Have)
- Experience in Telecom Domain and geospatial data is highly desirable
- Experience in working in a fast-paced Agile environment
- Strong problem solving and troubleshooting skills
- Excellent customer-facing communication skills
What we can offer you?
Working at Capgemini you'll find the rewards are more than just financial. Not only will you work alongside inspiring colleagues with a world of experience but you will also have access to great benefits including, salary continuance insurance, paid parental leave, education assistance, salary packaging, the ability to purchase additional leave, as well as, discounts on entertainment, financial and wellbeing services, travel and shopping.
Capgemini is one of the world's foremost providers of consulting, technology, outsourcing services and local professional services. Present in over 40 countries with more than 180,000 people, the Capgemini Group helps its clients transform in order to improve their performance and competitive positioning.
Ranked among Ethisphere's 2015 Most Ethical Companies in the Word. Our seven values are at the heart of everything we do - Honesty, Boldness, Trust, Team Spirit, Freedom, Fun and Modesty.
If you believe you have “La Niaque” to go the extra mile, then apply by submitting your resume and cover letter.
Want to know more? To learn more about Capgemini and find out about what makes our people unique, log onto www.capgemini.com.au
Proof of work entitlements and visa status will be required prior or at offer time. Successful applicants will be required to complete a Criminal Record and Reference checks prior to commencement of employment.