Big Data Engineer - Python | Melbourne
|Attachments:||No File Attached|
|Application Close Date:||31-Oct-2019|
A global leader in consulting, technology services and digital transformation, the Capgemini Group is at the forefront of innovation to address the entire breadth of clients' opportunities in the evolving world of cloud, digital and platforms. Building on its strong 50-year heritage and deep industry-specific expertise, Capgemini enables organizations to realize their business ambitions through an array of services from strategy to operations. Capgemini is driven by the conviction that the business value of technology comes from and through people. It is a multicultural company of over 200,000 team members in more than 40 countries. The Group reported 2018 global revenues of EUR 13.2 billion. People matter, results count. Learn more about us at www.capgemini.com
Let's talk about the team:
Our Insights and Data team helps our clients make better business decisions by transforming an ocean of data into streams of insight. Our clients are among Australia's top performing companies and they choose to partner with Capgemini for a very good reason - our exceptional people.Due to continued growth within Capgemini's Insights & Data practice we intend to recruit a Big Data Engineer with Python skills to join our team in Melbourne. If you are already working in a consultancy role, or. have excellent client-facing skills gained within large organizations, we would like to discuss our consultant opportunities with you.
Let's talk about the role and responsibilities:
The Big Data Engineer will collaborate with product owners, designers to build data engineering pipelines on various cloud platforms like GCP, AWS and Azure. Key responsibilities include
- Build robust, efficient and reliable data pipelines consisting of diverse data sources to ingest and process data into cloud-based data platforms preferably using Python as a programming language.
- Design and develop batch and real time data ingestion from various source systems to cloud based data and analytics platform.
- Transform data on at least one of the distributed data processing frameworks like map reduce, spark or beam.
- Building workflows for ingesting and processing data into SQL or NoSQL data warehouses.
- Collaborate with Architects to define the architecture and technology selection.
- Work with stakeholders including the Product Owner and data analyst teams to assist with data-related technical issues and support their data infrastructure needs
Let's talk about your qualifications and experience:
To be considered for this role you must have:
- Proven working experience as Big Data engineer for 3+ years preferably in building data lake solution by ingesting and processing data from various source systems
- Hands-on experience in programming languages like python, SQL, Spark. Experience working on developer tools such as Jupyter, pycharm, Git etc.
- Experience in distributed data processing in at least of these: EMR, Dataflow, Dataproc, Cloudera Spark etc.
- Experience in building SQL and NoSQL data warehouses in at least one of these: Hive/HBase/Redshift/Azure data warehouse/BigQuery.
- Experience with cloud based big data stack from one of the leading public cloud based provider such as GCP, AWS or Azure.
- Understanding of Data Engineering and Data Management practices in cloud.
- Experience in building various frameworks for Data quality and validation checks in Data Ingestion, Data Processing, Data Warehousing for enterprise data lake.
- Big Data Engineering certification is desirable but not imperative.
- Understanding of DevOps, Continuous Integration and Continuous Delivery principles to build automated pipelines for deployment and production assurance on the data platform.
- Knowledge of building self-contained applications using Docker and Kubernatives.
- Ability to articulate/relate complex technology material to a variety of different types of stakeholders.
- Excellent social and communication skills, with a passion for mentoring and knowledge sharing.
- Experienced in Agile / Scrum projects.
What happens next and what can we offer you?
Interested? Passionate people are Capgemini's Ace of spades. We believe that every one of us is an architect of positive futures. We invite you to join us to discover a career that will challenge, support and inspire you. Working at Capgemini you'll find the rewards are more than just financial. Not only will you work alongside inspiring colleagues with a world of experience, but you'll also have access to great benefits including, salary continuance insurance, paid parental leave, education assistance, salary packaging, the ability to purchase additional leave; as well as discounts on entertainment, financial and wellbeing services, travel and shopping. Talk to us about working part-time or. full time.
Ranked among Ethisphere's 2019 Most Ethical Companies in the World (for the 7th year running!), our seven values are at the heart of everything we do - Honesty, Boldness, Trust, Team Spirit, Freedom, Fun and Modesty. As Social Innovators, we are committed to making a positive impact on the planet and our communities. Capgemini is a proud to be Equal Opportunity Employer and all applicants must be legally entitled to work in the country the role is based out of, unless otherwise stated within the job advertisement. Proof of work entitlements and visa status will be required prior to and/or at time of offer. Successful applicants will be required to complete a security and reference checks prior to commencement of employment.
Please note unsolicited resumes from employment agencies or. other third parties will not be considered.