How To Succeed On Your Big Data Journey

“The world has become excited about Big Data and advanced analytics, not just because the data are big but also because the potential for impact is big.” – David Court: Director at McKinsey & Company.

Big Data Analytics is not just a project. It’s a journey, and there are steps you can take to improve your chances of success.



“How do you pick the framework that is here to stay? You don’t—because you can’t.” – Syncsort.

With rapidly evolving tools and frameworks, a challenge for businesses is to invest in applications that won’t need to be replaced in 12 months. Many are turning to Apache Hadoop for its speed and efficiency, but in an industry where change is the only constant, future-proofing Big Data Software has become a major investment for businesses.

“A recent Robert Half Management Resources Survey found that 41% of CFOs believe staying current with changing technology is the greatest pressure their accounting and finance teams face.” – Mark Sands: General Manager, Asia Pacific for BOARD International.



“Part of efficient Big Data Analytics is selecting the right platform to help you through it. But what should you look for? And do you want to build your solution, buy it, or bridge an available software with what you have in-house?” – Sherry Tiao: Content Marketing Manager at Datameer.

This is where research comes into play, and what tools and technologies you decide to integrate rely fundamentally on what problem you’re trying to solve. One of the key factors to consider is what your data platform drivers are – storage or advanced analytics?

“For organisations needing to store and process tens of terabytes of data, using an open-source distributed file system is a mature choice due to its predictable scalability over clustered hardware. However, if you’re looking to run analytics in online or real-time applications, consider hybrid architectures containing distributed file systems combined with distributed database management systems.” – Nick Millman: Data & Analytics Leader for Accenture.



“The earliest phase, where organisations experiment with and learn about their Big Data needs.” – Datameer.

This is is the initial step where the team is trying to understand what data can be analysed, who can analyse it, brainstorm ideas and identify challenges in a cost-effective and timely way.

“Typical problems encountered during the stage include missing or ill-prepared data, and the reliance on manual labor for data processing.” – Chris Raphael: Former Editorial Director & Content Strategist at RT Insights.

Experimentation is crucial for identifying problems early. It’s better to fail fast and fail cheap than to invest in the wrong platform and face disappointed customers.

“Fail often – obviously, try lots of things. As you discover what is working, do more of it. And what does not work gets cut and is not a failure, it is a learning of what does not work.” – Canrock Ventures.



“You need to know how, and why, Big Data is useful to your company.” – Talend.

What problem are you trying to solve, and what should you consider when looking for a Big Data solution? choosing the right use-case can be the difference between the success or failure of your Big Data project. Although it can be tempting to try and tackle the biggest and most complex business problem as soon as you’ve been given the go-ahead to implement a Big Data platform, the best approach is to start small.

“Go small. Very small. For example, starting with one low-key business problem and a few easily accessible datasets. If you don’t, you could unknowingly be winding down the path to failure.” – Ben Sharma: Co-Founder & CEO of Zaloni.



“At its core, data governance is about data trust and accountability, married with comprehensive data security best practices.” – Rob Marvin: Assistant Editor of PCMag.

A good data governance plan consists of a data management strategy, ongoing monitoring of data quality and selective access. “What’s the data you have, who has access to it, and how are you managing the lineage of that data over time?” Jack Norris: Senior VP of Data & Applications at MapR.

Data governance is not only used to manage risk, but also to make sure that there are as few errors as possible.“Through a proper process, companies can implement the appropriate data governance initiatives and framework, which creates structure and accountability to data.” – Desire Athow: Editor at TechRadar.


For more resources, please see below:


Big Data Projects

Five Phases of Big Data Projects

Getting Big Impact From Big Data

Big Data: Changing The Way Businesses Operate

11 Tips For Ensuring Your Big Data Initiative Succeeds

Five Big Data Challenges (Plus Free Resources To Help)

How To Turn Any Big Data Project Into A Success (And Key Pitfalls To Avoid)


Adapting To Change

8 Considerations When Selecting Big Data Technology

As a CFO, How Do You Keep Up With All The Technology Changes?

Keeping Up With Big Data Innovation Without Disrupting Your Business


Big Data Use Cases

Your First Big Data Success: Choosing The Right Use Case

How Is Big Data Used In Practice? 10 Use Cases Everyone Must Read


Data Governance

The Growing Importance Of Data Governance

Big Data Basics: How To Build A Data Governance Plan

How To Avoid Costly Data Errors In The Enterprise

Five Lessons In June 2017 On Big Data Success

At Contexti, we’re always striving to learn from our own experiences and from the insights of other industry leaders.

Here are five lessons we noted from our industry peers this month:



Launching Big Data projects & making data-driven decisions requires a team with a variety of technical, business and soft skills. When working on projects, it’s important to have different voices and skills at the table. “Marketing and data teams should move closer together and explain in simple terms the likely outcomes of the insights created,” – Sherine Yap: global head of CRM at Shell.



In Chapter 1 of ‘Learning to Love Data Science’ by Mark Barlow, he shares his insight on communication, a fundamental part of any project.

“After you’ve laid out a roadmap of the project so everyone knows where they are going, you need to provide them with regular updates. You need to communicate. If you stumble, you need to let them know why you stumbled and what you will do to overcome the barriers you are facing. Remember, there’s no clear path for Big Data projects. It’s like Star Trek – you’re going where no one has gone before.”



‘Every organization seeking to make sense of big data must determine which platforms and tools, in the sea of available options, will help them to meet their business goals.’ – Nick Millman: Data & Analytics Leader for Accenture.

Nick Millman goes on to discuss the importance of the structure of data.

‘How applications consume data should also be taken into consideration. For instance, some existing tools allow users to project different structures across the data store, giving flexibility to store data in one way and access it in another. Yes, being flexible in how data is presented to consuming applications is a benefit, but the performance may not be good enough for high velocity data. To overcome this performance challenge, you may need to integrate with a more structured data store further downstream in your data architecture.’ – Computerworld (from IDG)



“What’s really important about Big Data is to understand that there’s a lot of this data, most of it’s completely worthless to the business, but there are these gems, these nuggets of information, like the fact a customer just had a baby. You want to take that information, you want to integrate it to your business decisions and make more money for your company.” – Andy Mendelsohn: Senior VP of Database Server Technologies at Oracle.



Sample, test and learn – should be the nature of your Big Data project.

“You can only fail better only if you learn from failures. And then failing is something that prompts you to move ahead.” – Pearl Zhu, Digital Agility: The Rocky Road from Doing Agile to Being Agile.


For more resources, please see the links below:

Google Books – Learning to Love Data Science by Mark Barlow (O’Reilly Media)

Marketing Magic Meets Big Data: How To Make Technology and Creativity Work Together

8 Considerations When Selecting Big Data Technology

An Introduction to Big Data – Smart Insights Digital Marketing Advice

Contexti’s Big Data as-a-Service In The Cloud Just Got Better With Cloudera Altus!

We’re excited by the recent announcement of our partner Cloudera on the availability of Altus, which takes the deployment of data platforms and data pipelines in the cloud to the next level.

“Leveraging AWS cloud and Cloudera Enterprise, Contexti has a track record of providing big data-as-a-service / big data platform services for Australian customers including for Seven West Media’s coverage of the Rio Olympic games.” said Sidney Minassian, Founder & CEO of Contexti. “With the availability of Cloudera Altus we’re looking forward to enhancing our service offering for customers who are leveraging their data for value creation.”

Seven West Media taps Cloudera and Contexti for Big Data Solution for Rio Olympics

Cloudera Altus features include:

  • Managed service for elastic data pipelines
  • Workload orientation
  • Backward compatibility and platform portability
  • Built-in workload management and analytics
  • Faster cluster provisioning times
  • Integrated security with cloud service provider solutions

To learn more about Altus, read Cloudera’s blog: Simplifying Big Data in the Cloud

Data Science Workbench by Cloudera Ready for Prime Time

In a fast evolving ecosystem of tools and libraries, data scientists are finding it difficult to use their existing open source languages (e.g. Python, R) and libraries with Hadoop, and are striving to bridge the gaps between the language of the data scientist and the speak of distributed systems.

Contexti partner Cloudera have just announced the general availability of Data Science Workbench. This powerful, self-service tool allows people to accelerate data science from exploration to production using R,Python, Spark and more.

Data scientists now have the freedom to share, collaborate and manage their data in a way that best suits them, resulting in an easier and faster path to production that is is secure for the enterprise.

“We are entering the golden age of machine learning and it’s all about the data.”

– Charles Zedlewski, Senior Vice President of Products at Cloudera.


To find out more about the Data Science Workbench, visit our partner Cloudera’s site.

How Kudu Enables Fast Analytics on Fast Data

From our partner, Cloudera’s website:

Kudu is a columnar storage manager developed for the Hadoop platform that runs on commodity hardware, is horizontally scalable, and supports highly available operation.

Kudu shares the common technical properties of Hadoop ecosystem applications and targets support for families of applications that are difficult  to implement on current generation Hadoop storage technologies.


Kudu’s design sets it apart. Some of Kudu’s benefits include:

  • Fast processing of OLAP workloads.
  • Integration with MapReduce, Spark and other Hadoop ecosystem components.
  • Tight integration with Impala, making it a good, mutable alternative to using HDFS with Parquet.
  • Strong but flexible consistency model, allowing you to choose consistency requirements on a per-request basis, including the option for strict serialized consistency.
  • Strong performance for running sequential and random workloads simultaneously.
  • Easy to administer and manage with Cloudera Manager.
  • High availability. Tablet Servers and Master use the Raft consensus algorithm, which ensures availability even if f replicas fail, given 2f+1 available replicas. Reads can be serviced by read-only follower tablets, even in the event of a leader tablet failure.
  • Structured data model.

To read more about Kudu and find out how to install it, go to our partner Cloudera’s website.


6 Missing Drivers of Failed Big Data Projects in Australia

Given Contexti’s 50+ Big Data Solutions engagements and having provided Big Data Training to over 1,000 professional from over 200 enterprises across Australia, we’ve seen, heard and been involved with enough projects to draw some insights on what drives big data project success and failure.

At the same time, talking with our international business partners and counterparts, we find our observations, while made in Australia, are not unique to Australian enterprises and businesses across the globe will benefit by understanding the importance of the following drivers.

To help illustrate each driver below, we’ve provided an insight of how Contexti client Seven West Media positively addressed each driver, resulting in the delivery of one of the most successful Big Data Analytics projects in Australia.

#1 – Commercial Strategy

What’s the purpose of your big data project? Are you responding to business disruption? Are you leading with innovation? Which revenue lines have been impacted and which revenue lines do you intend to impact? Having a clear commercial strategy with the right narrative will help to align team members, limit initial scope and set the bar for getting a return on investment. No clear commercial strategy, a poorly defined strategy or the common ‘proof of concept’ approach of ‘let’s throw all our data in a data lake and see what we find’ will deliver a tyre kicking exercise, resulting in wasted time, money and missed opportunities.

Contexti Client Example: Seven West Media recognised ‘television was no longer just in the living room’ and they were now in the business of ‘quality content on any device, anywhere, any time’. This headline narrative guided the commercial strategy for Seven West Media’s big data project which was to find innovative and effective ways to engage with audiences on any platform. Engaged audiences directly correlate to increased revenue.

#2 – Defined Actions

Aligned to your commercial strategy, you need clarity on how you will take action with your new-found insights. Worse than not having the capability to find new insights from your data is when you have the insights but you’re unable to take actions on those insights. This require to step away from the data and think about systems, people, processes, customers and partners that will be impacted when you take action with your new insights.

Contexti Client Example: Seven West Media during the 2016 Rio Olympic Games leveraged their big data platform to find new audience insights and to personalise the viewer experience. Leveraging the insights they executed on a marketing program that delivered 2.7 million emails across 108 targeted campaigns. By defining and acting on their insights they achieved an ROI of a 29% lift in the average minutes streamed by users who were part of this targeted program.

#3 – Executive Engagement

All projects need executive sponsorship and support to get basic budget sign off, however some projects significantly benefit by having executives more actively engaged with the project. Big data projects fall into this category. With the right engagement, executives will provide headline messaging across the organisation of why this project exists, what are the benefits and importantly why departmental colleagues should pro-actively enable and support this project. There will come a time when requests will be made to departmental colleagues to make data available, make people available and make their expertise available and without the right mandate supported by an engaged executive your project is on-hold in the best case or dead in the extreme case.

Contexti Client Example: Seven West Media’s big data project was sponsored by its Chief Commercial Officer who was aware, active, vocal and visible across the project. With an engaged executive sponsor, the Director of Data and Business Intelligence who was driving the project had mandate to align internal departments and external services providers, ultimately delivering a successful project.

#4 – Team Composition

When considering the team composition for big data projects, many will be able to quickly identify the Data Scientist role, but will struggle to name other roles. Many project failures have been a result of unallocated or misallocated resources. A Data Scientist is a Data Scientist, they are not Platform Architects, Platform Engineers, Platform Administrators, Data Architects, Data Engineers, Commercial Strategist, Change Managers, Trainers or industry subject matter experts… to name just a few of the roles that need to be filed in a successful project. If you start with getting clarity on your commercial strategy (Driver #1) and work through how you will action your insights (Driver #2) then that should be a good start to thinking through your team composition. Of course depending on the size of your project and complexity of your mandate and budget you will take a different approach to composition of your team. Some roles may be internally filled, some will be full time roles, others will be part time roles, you might supplement your core team with contractors or you might outsource some functions to third party service providers. Key point here is don’t just hire a Data Scientist then wonder why the project failed.

Contexti Client Example: Seven West Media’s big data project team was comprised of many internally filled roles (Director of Data and BI, Data Architect, Data Scientist, Data Analyst), they also also engaged with other internal departments (e.g. marketing who executed on the targeted email campaigns) as well as third party providers such as Contexti who took care of the design, build and management of their big data platform (meaning Platform Architecture, Platform Engineering, Platform Administration, Data Engineering and Support was outsourced).

#5 – Technology Choice

Your big data solution needs to be powered by an appropriate technology that supports your commercial objectives and use cases (Driver #1). Further, the technology needs to be supported by a appropriately skilled, qualified and enabled resources (Driver #2). Too often we’ve seen technology choices being made in the absence of commercial objectives, use cases and clarity of how the technology will be supported in a production environment. What’s worse is we’ve also seen technology choices and decisions being made because of internal politics, pre-existing vendor relationships and by people who are not close enough to the issues of the project. This not only sets up the project for failure from a technology perspective, it also causes the project team to lose faith in the leadership and decision making process.

Contexti Client Example: Seven West Media had clarity on their technology choices. Leveraging the project leadership team’s prior experience and with clarity of their commercial objectives they identified there were two key challenges that they would focus on. Firstly, the variety and complexity of the data sources drove their decision towards selecting a mature, secure and scalable enterprise-grade data management platform at the core of their data solution (they selected Cloudera). Secondly, given the highly variable nature of the events they were covering (such as the Australian Open, Wimbledon, Rio Olympic Games etc) with variable amounts of traffic and data, they needed the ability to scale up and down in a flexible manner, driving their decision for a cloud-based solution (they selected Amazon Web Services). Further looking at the team composition, their skill-sets and familiarity with existing tools, the overall solution also integrated with Teradata on AWS, Tableau for visualisation and R for advanced analytics.

#6 – Agile Approach

With the first five drivers in place (Commercial Strategy, Defined Actions, Executive Engagement, Team Composition and Technology Choice) you want to execute effectively and demonstrate quick wins. Adopting an agile approach will enable to you test and learn, share the lessons-learned and move to your next iteration to meet your commercial objectives. Without these first five drivers in place, it’s difficult to be agile and we’ve seen many organisations fail as they have talked themselves into building what they perceived to be ‘the biggest, most flexible, most secure big data platform that could solve any big data use case’ but in reality solved none. These non-agile projects lost momentum and ultimately failed having spent so much time waiting for hardware to arrive, waiting for integrators to spend months building and securing their big data platforms, waiting for their internal departments to make data available, waiting for the newly hired data scientists to start and the list goes on. Without all six drivers in place all involved with the project had no sense of urgency, purpose or accountability.

Contexti Client Example: Seven West Media adopted an agile approach from day one. Within six weeks of Contexti being engaged (to design, build and manage a big data solution) an initial pilot program was run to support the coverage of the 2016 Australian Open Tennis event. Following that the solution was expanded to support the 2016 Wimbledon Tennis event, then more work was done to scale up the solution for Seven West Media’s coverage of the 2016 Rio Olympic Games in Australia.


By: Sidney Minassian – Founder & CEO, Contexti – Big Data Analytics


Subscribe for updates on the latest big data analytics training courses, industry events and career insights.

5 Tips On How To Land A Big Data Job In Australia

In today’s Australian job market if you’ve got some Big Data experience, you’re mostly likely getting approached by recruiters and are probably spoilt for choice. If that’s the case, you don’t need to read on. This article is for the rest of you, who have heard about this ‘Big Data’ thing and are wondering how to get your foot in the door.

Given Contexti’s focus on the Big Data Analytics market in Australia, we’re fortunate to be aware of and in many cases involved in a broad range of Big Data Analytics related conversations, deals, projects, partnerships, hires, fires and events across Australia. The single biggest challenge we constantly hear about is the shortage of qualified and experienced ‘Big Data’ people.

While we don’t advise our customers to drop their standards in the quality of their hires, we do strongly warn against holding on to the belief that there is a magical unicorn big data guru out there. Instead we suggest organisations hire professionals with the right fundamentals (e.g. fit for culture & values, coachable, possess skills in certain technologies or analytics methods, etc) and implement a plan to develop them into capable Big Data Analytics practitioners.

Similarly we’ve found ourselves having conversations with a broad range of professionals, some who are just starting our their careers and are thinking about graduate roles while others with decades of experience who now want to transition into a career in the growing Big Data Analytics space.

Like everything else in business and life there are no silver bullets, but if you approach this in a strategic and tactical manner, you will massively improve the odds in your favour.

So here are five tips to help you land a big data job in Australia:

#1 Define your target role

While ‘data scientist’ sounds like an exciting role, it may not be the right entry point for you. You want to get into a role where you will learn and where you will also quickly add value by bringing something to the table. To do this, think about your ‘home ground advantage’, what skills, experience or connections do you already have and map it to the closest Big Data role in the an industry most suitable to you.

Some real-world examples we are aware of:

  • Our own Damion Reeves at Contexti transitioned from being an experienced Database Administrator (DBA) with years of experience in infrastructure, Oracle and SQL to a Big Data Platform Engineer. While Hadoop and Spark were technologies he needed to learn, his underlying experience with Linux and UNIX, capabilities in shell scripting and knowledge of enterprise support and service protocols were immediate value-adds to Contexti and to our customers.
  • Our client Sharmaine Salis Head of Data Architecture at Seven West Media transitioned from a traditional Business Intelligence / Data Warehouse solutions role into a Big Data / Cloud Architect role, leading one of the most successful big data projects in Australia which underpinned Seven’s Rio Olympics games coverage.
  • One of our Hadoop & Spark training students, MingJian Tang currently a Cyber Security Data Scientist at the Commonwealth Bank of Australia (CBA), transitioned into this role from a statistics and data mining background.
  • Broader in the field we’ve seen someone with solid Telco background move into a Big Data Strategy role for one of the Telcos trying to monetise their data assets.

So the take-aways are:

  • There are many potential Big Data Analytics roles (Commercial Strategist, Platform Architect, Data Architect, Platform Engineer, Data Engineer, Analyst, Data Scientist, Project Manager, Quality Assurance, Sales, Business Development, Customer Success etc).
  • No one-person will be qualified to do all the available roles in Big Data.
  • Find your home ground advantage and target a role that gets you excited and one where you can add value quickly.

#2 Skill up

You will massively improve your chances in landing a role if you’ve invested in skilling yourself up. The one obvious benefit is the theoretical and in many cases the practical knowledge you will gain by attending formal training. The other not so obvious benefit is the network of relationships you will create with the instructor and other class participants. Depending on the role, your budget, time availability etc there are many courses to take advantage of. Here are some of the short to long training and certification programs we are aware of:

#3 Network

An important factor in landing a new role is ‘who you know and who knows you’. Networking enables you to to build relationships, get known, learn something new and contribute. There are many meetup groups and networking events. Here are some of the ones we attend:

#4 Be found

There are many ways to get your name out there and to be found. Speaking at events and meet ups, writing guest blogs posts, publishing your work in online forums (GitHub, Slideshare etc), getting active on Twitter and Quora. The simplest and most obvious one however is to put effort in your LinkedIn profile. After you consider your target role and your home ground advantages (existing skills, industry experience etc) as well as your training and up-skilling strategy, you should update your LinkedIn profile.

Your profile should be authentic. This means stating correctly what you have done, skills you possess and how much experience you actually have. Further an authentic profile should include objectives, aspirations and current activities you are undertaking to improve yourself giving the potential recruiter an idea of not only where you’ve been but a view of where you are headed.

A recent example was when I was doing a search on LinkedIn for anyone who had included “Data Science” in their profile. I came across a professional who had recently completed a data science course in addition to having a math and statistics major and hands-on actuarial work experience. His LinkedIn headline said ‘Aspiring Data Scientist’. The word ‘aspiring’ gave me an indication of where he was headed and what he was looking for yet it was authentic as he wasn’t claiming to be an experienced data scientist.

This approach can be applied to your LinkedIn headline and your summary where you can include your ‘elevator pitch’ of who you are, where you’ve been, what your great at and where you are heading.

#5 Look for early signals

To narrow down your targeting efforts and improve your odds, look for early signals that might lead you to a future job opportunity. Typically this will be keeping your eyes open on LinkedIn, subscribing to relevant industry news and blogs, reading mainstream business and technology news and being an active networker. Early signals you should keep your eyes on include: companies announcing changes in strategy, appointment of new leaders, new partnerships or vendors winning contracts.

For example in the last six months in Australia there have been a number of executive movements in the Chief Data Officer and Chief Digital Officer roles, this kind of appointment usually indicates a company is reprioritising ‘data’ as a strategic priority and is usually followed by a restructure and a recruitment drive. There have also been a number of public announcement of data deals and data partnerships as well vendors announcing contracts with new customers or publishing case studies of success stories with existing customers.

All of these are early signals that will give you hints on people, companies, technologies and deals to follow and target in order to land your next big data job.


By: Sidney Minassian – Founder & CEO, Contexti – Big Data Analytics


Subscribe for updates on the latest big data analytics training courses, industry events and career insights.