3 Strategies For Getting The Most Value From Your Data Lake

“Big Data’ and ‘data lake’ only have meaning to an organisation’s vision when they solve business problems by enabling data democratisation, re-use, exploration, and analytics.” – Carlos Maroto: Technical Manager at Search Technologies.

A data lake is a storage repository that acts as the central source of all your organisation’s current and historical data, both structured and unstructured. This data is transformed as it moves through the pipeline for things such as analysis, creating quarterly and annual reports, machine learning and data visualisation. The information contained in a data lake can be highly valuable asset, however, without the right structure, your data lake could turn into a data swamp.

Here’s three strategies for getting the most value from your data lake.

 

#1 – BUSINESS STRATEGY & TECHNOLOGY ALIGNMENT

“It’s important to align goals for your data lake with the business strategy of the organisation you’re working to support.” – Bizcubed.

What are the business goals you’re trying to achieve with your data lake? Operational efficiency? Better understanding of your customers? Will your current infrastructure help you achieve this while also maximising your profits? Aligning your goals with the technology you’re planning to implement will not only help you articulate what problem you’re trying to solve, but also improve your chances of gaining executive buy-in and winning the support of your team. The better the plan, the easier it is to identify possible roadblocks and the higher the chance of success.

“As technology teams continue to be influenced by the hype and disruption of Big Data, most fail to step back and understand where and how it can be of maximum business value. Such radically disruptive new business processes can’t be implemented without knowledge gathering and understanding how Big Data technology can become a catalyst for organisation and cultural change.” – Thierry Roullier: Director of Product Management at Infogix, Inc.

 

#2 – INTEGRATION & ARCHITECTURE

“You need to be able to integrate your data lake with external tools that are part of your enterprise-wide data view. Only then will you be able to build a data lake that is open, extensible, and easy to integrate into your other business-critical platforms.” – O’Reilly.

Technology is moving at a rapid place.The tools you use in your business may not cooperate well with your data lake, and may not support the data architectures of tomorrow. During the implementation process, one of the first things to look at is how adaptable your long-term technology investments are.

Big Data architectures are constantly evolving, and it’s important to select flexible data processing engines and tools that can handle changes to security, governance and structure without being too costly to the organisation. Before implementing anything, you need to have a clear vision of what you want the end technical platform to look like, and what components you will need to make that happen.

“Modern data onboarding is more than connecting and loading. The key is to enable and establish repeatable processes that simplify the process of getting data into the data lake, regardless of data type, data source or complexity – while maintaining an appropriate level of governance.” – Bizcubed.

 

#3 – DATA VIRTUALISATION & DEMOCRATISATION

“ Data virtualisation involves abstracting, transforming, federating and delivering data from disparate sources. The main goal of data virtualisation technology is to provide a single point of access to the data by aggregating it from a wide range of data sources.” – TechTarget.

Data lakes and data virtualisation tools work well together to solve different problems and provide a layer of intelligence that results in more agility and adaptability to change.

“ As an example, a virtual layer can be used to combine data from the data lake (where heavy processing of large datasets is pushed down) with golden records from the MDM that are more sensitive to stale copies. The advance optimisers of modern data virtualisation tools like Denodo make sure that processing is done where it is more convenient, leveraging existing hardware and processing power in a transparent way for the end user. Security and governance in the virtual layer also add significant value to the combined solution.” – datavirtualizationblog.com.

Data democratisation is the ability for information in a digital format to be accessible to the average end user. The goal of data democratisation is to allow non-specialists to be able to gather and analyse data without requiring outside help.

“Data must be freed from its silos. Today, it resides in a variety of independent business functions, such as HR, manufacturing, supply chain logistics, sales order management and marketing. To get a unified view of this data, businesses are engaging in a variety of ad-hoc, highly labor-intensive processes.” – Computer Weekly.

 

For more resources, please see below:

Best Practices For Data Lakes

How To Build A Successful Data Lake

Five Keys To Creating A Killer Data Lake

Avoiding The Swamp: Data Virtualisation & Data Lakes

Democratising Enterprise Data Access: A Data Lake Pattern

How To Successfully Implement A Big Data/ Data Lake Project

Top Five Differences Between Data Lakes & Data Warehouses

 

Data Empowering Artificial Intelligence & Machine Learning

#1 – FASTER & SMARTER DECISIONS

Digital transformation through Artificial Intelligence has led to more agile, productive and smarter businesses. Automation and machine learning are helping companies save time and money, personalise customer service and detect fraud while also improving work processes and expanding top-line growth.

“Artificial Intelligence or AI, has become pervasive in business in every industry where decision making is being fundamentally transformed by Thinking Machines. The need for faster and smarter decisions and the management of Big Data that can make the difference is what is driving this trend.” – James Canton: CEO & Chairman of The Institute of Global Futures.

 

#2 – DATA-DRIVEN AI & MACHINE LEARNING

“Machine Learning is a current application of AI based around the idea that we should really just be able to give machines access to data and let them learn for themselves.” – Bernard Marr: Founder & CEO of Bernard Marr & Co.

With data science reaching new capabilities for industry disruption, the correlation of data and Artificial Intelligence has powerful potential; and with advancements in machine learning becoming more accessible, it can now be applied to resolve actual business problems.

“The ability to access large volumes of data with agility and ready access is leading to a rapid evolution in the application of AI and machine-learning applications.” – Randy Bean: CEO of NewVantage Partners.

 

#3- ARTIFICIAL INTELLIGENCE VS. HUMAN INTELLIGENCE

“There have been multiple reports recently which claim that a major part of the human workforce will be replaced by automatons and machines in the years to come. With excessive research and development being conducted in the field of artificial intelligence, many fear that a major job crisis will unfold since multiple jobs are more accurately and efficiently performed with the utilisation of machines.” – Brent Morgan: Founder of Transcendent Designs LLC.

With all the benefits of Artificial Intelligence comes the growing fear of job crises. Will AI help or hinder our career opportunities? All though it’s hard to argue the fact that intelligent machines are in fact reliable when it comes to logical decision-making, there are still aspects of human intelligence that machines cannot mimic, like our emotional intelligence. Some argue that the combination of Human Intelligence and Artificial Intelligence will create more opportunities, not less.

“Machine Intelligence can help augment people to do their jobs by making them smarter in a situation, make better decisions, and offer greater engagement with customers.” – Charles Babcock: Editor at Information Week.

“HI is what defines us as humans and our relationship with everything on earth. Now, through the combination of HI and AI, we are at the brink of intelligence enhancement, which could be the most consequential technological development of our time, and in history.” – Bryan Johnson: Contributor at Techcrunch.

 

#4 – KEY TECHNOLOGIES

“The market for Artificial Intelligence (AI) technologies is flourishing. Beyond the hype and the heightened media attention, the numerous startups and the internet giants racing to acquire them, there is a significant increase in investment and adoption by enterprises.” – Gil Press: Managing Partner at gPress.

Australian startups such as Aipoly, creators of an app that combines image- recognition algorithms with smartphones to give instant feedback on surroundings for the visually impaired, have made a huge impact using Artificial Intelligence.

“People have told us that they’ve just started crying when they used it. They’ll say, ‘I have 200 apps on my phone and none of them have made the difference in my life that Aipoly has. It’s an amazing impact you can have on the life of someone that can’t see.” – Marita Cheng: Co-Founder of Aipoly.

There are a variety of AI tools and technologies taking the world by storm, among the most popular being deep learning platforms that provide algorithms such as FluidAI & MathWorks, biometrics for image and touch recognition like Affectiva and 3VR, and natural language processing tools used for fraud detection like Coveo and Sinequa.

 

For more resources, please see below:

The Business of Artificial Intelligence

Top 10 Hot Artificial Intelligence (AI) Technologies

These Emerging Technologies Will Play Critical Roles

Artificial Intelligence: Can It Replace Human Intelligence?

Data To Analytics To AI: From Descriptive To Predictive Analytics

How Big Data Is Empowering AI & Machine Learning At Scale

8 Ways Machine Learning Is Improving Companies’ Work Processes

Big Data & IoT Benefit From Machine Learning, AI Apocalypse Not Imminent

Meet The Australian Startup Using Artificial Intelligence To Help Blind People See

The Combination Of Human & Artificial Intelligence Will Define Humanity’s Future

Meet The Startups That Bring Artificial Intelligence To Log Management & Analysis

Six Reasons Why Data Visualisation Is The Future Of Storytelling

“A picture is worth a thousand words – especially when you are trying to find relationships and understand your data.” – SAS.

Gone are the days when businesses struggled to extract meaning from mass volumes of raw data and make insights come to light. Now, with the increasing number of data visualisation tools in the market, businesses are making faster and smarter decisions by engaging with data that’s easy to access and easy to understand.

Here’s 5 reasons why data visualisation is the future of storytelling.

 

#1- SIMPLICITY IS POWER

“Well designed data graphics are usually the simplest and at the same time, the most powerful.” – Centerline Digital.

Let’s say you’ve been given a short time frame to analyse thousands, or even millions of rows of numerical data, and make decisions based on the insights you’ve generated. How are you going to make it happen? Businesses have responded to this challenge by transforming data into visual content.

“Data visualisation turns numbers and letters into aesthetically pleasing visuals, making it easy to recognise patterns and find exceptions.” – Norman Shamas: Former Director of Curriculum at TechChange.

 

#2- CAPTURING ATTENTION

“The huge amounts of content available on the internet has significant implications for the modern day consumers’ attention span.” – Rob Weatherhead: The Guardian.

Speed, convenience and instant gratification enabled by technology are must-haves for today’s consumers. While trying to keep up with what’s in-demand, businesses have discovered the impact of visualising content in retaining engagement.

“Data visualisation excels in capturing a viewer’s attention and holding it through storytelling. It addresses a complex problem that could be easily looked over, and simplifies it using design.” – Elsa Wong: Author at Bridgeable.

 

#3 – SPEED & PRODUCTIVITY

“The faster a company can generate information from their sources, the faster they can generate their answers.” – Jessica Folliett: Author at Dataversy.

As the old saying goes, time is money. Many businesses struggle with the dilemma of whether or not to invest in Big Data due to the time and cost involved in generating insights. However, recent reports from Prysm Inc. show that data visualisation has significantly improved the decision making process when it comes to investing in data analytics.

“80 percent of organisations report more accurate decision making when using data visualisation tools. Further, 86 percent of companies report faster decision making through data visualisation, while 81 percent noted that the tools improved customer behavior insights.” – Prysm Inc.

 

#4 – UNDERSTANDING WHAT’S IMPORTANT

“The best data visualisations are ones that expose something new about the underlying patterns and relationships contained within the data. Understanding those relationships — and being able to observe them — is key to good decision making.” – Julie Steele: Author at O’Reilly.

Across many industries, data visualisation is being used to uncover hidden patterns and predict future trends and behaviors, helping businesses prepare for what’s to come.

“The primary objective of visual design is to present content to your readers in a manner that highlights what’s important, arranges it for clarity and leads them through it in a sequence that tells the story best.” – Stephen Few: Show Me The Numbers.

 

#5 – BREAKING THROUGH THE NOISE

“As we progress further into being an information society, data visualisation is becoming increasingly valuable.” – Elsa Wong: Author at Bridgeable.

In a world saturated with ever-increasing data and information, it’s now more important than ever for organisations to recognise what’s relevant and what isn’t. By combining data with creative visuals, businesses are able to find those golden nuggets of information amidst all the clutter.

“The beauty of data visualisation is that you can communicate far more information in a more digestible package than you can with text alone. While that doesn’t mean you have to use every piece of data at your disposal, you can tell a rich story in a single visualisation.” – Katy French: Author at Business 2 Community.

 

#6 – PERSUADING ACTION

“We’ve seen rapid growth in visualising as much data as possible to tell stories and present business results.” – Thomas Powell: CEO of ZingChart.

When done right, data visualisation tools can help you create engaging presentations that hold persuasive power with investors, managers and executives alike. Not only is visual data more engaging, it also helps you tell the strongest story and encourage a call-to-action.

“Unique and truly compelling visualisations are an underused, yet devastatingly effective tactic. They are equal parts rare and in demand.They are a catalyst for conversation, awareness, and action.” – Adam Singer: Author at ClickZ.

 

For more resources, please see below:

Storytelling With Data

Why Data Visualisation Matters

Why Data Visualisation Is Important

The Importance of Data Visualisation

Data Visualisation: What It Is And Why It Matters

Big Data Dilemma: Save Money Vs. Make Money

The Importance Of Big Data And Data Visualisation

How Time-To-Insight Is Driving Big Data Business Investment

7 Data-Storytelling Tips From Centuries-Old Data Visualisation

Data Visualisation: Your Secret Weapon In Storytelling and Persuasion

Say It Quick, Say It Well – The Attention Span Of A Modern Internet Consumer

Data Visualisation Holds The Key To More Efficient Decision-Making, New Report Reveals

The Increasing Importance of Data Visualisation: An Interview With ZingChart’s Thomas Powell

Rebounding From A Failed Venture & The Contexti Origin Story – An Interview With Sidney Minassian on Founder to Founder Podcast

Phil Hayes-St Clair, from Founder to Founder podcast recently interviewed Contexti – Big Data Analytics Founder & CEO Sidney Minassian about his entrepreneurial rollercoaster that started in 2000.

Prior to launching his current venture Contexti, Sidney built a project management and workflow software company, Think Software, serving financial markets, construction and professional services industries. He then moved to Silicon Valley, USA to launch his next venture, Liaise, a platform that used natural language processing of unstructured data in emails to improve personal and team productivity.

In this podcast, Sidney talks about:

  • Resilience
  • Learning from failure
  • The value of owning and accepting outcomes
  • The recovery process between ventures and how to reflect, recoup and rebuild
  • Why building a venture is all about people
  • Looking for scalable, repeatable business models
  • The Contexti origin story – positioning as a niche player in the emerging Big Data industry.

 

If you liked this episode of Founder to Founder, follow Phil Hayes- St Clair on soundcloud or download on iTunes.

Contexti Big Data Analytics Training

 

Request Training Course Syllabus

[activecampaign form=7]