Saturday, November 26, 2016

Data Analytics: - Are you ready for the Big Data era?


Many organizations are interested in leveraging their data in the purpose of decision making, exploring new opportunities and workflow optimization todays.  However, as the managements, do you know if your organization ready for this?  Or if you have already started your Big Data analytics plan, but it just not too effective as expected?  As a data consultant, the following are the three of the most common road-blocks of applying data analytics into the reality I saw in my experiences: -

1.      Too much technologies exists

New technologies appear every day.  Companies are normal in having multiple platforms together, include proper database driven platform and tailor made applications, where some of these are indeed overlapping in both functionality and information contained.  These indeed create data integrity issues and it would be a real headache when the data scientists try to gather the analyzable data and then resulting in false analytics insights and conclusion.  As to resolve this issue, my suggestion is that it would be always good to have an ETL data expert working together with the business managements and those subject matters expert as to consolidate and put the data into a proper, extendable and analyzable format before rolling out any Big Data analytics plan.  Put another way, data completeness and readiness would always be the very first thing to concern, confirm and establish before moving into any real analytics battles.

2.      Lack of business overview

This usually happened in large organizations especially for those organizations which are running with Target Operating Model (TOM).  Put another way, managements today in an organization are all talents and experienced subject matter expert but might too focus on their sole responsibilities, e.g. Procurement are not understand Manufacturing and, on the other hand, Manufacturing are not friends with sales and also logistic.  The fact is that, in most cases, there is no one know the true picture of the business from top to bottom from in to out.  Although this might be reasonable since it’s difficult to understand everything in the reality, this would still create significant impact on the accuracy and effectiveness of the analytics results.  As to resolve this issue, I believed that a proper business plan and a very clear business goal of the data analytics projects would definitely be essential for both the internal industrial subject matter experts the data scientist as to identify the right and appropriate analytics direction and approach.  Without a deep understands of the business, I believed that it is no way to be succeeded in any of the analytics projects.  And that’s why a successful analytics deployment would always require both the business and the data scientist to work closely together while I saw most the time these two parties appear to be in conflict.

3.      Lack of data compliance policy

People intend to use the easiest and the most effective ways to perform their jobs.  However, what is the easiest and the most effective are usually subjective.  For example, some people like to keep track of the data with Excel but others like using a giant databases or simple hard copy documents.  In fact all these are good in the normal circumstances as long as its helps in driving the business growth in my point of view.  However, if any unexpected things happened, for example regulatory requested look back review that happened quite often in banking industry these days, issues would then arise.  As to avoid this, I believed that it would be never too late to get a proper data compliance policy deployed within an organizations.  However, we must ensure that the data compliance policy are not trying to change or restrict how’s each of the management like to run their team, but a way to ensure that all the information and process are keep track properly in one places.  Otherwise, things deployed but no one follow, it will just again completely useless which I saw in many of the large organizations.  And this demonstrated that why not only the business team and the data scientists, but also the experts in executing deployment are indeed important.


In conclusion, if you believe in data and would like to deploy any data analytics in your organization, it’s not only about the data expert, but also your business team and numbers of parties involved.  Big Data is not magic that will only drive advantages towards you, without a proper plan and understanding Big Data might only create a nightmare for your organization.  If you are interested in learning or discuss more on any of your Big Data plan, please feel free to drop me a note and I am always happy to discuss and help.