Basic of Big Data Analytics

The rise of Big Data concept is rapidly engulfing the information world. Basic of Big Data Analytics

Predictive Data Analytics

Arguably, predictive analytics is the most commonly used category of data analytics as it is used to identify trends, correlations, and causation. The category can be further broken down into predictive modelingand statistical modeling. But, it’s important to know that these two really go hand in hand.

Prescriptive Data Analytics

Prescriptive analytics is where AI and big data meet to help predict outcomes and what actions to take. This category of analytics can be further broken down into optimization and random testing. Using advancements in machine learning, prescriptive analytics can help answer questions like “What if we try this?” and “What is the best action” without spending the time actually trying out each variable.

Basically, it can help you test the right variables and even suggest new variables with a higher chance of generating a positive outcome.

Diagnostic Data Analytics

Diagnostic data analytics help answer why something occurred. Like the other categories, it too is broken down into two even more specific categories: discover and alerts and query and drilldowns.

Query and drilldowns are what you’ll use to get more detail from a report. For example, let’s say that one of your sales reps closed significantly fewer deals last month. A drilldown could show fewer work days, reminding you that they had used 2 weeks vacation that month explaining the dip.

Descriptive Data Analytics

Descriptive analytics are the backbone of reporting—it’s impossible to have BI tools and dashboards without it. It addresses your basic how many, when, where, and what questions. Once again, this can be further separated into two categories: ad hoc reporting and canned reports. A canned report is one that has been designed previously and contains information around a given subject. An example of this a monthly report sent by your ad agency or ad team that details performance metrics on your latest ad efforts.

Intelligence Processing

Comprehension of the various stages of the analytical process from collation, through evaluation, analysis and integration to the production of an assessment

Analytical techniques will encompass: the use of analytical software, statistical analysis, temporal, geospatial and link analysis techniques

Understanding the biases and fallacies that can affect their products and the techniques that can mitigate biases and fallacies

The Analytic Process

Information Access > Quality Verification
Analysis > Contextualise
Actionable Knowledge > Impact Measurement

Information Access & Quality Verification

The Information Age
Uncontrolled Information
The Information Fog

Analysis

Derive meaningful substance from information.
Provide the right context in which that information can be used.
Determine the value of that information.

The context creation

These six points are essential to make any information meaningful.

These points create contexts

These points keep a rational mindset intact.

These points help breakdown large chunks of information and accommodate fair analyses.

Actionable Knowledge

The knowledge must be actionable.

It ensures the credibility of the information gathering process.

It helps maintain standards within the sector.

Impact Measurement

The impact has to be measured against pre-defined benchmarks.

This is how the success of the entire exercise is determined.

Analytical Techniques

SWOT

Mind Mapping / Brain Storming

PESTEL

SWOT

MIND MAPPING / BRAIN STORMING

PESTEL

PROCESSING

Waterfall
Agile

There are two main approaches to Intelligence Analysis processing.

Advantages of Waterfall Model

It is one the easiest model to manage. Because of its nature, each phase has specific deliverables and a review process.
It works well for smaller size projects where requirements are easily understandable.
Faster delivery of the project (this is questionable!)
Process and results are well documented.
Easily adaptable method for shifting teams
This project management methodology is beneficial to manage dependencies.

Limitations of Waterfall Model

It is not an ideal model for a large size project
If the requirement is not clear at the beginning, it is a less effective method.
Very difficult to move back to makes changes in the previous phases.
The testing process starts once development is over. Hence, it has high chances of inaccuracies to be found in the final product, where they are expensive to fix/alter.

AGILE (Came Into Existence in 2001 and Recently Became Part of the Intelligence World)

Advantages of the Agile model

It is focused process. So, it makes sure that the director is continuously involved during every stage.
Agile teams are extremely motivated and self-organized so it likely to provide a better result from the development projects.
Agile software development method assures that quality of the development is maintained

The process is completely based on the incremental progress. Therefore, the director and team know exactly what is complete and what is not. This reduces risk in the development process.

Limitations of AGILE Model

It is not useful method for small operations.
It requires an expert to take important decisions in the meeting.
Cost of implementing an agile method is little more compared to other methodologies.
The project can easily go off track if the project manager (director) is not clear what outcome he/she wants.
Initial results can creates doubts about the entire operation

Bias and Fallacies

the biases and Fallacies that can affect IA

A cognitive bias is a systematic pattern of deviation from rationality, which occurs due to the way our cognitive system works. Accordingly, cognitive biases cause us to be irrational in the way we search for, evaluate, interpret, judge, use, and remember information, as well as in the way we make decisions.

Theses could be pre-conceive ideas that point towards a conclusion.

Do you see what you see or do you see what you WANT to see?

Imagine thousands of hours, engagement of resources and workforce that counts for nothing because of a bias!

How to Minimize these Biases

A system based approach!
An objective mindset
Cross checking in place
Debrief and Feedback

Practical Application

Develop skills in written, graphic and verbal communication that enhance product dissemination
Understanding the best practices and skills needed for working in an intelligence team
Understanding the role of intelligence on operational planning