Marriage of BI and Big Data – Webcast Summary

I attended this ACM webcast given by Barry Devlin of 9sight Consulting.  It is an update from his webcast last April Why Big Data Analytics needs BI Too

Figure 1: Source: 9sight Consulting

He asked 3 questions.  The first one: What difference does big data make for business?

Today we have to understand, track sentiment, real time insight on web site activities.

Figure 2: Source: 9sight Consulting

Internet of Things (IoT) adds new urgency – a new view what is going on.  It extends existing processes.

Today we have new micro management of supply chains, extension of supply chain all the way to consumer

With the Internet connected refrigerator, the fridge knows milk orders for you

As the news reports, a  refrigerator has already been hacked

It extends existing processes – doing more of the same

New business models are depending on analytics.

Also motor insurance with actuarial work to encourage better driving prevention of accidents

How use data? For ongoing health monitoring with wearable devices

Figure 3: Source: 9sight Consulting

Figure 3 shows the “biz-tech” ecosystem, summarized in a digestible manner

Externally sourced information is in an abundance and variety.  Mobile technology leads to a different level of interaction, interacting with businesses

Today Mobile technology is coming with a lot of information with smart phones and mobile devices

There is extreme flexibility in marketplace – uncertainty, competition

You also want to integrate closely to enable speed and real-time decision making to understand what to do next, long term strategy

Environment and IT work need to work “symbiotically”

Figure 4: Source: 9sight Consulting

Figure 4 shows “process mediated data”

It is created and managed through processes we defined in business

It forms the legal basis of business – this is a transaction, this is a customer

Business value of Big Data relies on relating it to the traditional processes

Figure 5: Source: 9sight Consulting

Figure 5 shows the central question, covered in detail in Barry’s book (see below)

Figure 6: Source: 9sight Consulting

Figure 6 shows a layered architecture from the IBM systems journal in 1988

To the right shows the schematic view, separated data into vertical and horizontal segments, with a uni-directional data flow

It focuses entirely on process mediated data

How get data to flow through quickly enough? How get over operational divide? It is bi directional data flow

Figure 7: Source: 9sight Consulting

Figure 7 shows a conceptual architecture which is the foundation for biz tech ecosystem – simple enough for business people, and deep enough for IT

IDEAL – acronym – Integrated Distributed Emergent Adaptive Latent

Integrate from all parts of business

Distributed – across all the palces

Emergent  – don’t’ know how will evolve

Latent – hidden, not way build architecture – foundation behind what build

Read as a story – People, Process, Information – how do people process information?  Structure it within spaces

Figure 8: Source: 9sight Consulting

Figure 8 shows the way information has to be thought about

X axis – process go through from real time operational data into data warehouses

Paradox between timeliness and consistency

Y axis into the slide – structure and context – “structured and unstructured” structured context deeply interrelated and co-dependent

Z axis – reliance and usage – where get data from, how far rely on it – different

The idea is the possibility to break into different blocks or categories

Figure 9: Source: 9sight Consulting

Real stands for realistic extensible actionable labile

Logical architecture – how build architecture that has the breadth of information that have in world and the demands in real time

Blue is data

Brown is process

It does not contain people – it is not possible for IT to implement people in this sense

Process includes measures, events, turns them into transactions – data stores that creates and assimilates – mixes and matches – makes it available via reification

Data flows through pillars as quickly as needed and is reconciled only when necessary

Three pillars includes machine generated

Process data spans both BI and transaction

Process mediated data will most logically reasonably implemented in a relational model as process mediated data needs to be related to one another

Machine generated – IoT, ACMs, Telcos networks – NoSQL, Hadoop, Streaming

Human sourced – videos, tweets, social networks – Hadoop

CSI stands for context setting information

Figure 10: Source: 9sight Consulting

Online transaction processing system is OLTP

EDW includes standard BI data marts fits into middle pillar

Operational and predictive maps to the third pillar

Figure 11: Source: 9sight Consulting

Figure 11 is Barry Devlin’s book, which was highly recommended to me.  Why leaf growing out of root? Rationality – logical process – think inspiration – move to environment – the human aspect is more important than technology aspect

Figure 12: Source: 9sight Consulting

Figure 12 covers the stealing of personal information – in general

The speaker said we should “Stop using word metadata – data, information, processes, people and information.

NSA has repurposed it

Use the word  context setting information instead.

Figure 13: Source: 9sight Consulting

Figure 13 shows the 1989 Ackoff pyramid

We need a different way

We need to look at relationships between information, knowledge and meaning

At the bottom of Figure 13 shows information, see data – data is information optimized for computers

Information precedes data

There is a relationship between modeling and interpreting data

We move from information – physical objective storage on computers to knowledge in our minds

Knowledge management is nonsensical- it is really Information Management

Figure 14: Source: 9sight Consulting

Figure 14 shows that “Every piece of information adds context to something else”

Figure 15: Source: 9sight Consulting

Figure 15 shows Rodin’s the Thinker

Rationality is only one part of what do

This applies to business and decision makers as we are social animals and business is a social enterprise

Layers of information and process but have to take account what goes on in mind.

Figure 16: Source: 9sight Consulting

Figure 16 covers the dangers of business data

Data brokers create marketing lists

Figure 16 shows an address to “daughter killed in car crash”

Figure 17: Source: 9sight Consulting

Figure 17 shows something more disconcerting (my take)

People wonder how ads came up

The Internet sends back files through the TV – this is data snatching with no control

Figure 18: Source: 9sight Consulting

Figure 18 shows even more of “Invasion of data snatchers:

There is a game device for content delivery device – creating ambient action

The device detects ambient action and identifies an ad with action

It is then presents an identified ad

Figure 19: Source: 9sight Consulting

Figure 19 shows algorithms based on your face

Big data extends BI

It will change the way we do business and our lives

Figure 20: Source: 9sight Consulting

The speaker felt the path we are going down will be the “death of democracy”.

On the right of Figure 20 is Pandora’s box.  The gods gave Pandora box, don’t open –she opened – all ills of world  – tried to close and too late

One thing left in the box was hope

Question & Answer

Q: Definition of REAL

A: Realistic

Extensible – grow to handle

Actionable – do things with it

Labile – flexible – grow and change

Q: Relating mean normalized?

A: Normalized – not necessarily – information in process mediated area is understood how relates together

Nothing against star schemas – give a certain level of speed that you don’t get a normalized database

Resource:

Business unIntelligence: Insight and Innovation beyond Analytics