Business intelligence

Business Intelligence: What It Is, How It Works, Its Importance, Examples, & Tools

https://www.tableau.com/learn/articles/business-intelligence

Please explain, why BI cannot be purchased as an off-the-shelf product, but should always be implemented as an individual solution.

Your argumentation could be based on the following aspects:

(1) BI applications need to be integrated in and tailored to the company-specific context – as all

IT systems. This includes the adaptation of technical components, but primarily refers to

aligning the BI system with corporate culture, structural and organizational conditions, as

well as socio-psychological aspects.

(2) Contrary to statements made by some vendors, BI solutions cannot be purchased as finished,

pre-configured products. Only tools or basic solutions (templates) for building company-

specific solutions are available on the market.

Which of the following properties apply to analytical data management?

Data are organized and modeled according to topics or application areas.

  • Existing data are primarily supplemented rather than overwritten.

    Correct answer.
  • Stores mostly aggregated data.

Please name serious problems that may occur when granting management support systems direct access to data bases for operational transaction processing systems.

Major problems include:

1. Interfering with the operational systems‘ performance, as management support often requires complex queries, which are hardly known in advance.

2. Data of various operational systems are not harmonised and typically inconsistent.

3. For every management support application, processes for data extraction and integration must be designed and implemented.

Which steps are necessary when transforming operational data into managerial information?

Major steps of transforming operational data for BI purposes:

1. Filtering:

Extracting data from several source systems as well as cleansing syntactical and semantical defects present in the source data;

2. Harmonization:

Syntactical and semantical alignment of filtered data from separate sources as well as their representation on a unified level of granularity;

3. Aggregation:

Summarising data in accordance with defined aggregation paths (e.g., by totalling or averaging);

4. Augmentation:

Enhancing the database by calculating business metrics and KPIs.

Designate some examples for cases, where automatic detection and automatic correction of data defects are required.

Examples for a):

• Re-coding of units, e.g. transforming „€“to „EUR“;

• Re-coding of number representations, e.g. transforming character strings for age statements into numeric values;

• Changing attribute lengths, e.g. by using longer fields for serial numbers;

• Filling missing data by inserting interpolated values, e.g., inserting the average of surrounding months for missing monthly fixed costs values;

• Filling data violating plausibility rules by extracting correct values from additional sources, e.g., when supplier details are missing.

(b) Designate some examples for cases, where automatable detection and manual correction of data defects are required.

Examples for b):

• Exceeding an anticipated maximum value, e.g., when an article‘s sales exceed the available amount on stock;

• Uncommonly high or low sales or cost values;

• Non-matching values of debit and credit;

• Unusual age statements (e.g. „98“ within login details for an online game aimed at teenagers).

Examples for c):

Designate some examples for cases, where manual detection and correction of data defects are required.

• Technically caused errors in the data, e.g. when overwriting historical values by current values;

• Faulty data due to misunderstandings during data capture, e.g., erroneous interpretation of statements concerning future demand values;

• Typing errors, e.g. transposed digits in serial numbers;

• Intentional false statements, e.g. when customer satisfaction ratings are recorded by field representatives;

• Made-up values, e.g. by interviewers during consumer surveys.

Distinguish cases, where syntactical harmonization becomes necessary when integrating heterogeneous data sources.

Typical cases include the following:

1. Inconsistencies concerning keys, i.e., different source systems utilize diverging keys for identifying one object;

2. Diverging data representation, i.e., identical properties are represented by different values (attribute names and semantics are identical, value ranges resp. domains differ);

3. Synonymes, i.e., diverging attribute names represent identical issues (semantics and domains are identical);

4. Homonymes, i.e., attribute names are identical, but describe different aspects.

Which data transformation task is required in the following use case example:

(3) As identifying key attribute of articles, the inventory control system uses an EAN number, but the PPC system (production planning) prefers a serial number

instead.

  • a) Filtering: Automatable detection and automatable correction

  • b) Filtering: Automatable detection and manual correction

  • c) Harmonizing granularity

  • d) Eliminating inconsistencies of keys

    Correct answer.
    Correct! The keys of the articles are represented inconsistently in the source systems. Therefore, consolidation is required.

    Moreover, this case also shows two different levels of abstraction: In inventory control, articles are stored on type level with their EAN, whereas in PPC, every item produced is represented by its serial number on instance level (an individual copy of an article type). Depending on the intended semantics for reporting/analytics, either type level or instance level should be chosen.

     
  • e) Harmonising synonymes

  • f) Semantical harmonisation

  • g) Augmentation

  • h) Harmonising domains

Which data transformation task is required in the following use case example:

(4) The company lately acquired by our group has quantity-depending direct costs of sales (e.g., freight

charges) already included in the values of the „net yield“ attribute. All other group companies do not include these individual costs.

  • a) Filtering: Automatable detection and automatable correction

  • b) Filtering: Automatable detection and manual correction

  • c) Harmonising granularity

  • d) Eliminating inconsistencies of keys

  • e) Harmonising synonymes

  • f) Semantical harmonisation

    Correct answer.
    That's right. The stored values have different semantics: In the newly acquired company, direct costs of sales are included, but in the other departments, this has never been done. In such cases, the agreement will usually be to chose the common way of calculation. Alternatively, BI systems could also keep two different versions of cost values for all departments.

Which data transformation task is required in the following use case example:

(5) Due to a technical defect, the call centre in Cottbus is missing data on call durations for month 5/2017. For

missing time data, an approximation by averages is commonly accepted.

a) Filtering: Automatable detection and automatable correction

Correct answer.

This is correct. Missing values can be detected automatically. If, as in this case, filling in averages is considered acceptable, then this task is also automatable.

Which data transformation task is required in the following use case example:

(7) For a particular private customer, sales figures for June 2017 are as high as 18,000 EUR for product group „Various accessories for vacuum cleaners“. However, this

value usually does not exceed 1,000 EUR.

  • a) Filtering: Automatable detection and automatable correction

  • b) Filtering: Automatable detection and manual correction

    Correct answer.
    That's right. Detecting unusual values or outliers is automatable with appropriate methods. Correcting such values has to be done manually, as here, some research will be required: What are the reasons for such high values for accessories -  and how should this case be treated?

Which data transformation task is required in the following use case example:

(8) For each trading partner, the ERP system captures in the attribute „PARTNER“ the German name of the country where their headquarters are located (e.g., „Frankreich“). The data from marketing research that is to be integrated identifies countries by their ISO-3166 code (e.g., „fr“) in

the attribute called „PARTNER“.

  • h) Harmonizing domains

    Correct answer.
    Good! Here we have non-matching data representations/encodings. The domains of the sources have to be harmonized.

Data Transformation Defined

The ever-increasing volume of data offers your business limitless opportunities to make better decisions and improve results. But how can you take what you know about your business, customers, and competitors and make it more accessible to everyone in your enterprise? The answer is data transformation.

Data Transformation Defined

Data transformation is the process of converting data from one format to another, typically from the format of a source system into the required format of a destination system. Data transformation is a component of most data integration and data management tasks, such as data wrangling and data warehousing.

One step in the ELT/ETL process, data transformation may be described as either “simple” or “complex,” depending on the kinds of changes that must occur to the data before it is delivered to its target destination. The data transformation process can be automated, handled manually, or completed using a combination of the two.

Today, the reality of big data means that data transformation is more important for businesses than ever before. An ever-increasing number of programs, applications, and devices continually produce massive volumes of data. And with so much disparate data streaming in from a variety of sources, data compatibility is always at risk. That’s where the data transformation process comes in: it allows companies and organizations to convert data from any source into a format that can be integrated, stored, analyzed, and ultimately mined for actionable business intelligence.

How Data Transformation Works

The goal of the data transformation process is to extract data from a source, convert it into a usable format, and deliver it to a destination. This entire process is known as ETL (Extract, Load, Transform). During the extraction phase, data is identified and pulled from many different locations or sources into a single repository.

Data extracted from the source location is often raw and not usable in its original form. To overcome this obstacle, the data must be transformed. This is the step in the ETL process that adds the most value to your data by enabling it to be mined for business intelligence. During transformation, a number of steps are taken to convert it into the desired format. In some cases, data must first be cleansed before it can be transformed. Data cleansing prepares the data for transformation by resolving inconsistencies or missing values. Once the data is cleansed, the following steps in the transformation process occur:

  1. Data discovery. The first step in the data transformation process consists of identifying and understanding the data in its source format. This is usually accomplished with the help of a data profiling tool. This step helps you decide what needs to happen to the data in order to get it into the desired format.
  2. Data mapping. During this phase, the actual transformation process is planned.
  3. Generating code. In order for the transformation process to be completed, a code must be created to run the transformation job. Often these codes are generated with the help of a data transformation tool or platform.
  4. Executing the code. The data transformation process that has been planned and coded is now put into motion, and the data is converted to the desired output.
  5. Review. Transformed data is checked to make sure it has been formatted correctly.

In addition to these basic steps, other customized operations may occur. For example,

  • Filtering (e.g. Selecting only certain columns to load).
  • Enriching (e.g. Full name to First Name , Middle Name , Last Name).
  • Splitting a column into multiple columns and vice versa.
  • Joining together data from multiple sources.
  • Removing duplicate data.

After it has been transformed, the data is ready to be loaded into its target destination so it can be put to work.

Finally, it’s important to note that not all data needs to be transformed. In some cases, data from the source will already be in a usable format. This is referred to as “direct move” or “pass-through” data.

Benefits of Data Transformation

Whether it’s information about customer behaviors, internal processes, supply chains, or even the weather, businesses and organizations across all industries understand that data has the potential to increase efficiencies and generate revenue. The challenge here is to make sure that all the data that’s being collected can be used. By using a data transformation process, companies are able to reap massive benefits from their data, including:

  • Getting maximum value from data: Forrester reports that between 60 percent and 73 percent of all data is never analyzed for business intelligence. Data transformation tools allow companies to standardize data to improve accessibility and usability.
  • Managing data more effectively: With data being generated from an increasing number of sources, inconsistencies in metadata can make it a challenge to organize and understand data. Data transformation refines metadata to make it easier to organize and understand what’s in your data set.
  • Performing faster queries: Transformed data is standardized and stored in a source location, where it can be quickly and easily retrieved.
  • Enhancing data quality: Data quality is becoming a major concern for organizations due to the risks and costs of using bad data to obtain business intelligence. The process of transforming data can reduce or eliminate quality issues like inconsistencies and missing values.

More related articles

What exactly is machine learning?

Machine learning is a method of data analysis that automates analytical model building. It is a branch of artificial intelligence based on the idea that systems can learn from data, identify patterns and make decisions with minimal human intervention.

What is machine learning with example?

Image result for what is machine learning

For example, an algorithm would be trained with pictures of dogs and other things, all labeled by humans, and the machine would learn ways to identify pictures of dogs on its own. Supervised machine learning is the most common type used today.

https://mitsloan.mit.edu/ideas-made-to-matter/machine-learning-explained

5 Things You Should Know About Data Mining           

Data mining task in which the goal is to build a model that describes the most significant changes in the data from previously measured or normative values.

Detect objects that regularly occur together that show causal correlations, or that tend to occur in regular sequences.

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense) to each other than to those in other groups (clusters). ... Clustering can therefore be formulated as a multi-objective optimization problem.

What is a cluster analysis example?

Many businesses use cluster analysis to identify consumers who are similar to each other so they can tailor their emails sent to consumers in such a way that maximizes their revenue. For example, a business may collect the following information about consumers: Percentage of emails opened. Number of clicks per email.

5 Examples of Cluster Analysis in Real Life

https://medium.com/analytics-vidhya/customer-segmentation-for-differentiated-targeting-in-marketing-using-clustering-analysis-3ed0b883c18b

A data model organizes data(link is external) elements and standardizes how the data elements relate to one another. Since data elements document real life(link is external) people, places and things and the events between them, the data model represents reality. 

What is a complex time series?

Such complex time series include time series with multiple seasonal periods, high frequency seasonality, non-integer seasonality and dual-calendar effects. Our new modelling framework provides an alternative to existing exponential smoothing models, and is shown to have many advantages.

https://www.acquia.com/blog/difference-between-segmentation-and-clustering

Supervised and Unsupervised learning

https://www.geeksforgeeks.org/supervised-unsupervised-learning/

Reinforcement learning

https://www.geeksforgeeks.org/what-is-reinforcement-learning/

Explanation of Collaborative Filtering vs Content Based Filtering

https://codeburst.io/explanation-of-recommender-systems-in-information-retrieval-13077e1d916c

What are the uses of a recommender system?

Recommender system has the ability to predict whether a particular user would prefer an item or not based on the user's profile. Recommender systems are beneficial to both service providers and users [3]. They reduce transaction costs of finding and selecting items in an online shopping environment [4].

How To Design A Dashboard – The Top 23 Best Practices To Empower Your Business

What are the three types of BI?

In the vacation rental industry, there are three broad types of BI:

  • Internal Intelligence.
  • Market Intelligence.
  • Comparative/Competitive Intelligence.

What are the concepts of BI?

Business intelligence is the process by which enterprises use strategies and technologies for analyzing current and historical data, with the objective of improving strategic decision-making and providing a competitive advantage.

What makes a good information management system?

Accuracy: Data gathered by the system should be error free. Completeness: The software should be designed to gather as much data as required. Relevance: Data gathered should fulfill specific need. Accessibility: The software should allow the correct user to retrieve the data when required.

Benefits of SSOT

A robust data and business intelligence infrastructure that runs on an SSOT can:

  • Eliminate duplicate entries of data
  • Provide decision-makers with the right data at the right time
  • Substantially reduce the time spent on identifying which recorded data is correct
  • Iteratively improve the data intelligence capabilities of the company

With faster and more timely decision-making, the company will be in a better position to implement new strategies based on accepted data points.

What is a single source of truth (SSOT)?

Single source of truth (SSOT) is a concept used to ensure that everyone in an organization bases business decisions on the same data. Creating a single source of truth is straightforward. To put an SSOT in place, an organization must provide relevant personnel with one source that stores the data points they need.

Data-driven decision making has placed never-before-seen levels of importance on collecting and analyzing data. While acting on data-derived business intelligence is essential for competitive brands today, companies often spend far too much time debating which numbers, invariably from different sources, are the right numbers to use. Metrics from social platforms may paint one picture of a company’s target demographics while vendor feedback or online questionnaires may say something entirely different. How are corporate leaders to decide whose data points to use in such a scenario?

Establishing a single source of truth eliminates this issue. Instead of debating which of many competing data sources should be used for making company decisions, everyone can use the same, unified source for all their data needs It provides data that can be used by anyone, in any way, across the entire organization.

How to Ensure Data Consistency and Quality with Web Data Integration ?

6 Ways to Transform Data into Real Information That Drives Decision-Making. 

https://www.allbusiness.com/transform-data-real-information-drives-decision-making-16096-1.html

Tips to Convert Data Into Information

Meanwhile, let’s take a look at some tips to convert data into information that will help to drive decision making for most businesses:

1. Gather only the relevant or valid data

Now, when there is a huge volume of data available, what you first need to do is to see how much of it is relevant, valid or accurate. Identify the amount of data that can be used as information to arrive at decisions, which help to enhance services or cut down costs. This will also help you to ensure that only the most valid and relevant data is collected.

2. Employ tools that help you to analyze data

Make use of or employ tools or technologies, which can help you to analyze the information or the data that you collect. The data extracted from your system can be uploaded into the excel sheet as required and then transformed to the required information making use of its cutting-edge tool.

3. Collect only the data that is accurate

See, first how accurate the data that you have in hand is and how much of a difference it will make on your decision when correct data is being used. Good data is what needs to be used. However, analyzing the data and its accuracy for the purpose at hand is required or you could be spending time trying to get accuracy for data, which may not be relevant or valid for your purpose.

4. Transform the data you collect into valid information

When you have a huge volume of data available from various resources, you will need to analyze, process, and organize it into the most relevant, valid, and accurate format, suited for the business purpose. This combined with added data and business strategies help businesses to arrive at insightful information that drives marketing campaigns to success.

What is an operational data example?

Introduction to Operational Database. An Operational Database or OLTP (On-Line Transaction Processing) is a database management system where data is stored and processed in real-time. ... Some examples of Operational Databases are Microsoft SQL Server, AWS Dynamo, Apache Cassandra, MongoDB, etc.

While operational data tells a utility what is happening, non-operational data can explain why things are happening.

An OLTP system is an accessible data processing system in today's enterprises. Some examples of OLTP systems include order entry, retail sales, and financial transaction systems. ... OLTP is often integrated into service-oriented architecture (SOA) and Web services.

What are data augmentation techniques?

Data augmentation techniques generate different versions of a real dataset artificially to increase its size. Computer vision and natural language processing (NLP) models use data augmentation strategies to handle data scarcity and insufficient data diversity.

Data augmentation is useful to improve performance and outcomes of machine learning models by forming new and different examples to train datasets. ... Data augmentation techniques enable machine learning models to be more robust by creating variations that the model may see in the real world.

Why do we need business metrics?

Tracking metrics lets you improve overall results and align your people and processes with your organizational objectives, as well as giving you the following benefits: Measure financial performance – vital for keeping your cash flow healthy. ... Provide an actionable way to achieve overall business strategies and goals.

What role does data play in digital transformation?

The Importance of Data

Digital transformation provides businesses with real-time information and greater visibility into operations, particularly when it comes to the performance of people and assets. ... Data is at the heart of the digital government transformation.

The role of data is to empower business leaders to make decisions based on facts, trends and statistical numbers. But with so much information out there, business leaders must be able to sift through the noise, and get the right information, so that they can make the best decisions about strategy and growth.

Why is digital data important in business?

Good data can help you: ... Companies that use the information their data gives them to guide their way forward as an organization grow 8 times faster than global GDP, proving that evidence-based strategies are far more successful than your average digital growth plan.

There can be two types of reports, i.e. Static reports and Interactive reports.

Static reports cannot be altered by the end-user, and Interactive reports allow you to get detailed insights by drilling down to the data. These reports also provide the facility to navigate, filter, sort, & view the data.

These reporting tools can generate different types of the report as shown below:

  • Reporting for business intelligence,
  • Visualization and reporting,
  • Self-Service reporting,
  • Enterprise reporting,
  • Application performance reporting,
  • Finance related reporting.

Improve Operations

The key idea is to collect data about the organization and use it to improve operations. This is the most widely discussed benefit of analytics. The unstated assumption is that the organization has a lot of friction and inefficiencies. I am sure most of you will agree that this is true.

There are many use cases that fall under this. Following are a few.

  • Predictive maintenance i.e. production lines, equipment, vehicle fleets, and sites.
  • Optimizations, scheduling, and responding to issues i.e. making sure the right person is playing the right role and has access to everything they need.
  • Fraud detection and prevention.
  • HR analytics i.e. finding the best candidates, filtering candidates, performance appraisal analytics and proactive intervention, andchurn prediction.
  • Security and surveillance.

Knowledge Discovery in Databases

Leave a Reply

Your email address will not be published.