Data Fundamentals for Analytical Processing

Data is the raw material of every analytics project. Before you can find patterns, make predictions, or recommend actions, you need to understand what you’re working with — and whether it’s even ready to be used. This chapter shows you how to think about data like a skilled analyst: its types, its quality, and how to bring it all together.

The Big Picture#

Every business decision that relies on data starts with a simple question: can I trust this data to tell me something true? In this chapter, we’ll explore the different shapes data can take, how to measure whether it’s good enough to use, and how to combine scattered information into a single, reliable view. By the end, you’ll have a mental toolkit for checking any dataset, no matter where it comes from.

Data: The Raw Material of Analytics#

Think of a chef preparing a meal. Even the most talented chef cannot create a great dish from spoiled vegetables or missing ingredients. In the world of business analytics, data works the same way. It is the raw ingredient that gets transformed — through cleaning, analysis, and modeling — into insights, predictions, and decisions. If the data is flawed, the decisions will be flawed too. This is often called the “garbage in, garbage out” principle.

Data, in its simplest form, is a collection of facts. These facts can be numbers, words, measurements, observations, or even images and sounds. On its own, data is just a pile of raw material. Analytics is the process of cooking that material into something useful: a trend spotted in sales figures, a forecast of next month’s demand, or a recommendation for which product to suggest to a customer.

But before any cooking can begin, we need to know what kind of ingredients we have and whether they are fresh, complete, and safe to use. That’s what this chapter is all about — understanding the nature of data and checking if it’s ready for analytical work.

Data: A collection of raw facts, figures, or observations that can be processed to produce meaningful information.

📝 Section Recap: Data is the raw material for all analytics, and its quality directly determines the quality of the insights we can extract. Treating data like a perishable ingredient helps you remember to check it carefully before using it.

The Many Shapes of Data#

Not all data is created equal. To work with data effectively, you need to recognise its structure and what kind of measurement it uses. These characteristics determine which analytical methods are right and how we should understand the results.

Structured vs. Unstructured Data#

The first big division is between structured and unstructured data.

Structured data: Data that fits neatly into rows and columns, like a spreadsheet or a database table. Each row is a record, and each column is a field with a clearly defined data type (e.g., date, number, text).

Think of a customer list: each row is one customer, and the columns hold their name, email, sign‑up date, and total purchases. This kind of data is easy to search, sort, and analyse with traditional tools.

Unstructured data: Data that does not have a predefined row‑and‑column format. Examples include email text, social media posts, images, audio recordings, and video files.

Unstructured data is messy but very rich. It makes up the vast majority of data in the world today. Analysing it requires special techniques (which we’ll touch on later in the course), but recognising it is the first step.

Categorical Data: Names and Labels#

Categorical data describes qualities or groups. It answers the question “what kind?”. There are two subtypes.

Nominal data: Categories that have no natural order. They are simply names or labels. Examples: customer city (London, Tokyo, Nairobi), product colour (red, blue, green), or department name (Sales, HR, Engineering).

With nominal data, you can count how many items fall into each category and compare frequencies, but you cannot say that one category is “greater than” another.

Ordinal data: Categories that have a meaningful order or ranking, but the gaps between the ranks are not necessarily equal. Examples: survey responses (strongly disagree, disagree, neutral, agree, strongly agree), education level (high school, bachelor’s, master’s, PhD), or a restaurant’s star rating (1 to 5 stars).

Ordinal data lets you sort items from lowest to highest, but you cannot measure the exact distance between “agree” and “strongly agree” the way you can measure the distance between 10 kg and 12 kg.

Numeric Data: Measurements and Counts#

Numeric data describes quantities. It answers “how much?” or “how many?”. Again, there are two important levels.

Interval data: Numeric data where the difference between values is meaningful, but there is no true zero point. The classic example is temperature in degrees Celsius or Fahrenheit. A temperature of 0 °C does not mean “no temperature”; it is just a point on the scale. Because there is no true zero, ratios are not meaningful — 40 °C is not “twice as hot” as 20 °C in any physical sense.

Ratio data: Numeric data that has a true, meaningful zero. Examples: sales revenue, weight, height, time, number of website visitors. A revenue of 100 is twice as much as $50, and 0 kg is the complete absence of weight.

Why do these types matter? Because the type of data dictates the kinds of statistics and visualisations you can use. You can calculate an average rating (ordinal) but need to be cautious; you can safely calculate an average salary (ratio). You can make a bar chart for nominal data, but a histogram only makes sense for numeric data. Knowing these types is like knowing whether you’re working with flour or sugar — it changes the recipe.

📝 Section Recap: Data comes in two structural forms (structured and unstructured) and four measurement levels (nominal, ordinal, interval, ratio). Recognising these differences tells you how to handle, summarise, and analyse a dataset correctly.

Is Your Data Ready? Essential Quality Metrics#

Having the right type of data is only half the battle. The data must also be good enough to use. Imagine you’re buying a used car: you’d check the mileage, service history, and whether it starts reliably. Data needs a similar health check. Over the years, data professionals have agreed on a list of checks — we’ll call them data readiness metrics — that help us decide if a dataset is trustworthy and usable.

Here are the ten key metrics, explained in plain English.

Reliability: Can you depend on the data to be consistent over time and across different sources? Reliable data comes from stable, well‑kept collection processes. If a sensor sometimes records temperatures and sometimes doesn’t, the data is unreliable.

Accuracy: Does the data reflect the real‑world value it is supposed to represent? If a customer’s recorded age is 142, that’s inaccurate. Accuracy is about correctness.

Accessibility: Can the people who need the data actually get to it, when they need it? Data locked in an old system that only one person knows how to get data from is not accessible. Accessibility also includes whether the data is in a usable format.

Security: Is the data protected from unauthorised access, tampering, or theft? Security is especially critical for personal or financial information. Even if data is accurate and accessible, it must be handled in a way that respects privacy and legal requirements.

Richness: How much detail and context does the data contain? A dataset with only customer IDs and purchase amounts is less rich than one that also includes timestamps, product categories, and customer locations. Richer data allows deeper analysis.

Consistency: Does the same piece of information appear in the same way across all systems? If one database stores dates as “MM/DD/YYYY” and another as “DD‑MM‑YYYY”, the data is inconsistent. Inconsistent data leads to errors when datasets are combined.

Currency: How up‑to‑date is the data? A sales report from three years ago is not current. The required level of currency depends on the decision: stock‑market trading needs data in milliseconds; a long‑term demographic study can tolerate older figures.

Granularity: At what level of detail is the data captured? Granularity refers to the finest level of detail available. Daily sales data is more granular than monthly totals. Individual transaction records are more granular than daily summaries. Higher granularity gives you more flexibility, but it also means more data to store and process.

Validity: Does the data conform to defined business rules and formats? For example, an email field should contain a properly formatted email address; a date of birth should not be in the future. Validity checks catch data that is wrong in format or logic, even if it might be accurate in some other sense.

Relevancy: Is the data actually useful for the question you’re trying to answer? You might have perfectly accurate, timely, and granular data about rainfall in Brazil, but it’s irrelevant if you’re trying to forecast ice‑cream sales in Canada. Relevancy ties the data directly to the decision at hand.

You can think of these ten metrics as a checklist. Before you start any analysis, run through them. If your data fails on several dimensions, you’ll need to clean, add to, or supplement it before you can trust the results.

📝 Section Recap: Ten readiness metrics — reliability, accuracy, accessibility, security, richness, consistency, currency, granularity, validity, and relevancy — form a complete health check for any dataset. Good analytics starts with data that passes this check.

Bringing It All Together: Data Consolidation and Integration#

In a typical business, data is scattered everywhere. Sales data lives in one system, customer complaints in another, inventory levels in a third, and social media mentions somewhere out on the web. Each system was probably designed for a different purpose, at a different time, by a different team. To get a complete picture, we need to bring all these pieces together. This process is called data consolidation and data integration.

Data consolidation: Moving data from multiple sources into one central storage location.

Data integration: The wider process of combining data from different sources, fixing mismatches, and giving a single view — whether the data is moved or just accessed together.

Think of it like building a jigsaw puzzle. The pieces come from different boxes (the source systems), but you need to fit them together on one table (the consolidated view) to see the whole picture. Sometimes the pieces have slightly different cuts (inconsistent formats), so you have to trim or adjust them — that’s the integration work.

How It Works: ETL#

The most common recipe for consolidation is called ETL, which stands for Extract, Transform, Load.

1. Extract: Pull the raw data out of the source systems. This might mean reading from a database, downloading CSV files, or calling an API.
2. Transform: Clean, standardise, and reshape the data. This is where you fix inconsistencies (e.g., convert all dates to the same format), remove duplicates, fill in missing values where possible, and combine related tables. The transformation step applies many of the readiness checks we discussed earlier.
3. Load: Write the cleaned, transformed data into the target destination, usually a data warehouse or a data lake.

Where the Data Lands#

The destination matters because it shapes how easily you can analyse the data later.

• A data warehouse is a central storage place built for analytics. It holds structured, cleaned, and organised data that is ready for asking questions and making reports. Think of it as a well‑organised pantry: everything is labelled, stored in matching containers, and easy to find.
• A data lake stores huge amounts of raw data in its original format — structured, semi‑structured, and unstructured. It’s more like a large reservoir: you can dump everything in, but you’ll need to do some processing before you can use it for a specific analysis. Data lakes are popular when you want to keep all your data, cheaply, and decide later what to do with it.

Consolidation and integration are not one‑time projects; they are ongoing processes. As new data sources appear and business questions evolve, the integrated view must be updated. But getting this foundation right is what turns a messy collection of facts into a valuable resource.

📝 Section Recap: Data consolidation and integration bring scattered data together into a single, consistent view. The ETL process (Extract, Transform, Load) feeds cleaned data into a warehouse or a lake, making it ready for serious analysis.

Summary#

We started with a simple idea: data is the raw material of analytics, and like any raw material, its quality determines the quality of the finished product. Then we learned to recognise the many shapes data can take — from neat rows and columns to free‑form text, and from labels without order to measurements with a true zero. We armed ourselves with a ten‑point health checklist to judge whether data is ready for use. Finally, we saw how to bring scattered data together through consolidation and integration, turning separate pieces into a single, trustworthy source of truth. These fundamentals are the foundation of every data‑driven decision.