Chapter 2: Data Types and Structures#

Data can be messy like a pile of receipts, or neat like a spreadsheet. It can also stream in fast like social media posts. How you handle it decides whether you guess or make a decision you can trust.

The Big Picture#

Before you can make a chart or calculate a trend, you need to know what kind of data you have and if you can trust it. This chapter sorts data into structured tables and free‑form text. It also explains what makes data “big” and gives you a way to spot data that is incomplete, unreliable, or just wrong. For an accountant, this skill protects every audit opinion, budget forecast, and tax filing from silent errors.

Structured and Unstructured Data#

Pick up any accounting system — an invoicing module, a payroll file, or a general ledger — and you will notice the same pattern: rows and columns. Each row is a transaction or a record, and each column holds a single, predictable piece of information: a date, an amount, a vendor name, an account code. That is structured data. It follows a fixed layout, which makes it easy to search, sort, and plug into formulas. Think of a stack of perfectly identical filing cabinets, where every drawer is labeled and every folder sits in the same place.

Structured data: Information that fits neatly into tables with rows and columns, where every field has a fixed format and expected type of value.

In accounting, structured data is everywhere: trial balances, depreciation schedules, inventory counts, and compliance reports all come in tabular form. Because the structure is known in advance, software can read a date field as an actual date, add a column of numbers automatically, and join two tables on a common customer ID without guessing.

Most data in the world, however, refuses to sit in tidy rows. An email between a client and an auditor, a scanned PDF of a contract, a voice recording of a board meeting, or a satellite photo of a warehouse — these are all unstructured data. The information is there, but there is no pre‑defined column that says “here is the contract value” or “the tone of this email is hostile.” You cannot simply sum a column of text.

Unstructured data: Information that does not follow a fixed row‑and‑column model, such as free‑form text, images, audio, and video.

To make unstructured data useful for analysis, you typically need an extra step — something like a program that reads the contract and pulls out the dollar amounts, or a sentiment tool that scores the email’s language. Even after that step, the result might be stored in a structured table, but the original raw material was unstructured.

Many real‑world accounting tasks live in a middle ground. An invoice is usually structured when it comes from a system, but an emailed PDF of an invoice is a hybrid: the PDF is an unstructured file, but inside it there are recognizable patterns like “Invoice #,” “Date,” and “Total Due.” Some people call this semi‑structured data. It has some labels or keys that help organize it, but it is not as rigidly uniform as a database table. For example, a photo might have metadata like the date and location, but the image itself is unstructured. For an accountant, the practical point is simple: structured data is ready for analysis right away; unstructured and semi‑structured data need a bit of wrangling first.

📝 Section Recap: Structured data lives in neat tables and is ready for immediate use; unstructured data is free‑form and requires extra processing before analysis. Understanding the difference helps you know what you can trust at a glance — and what needs a closer look.

Big Data: The Four V’s#

A small business might track a few thousand sales transactions a year. A multinational retailer, on the other hand, might record every click on its website, every item scanned at millions of checkouts every second, and every social media post that mentions its brand. When data becomes so massive, fast, and varied that ordinary spreadsheet tools buckle under the load, we have entered the world of big data. The term is often unpacked through four V’s: volume, velocity, variety, and veracity.

Volume is the sheer scale. Decades ago, accounting data lived in ledgers that a person could carry. Today, a single large corporation’s sales system might generate so much data in a day that it takes special storage and processing tools to handle. High volume alone does not make data valuable, but it forces us to think about how we store and query it.

Volume: The enormous amount of data generated, often exceeding the capacity of a single desktop computer.

Velocity is about speed of arrival. A payroll system might update once a month; that is low velocity. A credit card fraud detection system must evaluate each swipe in milliseconds — high velocity. When data arrives in a continuous stream, analysis often has to happen in near‑real time, not in a batch report run overnight.

Velocity: The rate at which new data is created and needs to be processed.

Variety captures the mix of formats that pour in. An integrated audit might combine structured transaction logs, semi‑structured bank feeds, and unstructured email threads. The variety of data types — numbers, text, images, sensor readings — challenges any single tool that was designed only for rows and columns.

Variety: The range of different data formats, from tables and spreadsheets to free‑form text, images, and video.

Veracity addresses the trustworthiness of the data. When you have a flood of information from many sources, some of it will be noisy, biased, or just wrong. A social media scrape may contain spam; a sensor might malfunction and report impossible temperatures. Veracity is the degree to which the data is accurate and reliable. For accountants, this V is especially critical — a forecast built on low‑veracity data can be worse than no forecast at all.

Veracity: The level of accuracy, consistency, and trustworthiness of the data.

These four V’s remind us that “big data” is not just about size. A large but entirely clean and slow‑moving dataset might strain storage but not need real‑time analytics; a small but messy and fast dataset can be just as challenging in its own way. In accounting, you are far more likely to wrestle with veracity and variety than with massive volume.

📝 Section Recap: Big data is characterized by volume, velocity, variety, and veracity. Together they explain why some datasets are hard to handle — and why building trustworthy analysis requires more than just a powerful computer.

Data Quality Dimensions#

Imagine you are preparing a year‑end financial statement. You pull a trial balance, and you notice that several account balances are empty, one vendor name appears as “UNKNOWN,” and a sales figure seems to have been entered in the wrong currency. Before you can use that data, you need to judge its quality. Data quality is not a single yes/no question; it sits on several dimensions. The most foundational for accounting work are completeness, reliability, and validity.

Completeness asks: is anything missing? A table with 10,000 rows where 500 rows have a blank value in the “amount” column is incomplete. Incompleteness might be obvious, like a missing field, or subtle, like an entire day’s transactions that failed to load. When data is incomplete, any average, total, or trend you calculate will be distorted.

Completeness: The degree to which all required data is actually present, with no missing fields or hidden gaps.

Reliability (often intertwined with accuracy) asks: can we trust what is there? A vendor’s address might be present and valid in format, but if it points to the wrong building, the data is unreliable. Reliability covers both correctness and the consistency of the data across different systems. For example, if the same customer is recorded as “Acme Inc.” in the billing system and “ACME Corp.” in the CRM, the inconsistency makes the data less reliable.

Reliability: The overall trustworthiness of the data — does it reflect reality correctly and consistently over time and across sources?

Validity is about whether the data follows defined rules. A date field that contains “February 30” is invalid. A “payment method” field that allows “Credit Card” but someone typed “creditcard” violates a validity rule. Validity is often enforced by the system itself — for instance, a drop‑down menu that limits choices — but when data comes from external sources or manual entry, invalid values slip through.

Validity: Whether the data conforms to expected formats, ranges, and business rules.

These three dimensions interact. A dataset can be complete (no blanks) and valid (all dates are real dates) yet still unreliable (the amounts are systematically overstated because of a double‑counting bug). When you are handed a dataset for analysis, your first instinct should be to check these three areas, not to jump straight into making charts or summaries.

📝 Section Recap: Data quality rests on completeness, reliability, and validity. Missing values, untrustworthy entries, and rule‑breaking formats each undermine analysis and must be checked before any meaningful insight is drawn.

Spotting Data Quality Problems#

Accounting professionals develop a sixth sense for numbers that “feel wrong.” Transfer that instinct to a data table, and you are already halfway to recognizing quality problems. The trick is to apply simple, systematic checks rather than relying on gut feel alone.

The first and easiest red flag is blank or placeholder values. Open a spreadsheet and apply a filter to every column — nulls, “N/A,” “000-00-0000,” or strangely uniform entries like “Employee 999” often signal missing data that was filled with a dummy value. In a general ledger extract, a blank account code makes it impossible to map the transaction to the correct line of the financial statements.

Next, watch for out‑of‑range values. A payroll earnings record that shows an hourly rate of 50,000 is suspect. You do not need to know the exact correct value; you only need to know the reasonable boundaries. Similarly, a date of “1901‑01‑01” or a future date five years from now usually comes from a system default or a typo, not from a real transaction.

Duplication is another silent killer. The same invoice might appear twice in a dataset because the export overlapped two date ranges or because two different source systems fed the same record. Duplicate rows inflate counts and distort sums, and they are notoriously easy to overlook. A quick grouping by a unique identifier (like an invoice number) will often reveal them.

Then there are format inconsistencies. One column may hold dates in both “MM/DD/YYYY” and “DD/MM/YYYY” without warning — a classic source of catastrophic error when the day is 12 or less. A tax identification number might be stored with and without dashes. These small inconsistencies may seem trivial, but they can cause errors when you try to combine data from different sources.

Orphan records appear when one table references something that does not exist in another. For example, a sales transaction points to a customer ID that has been deleted from the customer master file. That transaction is still valid, but any report that tries to pull the customer name will return nothing. Orphan records scream “the data was not maintained as a unified whole.”

Finally, a subtler problem is data drift — when the meaning of a field changes over time without documentation. A “status” column might once have used “A” for “Approved” and “P” for “Pending,” but a system upgrade started using “AP” and “PE.” If you merge old and new data without normalizing those codes, half your history will look like a different category.

To catch these problems, adopt the habit of data profiling — a quick, structured review before any analysis. Sort numeric columns by descending value and look at the extremes; count distinct values in text columns to see if codes are messy; join key tables and check for orphans. This profiling takes far less time than fixing an error that spreads through a final report.

Data profiling: A quick, structured review of a dataset before analysis that uses sorting, filtering, counting, and joining to spot quality problems.

📝 Section Recap: Quality problems like blanks, impossible values, duplicates, inconsistent formats, orphan records, and data drift often hide in plain sight. A few minutes of disciplined profiling — sorting, filtering, and joining — can save hours of detective work down the road.

Summary#

You now see data not as one simple thing, but as a mix of forms. Some are neat and ready, some are messy and need extra care. Even a clean spreadsheet can hide flaws that would ruin a financial conclusion. Understanding structured and unstructured data, the challenges of big data, and the dimensions of data quality turns you from a passive data consumer into a careful, questioning professional. That mindset is the foundation of every analysis you will build.