We took Machiavelli’s advice to heart. The algebra of data is a new application of mathematics. It has existed as a set of mature usable techniques since 2012. The few experienced mathematicians who are currently familiar with it will tell you there is nothing new about it, in the pure mathematical sense. For example, there was no need to adjust or extend set theory to formulate the various algebras that constitute the algebra of data — yes, indeed, data algebra comprises more than one algebra.

It applies set theory in a new way, and for that reason, it constitutes a new branch of applied mathematics. It was first formulated as a collection of mathematical constructs that were thought to provide a practical foundation for data management software. It was gradually tested and applied to data management problems, first to “relational” data structures and then to “graph” data structures.

This initial chapter is devoted to answering several questions that all begin with the words “What is…” We believe they are worth discussing before we introduce the algebra of data. The first such question is:

What is mathematics?

The New Oxford American Dictionary defines it as:

Alternatively, the Merriam-Webster dictionary says:

In fact, neither of these definitions is entirely adequate. The first confines mathematics to number, quantity and space, when in practice, some areas of mathematics deal with “things” that may not be spatial or numeric or have the attribute of quantity. Data can be such a thing.

The second definition is slightly better, as it refers directly to operations, interrelations, combinations, generalizations and abstractions, which are the beating heart of mathematics. Nevertheless, this definition also confines mathematics to numeric and spatial concerns.

The power of mathematics lies partly in its ability to construct useful models of real things. Using scale measurements, geometry and various calculations, you can design buildings. With more sophisticated mathematics, you can design chemical plants or airplanes. Mathematics is both the toolbox of innovators and the workhorse of engineers.

Even where mathematics has failed to model a domain perfectly — for example, weather prediction — it has no peer. Weather forecasts still rely on mathematical models, and nowadays, given sufficient computer power and historical data, they are usually accurate for two days or more.

In fact — and this is important to understand — mathematics is never wrong per se. As Isaac Newton remarked, “Errors are not in the art but in the artificers.” A mathematical statement can be incorrect if someone makes an error in formulating it. A mathematical model can be horribly inaccurate if the modeling work is done poorly. But the mathematics itself is blameless. The tool is either used well or badly. This can be said of very few areas of study.

Data algebra models data, and it does so with rigorous accuracy. This attention to detail makes it an extremely useful tool for anyone who wants to define data structures and manipulate them algebraically.

The next question is also a fundamental one:

What is a computer?

Dictionaries focus on the fact that a computer is an electronic machine that runs programs to process information. However, here we need to be more particular and define a computer in terms that have relevance to data algebra.

So think of a computer as an electronic machine that runs programs that act on data. This brief, accurate definition allows us to make an important point:

The only thing a computer ever does is take data and transform it into other data.

Every program that you have ever used or can conceive of does this and only this. It is true of operating systems, spreadsheets, email and big databases. It is true of every executable program.

There is an extremely powerful corollary to this: given that a data algebra exists, all programs can be written in algebraic terms as a set of mathematical functions that transform various algebraic collections of data into other algebraic collections of data.

Whether this means that all programs should be written in a strict algebraic manner is a moot point. In reality, simple programs will gain little from it.

What is an algebra?

Back to the dictionaries. Merriam-Webster’s answer is:

This is quite an accurate description of algebra. Algebraic statements use symbols (+, −, ×, ÷, ±, =, ≠, ∈, ∪, and many others) as a shorthand to express specific relationships quickly and succinctly. Such symbols form a kind of lexicon that mathematicians quickly learn. But some people find these symbols difficult to follow and are turned off, perhaps unnecessarily, from mathematics.

To minimize this problem for readers, we have gone to some lengths to translate algebraic statements into words and provided illustrations as well. Of course, those who quickly pick up the symbols can skip these aids.

There are many algebras

Wikipedia provides a list of over fifty distinct algebras, each focusing on different targets — and it is not an exhaustive list. Data algebra itself is not a single algebra, as we have already noted. You will appreciate what this means as you progress through the book. You will realize that it is, in reality, a number of algebras that are included, one within another, like Russian nesting dolls.

Algebras are specifically built to enable abstraction and generalization. In algebra, the use of alphabetic letters or other symbols to represent real things is a fundamental tactic. So when we say, for example, a locomotive is travelling due west at 60 mph, we may refer to the locomotive as, say, L and its velocity as v. Thus we abstract a real-world object by using a single letter and its velocity by using another letter.

If we manage through various manipulations to find relationships between some of these letters, regardless of their value in any given context, we have a generalization that can be applied to many contexts, which may be very useful. One simple example is the equation you probably learned at school:

v(t) = v₀ + at

where v = velocity, v₀ = initial velocity, a = acceleration and t = time.

Discovering such relationships is the primary purpose of any given algebra when it is used to model some real-world situation. However, with data algebra the situation is slightly different. The goal of the algebra is to enable data to be handled algebraically. It does not model the data, it defines it.

The rewards that the algebra naturally bestows are generalizations that can be applied to all data. It can solve problems in data integration and software optimization that might otherwise be intractable.

What is data?

In order to construct a data algebra, it is necessary to find an answer to the question “What is data?” and, having found an answer, to discover productive ways to represent data algebraically. Let’s satisfy your immediate curiosity — assuming you have some — by discussing the question.

So, if you are willing to play, you are up. Think about the question “What is data?” for a moment or two, and then write your answer in the space below, without peeking at the text that follows:

Dictionary definitions of data make assertions of the following kind:

• Facts and statistics collected together for reference or analysis.
• Factual information used as a basis for reasoning.
• Any representations to which meaning is or might be assigned.
• The plural of datum.

Thesauri suggest confusing synonyms including: information, aggregation, collection, accumulation, assemblage, fact, details and even input. Perhaps this is what we should expect from the compilers of dictionaries and thesauri, but it’s a little disappointing, and it doesn’t fry any fish.

The responses to this question from IT experts in the field, often preceded by a long pause and punctuated with some facial contortions and guttural utterings, are more telling. We have experienced the following replies:

• “Knowledge.”
• “Facts.”
• “Files, digital information.”
• “A database.”
• “Anything that somebody might want to know that is enumerated, listed or quantified.”
• “It is the answer to a question (may be a hypothetical question).”
• “It can be written (this includes that it is limited).”
• “It’s data. It is what it is.”
• “In computer parlance, it literally means anything that has been captured in any form on any digital recording medium.”

We have even met with the somewhat defensive, “What do you mean by that?” and the somewhat facetious, “A character on Star Trek: The Next Generation.”

This sample of responses is a hodgepodge of ambiguity and circularity which, if left undisturbed, precludes a rigorous mathematical discussion of data, and since that’s what we’re focused on, we must do better. If we do not, we will simply repeat the previous errors of the IT industry, which, for its sins, gave birth to SQL, a mathematical disaster of the first order.

To paraphrase Carl Sagan (Dragons of Eden), those closest to the intricacies of data seem to have a more highly developed (and ultimately erroneous) sense of its mathematical intractability than those at some remove.

As it turns out, the wife of one of the authors, a notorious data-phobe, provided the most suggestive responses to both questions.

“What is data?” was met with a quizzical look, then, “Well, come on now, you and I both know what it means.”

And the subsequent question “What is a datum?” was met with a more menacing look, and “You and I both know what it means!”

Indeed, anybody making a living in the data world knows what data is, and knows what a datum is, and uses the terms with reckless abandon in discussions with colleagues who also know what data is and know what a datum is, even though such “undoubted knowledge” calls into question the very definition of the word “know.”

The only implied constant in such discussions is suggested by the surprisingly useful dictionary definition: “data is the plural of datum.” That is, data (whatever it is) is a collection of “datums,” and a datum (whatever that is) may be collected with other “datums” to create data.

Our assertion here is that the key to providing a practical mathematical understanding of data is to take data and datum as primitive undefined terms and axiomatize their relationship, i.e., the notion of belonging.

Wait! What?

To tell the truth, this was done long ago, in the 19th century by Georg Cantor, as we shall explain in Chapter 3, and the only difference with what we assert and what Georg Cantor asserted is that we are using the terms data and datum, whereas he used the concepts of set and element, respectively.

To put it bluntly:

Data is to datum as set is to element.

It naturally follows that any mathematically legitimate notion of data algebra must be couched in terms of (and subject to the rigor of) set theory and be independent of any preconceived notions about data: visual, structural or otherwise.

We hope to explain, as simply as we can, the algebraic tale of data. It is a tale of sets recounted by ascending towers of sets of ever increasing cardinality, in the words of Paul Halmos, “of frightening height and complexity.” But don’t be too concerned; all algebraic tales are like that.

A final note

This book adopts the style of ending this chapter and each one that follows, with one or more pages entitled Taking Stock. They are provided so that readers can go to these pages to remind themselves of the major points covered in the chapter.

And so…