ow odd it is that anyone should not see that all observations must be for or against some view if it is to be of any service.” - Charles Darwin.
Speak to exploration geologists and you will find two opposing views about how to observe outcrop or drill core. Some seek merely to be unbiased objective recorders of what they see. Others observe the rock in the wider context of their theories about region or prospect geology and ask questions of the exposure to help choose between these theories.
I characterize these two approaches with the metaphors of geologist-as-camera, and geologist-as-interrogator.
The Geologist as camera
I arrive at the exploration site to find a team of three young geologists engaged in making a geological map of their large exploration acreage. Using GPS, they walk predetermined traverses spaced 2-400m apart. Each aim to average 8km a day and between them, by taking alternate lines, they cover a large swathe of country before their next field break. Observations on each geological feature along the line of march are logged into a field-hardened tablet computers by going through a series of pull-down menus and checking boxes on pre-determined questions. The geologists have only the vaguest ideas about the geology of the property they are mapping. Two of them have never thought much about this: the brightest of the three specifically rejects the notion of understanding her observations: she sees herself as an unprejudiced objective observer: confirmation bias is not for her. Eventually all this data is computer-plotted to 2D images as linear sequences of point observation. Another more senior geologist might eventually produce a 2D geological map by joining the dots. Probably there is a software program that can do much of this task for him (or her) and thus obviate the need for too much thought or hard work.
I travel to an exploration site on the other side of the country. Here, a much larger company have been drill testing a substantial metal deposit for several years. Four diamond rigs are going 24/7 and have drilled well over 300 holes. Stacked on pallets, several kilometers of core are waiting to be logged. Four geologists are hard at work logging core laid out in the open on long racks. It seldom happens that any one hole is logged in its entirety by the same person. Logging is done on to spread sheets in laptop computers using alpha-numeric codes on pre-determined menu options. There are more than 50 columns on their spreadsheet and the same digital form has been used since hole one. The geologists have never seen a drill section of the prospect (there are none). There are no detailed maps or level plans either. The geological model, such as it is, was produced a year before by an outside consultant who spent ten days on site. Back in the head office, a specialist Ore Reserve Geologist ignores 95% of the vast data base and calculates a resource based on assay numbers, virtual reality 3D string models and geostatistics. For the young geologists at point, labouring in the core yard, their job is 100-meter-a-day, core-logging automata. It is mind-numbingly boring. They count the days till their next field break, or until they have earned enough money to be able to resign and find some other more intellectually stimulating employment. Taxi driving, anyone?
After a long day core logging, Susan contemplates a career change.
These are anonymized projects, extreme examples perhaps, but based on observations of many actual projects, with many companies in many different countries over many years. Not all exploration projects are conducted like this of course, but I am sure readers will recognize the method of doing geology that I describe.
The geologists themselves are not to blame. In many cases it is their first job, they are on short-term contracts, and they are doing what their employer asks and expects of them. They probably wonder what the point was of much of the time they spent learning geological theory and critical thinking at university and have come to believe that the essence of their job as exploration geologists is to convert light signals from their eyes into digital computer feed according to pre-set formulae. A kind of biological digital camera. Without a pre-existing context in their brain, each observation they make has equal importance, each pixel equal weight.
There is a better way.
The geologist as interrogator
All observation is made in a particular context. The context which should guide the exploration geologist is a matrix of different competing theories about the true nature of the geology being observed. This context provides the questions that must be asked of each outcrop or each piece of drill core. It defines the strategy to be followed in the search for those critical observations that allow selection between multiple working hypotheses. The idea of multiple working hypotheses was first propounded by 19th Century USGS geologist Thomas Crowther Chamberlin (1). It is a methodology now used in all fields of science research (2), but geologists can be proud that the first clear statement and naming of this procedure came from their own profession.
Critical observations are those that fit into a pre-existing context or, just as importantly, those that do not. These are prioritized and not lost in a sea of trivial observation.
The method does not guarantee that you will arrive at the correct solution. Your evidence may be less than ideal, the expression in the rocks of geological events may be atypical. Nature can be chaotic and unpredictable: the true hypothesis you should be testing may be a Black Swan event - the one you have not thought of. In all science, and especially geological science, neat, clean results where all observations slots in exactly are rare and, when this happens, should be regarded with some skepticism (maybe you are unconsciously rejecting or downplaying some observations?). But despite all that, the method of multiple working hypotheses is still the best-known procedure for navigating apparent (or real) chaos and approaching the truth.
After a morning of fieldwork, Robert interrogates his lunchtime sandwich using the method of Multiple Working Hypotheses.
All geologists have biases. These were imprinted through their university training and reinforced by early-career mentoring and by reading geological literature relevant to the job. Collecting data to test hypotheses is biased observation. But this bias is a good thing where it is up-front, acknowledged and constantly revised and re-focused in the face of evidence. Without bias, there is no way of separating signal from noise. This is a totally different kind of bias from the one that uncritically accepts only evidence that confirms a single pre-existing theory – one you might have fallen in love with – and ignores, or explains away, evidence which does not. That is Confirmation Bias, and is rightly condemned.
All scientists have a point of view. All views must have come from somewhere. Anyone who claims to make unbiased observation merely lacks self-awareness. Their biases are unconscious or lie in the biases of whoever drew up the detailed procedure manual which they follow.
The metaphor I use for this approach is the geologist as an interrogator of each rock exposure or core piece, asking a series of relevant questions that come from his or her views as to what might be happening, with each question determined by the answer to the previous question.
Interrogating drill core.
The Scientific Method
The true nature of scientific investigation is focused observation guided by emergent theory. This is a long established and respectable idea and contrasts with seeking to find your theory in your data after you have completed your observations.
Collecting data should not be a fishing expedition that hopes to fortuitously entangle new understanding or ideas on its line. Its purpose should be to provide evidence to choose between a wide range of pre-existing hypotheses. It therefore follows that the hypotheses must come first.
Seeking your hypothesis in your results is an acknowledged error in many scientific disciplines. So much so that it been given its own acronym – HARKing (from Hypothesizing After Results are Known (3)). In statistics-based research the same error is known as p-hacking (4). Both techniques have been, and undoubtedly still are, widely used in science (5).
The parable of the Texas Sharpshooter provides a dramatic example of the HARKing at work. As the cited Nature article (5) explains:
“The Texan directs multiple shots at a barn door. He then selects the closest grouping of the random impacts and draws a target around them. A carefully cropped photograph is then used to establish his sharpshooting credentials.”
The Texas Sharpshooter
This idea that theories should precede observation is the exact opposite of what is popularly believed, including by many scientists. Sherlock Holmes, for example, is usually quoted approvingly:
“It is a capital mistake to theorize before one has data”.
But Sherlock’s patronizing throw-away line to Watson could not be more wrong. Contrast it with these quotes from real scientists – as opposed to a fictional one.
In Biology:
“How odd it is that anyone should not see that all observations must be for or against some view if it is to be of any service.” Charles Darwin.
In Physics:
“We never draw inferences from observations alone, but observations can become significant when they reveal deficiencies in some of the contending explanations.” David Deutsch, The Fabric of Reality, 1998.
In the Philosophy of Science:
“The facts that we measure or perceive never just speak for themselves but must be interpreted through the coloured lens of ideas…. We can no more separate our theories and concepts from our data than we can find a true Archimedean viewpoint – a God’s eye view – of ourselves and the world”. Michael Schermer, Scientific American, 2007
In Medical Research:
“…you cannot find your hypothesis in your results. Before you go to your data…you have to have a specific hypothesis to check. If your hypothesis comes from analyzing your data, then there is no sense in analyzing the same data again to confirm it.” Ben Goldacre, Bad Medicine, 2008.
In Psychology:
“I am more and more convinced that the only way to obtain clear answers from Nature is to ask her clear questions.” Eric-Jan Wagenmakers, Professor of Neuro-Cognitive Modelling, University of Amsterdam, 2014.
Final words
But despite this, it is my observation that the geologist-as-camera view is becoming increasingly common in exploration geology. This slows down the acquisition of knowledge and can be disastrous for understanding.
One should always remember the quatrain:
Data is not information
Information is not knowledge
Knowledge is not understanding
Understanding is not wisdom (7)
Without geological understanding, drill holes are put in the wrong place, or ten are drilled where one would have sufficed. Without good geological models reflecting reality, the certainty required to convert an Ore Resource into a bankable Proven Reserve cannot be easily or cheaply achieved.
No one – beyond its most starry-eyed proponents – believe that AI will one day be able to fill in all the steps that lie between data and wisdom.
N.B. This is an updated and amended version of a previous post entitled The Camera and the Interrogator
(1) Revisiting Chamberlin: Multiple Working Hypotheses for the 21st Century. LP Elliot & BW Brook. BioScience 57 (7), 608-614 https://doi.org/10/10.1641/B570708
(2) The method of multiple working hypotheses. Thomas Crowther Chamberlin 1890 Science 15 92-96 (reprinted in Science 148, 754-759, 1965). https://doi:10.1126/science.ns-15.366.92
(3) HARKing - an acronym for Hypothesizing After Results are Known.
(4) p-hacking - carrying out multiple sets of analysis on massive multivariate data sets until the one analysis is found that has “statistical significance” (i.e., p ≥ 0.05, meaning: 5 or fewer chances in a hundred that the result – or indeed any publishable result – was generated by pure chance. The “negative” or “null” analyses go to the filing cabinet or trash can): the “positive” result is published. This leads to what is known publication bias – see footnotes 5 and 6.
(5) How scientists fool themselves – and how they can stop. 2015 paper by Regina Nuzzo. Nature 526,182-185; https://doi.org/10.1038/52618a
(6) The cumulative effect of reporting and citation biases. 2018 by Y.A. De Vries (and five co-authors). Psychological Medicine 48, 2453-2455 https://doi.org/10.1017/S0033291718001873
(7) These oft-quoted lines, which are usually attributed to American physicist Clifford Stoll or rock poet Frank Zappa, almost certainly were inspired by the 1934 T S Elliot poem, The Rock:
“Where is the wisdom we have lost in knowledge?
Where is the knowledge we have lost in information?”