The Camera and the Interrogator

The camera and the interrogator

Speak to exploration geologists and you will find two opposing views about what a geologist should do when observing outcrop or drill core in the field.  Some seek merely to be unbiased objective recorders of what they see.  Others observe the rock in the wider context of their theories about region or prospect geology and ask questions of the exposure to help choose between them. I characterize these two approaches with the metaphors of  geologist-as-camera, and geologist-as-interrogator.

Geologist as camera – 1

I arrive at the exploration site to find a team of three young geologists engaged in making a geological map of their large property. Using GPS, they walk predetermined traverses spaced 400m apart. They each aim to average 8km in a day and, between them, by taking alternate lines, they cover a large swathe of country before their next field break. Observations on each geological feature along the line of march are logged into a field-hardened tablet computers by going through a series of pull-down menus and checking boxes on pre-determined questions. The geologists have only the vaguest ideas about the geology of the property they are mapping. Two of them have never thought much about this: the brightest of the three specifically rejects the notion of understanding her observations: she sees herself as an unprejudiced, objective observer – confirmation bias is not for her. Eventually all this data is computer-plotted to 2D images as a linear sequences of point observation.  Another, more senior geologist, might eventually produce a geological map by joining the dots. Probably there is a software program that can do much of this task for him (or her) and thus obviate the need for too much thought or hard work.

Geologist as Camera – 2

I travel to an exploration site on the other side of the country. Here, a much larger company is at the final stage of drill testing a substantial metal deposit. Four diamond drill rigs are going 24/7 and have reached hole number 300 and something. Stacked on pallets, kilometers of core are waiting to be logged. Four geologists are hard at work logging core laid out on long racks in a big shed. As geologists come and go on field break, or come and go through resignations and new hires, it seldom happens that any one hole is logged in its entirety by the same person. They log on to analytical spread sheets in laptop computers using pre-determined menu options – there are more than 50 columns on their spreadsheet. The same log form has been used since DDH 001. The geologists have never seen a drill section of the prospect (there are none). There are no detailed maps or level plans either. The geological model, such as it is, was produced a year before by an outside consultant who spent ten days on site. Back in the head office a specialist Ore Reserve Geologist ignores most of the vast data base of geological observations and calculates a resource based on assay numbers, virtual reality 3D string models and geostatistical techniques. For the geologists at point, slaving in the core yard, their job as 100-meter-a-day, core-logging automata is mind-numbingly boring. They count the days till their next field break, or until they have earned enough money to find some other more intellectually stimulating employment.

After a day data logging, Susan contemplates a career change

These are anonymized projects, extreme examples perhaps, but based on observations of many actual projects, with many companies in different countries over many years. Not all exploration projects are conducted like this, but I am sure that readers will recognize the method of doing geology that I describe. The geologists are not to blame. In many cases it is their first job, they are on short-term contracts, and they are doing what their employer asks and expects of them. They probably wonder what the point was of much of the time they spent learning at university and have come to believe that the essence of their job as exploration geologists is to convert light signals from their eyes into digital computer feed according to pre-set formulae. A kind of biological digital camera.  Without a pre-existing context in their brain, each observation they make has equal importance, each pixel equal weight.

There is a better way.

The geologist as interrogator

All observation is made in a particular context. The context which should guide the exploration geologist is a matrix of different competing theories about the true nature of the geology being observed. This context provides the questions that must be asked of each outcrop or each piece of drill core. It defines the strategy to be followed in the search for those critical observations that allow selection between multiple working hypotheses (1), (2). The idea of multiple working hypotheses was first propounded by 19th Century US Geological Survey geologist Thomas Crowther Chamberlin. It is a methodology now used in all fields of science research, but geologists can be proud that the first clear statement of the idea came from their profession.

Critical observations are those that fit into a pre-existing context or, just as importantly, those that do not. These are prioritized and not lost in a sea of trivial observation. The method does not guarantee the you will arrive at the correct solution. Your evidence may be less than ideal, the expression in the rocks of geological events may be atypical.  Nature can be chaotic and unpredictable: the true hypothesis you should be testing  may be a Black Swan – the one you have never thought of. In Geology – indeed in all science – neat, clean results where all the data slots in exactly are rare and should be always regarded with some skepticism. But in spite of all that, the method of multiple working hypotheses is the best known procedure for approaching the truth.

After a morning of  fieldwork, Robert interrogates his lunchtime sandwich using the method of multiple working hypotheses

Collecting data to test hypotheses in this way is biased observation. All geologists have cognitive biases which were imprinted when they were taught to be geologists at University, and reinforced by early-career mentoring and  reading geological literature relevant to the job.  But this bias is a good thing where it is up-front, acknowledged and constantly revised and re-focused in the face of evidence. Without bias, there is no way of separating signal from noise – not even by the most sophisticated of statistical procedures. This is a quite different kind of bias from the one that uncritically accepts evidence that confirms a single pre-existing theory, or a theory you have fallen in love with,  and ignores, or explains away, evidence which does not. That is Confirmation Bias and is rightly condemned.

All scientists have a point of view and all views must be from somewhere. Anyone who claims to make unbiased observations merely lacks self-awareness. Their biases are unconscious or lie in the biases of whoever drew up the detailed procedure manual which they follow.

The metaphor I use for this approach is the geologist as an interrogator of each rock exposure or core piece, asking a series of relevant questions that come from his or her views as to what might be happening, with each question determined by the answer to the previous question.

RM ’14

In a previous post I explore how using the technique of multiple working hypotheses can be used when constructing a geological map (LINK)

The wider context

The true nature of scientific investigation is focused observation guided by emergent theory. This is a long established and respectable idea and contrasts with seeking to find your theory in your data after you have completed your observations.

Collecting data should not be a fishing expedition that hopes to fortuitously entangle a new understanding or ideas on its line.  Its purpose should be to provide evidence to choose between a range of pre-existing hypotheses.  It therefore follows that the hypotheses must come first.

Seeking your hypothesis in your results is such a widespread and acknowledged error in many scientific disciplines that it has its own acronym - HARKing[3] (Hypothesizing After Results are Known). In statistics-based research the same error is known as p-hacking [4].  Both are surprisingly widespread [6].

The parable of the Texas Sharpshooter provides a good example of  the HARKing technique at work [5] (6) :

 The Texan draws his revolver and, loading and firing, directs sixty shots at a distant barn door. He then selects the best grouping from the impacts and draws a target around them. He takes a carefully-cropped photograph and uses this to establish his sharpshooting credentials.

This idea that theories should precede observation is the exact opposite of what is popularly believed. Sherlock Holmes, for example, is often quoted approvingly:

“It is a capital mistake to theorise before one has data”.

But Sherlock’s patronising throw-away line to Watson could not be more wrong. Contrast it with these quotes from real scientists – as opposed to a fictional one.

In Biology:

“How odd it is that anyone should not see that all observations must be for or against some view if it is to be of any service.”  Charles Darwin.

In Fundamental Physics:

“We never draw inferences from observations alone, but observations can become significant when they reveal deficiencies in some of the contending explanations.” David Deutsch, The Fabric of Reality, 1998.

In the Philosophy of Science:

“The facts that we measure or perceive never just speak for themselves but must be interpreted through the coloured lens of ideas…. We can no more separate our theories and concepts from our data than we can find a true Archimedean viewpoint – a God’s eye view – of ourselves and the world”. Michael Schermer, Scientific American, 2007

In Medical Research:

“…you cannot find your hypothesis in your results. Before you go to your data…you have to have a specific hypothesis to check. If your hypothesis comes from analysing your data, then there is no sense in analysing the same data again to confirm it.”  Ben Goldacre, Bad Medicine, 2008.

In Psychology

“I am more and more convinced that the only way to obtain clear answers from Nature is to ask her clear questions.” Eric-Jan Wagenmakers, Professor of Neuro-Cognitive Modelling, University of Amsterdam, 2014. 

 Final words

But in spite of this, it is my observation that the geologist-as-camera view is becoming increasingly common in our profession.  This slows down the acquisition of geological knowledge and can be disastrous for understanding.

Data is not information

Information is not knowledge

Knowledge is not understanding

Understanding is not wisdom (8)

Without geological understanding, drill holes are put in the wrong place, or ten are drilled where one would have sufficed. Without good geological models that reflect reality, the certainty required to convert an Ore Resource into a bankable Proven Reserve cannot be easily or cheaply achieved.

 

This is a modified and updated version of an essay first posted in 2014