There is still little scientific progress in explaining anomalous phenomena. While part of the reason undoubtedly lies in the apparent elusiveness of the phenomena, investigators are not blameless. The standard of case investigation is highly variable, and theoretical efforts often leave much to be desired. Indeed, it could be argued that poor theories, in particular, have done more to hold our subject back than any other single factor.

Science has been conspicuously successful in explaining many of the phenomena humans observe, especially physical effects. However, progress in the field of anomaly study remains slight. Significant reasons for this include the lack of funded research by recognised scientists (the area is still stigmatised by many professionals as 'cranky') as well as the comparative rarity of the phenomena. The fact that most phenomena are witnessed unexpectedly by untrained observers in uncontrolled circumstances is also a severe problem. The frequent lack of any physical, measurable residue is also distinctly unhelpful.

Science works through observation, followed by hypothesis and experiment (to test the hypothesis). This is repeated in an endless cycle, producing a steadily more and more accurate, though always provisional, model of reality. In the case of the paranormal, the experimental phase has not been promising. Decades of laboratory experiments have produced a slight statistical deviation from randomness pointing towards the existence of very weak ESP (extra-sensory perception) and PK (psychokinesis). This is not hugely surprising when you consider that many of these experiments were undertaken with individuals who did not regard themselves as psychic. With such slight evidence in favour of the very existence of the paranormal, a scientific explanation is still seemingly some way off.

It is legitimate to ask whether a slight statistical tendency in laboratory experiments is really the same thing that allows poltergeists to throw chairs about or people to accurately predict accidents. Are ESP and PK always present at a low background level, occasionally flaring up to become dramatically obvious? Is there any evidence that the weak laboratory phenomena are the same as the amazing paranormal happenings reported in the press? In fact, there is some. Such things as the sheep/goat effect ('believers' producing more phenomena than 'non-believers') seem to occur both in the laboratory and in spontaneous cases. The elusiveness factor (where the more controls you introduce to eliminate the possibility of natural causes the more effects are diminished) also seems common to both situations. So laboratory and spontaneous paranormal effects probably are one and the same thing. If so, then it could be said that we in ASSAP are mainly concerned with finding out what occasionally amplifies these phenomena to become the stuff of poltergeists.

I would suggest that the reason for a lack of scientific progress in our understanding of anomalous phenomena lies somewhere in the following alternatives:

• that there are simple explanations for paranormal phenomena but they are obscured by the rarity and poor recording of data
• that there are complex explanations for the paranormal, possibly involving many multiple (probably interacting) effects
• that paranormal effects lie completely outside current scientific understanding.

Given that none of these possibilities has yet been eliminated, we must proceed with due caution.

As researchers, we should concentrate on the practical aspects of the problems outlined above. There are two problems that we can and should tackle if we are to progress. One is poor data collection and the other is inadequate theory based on those data. Theories are the driving force of science, but if they are based on inadequate data then the subject can be driven down a cul-de-sac. Science is self- correcting because new evidence pushes inadequate theories into extinction. That is not happening as it should in our subject, due mainly to the lack of decisive, high quality evidence.

Problems with data collection

As if it is not bad enough that paranormal phenomena are rare, they are often also inadequately recorded. Many of our cases rely solely on witness testimony. People are not good recorders of phenomena at the best of times. When faced with unexpected, sometimes distressing one-off events they become downright unreliable. In addition, observing conditions (lighting, weather, distractions etc.) are often far from ideal. The fact that these events are sometimes interpreted in terms of personal beliefs and convictions further muddies the water. The stuff of our subject tends to overlap with the religious and philosophical beliefs of many witnesses.

Witnesses often do not report their experiences for a significant time. They are unlikely to have heard of ASSAP or similar organisations and will usually discuss their case with various friends and outside agencies first. These people will often bring their own beliefs and philosophies into the equation, which may actually alter some of the details that the witness later recalls. Quite often the press gets hold of a case fairly early on and is likely to be either highly skeptical or take round 'Psychic Fred' to add 'spice' and bring further 'facts' to light. All of this can bury the original facts of a case under a heavy load of premature interpretations.

If witnesses are lucky, they will then be investigated by an organisation like ASSAP which tries to rescue, in an unbiased way, whatever fragments of the original observation are left. If the witnesses are unlucky, the case will be investigated by other groups or individuals who bring certain preconceptions about the nature of the phenomena with them. Whether many of the original unvarnished facts survive such an onslaught (by people usually accepted by the witnesses as experts) untarnished is doubtful. Many groups publish the results of their investigations. How is the casual reader to know how a case has been investigated? Sadly, much of the biased data is given equal prominence to material investigated in a more neutral manner. Equally serious, biased investigators may antagonise witnesses with their views and make people unwilling to come forward in future. Incidentally, really unlucky witnesses may be disturbed by many different groups of investigators, none of which knows about the others, causing some irritation.

Some methods of investigation, while not actually motivated by belief, nevertheless show clear signs of assumptions beforehand, eg. a purely psychological approach. Such assumptions, however reasonable they may appear to serious experienced researchers, are usually not proven. Thus their use seems premature, to say the least. Will we miss vital clues because of their use? Even if a conspicuously successful theory does emerge from the evidence, we should continue to record ALL facts for posterity. All theories, however successful, are provisional and oddities and anomalies in the evidence may be the basis for the next generation of theories. As an example, if we accept that those UFO experiences with no obvious explanation are psychological or physiological in origin should we stop looking for physical traces of 'landings'? If we did we might miss an important factor which could decide between competing 'natural' theories. If aliens ever did land on the White House lawn, they might be surprised to be dismissed as hoaxers by some anomaly researchers!

Inadequate theories

The inadequacies of observation in our field are well-known. But we also suffer from a second significant problem which is rarely discussed - inadequate theories. In most fields of science there is a substantial body of accepted theory. In our field this body is very small indeed. Nature abhors a vacuum and so makes up for this lack of accepted theory with unsubstantiated speculation. Exciting though such speculation is, it does not get us anywhere much in a strictly scientific sense.

I will now list some of the problems which afflict many theories in our field making them, strictly speaking, unscientific. That is not to say that these theories are necessarily false, but they are never likely to progress to general acceptance without a firm dose of critical analysis. Such theories should be regarded as interesting speculation which can stimulate adventurous research but nothing more. They are never likely to be accepted by the scientific establishment. Sadly, some popular writers propagate these theories, which can confuse newcomers to our subject, who may be led to think the theories have scientific status. It is not unusual to meet a newcomer to our subject who confidently asserts that 'it has been proved' (by a mysterious 'they') that Elvis was in fact abducted by vampires, or some such similarly unlikely claim?

Inelastic assumptions

All theories rest on assumptions but in truly scientific hypotheses these can be modified by new evidence. This is sadly untrue in the case of many theories of the paranormal. Skeptical theories typically assume that nothing can occur that is not explicable by current science (often forgetting that 'current science' is forever changing) and so any 'paranormal' effects are essentially illusory in nature. Similarly, there are 'believer' theories which treat 'current science' as an inconvenient restriction to be entirely ignored when necessary. Other theories try to make no assumptions concerning the true nature of the paranormal and, due to their flexibility, are more likely to be successful.

A typical symptom of inelastic assumptions is the problem of falsifiability. In a truly scientific theory, it should be possible to devise an experiment that could prove the theory wrong (otherwise it would explain 'everything' and hence nothing). Instead, many theories march on year after year in the face of all contradictory evidence. They are either superficially modified to 'explain' glaring inconsistencies (often by bolting on highly unlikely riders) or the new evidence is simply ignored or declared suspect! The basic assumptions invariably remain unchanged, however.

It may come as a surprise to non-scientists that theoretical assumptions are constantly under test in the profession. Whenever a new experimental technique that can better test the accuracy of a theoretical prediction becomes available, it is seized upon rapidly. Einstein's relativity theory is regularly retested whenever some new method appears capable of doing so. There seems to be nothing a jobbing scientist would like more than to bring down a widely accepted theory with a clever new experiment. These experiments can often seem a bit negative - they show that an alternative theory does NOT work. Our field could benefit hugely both by challenging what theories we already have and by demonstrating those commonly believed aspects which are NOT true. By eliminating the impossible we can move, if rather slowly, closer to genuinely workable theories.

Poor scientific method

Anomaly theories typically appear in unrefereed journals. Scientific theories usually appear in refereed journals where they have been reviewed by expert peers for originality and scientific rigour. There are regular debates about the validity of this method but it continues to be generally accepted. I would not suggest that anomaly theories should only appear in refereed journals (Anomaly is not refereed). However, it does mean that some theories are published which have never been checked by experts for obvious factual errors and logical inconsistencies. Once published, such theories are often treated as valid by non-expert readers.

People without formal scientific training often have a poor grasp of how to construct and communicate a coherent theory. For instance, they often see no reason to demonstrate that the hypothetical mechanism they use to explain phenomena: (a) can actually exist, even in theory, (b) has ever been recorded in nature and (c) actually occurred in the case(s) described.

In their enthusiasm to demonstrate a new theory, some researchers consult extremely specialised scientific papers. They then suggest novel effects by extrapolating the results of these papers far beyond what was intended by the authors. This problem typically arises when theorists go straight to the advanced literature of an unfamiliar field without first obtaining an understanding of the basics. Writers of such scientific papers tend to assume that their readers will understand such basics and so appreciate, without having to be told explicitly, any unspoken limitations to their conclusions. Results may have been obtained in extreme laboratory conditions which simply never occur in everyday life, a fact which may not be appreciated by eager theoreticians. In addition, keen theorists may not have adequately surveyed the whole range of scientific sources. Science is a controversial subject, and there may well be papers whose results contradict those used to produce a theory. These should at least be consulted and taken into account.

Poor data selection

Data from cases and experiments (as well as scientific papers and books) are naturally the source of novel theories and can also be used to test them. However, data are often selected in ways that are not strictly statistically valid. Typical examples are the inclusion of poor quality data, eg. through the use of unchecked newspaper articles or investigation reports from biased sources. Anyone who has had their 'story' reported in the press will know only too well how facts can be distorted. Books and magazines can often contain similar inaccuracies. These sources have the additional problem of frequently being out of date. Sometimes cases which are known to be fraudulent by most researchers are still doing the rounds of popular books years later.

Some theoreticians have attempted to analyse Fortean data in terms of the paranormal. An important point to note here is that Forteans are interested in the unusual. The data they study may not all necessarily be paranormal. Many perfectly natural, if unusual, effects are regularly recorded by Forteans. Thus to include such data in any theory of the paranormal is unlikely to produce any sensible results.

In anomaly research data sample sizes are often too small for valid scientific statistical analysis. While this not the fault of the theoretician it nevertheless cannot simply be ignored. Some people try to 'hide' this problem by using percentages rather than absolute totals (always a bad sign to look out for). Thus 40% sounds a respectable number whereas just 2 out of a miserably small sample of 5 sounds less so. With such small sizes, samples might happen to be taken at time periods or places which fortuitously appear to back a particular theory. As many different samples as possible should be taken to justify a theory.

Anomalous phenomena often occur bunched in groups over a short time period and in a restricted geographical area. These are called 'flaps' when applied to UFOs. Some theoreticians have based their samples largely on such flaps, as they are a rich source of material. However, there is a danger here. Such flaps are often known to be spuriously generated by publicity. They will typically include many mistaken observations generated on a wave of enthusiasm, fraudulent reports from publicity- seekers and ancient cases dug up to fill column inches. Even if most of the cases in a flap are entirely genuine, they may have a very particular local cause that is not typical of the phenomenon in general. It has been suggested that some geographical concentrations of cases can occur when a particularly active investigator lives in the area. This can give rise to spurious associations of cases with local geographical features.

The means of classifying data type (eg. what set of reported phenomena constitute a poltergeist) to test theories can also produce spurious results (particularly when using small samples). Such classifications may unconsciously be defined by the theoretician using the in-built assumptions of the theory to be tested. When analysis using these classifications confirms the theory it should come as no real surprise. Thus it might be 'convenient' to classify a haunting as different from a poltergeist if all cases of this type happen to occur in houses with streams next to them. Trying different classifications for poltergeists (for instance) may make other conclusions appear. To avoid such a bias it would be sensible to remodel the data trying several different classification systems to see whether apparent initial associations are real or not.

The study of anomalous phenomena is based around the idea of examining the odd bits and pieces which do not fit into current science. This can lead to a tendency for anomaly researchers to try to include every single detail of a case, however trivial, and fit it in somehow. This inhibits the idea of sensible working theories. Only if particular factors occur consistently in many cases need they be taken into account in the development of an initial working theory. Other oddities may simply arise from genuinely mistaken observation. All the details of a case should always be recorded, but not all should necessarily be included in a provisional theory.

Poor discrimination

When one is presented with a novel theory and the data to back it up, it is easy to be led into believing that theory to be the only one possible to explain the cases presented. In their enthusiasm to present their case, some theorists may not examine alternatives thoroughly. Ask yourself if there are alternative explanations for the cases presented. If so, have these possibilities been eliminated satisfactorily (or even presented for comparison)? Is the test being used to detect the proposed theoretical mechanism sufficiently sensitive to actually exclude other possibilities? If such a theoretical mechanism is detected, is it present in sufficient quantity to cause what the theory predicts? Does the theory explain ALL such cases satisfactorily (even those NOT presented by the theoretician as examples)? Has the 'null theory' (that the apparent connection between proposed cause and effect is just a coincidence) been convincingly eliminated? This could be done by looking for similar cases in areas or at times when the proposed theory predicts they should NOT be present. You could also look for cases where the theory predicts they ought to be present but have not so far been reported (this approach is more difficult because you are trying to prove a negative - if you do not find anything, you may just have missed it). Sadly such questions are rarely asked of those that propose theories in our field.

Towards a gold standard

Where does all this leave ASSAP? I do not want to discourage non-scientists, or indeed scientists, from investigation and theorising. However, theorists should really apply scientific standards if they want credible hypotheses. ASSAP has many scientifically trained members who can help aspiring theoreticians. ASSAP is attempting to use scientific standards to investigate cases. It is hoped that these will form a solid foundation on which rational theories can be built which will advance our subject and not hinder it. I would like to suggest that researchers adopt a 'gold standard' in both data collection and theory development. I will now list some essential points in such a gold standard. ASSAP already attempts to follow the points on data collection. We are hoping to develop the ideas on theories in future.

The 'gold standard' points are as follows:

On data collection:

• investigate in an unbiased way using trained individuals
• look for ALL possible evidence, however seemingly irrelevant, and record it carefully
• corroborate all recorded facts if at all possible
• get as much physical evidence as possible (eg. vigils, site examination etc.)
• try to eliminate natural causes and coincidences
• use the same basic methods for investigating all cases so that they can be meaningfully compared
• if possible, experiment on the live phenomena

ASSAP has already improved standards in its case work, through training and careful methodology. No would-be investigators should be put off by this serious approach. Instead they should consider the possibility that they may be making a real contribution to science by doing things in a careful, disciplined way.

• look critically at the evidence and theories
• get other (especially scientifically trained) people to check theories before publication
• test theories against the points mentioned in this article

I would urge all ASSAP members to view critically any investigation reports, accounts of experiments or theories they are presented with in our field. Sadly, many fail to stand up to such careful examination and simply serve to confuse and retard the scientific study of anomalous phenomena.

Problems with data collection

Given what has been said in this article concerning inadequate theories, it is legitimate to ask where we should go from here. Also, given the singular lack of progress, we might as well go back to the beginning. ASSAP and others, aided by new cheap, highly accurate electronic equipment, are attempting to collect a higher standard of case data than has ever been possible before. With this information, we can examine the evidence afresh.

There is, for a start, the problem of classification. Take as an example three hypothetical cases. In one, a transparent human figure is seen. In another, an apparently solid figure is seen to walk through a wall. In the third, the figure of someone who is known to be dead is seen, apparently alive. All could be classified as ghosts or apparitions. But are they the same phenomenon and do they share the same cause? By lumping them together under the label 'ghost' we may be obscuring their cause. Is it perhaps time to re-examine the labels we use for the phenomena?

A possibly better approach could be as follows. Supposing we simply record the actual events that appear to violate current laws of science, without attempting to label them. Thus the three examples above would be recorded as: a transparent human figure seen, a solid human figure seen to pass through a solid object and a dead person seen apparently alive. We could then quantify the event, eg. size of figure, apparent age, degree of solidity etc. Then we could record the circumstances of the sighting, eg. position, time, lighting, weather, witnesses, etc. We could also record any history of such sightings in the area and see if the witness had any history of similar experiences there or elsewhere. Finally we could attempt to quantify the percentage reliability of the sighting (based on credibility of witness, sighting conditions, reliability of report sources, factors ruling out misobservation, etc.). This quality rating could be used to assign relative weight when classifying phenomena.

Armed with such simple facts and a quality assessment, we could begin to reclassify the phenomena. We would allow the data themselves to dictate the classes. Supposing we record one case of a transparent figure being seen. This would form a 'base class'. A base class is one derived from answering the question 'what single event actually violates the current laws of science?' We could trawl case reports for other such base classes to form the basic units from which to construct new classifications. Many base classes would, however, almost certainly not represent sensible units to classify real-life phenomena. We need to go to a higher level of classification to find such 'real-life' entities.

We could search for other instances of the same base classes in case reports. By then analysing such groups of occurrences we could look for patterns. We might find, for instance, many transparent figures but very few solid ones. This would mean that transparent figures are some kind of naturally occurring class, whereas the solid figure might be a misobservation (though it could simply be very rare).

Towards new theories

Once we have obtained new, realistic classifications all sorts of interesting possibilities open up. We could look for consistent factors occurring when particular classes of phenomena are seen. We might find that our transparent greenish examples tend to turn up on wet Wednesdays in September to children. All of these factors might pose further questions for theorists to puzzle over. Such searches for consistent factors have been done before. However, they suffered from poor quality of data and lack of any good definition of the 'entity' being sought. It is no use looking for consistent factors if you are treating both fully formed solid figures and the sounds of footsteps equally as 'ghosts'. The definition is far too loose to be meaningful. It should come as no surprise when such studies produce little of value. Unless you can definitively tie figures to footsteps you do not know if they are likely to have the same cause.

Once common factors have been identified, this information can be fed back to investigators. They can then deliberately look for such factors in apparently typical cases. This is not to say that other factors should then be ignored. If there is one thing we have learned over the years of studying the paranormal, it is to expect the unexpected. Other factors should be recorded, even if they do not appear to fit the theory.

Cases with 'negative' results will be particularly useful in such studies. By negative results, I mean apparently paranormal sounding phenomena which turn out to have natural causes. The very fact that such cases produce phenomena that make them look like the real thing makes them excellent 'control trials'. Just as in drug trials, these cases should be found NOT to have the factors we have previously identified as occurring in genuine cases. If this is so, it reinforces the validity of the original theory. So please do write up those 'negative' cases, as they are every bit as important as the 'genuine' ones.

Once apparently fundamental classes have been identified we can look for connections between them. This could form another fruitful source of new theories. Suppose that a figure is seen simultaneously by two witnesses. If one witness sees a transparent greenish figure while the other sees a solid one, there is clearly a link. It may hint at an even more fundamental classification. Trying to unite the two classes could give rise to new theories.

In the long term we could even try to recreate the phenomena artificially. Supposing we get together a group of children on a wet Wednesday in September, would one of them see something? We could manipulate the various factors we have identified to see which of them is truly significant and which just coincidental.

I hope you will see from this that approaching paranormal data from the lowest level might be more fruitful than the existing methods. Until we have identified the fundamental phenomena we are studying, we might be comparing chalk with cheese. This, coupled with poor data and inadequate theories, makes it unsurprising that only limited progress has been made in getting scientific explanations for the phenomena that fascinate us.

[This article first appeared in ANOMALY number 17 as 'The Gold Standard']