User:Matthias Brendel/Scientific method

Scientific method is the method of science. In most cases it refers only to the method of empirical sciences, which include natural and social sciences, but do not include mathematics. We do not describe here the method of mathematics, and use the word "science" for "natural sciences" in the following. There is also a different opinion if the followings shall be understood in a descriptive way, i.e. describing the actual method of science, or prescriptive, i.e. describing, how science should be done ideally. In the second case it may be disputed, how close actual science is to this ideal description.

Some scholar doubt that one or some methods of science exists, may exist or may be described. Most of those thinkers, who accept that there is a method of science describe it like it follows in this introduction. We will also point to different opinions later.

The method of empirical sciences describes processes for investigating phenomena and acquiring new knowledge, as well as for correcting and integrating previous knowledge. It is based on observable, empirical evidence, and subject to laws or reasoning.

Although specialized procedures vary from one field of inquiry to another, there are identifiable features that distinguish scientific inquiry. Scientific researchers propose specific hypotheses as explanations of phenomena, and carry oout more observations or design experimental studies that test these predictions. These steps are repeated in order to create better theories. There is no perfect consensus what exactly the definition of "better" is for theories.

Theories that encompass whole domains of inquiry serve to bind more specific hypotheses together into logically coherent wholes. This in turn aids in the formation of new hypotheses, as well as in placing groups of specific hypotheses into a broader context of understanding.

Among other facets shared by the various fields of inquiry is the conviction that the process must be objective so that the scientist does not bias the interpretation of the results or change the results outright. Another basic expectation is that of making complete documentation of data and methodology available for careful scrutiny by other scientists and researchers, thereby allowing other researchers the opportunity to verify results by attempted reproduction of them. Note that reproducibility can not be expected in all fields of science.

Scientific publication of measurement and thopries also allows statistical measures of the reliability of the results to be established.

Elements of scientific method

There are multiple ways of outlining the basic method shared by all of the fields of scientific inquiry. The following examples are typical classifications of the most important components of the method on which there is very wide agreement in the scientific community and among philosophers of science, each of which are subject only to marginal disagreements about a few very specific aspects.

Experiments

Once predictions are made, they can be tested by experiments. If test results contradict predictions, then this constitutes an anomaly, and explanations may be sought. Sometimes experiments are conducted incorrectly and are at fault. If the results confirm the predictions, then the hypotheses are considered likely to be correct but might still be wrong and are subject to further testing.

Depending on the predictions, the experiments can have different shapes. It could be a classical experiment in a laboratory setting or a double-blind study. Even taking a plane from New York to Paris is an experiment which tests the aerodynamical hypotheses used for constructing the plane.

Scientists assume an attitude of openness and accountability on the part of those conducting an experiment. Detailed recordkeeping is essential, to aid in recording and reporting on the experimental results, and providing evidence of the effectiveness and integrity of the procedure. They will also assist in reproducing the experimental results. This tradition can be seen in the work of Hipparchus (190 BCE - 120 BCE), when determining a value for the precession of the Earth over 2100 years ago, and 1000 years before Al-Batani.

Note that experiments are not a necessary part of scientific method. There are purelly observational sciences, or fields of science, like history and astronomy, where only observation is possible.

Hypothesis development

A hypothesis is a suggested explanation of a phenomenon, or alternately a reasoned proposal suggesting a possible correlation between or among a set of phenomena.

Normally hypotheses have the form of a mathematical model. Sometimes, but not always, they can also be formulated as existential statements, stating that some particular instance of the phenomenon being studied has some characteristic and causal explanations, which have the general form of universal statements, stating that every instance of the phenomenon has a particular characteristic.

According Hans Reichenbach's distinction of context of justification and discovery, there is no single method describing how to create hypotheses. Scientists are free to use whatever resources they have — their own creativity, ideas from other fields, induction, Bayesian inference, and so on — to imagine possible explanations for a phenomenon under study. Charles Sanders Peirce, borrowing a page from Aristotle (Prior Analytics, 2.25) described the incipient stages of inquiry, instigated by the "irritation of doubt" to venture a plausible guess, as abductive reasoning. The history of science is filled with stories of scientists claiming a "flash of inspiration", or a hunch, which then motivated them to look for evidence to support or refute their idea. Michael Polanyi made such creativity the centrepiece of his discussion of methodology.

Karl Popper, following others, notably Charles Peirce, has argued that a hypothesis must be falsifiable, and that a proposition or theory cannot be called scientific if it does not admit the possibility of being shown false. It must at least in principle be possible to make an observation that would show the proposition to be false, even if that observation had not yet been made. Popper was criticised for this demarcation of scientific and non-sicientific hipotheses.

In general scientists tend to look for theories that are "elegant" or "beautiful". In contrast to the usual English use of these terms, they here refer to a theory in accordance with the known facts, which is nevertheless relatively simple and easy to handle. If a model is mathematically too complicated, it is hard to deduce any prediction. Note that 'simplicity' may be perceived differently by different individuals and cultures.

For example Linus Pauling proposed that DNA was a triple helix. Francis Crick and James Watson learned of Pauling's hypothesis, understood from existing data that Pauling was wrong and realized that Pauling would soon realize his mistake. So the race was on to figure out the correct structure. Except that Pauling did not realize at the time that he was in a race.

Predictions from the hypotheses

Any useful hypothesis will enable predictions, by reasoning including deductive reasoning. It might predict the outcome of an experiment in a laboratory setting or the observation of a phenomenon in nature. The prediction can also be statistical and only talk about probabilities. Note, that generally a single hipothesis is not enough for a prediction, but many hipotheses and assumptions are needed for deducing it. Also note, that a prediction must not always point to the future. There might be a prediction refering to an event in the past, or a property of an object in the past, which is unknown, but may be measured or observed later.

It is essential that the outcome be currently unknown. If the outcome is already known, it's called a consequence and should have already been considered while formulating the hypothesis.

If the predictions are not accessible by observation or experience, the hypothesis is not yet useful for the method, and must wait for others who might come afterward, and perhaps rekindle its line of reasoning. For example, a new technology or theory might make the necessary experiments feasible.

For example,when Watson and Crick hypothesized that DNA was a double helix, Francis Crick predicted that an X-ray diffraction image of DNA would show an X-shape. Also in their first paper they predicted that the double helix structure that they discovered would prove important in biology writing "It has not escaped our notice that the specific pairing we have postulated immediately suggests a possible copying mechanism for the genetic material".

Testing and improvement

The scientific process is iterative. At any stage it is possible that some consideration will lead the scientist to repeat an earlier part of the process. Failure to develop an interesting hypothesis may lead a scientist to re-define the subject they are considering. Failure of a hypothesis to produce interesting and testable predictions may lead to reconsideration of the hypothesis or of the definition of the subject. Failure of the experiment to produce interesting results may lead the scientist to reconsidering the experimental method, the hypothesis or the definition of the subject.

Other scientists may start their own research and enter the process at any stage. They might adopt the characterization and formulate their own hypothesis, or they might adopt the hypothesis and deduce their own predictions. Often the experiment is not done by the person who made the prediction and the characterization is based on experiments done by someone else. Published results of experiments can also serve as a hypothesis predicting their own reproducibility.

For example, after considerable fruitless experimentation, being discouraged by their superior from continuing, and numerous false starts, Watson and Crick were able to infer the essential structure of DNA by concrete modelling of the physical shapes of the nucleotides which comprise it. They were guided by the bond lengths which had been deduced by Linus Pauling and the X-ray diffraction images of Rosalind Franklin.

Peer review evaluation

Scientific journals use a process of peer review, in which scientists' manuscripts are submitted by editors of scientific journals to (usually one to three) fellow (usually anonymous) scientists familiar with the field for evaluation. The referees may or may not recommend publication, publication with suggested modifications, or, sometimes, publication in another journal. This serves to keep the scientific literature free of unscientific or crackpot work, helps to cut down on obvious errors, and generally otherwise improve the quality of the scientific literature. Work announced in the popular press before going through this process is generally frowned upon. Sometimes peer review inhibits the circulation of unorthodox work, and at other times may be too permissive. The peer review process is not always successful, but has been very widely adopted by the scientific community.

Documentation and replication

Sometimes experimenters may make systematic errors during their experiments, or (in rare cases) deliberately falsify their results. Consequently, it is a common practice for other scientists to attempt to repeat the experiments in order to duplicate the results, thus further validating the hypothesis.

As a result, experimenters are expected to maintain detailed records of their experimental procedures, in order to provide evidence of the effectiveness and integrity of the procedure and assist in reproduction. These procedural records may also assist in the conception of new experiments to test the hypothesis, and may prove useful to engineers who might examine the potential practical applications of a discovery.

Note that it is not possible for a scientist to record everything that took place in an experiment. He must select the facts he believes to be relevant to the experiment and report them. This may lead, unavoidably, to problems later if some supposedly irrelevant feature is questioned. For example, Heinrich Hertz did not report the size of the room used to test Maxwell's equations, which later turned out to account for a small deviation in the results. The problem is that parts of the theory itself need to be assumed in order to select and report the experimental conditions. The observations are sometimes hence described as being 'theory-laden'.

Models of scientific inquiry

The deductive-nomological model

The deductive-nomological model describes theories as a set of hypotheses, which are used, to deduce some particular statements, which can be compared to empirical data. If the empirical data is already known, then this way the theory explains them, if they are nort yet known, then this is a prediction of the theory, which can be tested.

There are however different views about how the result of testing imfluences the improvement of the theory.

Positivist Model

Most of the logical positivists focused to justification, or confirmation. Rudolf Canrap developed a model, where the degree of confirmation of a theory is defined. Carnap stated that a potive test result increases this degree of confirmation, and after a reasonable degre, the theory can be considered to be confirmed. There is no exact rule about the treshhold of degree of conformation. Carnap admitted that other factors also influence, if we accept a theory, like simplicity. In his views theory acceptance is a practical issue. This view resembles to the model of Thomas Kuhn, who is however not considered to be a positivist (see later).

Critical Rationalist Model

Karl Popper's model focuses on falsification instead of confirmation. Popper emphasized that no general statement can be justified wtih a finite number of tests without doubt. Because of this, he focused on falsification. In Poppers view, scientist just create some hipotheses (it is not a question of scientific method, how they do it), which have to be falsified. After testing we either succed, to falsify the theory, or not. All the theories, whihc are not falsified, are possible theoiries to accept. Popper's method is called critical ratinalist, because the focus is on the critique of the theory.

Later he defined the empirical content of the theories, whoich is the set of deducable statements. I.e. the empirical content of a theory is proprotional to the number of phenomena, which can be explained or predicted by the theory. Popper's norm was to accept the theory, which is not falsified and has the largest empirical content. In Poppers view, a main virtue of a theory is how many phenomena it is excluding by making it impossible.

If a theory is falsified, then the test do not define, how the theory shall be changed. Tere can be many ways to save a theory. Popper described that in some cases ad hoc changes are done, which are only for saving the theory, but do destruct the empirical content of the theory. For example to hipothise a sorcerer, who distuirbed an experiment is an ad hoc hipothesis, since this sorcerer is not described by rules, so a lot of things may be possible after accepting a hipothesis like this. And as we saw it this is not good in Popper's model.

It was shown that one problem with Popper's theory is, that existential statements are not falsifiable. On the opther hand, as we saw, universal statements are not proovable. Both kind of statements are used in a theory, and this way neither justification, nor falsification is possible for all the statements. This means that both conformation and falsification is used in the method of science. Moreover, they are the two sides of testing.

So, if Popper's and the logical positivist model is unified, it gives us quite a good model for the logical part of scientific method.

Pragmatic model

Charles Peirce considered scientific inquiry to be a species of the genus inquiry, which he defined as any means of fixing belief, that is, any means of arriving at a settled opinion on a matter in question. He observed that inquiry in general begins with a state of uncertainty and moves toward a state of certainty, sufficient at least to terminate the inquiry for the time being. He graded the prevalent forms of inquiry according to their evident success in achieving their common objective, scoring scientific inquiry at the high end of this scale. At the low end he placed what he called the method of tenacity, a die-hard attempt to deny uncertainty and fixate on a favored belief. Next in line he placed the method of authority, a determined attempt to conform to a chosen source of ready-made beliefs. After that he placed what might be called the method of congruity, also called the a priori, the dilettante, or the what is agreeable to reason method. Peirce observed the fact of human nature that almost everybody uses almost all of these methods at one time or another, and that even scientists, being human, use the method of authority far more than they like to admit. But what recommends the specifically scientific method of inquiry above all others is the fact that it is deliberately designed to arrive at the ultimately most secure beliefs, upon which the most successful actions can be based.

Historic Turn

After Thomas Kuhn's famous work, Structure of Scientific Revolutions the main focus shifted from methodology of science to history of science. Kuhn gave a description of scientific processed, based more on history than on logic. This means also a shift from prescriptive to descriptive philosophy. Kuhn's theory distinguishes between normal science and revolutionary periods. Normal science is made based on a so called paradigm, which is a big theoretical framework for developing theopries. For example Newtonian physics was a paradigm. Scientific revolutions are paradigm changes. According to Kuhn, there are some scientific values, like simplicity, which govern theory development, but scientific revolutions are like geschtalt switch. There is no exact methodology to compare theories before and after a revolution, since the different theories are like written in different languages, and noone can understand perfectly both languages. This is the incommensurability thesis of Kuhn. Kuhn was regarded as relativist because of this, but he himself rejected this. It is not clear, how relativist Kuhn was, since the mentioned constant values of science suggest also some ant-relativism. After Kuhn however the focus of history of science was permanent until the focus shifted even more to sociology of science. Kuhn himself take part also in this second shift.

Sociological Turn

The strong programme is a variety of the sociology of scientific knowledge (SSK) particularly associated with David Bloor, Barry Barnes, Harry Collins, Donald MacKenzie, and John Henry. The largely Edinburgh-based school of thought has illustrated how the existence of a scientific community, bound together by allegiance to a shared paradigm, is a pre-requisite for normal scientific activity.

The strong programme proposed that both 'true' and 'false' scientific theories should be treated the same way -- that is, symmetrically. Both are caused by social factors or conditions, such as cultural context and self interest. All human knowledge, as something that exists in the human cognition, must contain some social components in its formation process. The presence of social factors alone is not enough to falsify a scientific theory.

Models based on research programmes

At the end of tho era of logical positivism, philosophy of science focused not only on theories, but bigger parts of scientific research. They named them as research programmes, which are quite close to Kuhn's paradigm. Some post-positivistic philosophers started a detailed analysis of these research programmes. In a research program a main theory is developed and several variants are developed. We have seen models on acceptance of theories, but the new question was the acceptance or rejection of a research program. In the context of research programmes, rejection means that the scientist does not work any mor in the research program. As it was shown however, it is possible for a researcher to work in even rival research programmes, since this is not a question of beleive, but a mor practical question. Larry Laudan and Imre Lakatos tried to describe some criteria oiof how good a research program is. Lakatos defined progressive and degenerative research programmes. Lakatos, as a student of Popper connected these concept to the empirical content of the theory. A research program is degenerative, if the empirical content of the theory gets lower. Lakatos stated taht there is no rule, of which degree of degeneration is a must for quit the program. That is why this is a practical issue.

Larry Laudan defined the degree of progress of a theory by how much problems it can solve over the time. Again, a scientist estimates this measure and decides which program to quit, which to maintain. It is again apractical decision.

Evolutionary models

In 1972, Toulmin published Human Understanding which asserts that conceptual change is an evolutionary process. In contrast to Kuhn’s revolutionary model, Toulmin proposed an evolutionary model of conceptual change comparable to Darwin’s model of biological evolution. Toulmin states that conceptual change involves the process of innovation and selection. Innovation accounts for the appearance of conceptual variations, while selection accounts for the survival and perpetuation of the soundest conceptions. The soundest concepts will survive the forum of competition as replacements or revisions of the traditional conceptions.

The later Karl Popper also developed a view, which describes the growth of knowledge as an evolutionary process. He used Newtonian physics as an example of a body of theories so thoroughly confirmed by testing as to be considered unassailable, but which were nevertheless overturned by Einstein's bold insights into the nature of space-time. For the evolutionary epistemologist, all theories are true only provisionally, regardless of the degree of empirical testing they have survived.

Computational approaches

Many subspecialties of applied logic and computer science, to name a few, artificial intelligence, computational learning theory, inferential statistics, and knowledge representation, are concerned with setting out computational, logical, and statistical frameworks for the various types of inference involved in scientific inquiry, in particular, hypothesis formation, logical deduction, and empirical testing. Some of these applications draw on measures of complexity from algorithmic information theory to guide the making of predictions from prior distributions of experience, for example, see the complexity measure called the speed prior from which a computable strategy for optimal inductive reasoning can be derived.

Philosophical issues

Problem of demarcation

The problem of evaluating a system of thought with regard to its status as science is often called the demarcation problem. The criteria for a system of assumptions, methods, and theories to qualify as science vary in their details from application to application, but they typically include (1) the formulation of hypotheses that meet the logical criterion of contingency, defeasibility, or falsifiability and the closely related empirical and practical criterion of testability, (2) a grounding in empirical evidence, and (3) the use of scientific method. The procedures of science typically include a number of heuristic guidelines, such as the principles of conceptual economy or theoretical parsimony that fall under the rubric of Ockham's razor. A conceptual system that fails to meet a significant number of these criteria is likely to be considered non-scientific. The following is a list of additional features that are highly desirable in a scientific theory.

• Consistent. Generates no obvious logical contradictions, and 'saves the phenomena', being consistent with observation.

• Parsimonious. Economical in the number of assumptions and hypothetical entities.

• Pertinent. Describes and explains observed phenomena.

• Falsifiable and testable. See Falsifiability and Testability.

• Reproducible. Makes predictions that can be tested by any observer, with trials extending indefinitely into the future. (This may not be expected in all empirical sciences. )

• Correctable and dynamic. Subject to modification as new observations are made.

• Integrative, robust, and corrigible. Subsumes previous theories as approximations, and allows possible subsumption by future theories. See Correspondence principle

• Provisional or tentative. Does not assert the absolute certainty of the theory.

Cuurently, there is no consensus among scientists and philosophers about an exact formulation of criteria for the demarcation of science and non-science. Some of these criteria werre suggested by some philosophers, but rejected by other, significant philosophers.

Problem of foundation

Protocol sentence debate

Theories of truth

Coherence versus correspondence theory of truth