Cyril Cleverdon
Encyclopedia
Cyril Cleverdon was a British
librarian and computer scientist who is best known for his work on the evaluation of information retrieval
systems.
Cyril Cleverdon was born in Bristol
, England
. He worked at the Bristol Libraries from 1932 to 1938, and from 1938 to 1946 he was the librarian of the Engine Division of the Bristol Aeroplane Co. Ltd. In 1946 he was appointed librarian of the College of Aeronautics at Cranfield (later the Cranfield Institute of Technology and Cranfield University
), where he served until his retirement in 1979, the last two years as professor of Information Transfer Studies.
With the help of NSF funding, Cleverdon started a series of projects in 1957 that lasted for about 10 years in which he and his colleagues set the stage for information retrieval research. In the Cranfield project, retrieval experiments were conducted on test databases in a controlled, laboratory-like setting. The aim of the research was to improve the retrieval effectiveness of information retrieval systems, by developing better indexing languages and methods. The components of the experiments were: 1) a collection of documents, 2) a set of user requests or queries, and 3) a set of relevance judgments—that is, a set of documents judged to be relevant
to each query. Together, these components form an information retrieval test collection. The test collection serves as a standard for testing retrieval approaches, and the success of each approach is measured in terms of two measures: precision and recall. Test collections and evaluation measures based on precision and recall are driving forces behind modern research on search systems. Cleverdon's approach formed a blueprint for the successful Text Retrieval Conference
series that began in 1992.
Not only did Cleverdon's Cranfield studies introduce experimental research into computer science, the outcomes of the project also established the basis of the automatic indexing as done in today's search engines. Essentially, Cleverdon found that the use of single terms from the documents achieved the best retrieval performance, as opposed to manually-assigned thesaurus terms, synonyms, etc. These results were very controversial at the time. In the Cranfield 2 Report, Cleverdon said:
This conclusion is so controversial and so unexpected that it is bound to throw considerable doubt on the methods which have been used (...) A complete recheck has failed to reveal any discrepancies (...) there is no other course except to attempt to explain the results which seem to offend against every canon on which we were trained as librarians.
Cyril Cleverdon also ran, for many years, the Cranfield conferences, which provided a major international forum for discussion of ideas and research in information retrieval. This function was taken over by the SIGIR
conferences in the 1970s.
United Kingdom
The United Kingdom of Great Britain and Northern IrelandIn the United Kingdom and Dependencies, other languages have been officially recognised as legitimate autochthonous languages under the European Charter for Regional or Minority Languages...
librarian and computer scientist who is best known for his work on the evaluation of information retrieval
Information retrieval
Information retrieval is the area of study concerned with searching for documents, for information within documents, and for metadata about documents, as well as that of searching structured storage, relational databases, and the World Wide Web...
systems.
Cyril Cleverdon was born in Bristol
Bristol
Bristol is a city, unitary authority area and ceremonial county in South West England, with an estimated population of 433,100 for the unitary authority in 2009, and a surrounding Larger Urban Zone with an estimated 1,070,000 residents in 2007...
, England
England
England is a country that is part of the United Kingdom. It shares land borders with Scotland to the north and Wales to the west; the Irish Sea is to the north west, the Celtic Sea to the south west, with the North Sea to the east and the English Channel to the south separating it from continental...
. He worked at the Bristol Libraries from 1932 to 1938, and from 1938 to 1946 he was the librarian of the Engine Division of the Bristol Aeroplane Co. Ltd. In 1946 he was appointed librarian of the College of Aeronautics at Cranfield (later the Cranfield Institute of Technology and Cranfield University
Cranfield University
Cranfield University is a British postgraduate university based on two campuses, with a research-oriented focus. The main campus is at Cranfield, Bedfordshire and the second is the Defence Academy of the United Kingdom based at Shrivenham, Oxfordshire. The main campus is unique in the United...
), where he served until his retirement in 1979, the last two years as professor of Information Transfer Studies.
With the help of NSF funding, Cleverdon started a series of projects in 1957 that lasted for about 10 years in which he and his colleagues set the stage for information retrieval research. In the Cranfield project, retrieval experiments were conducted on test databases in a controlled, laboratory-like setting. The aim of the research was to improve the retrieval effectiveness of information retrieval systems, by developing better indexing languages and methods. The components of the experiments were: 1) a collection of documents, 2) a set of user requests or queries, and 3) a set of relevance judgments—that is, a set of documents judged to be relevant
Relevance (information retrieval)
In information science and information retrieval, relevance denotes how well a retrieved document or set of documents meets the information need of the user.-Types:...
to each query. Together, these components form an information retrieval test collection. The test collection serves as a standard for testing retrieval approaches, and the success of each approach is measured in terms of two measures: precision and recall. Test collections and evaluation measures based on precision and recall are driving forces behind modern research on search systems. Cleverdon's approach formed a blueprint for the successful Text Retrieval Conference
Text Retrieval Conference
The Text REtrieval Conference is an on-going series of workshops focusing on a list of different information retrieval research areas, or tracks. It is co-sponsored by the National Institute of Standards and Technology and the Intelligence Advanced Research Projects Activity , and began in 1992...
series that began in 1992.
Not only did Cleverdon's Cranfield studies introduce experimental research into computer science, the outcomes of the project also established the basis of the automatic indexing as done in today's search engines. Essentially, Cleverdon found that the use of single terms from the documents achieved the best retrieval performance, as opposed to manually-assigned thesaurus terms, synonyms, etc. These results were very controversial at the time. In the Cranfield 2 Report, Cleverdon said:
This conclusion is so controversial and so unexpected that it is bound to throw considerable doubt on the methods which have been used (...) A complete recheck has failed to reveal any discrepancies (...) there is no other course except to attempt to explain the results which seem to offend against every canon on which we were trained as librarians.
Cyril Cleverdon also ran, for many years, the Cranfield conferences, which provided a major international forum for discussion of ideas and research in information retrieval. This function was taken over by the SIGIR
Special Interest Group on Information Retrieval
SIGIR is the Association for Computing Machinery's Special Interest Group on Information Retrieval. The scope of the group's specialty is the theory and application of computers to the acquisition, organization, storage, retrieval and distribution of information; emphasis is placed on working with...
conferences in the 1970s.