Legal information retrieval
Encyclopedia
Legal information retrieval is the science of information retrieval
applied to legal text, including legislation
, case law
, and scholarly works. Accurate legal information retrieval is important to provide access to the law to laymen and legal professionals. Its importance has increased because of the vast and quickly increasing amount of legal documents available through electronic means.
obligated to be reasonably informed as to relevant legal documents.
Legal Information Retrieval attempts to increase the effectiveness of legal searches by increasing the number of relevant documents (providing a high recall rate) and reducing the number of irrelevant documents (a high precision rate). This is a difficult task, as the legal field is prone to jargon
, polysemes (words that have different meanings when used in a legal context), and constant change.
Techniques used to achieve these goals generally falls into three categories: boolean retrieval, manual classification of legal text, and natural language processing
of legal text.
techniques to legal text can be more difficult than application in other subjects. One key problem is that the law rarely has an inherent taxonomy
. Instead, the law is generally filled with open-ended terms, which may change over time. This can be especially true in common law
countries, where each decided case can subtly change the meaning of a certain word or phrase.
Legal information systems must also be programmed to deal with law-specific words and phrases. Though this is less problematic in the context of words which exist solely in law, legal texts also frequently use polysemes, words may have different meanings when used in a legal or common-speech manner, potentially both within the same document. The legal meanings may be dependent on the area of law in which it is applied. For example, in the context of European Union legislation, the term "worker" has four different meanings:
In addition, it also has the common meaning:
Though the terms may be similar, correct information retrieval must differentiate between the intended use and irrelevant uses in order to return the correct results.
Even if a system overcomes the language problems inherent in law, it must still determine the relevancy of each result. In the context of judicial decisions, this requires determining the precedential value of the case. Case decisions from senior or superior court
s may be more relevant than those from lower court
s, even where the lower court's decision contains more discussion of the relevant facts. The opposite may be true, however, if the senior court has only a minor discussion of the topic (for example, if it is a secondary consideration in the case). A information retrieval system must also be aware of the authority of the jurisdiction. A case from a binding authority is most likely of more value than one from a non-binding authority.
Additionally, the intentions of the user may determine which cases they find valuable. For instance, where a legal professional is attempting to argue a specific interpretation of law, he might find a minor court's decision which supports his position more valuable than a senior courts position which does not. He may also value similar positions from different areas of law, different jurisdictions, or dissenting opinions.
Overcoming these problems can be made more difficult because of the large number of cases available. The number of legal cases available via electronic means is constantly increasing (in 2003, US appellate courts handed down approximately 500 new cases per day), meaning that an accurate legal information retrieval system must incorporate methods of both sorting past data and managing new data.
, LexisNexis
, and Findlaw
. However, they overcome few of the problems discussed above.
The recall and precision rates of these searches vary depending on the implementation and searches analyzed. One study found a basic boolean search's recall rate to be roughly 20%, and its precision rate to be roughly 79%. Another study implemented a generic search (that is, not designed for legal uses) and found a recall rate of 56% and a precision rate of 72% among legal professionals. Both numbers increased when searches were run by non-legal professionals, to a 68% recall rate and 77% precision rate. This is likely explained because of the use of complex legal terms by the legal professionals.
to classify the texts, based on the way a legal professional might think about them. These attempt to link texts on the basis of their type, their value, and/or their topic areas. Most major legal search providers now implement some sort of classification search, such as Westlaw
's “Natural Language” or LexisNexis
' Headnote searches. Additionally, both of these services allow browsing of their classifications, via Westlaw's West Key Numbers or Lexis' Headnotes. Though these two search algorithms are proprietary and secret, it is known that they employ manual classification of text (though this may be computer-assisted).
These systems can help overcome the majority of problems inherent in legal information retrieval systems, in that manual classification has the greatest chances of identifying landmark cases and understanding the issues that arise in the text. In one study, ontological searching resulted in a precision rate of 82% and a recall rate of 97% among legal professionals. The legal texts included, however, were carefully controlled to just a few areas of law in a specific jurisdiction.
The major drawback to this approach is the requirement of using highly skilled legal professionals and large amounts of time to classify texts. As the amount of text available continues to increase, some have stated their belief that manual classification is unsustainable.
(NLP) techniques that are adapted to the legal domain, and also require the creation of a legal ontology
. Though multiple systems have been postulated, few have reported results. One system, “SMILE,” which attempted to automatically extract classifications from case texts, resulted in an f-measure (which is a calculation of both recall rate and precision) of under 0.3 (compared to perfect f-measure of 1.0). This is probably much lower than an acceptable rate for general usage.
Despite the limited results, many theorists predict that the evolution of such systems will eventually replace manual classification systems.
Information retrieval
Information retrieval is the area of study concerned with searching for documents, for information within documents, and for metadata about documents, as well as that of searching structured storage, relational databases, and the World Wide Web...
applied to legal text, including legislation
Legislation
Legislation is law which has been promulgated by a legislature or other governing body, or the process of making it...
, case law
Case law
In law, case law is the set of reported judicial decisions of selected appellate courts and other courts of first instance which make new interpretations of the law and, therefore, can be cited as precedents in a process known as stare decisis...
, and scholarly works. Accurate legal information retrieval is important to provide access to the law to laymen and legal professionals. Its importance has increased because of the vast and quickly increasing amount of legal documents available through electronic means.
Overview
In a legal setting, it is frequently important to retrieve all information related to a specific query. However, commonly used boolean search methods (exact matches of specified terms) on full text legal documents have been shown to have an average recall rate as low as 20 percent, meaning that only 1 in 5 relevant documents are actually retrieved. In that case, researchers believed that they had retrieved over 75% of relevant documents. This may result in failing to retrieve important or precedential cases. In some jurisdictions this may be especially problematic, as legal professionals are ethicallyLegal ethics
Legal ethics encompasses an ethical code governing the conduct of persons engaged in the practice of law and persons more generally in the legal sector.-In the United States:...
obligated to be reasonably informed as to relevant legal documents.
Legal Information Retrieval attempts to increase the effectiveness of legal searches by increasing the number of relevant documents (providing a high recall rate) and reducing the number of irrelevant documents (a high precision rate). This is a difficult task, as the legal field is prone to jargon
Jargon
Jargon is terminology which is especially defined in relationship to a specific activity, profession, group, or event. The philosophe Condillac observed in 1782 that "Every science requires a special language because every science has its own ideas." As a rationalist member of the Enlightenment he...
, polysemes (words that have different meanings when used in a legal context), and constant change.
Techniques used to achieve these goals generally falls into three categories: boolean retrieval, manual classification of legal text, and natural language processing
Natural language processing
Natural language processing is a field of computer science and linguistics concerned with the interactions between computers and human languages; it began as a branch of artificial intelligence....
of legal text.
Problems
Application of standard information retrievalInformation retrieval
Information retrieval is the area of study concerned with searching for documents, for information within documents, and for metadata about documents, as well as that of searching structured storage, relational databases, and the World Wide Web...
techniques to legal text can be more difficult than application in other subjects. One key problem is that the law rarely has an inherent taxonomy
Taxonomy
Taxonomy is the science of identifying and naming species, and arranging them into a classification. The field of taxonomy, sometimes referred to as "biological taxonomy", revolves around the description and use of taxonomic units, known as taxa...
. Instead, the law is generally filled with open-ended terms, which may change over time. This can be especially true in common law
Common law
Common law is law developed by judges through decisions of courts and similar tribunals rather than through legislative statutes or executive branch action...
countries, where each decided case can subtly change the meaning of a certain word or phrase.
Legal information systems must also be programmed to deal with law-specific words and phrases. Though this is less problematic in the context of words which exist solely in law, legal texts also frequently use polysemes, words may have different meanings when used in a legal or common-speech manner, potentially both within the same document. The legal meanings may be dependent on the area of law in which it is applied. For example, in the context of European Union legislation, the term "worker" has four different meanings:
- Any worker as defined in Article 3(a) of Directive 89/391/EECDirective 89/391/EECThe EU Workplace Health and Safety Directive is a European Union directive that sets out general principles for protection of workers' Occupational safety and health...
who habitually uses display screen equipment as a significant part of his normal work. - Any person employed by an employer, including trainees and apprentices but excluding domestic servants;
- Any person carrying out an occupation on board a vessel, including trainees and apprentices, but excluding port pilots and shore personnel carrying out work on board a vessel at the quayside;
- Any person who, in the Member State concerned, is protected as an employee under national employment law and in accordance with national practice;
In addition, it also has the common meaning:
- A person who works at a specific occupation.
Though the terms may be similar, correct information retrieval must differentiate between the intended use and irrelevant uses in order to return the correct results.
Even if a system overcomes the language problems inherent in law, it must still determine the relevancy of each result. In the context of judicial decisions, this requires determining the precedential value of the case. Case decisions from senior or superior court
Superior court
In common law systems, a superior court is a court of general competence which typically has unlimited jurisdiction with regard to civil and criminal legal cases...
s may be more relevant than those from lower court
Lower court
A lower court is a court from which an appeal may be taken. In relation to an appeal from one court to another, the lower court is the court whose decision is being reviewed, which may be the original trial court or an appellate court lower in rank than the superior court which is hearing the...
s, even where the lower court's decision contains more discussion of the relevant facts. The opposite may be true, however, if the senior court has only a minor discussion of the topic (for example, if it is a secondary consideration in the case). A information retrieval system must also be aware of the authority of the jurisdiction. A case from a binding authority is most likely of more value than one from a non-binding authority.
Additionally, the intentions of the user may determine which cases they find valuable. For instance, where a legal professional is attempting to argue a specific interpretation of law, he might find a minor court's decision which supports his position more valuable than a senior courts position which does not. He may also value similar positions from different areas of law, different jurisdictions, or dissenting opinions.
Overcoming these problems can be made more difficult because of the large number of cases available. The number of legal cases available via electronic means is constantly increasing (in 2003, US appellate courts handed down approximately 500 new cases per day), meaning that an accurate legal information retrieval system must incorporate methods of both sorting past data and managing new data.
Boolean searches
Boolean searches, where a user may specify terms such as use of specific words or judgments by a specific court, are the most common type of search available via legal information retrieval systems. They are widely implemented by services such as WestlawWestlaw
Westlaw is one of the primary online legal research services for lawyers and legal professionals in the United States and is a part of West. In addition, it provides proprietary database services...
, LexisNexis
LexisNexis
LexisNexis Group is a company providing computer-assisted legal research services. In 2006 it had the world's largest electronic database for legal and public-records related information...
, and Findlaw
FindLaw
FindLaw is a business of Thomson Reuters that provides online legal information and online marketing services for law firms. FindLaw was created by Stacy Stern, Martin Roscheisen and Tim Stanley in 1995, and was acquired by Thomson West in 2001....
. However, they overcome few of the problems discussed above.
The recall and precision rates of these searches vary depending on the implementation and searches analyzed. One study found a basic boolean search's recall rate to be roughly 20%, and its precision rate to be roughly 79%. Another study implemented a generic search (that is, not designed for legal uses) and found a recall rate of 56% and a precision rate of 72% among legal professionals. Both numbers increased when searches were run by non-legal professionals, to a 68% recall rate and 77% precision rate. This is likely explained because of the use of complex legal terms by the legal professionals.
Manual classification
In order to overcome the limits of basic boolean searches, information systems have attempted to classify case laws and statutes into more computer friendly structures. Usually, this results in the creation of an ontologyOntology
Ontology is the philosophical study of the nature of being, existence or reality as such, as well as the basic categories of being and their relations...
to classify the texts, based on the way a legal professional might think about them. These attempt to link texts on the basis of their type, their value, and/or their topic areas. Most major legal search providers now implement some sort of classification search, such as Westlaw
Westlaw
Westlaw is one of the primary online legal research services for lawyers and legal professionals in the United States and is a part of West. In addition, it provides proprietary database services...
's “Natural Language” or LexisNexis
LexisNexis
LexisNexis Group is a company providing computer-assisted legal research services. In 2006 it had the world's largest electronic database for legal and public-records related information...
' Headnote searches. Additionally, both of these services allow browsing of their classifications, via Westlaw's West Key Numbers or Lexis' Headnotes. Though these two search algorithms are proprietary and secret, it is known that they employ manual classification of text (though this may be computer-assisted).
These systems can help overcome the majority of problems inherent in legal information retrieval systems, in that manual classification has the greatest chances of identifying landmark cases and understanding the issues that arise in the text. In one study, ontological searching resulted in a precision rate of 82% and a recall rate of 97% among legal professionals. The legal texts included, however, were carefully controlled to just a few areas of law in a specific jurisdiction.
The major drawback to this approach is the requirement of using highly skilled legal professionals and large amounts of time to classify texts. As the amount of text available continues to increase, some have stated their belief that manual classification is unsustainable.
Natural language processing
In order to reduce the reliance on legal professionals and the amount of time needed, efforts have been made to create a system to automatically classify legal text and queries. Adequate translation of both would allow accurate information retrieval without the high cost of human classification. These automatic systems generally employ Natural Language ProcessingNatural language processing
Natural language processing is a field of computer science and linguistics concerned with the interactions between computers and human languages; it began as a branch of artificial intelligence....
(NLP) techniques that are adapted to the legal domain, and also require the creation of a legal ontology
Ontology
Ontology is the philosophical study of the nature of being, existence or reality as such, as well as the basic categories of being and their relations...
. Though multiple systems have been postulated, few have reported results. One system, “SMILE,” which attempted to automatically extract classifications from case texts, resulted in an f-measure (which is a calculation of both recall rate and precision) of under 0.3 (compared to perfect f-measure of 1.0). This is probably much lower than an acceptable rate for general usage.
Despite the limited results, many theorists predict that the evolution of such systems will eventually replace manual classification systems.