Frederick Jelinek
Encyclopedia
Frederick Jelinek was a Czech American
researcher in information theory
, automatic speech recognition, and natural language processing
. He was well-known for his oft-quoted quip that "Every time I fire a linguist, the performance of the speech recognizer goes up".
Born in Czechoslovakia
just before the war, his family managed to emigrate to the United States in the early years of the communist regime. He studied engineering at the Massachusetts Institute of Technology
and taught for 10 years at Cornell University
before being offered a job at IBM Research
. There his team essentially revolutionized approaches to computer speech recognition and machine translation. After IBM, he went to head the Center for Language and Speech Processing at Johns Hopkins University
for 17 years, and he was still working on the day of his death. He had been married since 1961 to Czech screenwriter Milena Jelinek
.
a decade before WWII to Vilém and Trude Jelinek. His father was Jewish, but his mother was a Catholic born in Switzerland who converted to Judaism. Jelinek senior, a dentist, had planned early for an escape to England, arranging for a passport, visa, and the shipping of his dentistry materials; the couple planned to send their son to an English private school. However, Vilém decided to stay at the last minute, and was eventually sent to the Theresienstadt concentration camp
, where he died of disease in 1945. The family was forced to move to Prague
in 1941, but Frederick, his sister and mother, thanks to the latter's background, escaped the concentration camps.
After the war, Jelinek successfully entered in the gymnasium
despite having missed several years of schooling (as education of Jewish children had been forbidden since 1942). His mother, anxious for her son to get a good education, made great efforts for their emigration,As he put it, "she didn't want to emulate my father's big mistake." particularly as it became clear he would not be allowed to even attempt the graduation examination. His mother hoped for her son to become a physician, but Jelinek dreamed of being a lawyer; he ended up studying engineering in evening classes at the City College of New York
. He received stipends from the National Committee for a Free Europe
that allowed him to study at the Massachusetts Institute of Technology
. About his choice of specialty, he joked: "Fortunately, to electrical engineering there belonged a discipline whose aim was not the construction of physical systems: the theory of information." He obtained his Ph.D. in 1962, with Robert Fano
as his adviser: "Not daring to approach Shannon himself, I asked Professor Fano to be my thesis adviser.".
In 1957, Jelinek paid an unexpected visit to Prague. He had been in Vienna
and, hoping to see his former acquaintances again, applied for a visa. He met with his old friend Miloš Forman
, who introduced him to film student Milena Tabolova
, whose screenplay had been the basis for the just released movie Easy Life (Snadný život). His flight back to the U.S. had a stopover in Munich, during which he called her to propose. Tabolova was considered a dissident, and her movie did not sit well with the authorities. Jelinek asked for help from Jerome Wiesner
and Cyrus Eaton, the latter who lobbied Nikita Khrushchev
. Following the inauguration of John F. Kennedy
, a group of Czech dissidents were allowed to emigrate in January 1961; thanks to the lobbying, the future Milena Jelinek was one of them.
After completing his graduate studies, Jelinek, who had developed an interest in linguistics
, had plans to work with Charles F. Hockett
at Cornell University
. Unfortunately for him, these fell through and during the next ten years he continued to devote himself to information theory. Having previously worked at IBM
during a sabbatical, he began work there in 1972, at first on leave for Cornell, but permanently from 1974 on; he remained there for over twenty years. Although at first he was to hold a regular research job, upon his arrival he learned that Josef Raviv had just been promoted to head of the newly opened IBM Haifa Research Laboratory
, and found himself head of the Continuous Speech Recognition group at the Thomas J. Watson Research Center
. Despite his team's successes in this area, his work remained little known in his home country, as scientists were not allowed to participate in key conferences.
After the 1989 fall of communism, he helped with establishing scientific relationships, regularly visiting to lecture and helping to convince IBM to establish a computing centre at Charles University. In 1993 he retired from IBM and went to Johns Hopkins University
's Center for Language and Speech Processing, where he was director and Julian Sinclair Smith Professor of Electrical and Computer Engineering. He was still working there at the time of his death; Jelinek died of a heart attack at the close of an otherwise normal workday in mid-September 2010. He was survived by his wife, daughter and son, sister, stepsister, and three grandchildren.
was a fashionable scientific approach in the mid '50s. However, pioneer Claude Shannon mused in 1956 that this trendiness was dangerous: "Our fellow scientists in many different fields, attracted by the fanfare and by the new avenues opened to scientific analysis, are using these ideas in their own problems. [...] It will be all too easy for our somewhat artificial prosperity to collapse overnight when it is realized that the use of a few exciting words like information, entropy, redundancy, do not solve all our problems." Indeed over the next decade, a combination of factors would shut down application of information theory to natural language processing
(NLP) problems, in particular machine translation. One was the 1957 publication of Noam Chomsky
's Syntactic Structures
, which stated that "probabilistic models give no insight into the basic problems of syntactic structure". This accorded well with the philosophy of the artificial intelligence
research of the time, which promoted rule-based approaches. The other factor was to be the 1966 ALPAC
report, which recommended that the government stop funding research in machine translation
. ALPAC chairman John Pierce
later characterised that field as filled with "mad inventors or untrustworthy engineers". He argued that the underlying linguistic problems must be solved before attempts at NLP could be reasonably made. Combined, these elements essentially halted research in the field.
Jelinek had begun to develop an interest in linguistics after the immigration of his wife, who initially enrolled in the linguistics program of the MIT thanks to Roman Jakobson
's help. Jelinek often accompanied her to Chomsky's lecture, and even went so far as to discuss the possibility of changing orientation with his adviser. Fano was "really upset", and with the failure of his project with Hockett at Cornell, he did not return to this avenue of research until starting work at IBM. The scope of research at IBM was considerably different from that of most other teams: "While Fred was leading IBM’s effort to solve the general dictation problem during the decade or so following 1972, most other U.S. companies and academic researchers were working on very limited problems [...] or were staying out of the field entirely."
It was only natural for Jelinek to see speech recognition
as an information theory problem: a noisy channel (in this case the acoustic signal)—and yet this was a daring, or even anathema approach to observers. The concept of perplexity
was introduced in their first model, New Raleigh Grammar, itself published (1976) in the "now famous paper in the Proceedings of the IEEE called "Continuous Speech Recognition by Statistical Methods"'. The basic noisy channel approach "reduced the speech recognition problem to one of producing two statistical models." Whereas New Raleigh Grammar was a hidden Markov model
, Tangora (their next model) was broader and involved n-gram
s, specifically trigrams. Even though "it was obvious to everyone that this model was hopelessly impoverished", it would remained unimproved upon until another paper of Jelinek himself presented in 1999 (see under "selected publication"). The same trigram approach was applied to phones in single words. Although the identification of parts of speech turned out not to be very useful for speech recognition, tagging methods developed during these projects are now used in various NLP applications.
The incremental research techniques developed at IBM eventually became dominant in the field after DARPA, in the mid-80s, returned to NLP research and imposed that methodology
to participating teams, shared common goals, data, and precise evaluation metrics. The Continuous Speech Recognition Group's research, which required large amounts of data to train the algorithms, eventually led to the creation of the Linguistic Data Consortium
. In the 80s, although the broader problem of speech recognition remained unsolved, they sought to apply the methods developed to other problems, and came up with two: machine translation and stock value prediction. In fact, a group of IBM searchers eventually went to work for Renaissance Technologies
. Jelinek comments: "The performance of the Renaissance fund is legendary, but I have no idea whether any methods we pioneered at IBM have ever been used. My former colleagues will not tell me: theirs is a very hush-hush operation!" Methods very similar to those developed for achieving speech recognition are at the base of most machine translation systems today.
Observers have noted that Pierce's paradigm, according to which engineering achievements in this area would be built on scientific progress, has been inverted, with the achievements in engineering being at the base of a number of scientific findings.
Jelinek's works won "best paper" awards on several occasions, and he received a number of company awards while he worked at IBM. He received the Society Award (for "outstanding technical contributions and leadership") from the IEEE Signal Processing Society
for 1997, and the ESCA Medal for Scientific Achievement in 1999. He was a recipient of a IEEE Third Millenium Medal in 2000, the ELRA
's first (2004) Antonio Zampolli Prize, the 2005 James L. Flanagan Speech and Audio Processing Award
, and the 2009 Lifetime Achievement Award from the Association for Computational Linguistics
. He received a honoris causa Ph.D. from Charles University in 2001, was elected to the National Academy of Engineering
in 2006 and made one of twelve inaugural fellows of the International Speech Communication Association in 2008.
Czech American
Czech Americans are citizens of the United States who were born in, or who descended from, the territory of the historic Czech lands, , or succession states, now known as the Czech Republic...
researcher in information theory
Information theory
Information theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental limits on signal processing operations such as compressing data and on reliably storing and...
, automatic speech recognition, and natural language processing
Natural language processing
Natural language processing is a field of computer science and linguistics concerned with the interactions between computers and human languages; it began as a branch of artificial intelligence....
. He was well-known for his oft-quoted quip that "Every time I fire a linguist, the performance of the speech recognizer goes up".
Born in Czechoslovakia
Czechoslovakia
Czechoslovakia or Czecho-Slovakia was a sovereign state in Central Europe which existed from October 1918, when it declared its independence from the Austro-Hungarian Empire, until 1992...
just before the war, his family managed to emigrate to the United States in the early years of the communist regime. He studied engineering at the Massachusetts Institute of Technology
Massachusetts Institute of Technology
The Massachusetts Institute of Technology is a private research university located in Cambridge, Massachusetts. MIT has five schools and one college, containing a total of 32 academic departments, with a strong emphasis on scientific and technological education and research.Founded in 1861 in...
and taught for 10 years at Cornell University
Cornell University
Cornell University is an Ivy League university located in Ithaca, New York, United States. It is a private land-grant university, receiving annual funding from the State of New York for certain educational missions...
before being offered a job at IBM Research
IBM Research
IBM Research, a division of IBM, is a research and advanced development organization and currently consists of eight locations throughout the world and hundreds of projects....
. There his team essentially revolutionized approaches to computer speech recognition and machine translation. After IBM, he went to head the Center for Language and Speech Processing at Johns Hopkins University
Johns Hopkins University
The Johns Hopkins University, commonly referred to as Johns Hopkins, JHU, or simply Hopkins, is a private research university based in Baltimore, Maryland, United States...
for 17 years, and he was still working on the day of his death. He had been married since 1961 to Czech screenwriter Milena Jelinek
Milena Jelinek
Milena Jelinek is a Czech American screenwriter, playwright and teacher. She wrote the screenplay for the film Zapomenuté světlo , which was awarded three Czech Lions in 1997. Her name is associated with the golden generation of Czech filmmakers, known as Czech New Wave...
.
Biography
Bedřich Jelínek was born in KladnoKladno
Kladno is a city in the Central Bohemian Region of the Czech Republic. It is located 25 km northwest of Prague. Kladno is the largest city of the region and holds a population together with its adjacent suburban areas of more than 110,000 people...
a decade before WWII to Vilém and Trude Jelinek. His father was Jewish, but his mother was a Catholic born in Switzerland who converted to Judaism. Jelinek senior, a dentist, had planned early for an escape to England, arranging for a passport, visa, and the shipping of his dentistry materials; the couple planned to send their son to an English private school. However, Vilém decided to stay at the last minute, and was eventually sent to the Theresienstadt concentration camp
Theresienstadt concentration camp
Theresienstadt concentration camp was a Nazi German ghetto during World War II. It was established by the Gestapo in the fortress and garrison city of Terezín , located in what is now the Czech Republic.-History:The fortress of Terezín was constructed between the years 1780 and 1790 by the orders...
, where he died of disease in 1945. The family was forced to move to Prague
Prague
Prague is the capital and largest city of the Czech Republic. Situated in the north-west of the country on the Vltava river, the city is home to about 1.3 million people, while its metropolitan area is estimated to have a population of over 2.3 million...
in 1941, but Frederick, his sister and mother, thanks to the latter's background, escaped the concentration camps.
After the war, Jelinek successfully entered in the gymnasium
Gymnasium (school)
A gymnasium is a type of school providing secondary education in some parts of Europe, comparable to English grammar schools or sixth form colleges and U.S. college preparatory high schools. The word γυμνάσιον was used in Ancient Greece, meaning a locality for both physical and intellectual...
despite having missed several years of schooling (as education of Jewish children had been forbidden since 1942). His mother, anxious for her son to get a good education, made great efforts for their emigration,As he put it, "she didn't want to emulate my father's big mistake." particularly as it became clear he would not be allowed to even attempt the graduation examination. His mother hoped for her son to become a physician, but Jelinek dreamed of being a lawyer; he ended up studying engineering in evening classes at the City College of New York
City College of New York
The City College of the City University of New York is a senior college of the City University of New York , in New York City. It is also the oldest of the City University's twenty-three institutions of higher learning...
. He received stipends from the National Committee for a Free Europe
National Committee for a Free Europe
The National Committee for a Free Europe was an American anti-communist organization, founded on March 17, 1949 in New York, which worked for the spreading of American influence in Europe and to oppose Stalin's Soviet occupation and dictatorship...
that allowed him to study at the Massachusetts Institute of Technology
Massachusetts Institute of Technology
The Massachusetts Institute of Technology is a private research university located in Cambridge, Massachusetts. MIT has five schools and one college, containing a total of 32 academic departments, with a strong emphasis on scientific and technological education and research.Founded in 1861 in...
. About his choice of specialty, he joked: "Fortunately, to electrical engineering there belonged a discipline whose aim was not the construction of physical systems: the theory of information." He obtained his Ph.D. in 1962, with Robert Fano
Robert Fano
Robert Mario Fano is an Italian-American computer scientist, currently professor emeritus of Electrical Engineering and Computer Science at Massachusetts Institute of Technology. Fano is known principally for his work on information theory, inventing Shannon-Fano coding...
as his adviser: "Not daring to approach Shannon himself, I asked Professor Fano to be my thesis adviser.".
In 1957, Jelinek paid an unexpected visit to Prague. He had been in Vienna
Vienna
Vienna is the capital and largest city of the Republic of Austria and one of the nine states of Austria. Vienna is Austria's primary city, with a population of about 1.723 million , and is by far the largest city in Austria, as well as its cultural, economic, and political centre...
and, hoping to see his former acquaintances again, applied for a visa. He met with his old friend Miloš Forman
Miloš Forman
Jan Tomáš Forman , better known as Miloš Forman , is a Czech-American director, screenwriter, professor, and an emigrant from Czechoslovakia. Two of his films, One Flew Over the Cuckoo's Nest and Amadeus, are among the most celebrated in the history of film, both gaining him the Academy Award for...
, who introduced him to film student Milena Tabolova
Milena Jelinek
Milena Jelinek is a Czech American screenwriter, playwright and teacher. She wrote the screenplay for the film Zapomenuté světlo , which was awarded three Czech Lions in 1997. Her name is associated with the golden generation of Czech filmmakers, known as Czech New Wave...
, whose screenplay had been the basis for the just released movie Easy Life (Snadný život). His flight back to the U.S. had a stopover in Munich, during which he called her to propose. Tabolova was considered a dissident, and her movie did not sit well with the authorities. Jelinek asked for help from Jerome Wiesner
Jerome Wiesner
Jerome Bert Wiesner was an educator, a Science Advisor to U.S. Presidents Eisenhower and Kennedy and Johnson, an advocate for arms control, and a critic of anti-ballistic-missile defense systems...
and Cyrus Eaton, the latter who lobbied Nikita Khrushchev
Nikita Khrushchev
Nikita Sergeyevich Khrushchev led the Soviet Union during part of the Cold War. He served as First Secretary of the Communist Party of the Soviet Union from 1953 to 1964, and as Chairman of the Council of Ministers, or Premier, from 1958 to 1964...
. Following the inauguration of John F. Kennedy
John F. Kennedy
John Fitzgerald "Jack" Kennedy , often referred to by his initials JFK, was the 35th President of the United States, serving from 1961 until his assassination in 1963....
, a group of Czech dissidents were allowed to emigrate in January 1961; thanks to the lobbying, the future Milena Jelinek was one of them.
After completing his graduate studies, Jelinek, who had developed an interest in linguistics
Linguistics
Linguistics is the scientific study of human language. Linguistics can be broadly broken into three categories or subfields of study: language form, language meaning, and language in context....
, had plans to work with Charles F. Hockett
Charles F. Hockett
Charles Francis Hockett was an American linguist who developed many influential ideas in American structuralist linguistics. He represents the post-Bloomfieldian phase of structuralism often referred to as "distributionalism" or "taxonomic structuralism"...
at Cornell University
Cornell University
Cornell University is an Ivy League university located in Ithaca, New York, United States. It is a private land-grant university, receiving annual funding from the State of New York for certain educational missions...
. Unfortunately for him, these fell through and during the next ten years he continued to devote himself to information theory. Having previously worked at IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
during a sabbatical, he began work there in 1972, at first on leave for Cornell, but permanently from 1974 on; he remained there for over twenty years. Although at first he was to hold a regular research job, upon his arrival he learned that Josef Raviv had just been promoted to head of the newly opened IBM Haifa Research Laboratory
IBM Haifa Research Laboratory
IBM Haifa Research Laboratory is located in Haifa, Israel. It is one of several IBM R&D Labs in Israel.IBM Haifa Research Laboratory handles projects in the spheres of cloud computing , healthcare and life sciences, verification technologies , multimedia, event processing , information retrieval...
, and found himself head of the Continuous Speech Recognition group at the Thomas J. Watson Research Center
Thomas J. Watson Research Center
The Thomas J. Watson Research Center is the headquarters for the IBM Research Division.The center is on three sites, with the main laboratory in Yorktown Heights, New York, 38 miles north of New York City, a building in Hawthorne, New York, and offices in Cambridge, Massachusetts.- Overview :The...
. Despite his team's successes in this area, his work remained little known in his home country, as scientists were not allowed to participate in key conferences.
After the 1989 fall of communism, he helped with establishing scientific relationships, regularly visiting to lecture and helping to convince IBM to establish a computing centre at Charles University. In 1993 he retired from IBM and went to Johns Hopkins University
Johns Hopkins University
The Johns Hopkins University, commonly referred to as Johns Hopkins, JHU, or simply Hopkins, is a private research university based in Baltimore, Maryland, United States...
's Center for Language and Speech Processing, where he was director and Julian Sinclair Smith Professor of Electrical and Computer Engineering. He was still working there at the time of his death; Jelinek died of a heart attack at the close of an otherwise normal workday in mid-September 2010. He was survived by his wife, daughter and son, sister, stepsister, and three grandchildren.
Research and legacy
Information theoryInformation theory
Information theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental limits on signal processing operations such as compressing data and on reliably storing and...
was a fashionable scientific approach in the mid '50s. However, pioneer Claude Shannon mused in 1956 that this trendiness was dangerous: "Our fellow scientists in many different fields, attracted by the fanfare and by the new avenues opened to scientific analysis, are using these ideas in their own problems. [...] It will be all too easy for our somewhat artificial prosperity to collapse overnight when it is realized that the use of a few exciting words like information, entropy, redundancy, do not solve all our problems." Indeed over the next decade, a combination of factors would shut down application of information theory to natural language processing
Natural language processing
Natural language processing is a field of computer science and linguistics concerned with the interactions between computers and human languages; it began as a branch of artificial intelligence....
(NLP) problems, in particular machine translation. One was the 1957 publication of Noam Chomsky
Noam Chomsky
Avram Noam Chomsky is an American linguist, philosopher, cognitive scientist, and activist. He is an Institute Professor and Professor in the Department of Linguistics & Philosophy at MIT, where he has worked for over 50 years. Chomsky has been described as the "father of modern linguistics" and...
's Syntactic Structures
Syntactic Structures
Syntactic Structures is an seminal book in linguistics by American linguist Noam Chomsky, first published in 1957. It laid the foundation of Chomsky's idea of transformational grammar...
, which stated that "probabilistic models give no insight into the basic problems of syntactic structure". This accorded well with the philosophy of the artificial intelligence
Artificial intelligence
Artificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...
research of the time, which promoted rule-based approaches. The other factor was to be the 1966 ALPAC
ALPAC
ALPAC was a committee of seven scientists led by John R. Pierce, established in 1964 by the U. S. Government in order to evaluate the progress in computational linguistics in general and machine translation in particular...
report, which recommended that the government stop funding research in machine translation
Machine translation
Machine translation, sometimes referred to by the abbreviation MT is a sub-field of computational linguistics that investigates the use of computer software to translate text or speech from one natural language to another.On a basic...
. ALPAC chairman John Pierce
John Pierce
John Pierce is the name of:* John J. Pierce , writer* John M. Pierce , American amateur astronomer* John R. Pierce , American engineer, Professor, and author* John Pierce, member of the group Pablo CruiseSee also...
later characterised that field as filled with "mad inventors or untrustworthy engineers". He argued that the underlying linguistic problems must be solved before attempts at NLP could be reasonably made. Combined, these elements essentially halted research in the field.
Jelinek had begun to develop an interest in linguistics after the immigration of his wife, who initially enrolled in the linguistics program of the MIT thanks to Roman Jakobson
Roman Jakobson
Roman Osipovich Jakobson was a Russian linguist and literary theorist.As a pioneer of the structural analysis of language, which became the dominant trend of twentieth-century linguistics, Jakobson was among the most influential linguists of the century...
's help. Jelinek often accompanied her to Chomsky's lecture, and even went so far as to discuss the possibility of changing orientation with his adviser. Fano was "really upset", and with the failure of his project with Hockett at Cornell, he did not return to this avenue of research until starting work at IBM. The scope of research at IBM was considerably different from that of most other teams: "While Fred was leading IBM’s effort to solve the general dictation problem during the decade or so following 1972, most other U.S. companies and academic researchers were working on very limited problems [...] or were staying out of the field entirely."
It was only natural for Jelinek to see speech recognition
Speech recognition
Speech recognition converts spoken words to text. The term "voice recognition" is sometimes used to refer to recognition systems that must be trained to a particular speaker—as is the case for most desktop recognition software...
as an information theory problem: a noisy channel (in this case the acoustic signal)—and yet this was a daring, or even anathema approach to observers. The concept of perplexity
Perplexity
Perplexity is a measurement in information theory. It is defined as b raised to the power of entropy in base b, or more often as b raised to the power of cross-entropy in base b...
was introduced in their first model, New Raleigh Grammar, itself published (1976) in the "now famous paper in the Proceedings of the IEEE called "Continuous Speech Recognition by Statistical Methods"'. The basic noisy channel approach "reduced the speech recognition problem to one of producing two statistical models." Whereas New Raleigh Grammar was a hidden Markov model
Hidden Markov model
A hidden Markov model is a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobserved states. An HMM can be considered as the simplest dynamic Bayesian network. The mathematics behind the HMM was developed by L. E...
, Tangora (their next model) was broader and involved n-gram
N-gram
In the fields of computational linguistics and probability, an n-gram is a contiguous sequence of n items from a given sequence of text or speech. The items in question can be phonemes, syllables, letters, words or base pairs according to the application...
s, specifically trigrams. Even though "it was obvious to everyone that this model was hopelessly impoverished", it would remained unimproved upon until another paper of Jelinek himself presented in 1999 (see under "selected publication"). The same trigram approach was applied to phones in single words. Although the identification of parts of speech turned out not to be very useful for speech recognition, tagging methods developed during these projects are now used in various NLP applications.
The incremental research techniques developed at IBM eventually became dominant in the field after DARPA, in the mid-80s, returned to NLP research and imposed that methodology
Methodology
Methodology is generally a guideline for solving a problem, with specificcomponents such as phases, tasks, methods, techniques and tools . It can be defined also as follows:...
to participating teams, shared common goals, data, and precise evaluation metrics. The Continuous Speech Recognition Group's research, which required large amounts of data to train the algorithms, eventually led to the creation of the Linguistic Data Consortium
Linguistic Data Consortium
The Linguistic Data Consortium is an open consortium of universities, companies and government research laboratories. It creates, collects and distributes speech and text databases, lexicons, and other resources for linguistics research and development purposes. The University of Pennsylvania is...
. In the 80s, although the broader problem of speech recognition remained unsolved, they sought to apply the methods developed to other problems, and came up with two: machine translation and stock value prediction. In fact, a group of IBM searchers eventually went to work for Renaissance Technologies
Renaissance Technologies
Renaissance Technologies is a hedge fund management company of about 275 employees and more than $ billion in assets under management in three funds...
. Jelinek comments: "The performance of the Renaissance fund is legendary, but I have no idea whether any methods we pioneered at IBM have ever been used. My former colleagues will not tell me: theirs is a very hush-hush operation!" Methods very similar to those developed for achieving speech recognition are at the base of most machine translation systems today.
Observers have noted that Pierce's paradigm, according to which engineering achievements in this area would be built on scientific progress, has been inverted, with the achievements in engineering being at the base of a number of scientific findings.
Jelinek's works won "best paper" awards on several occasions, and he received a number of company awards while he worked at IBM. He received the Society Award (for "outstanding technical contributions and leadership") from the IEEE Signal Processing Society
IEEE Signal Processing Society
The IEEE Signal Processing Society is a society of the IEEE. It is also known by the acronym IEEE SPS. In the hierarchy of IEEE, the Signal Processing Society is one of close to 40 technical societies organized under the IEEE's Technical Activities Board...
for 1997, and the ESCA Medal for Scientific Achievement in 1999. He was a recipient of a IEEE Third Millenium Medal in 2000, the ELRA
ELRA
A not-for-profit organisation, the European Language Resources Association association is established under the law of the Grand Duchy of Luxembourg...
's first (2004) Antonio Zampolli Prize, the 2005 James L. Flanagan Speech and Audio Processing Award
IEEE James L. Flanagan Speech and Audio Processing Award
The IEEE James L. Flanagan Speech and Audio Processing Award is a Technical Field Award presented by the IEEE for an outstanding contribution to the advancement of speech and/or audio signal processing. It may be presented to an individual or a team of up to three people...
, and the 2009 Lifetime Achievement Award from the Association for Computational Linguistics
Association for Computational Linguistics
The Association for Computational Linguistics is the international scientific and professional society for people working on problems involving natural language and computation. An annual meeting is held each summer in locations where significant computational linguistics research is carried out...
. He received a honoris causa Ph.D. from Charles University in 2001, was elected to the National Academy of Engineering
National Academy of Engineering
The National Academy of Engineering is a government-created non-profit institution in the United States, that was founded in 1964 under the same congressional act that led to the founding of the National Academy of Sciences...
in 2006 and made one of twelve inaugural fellows of the International Speech Communication Association in 2008.
Selected publications
- Jelinek, Frederick (1968). Probabilistic Information Theory: Discrete and memoryless models. McGraw-Hill series in systems science. New York: McGraw-Hill. 689p.
- ———————- (1969). "Fast sequential decoding algorithm using a stack". IBM Journal of Research and Development 13(6):675–685. .
- ———————- (1969). "Tree encoding of memoryless time-discrete sources with a fidelity criterion". IEEE Transactions on Information Theory 15(5):584–590. . (received 1971 "Best Paper" award)
- Bahl, Lalit R.; John CockeJohn CockeJohn Cocke was an American computer scientist recognized for his large contribution to computer architecture and optimizing compiler design. He is considered by many to be "the father of RISC architecture."...
, Frederick Jelinek, Josef Raviv (1974). "Optimal decoding of linear codes for minimizing symbol error rate". IEEE Transactions on Information Theory 20(2):284–287. . (received Information Theory Society Golden Jubilee paper award) - ———————- (1976). "Continuous speech recognition by statistical methods". Proceedings of the IEEE 64(4):532–556. .
- Brown, P.; J. Cocke, S. Della Pietra, V. Della Pietra, F. Jelinek, R, Mercer and P. Roossin (1988). "A statistical approach to language translation". In Dénes Vargha, ed. Coling 88: Proceedings of the 12th conference on Computational linguistics, volume 1. Budapest: John Von Neumann society for computing sciences. pp. 71–76. . ISBN 963-8431-56-3.
- ———————- (1990). "Self-Organized Language Modeling for Speech Recognition". In Alex Waibel & Kai-Fu Lee, eds. Readings in speech recognition. San Mateo: Morgan Kaufmann. 629p. ISBN 1-55860-124-4.
- ———————-; John D. Lafferty and Robert L. Mercer. (1990) "Basic methods of probabilistic context free grammars". Technical Report RC 16374 (72684), IBM.
- Reprinted in Laface, Pietro; Renato De Mori (1992). Speech Recognition and Understanding: Recent advances, trends, and applications. NATO ASI series. Series F, Computer and systems sciences, 75. New York: Springer-Verlag. pp. 345–360. ISBN 0-387-54032-6.
- ———————- (1997). Statistical Methods for Speech Recognition. Cambridge, Mass.: MIT Press. 283p. ISBN 0-262-10066-5. (review) (review 2)
- Chelba, Ciprian; Frederick Jelinek (2000). "Structured Language Modeling". Computer Speech & Language 14(4):283–332. (received 2002 "Best Paper" award).
- Expanded version of a presentation at NLDB'99. Klagenfurt, Austria, June 17–19, 1999 .
- Xu, Peng; Ahmad Emami and Frederick Jelinek (2003). "Training Connectionist Models for the Structured Language Model". In Michael Collins and Mark Steedman, eds. EMNLP '03 Proceedings of the 2003 conference on Empirical methods in natural language processing. East Stroudsburg, Penn.: Association for Computational Linguistics. pp. 160–167. ISBN 1-932432-13-2. . (won "best paper" award)
External links
- Institutional page at Johns Hopkins university