Using Movie Subtitles for Creating Statistical Alignment Models

@mastersthesis{citeulike:2782630, abstract = {In this thesis we present a method for compiling a large-scale bilingual corpus from a database of movie subtitles. To create the corpus, we implemented an alignment algorithm which is based on Gale and Church’s sentence alignment algorithm (1993). Our algorithm not only relies on character length information, but also uses subtitle-duration information, which is encoded in the subtitle files. This results in a significant reduction in the alignment error rate. After creating a “subtitle-level” aligned corpus by finding the matching subtitle pairs, a statistical model is used for aligning the subtitles in the phrase level. The alignment is based on a joint probability model with second order dependence. Since the model is joint (as opposed to conditional) it is inherently symmetric, so no symmetrization step is required. To find the best alignment for the corpus, as well as the best parameters of the model, we use an adapted version of the EM algorithm.}, author = {Itamar, Einav }, citeulike-article-id = {2782630}, keywords = {alignment}, posted-at = {2008-05-10 10:27:19}, priority = {3}, school = {Israel Institute of Technology}, title = {Using Movie Subtitles for Creating Statistical Alignment Models} }

TY - THES ID - citeulike:2782630 L3 - citeulike-article-id:2782630 TI - Using Movie Subtitles for Creating Statistical Alignment Models N2 - In this thesis we present a method for compiling a large-scale bilingual corpus from a database of movie subtitles. To create the corpus, we implemented an alignment algorithm which is based on Gale and Church’s sentence alignment algorithm (1993). Our algorithm not only relies on character length information, but also uses subtitle-duration information, which is encoded in the subtitle files. This results in a significant reduction in the alignment error rate. After creating a “subtitle-level” aligned corpus by finding the matching subtitle pairs, a statistical model is used for aligning the subtitles in the phrase level. The alignment is based on a joint probability model with second order dependence. Since the model is joint (as opposed to conditional) it is inherently symmetric, so no symmetrization step is required. To find the best alignment for the corpus, as well as the best parameters of the model, we use an adapted version of the EM algorithm. KW - alignment AU - Itamar, Einav ER -