Language-Independent Sentence Boundary Detectionwith Automatic Feature Selection

다국어 초록 (Multilingual Abstract)

This paper proposes a machine learning approach for language-independent sentence boundary detection. The proposed method requires no heuristic rules and language-specific features, such as part-of-speech information, a list of abbreviations or proper names. With only the language-independent features, we perform experiments on not only an inflectional language but also an agglutinative language, having fairly different characteristics (in this paper, English and Korean, respectively). In addition, we obtain good performances in both languages. We have also experimented with the methods under a wide range of experimental conditions, especially for the selection of useful features.

참고문헌 (Reference)

1 Daelemans, W, "Timbl: Tilburg memorybased learner - version 4.3 reference guide" 2002

2 Shim, J, "Multi-strategic integrated web document pre-processing for sentence and word boundary detection" 38 (38): 509-527, 2002

3 Joachims,T, "Making large-scale support vector machine learning practical, Advances in Kernel Methods: Support Vector Machines" MIT Press, Cambridge, MA. 1999

4 Palmer, D. D, "Adaptive multilingual sentence boundary disambiguation" 23 (23): 241-267, 1997

5 Reynar, J, "A maximum entropy approach to identifying sentence boundaries" 16-19, 1997