Resources » History » Version 14
Version 13 (Prokopis Prokopidis, 2016-02-05 02:41 PM) → Version 14/15 (Prokopis Prokopidis, 2016-02-12 03:06 PM)
# Domain-specific resources acquired with ILSP-FC
ILSP-FC [1] has been used in order to acquire several domain-specific datasets for training and evaluating domain-specific SMT systems. These datasets include:
* [bilingual corpora](http://www.panacea-lr.eu/en/info-for-researchers/data-sets/bilingual-aligned-parallel-corpora/) corpora](http://panacea-lr.eu/en/info-for-researchers/data-sets/bilingual-aligned-parallel-corpora/) in EN-EL and EN-FR (for the environment and labor legislation domains) that were then used by the PANACEA consortium for domain adaptation SMT experiments [2] and the generation of domain specific [bilingual glossaries](http://www.panacea-lr.eu/en/info-for-researchers/data-sets/bilingual-glossaries/) glossaries](http://panacea-lr.eu/en/info-for-researchers/data-sets/bilingual-glossaries/) ; [monolingual corpora](http://www.panacea-lr.eu/en/info-for-researchers/data-sets/monolingual-corpora/) corpora](http://panacea-lr.eu/en/info-for-researchers/data-sets/monolingual-corpora/) in EL, EN, ES, FR, and IT, and in the same domains, used for the creation of domain-specific [ngram lists](http://panacea-lr.eu/en/info-for-researchers/data-sets/monolingual-corpora-n-grams/).
* all combinations of DE, EL, EN, PT for the [automotive and medical domains](http://qt21.metashare.ilsp.gr/repository/search/?q=qtlp) in QTLaunchPad
* EN-HR bilingual corpora for the tourist domain [3]; EN-FI bilingual corpora used for the Abu-MaTran project submissions in WMT 2015 [4];
Additionally, experiments involving crawling public administration websites for the purposes of ELRC have generated bilingual collections in several language pairs, some examples of which are available at the following links: [EN-DE](http://nlp.ilsp.gr/elrc/output_bundesregierung.tmx.html) ; [EN-LV](http://nlp.ilsp.gr/elrc/output_eu2015_en-lv.tmx.html) ; [EN-GA](http://nlp.ilsp.gr/elrc/output_citizensinformation_en-ga.tmx.html).
# References
1. V. Papavassiliou, P. Prokopidis, G. Thurmair. [A modular open-source focused crawler for mining monolingual and bilingual corpora from the web](http://www.aclweb.org/anthology/W13-2506.pdf). In the 6th Workshop on Building and Using Comparable Corpora. 2013.
2. P. Pecina, A. Toral, V. Papavassiliou, P. Prokopidis, A. Tamchyna, A. Way, J.V. Genabith. [Domain adaptation of statistical machine translation with domain-focused web crawling](http://link.springer.com/article/10.1007%2Fs10579-014-9282-3). Language Resources and Evaluation. Vol. 49:1. 2015.
3. M. Esplà-Gomis, F. Klubička, N. Ljubešić, S. Ortiz-Rojas, V. Papavassiliou, P. Prokopidis. [Comparing two acquisition systems for automatically building an English-Croatian parallel corpus from multilingual websites](www.lrec-conf.org/proceedings/lrec2014/pdf/529_Paper.pdf). In LREC 2014.
4. R. Rubino, T. Pirineny, M. Esplà-Gomis, N. Ljubešić, S. Ortiz-Rojas, V. Papavassiliou, P. Prokopidis, A. Toral. [Abu-MaTran at WMT 2015 Translation Task: Morphological Segmentation and Web Crawling](www.aclweb.org/anthology/W15-3022.pdf). In WMT2015
ILSP-FC [1] has been used in order to acquire several domain-specific datasets for training and evaluating domain-specific SMT systems. These datasets include:
* [bilingual corpora](http://www.panacea-lr.eu/en/info-for-researchers/data-sets/bilingual-aligned-parallel-corpora/) corpora](http://panacea-lr.eu/en/info-for-researchers/data-sets/bilingual-aligned-parallel-corpora/) in EN-EL and EN-FR (for the environment and labor legislation domains) that were then used by the PANACEA consortium for domain adaptation SMT experiments [2] and the generation of domain specific [bilingual glossaries](http://www.panacea-lr.eu/en/info-for-researchers/data-sets/bilingual-glossaries/) glossaries](http://panacea-lr.eu/en/info-for-researchers/data-sets/bilingual-glossaries/) ; [monolingual corpora](http://www.panacea-lr.eu/en/info-for-researchers/data-sets/monolingual-corpora/) corpora](http://panacea-lr.eu/en/info-for-researchers/data-sets/monolingual-corpora/) in EL, EN, ES, FR, and IT, and in the same domains, used for the creation of domain-specific [ngram lists](http://panacea-lr.eu/en/info-for-researchers/data-sets/monolingual-corpora-n-grams/).
* all combinations of DE, EL, EN, PT for the [automotive and medical domains](http://qt21.metashare.ilsp.gr/repository/search/?q=qtlp) in QTLaunchPad
* EN-HR bilingual corpora for the tourist domain [3]; EN-FI bilingual corpora used for the Abu-MaTran project submissions in WMT 2015 [4];
Additionally, experiments involving crawling public administration websites for the purposes of ELRC have generated bilingual collections in several language pairs, some examples of which are available at the following links: [EN-DE](http://nlp.ilsp.gr/elrc/output_bundesregierung.tmx.html) ; [EN-LV](http://nlp.ilsp.gr/elrc/output_eu2015_en-lv.tmx.html) ; [EN-GA](http://nlp.ilsp.gr/elrc/output_citizensinformation_en-ga.tmx.html).
# References
1. V. Papavassiliou, P. Prokopidis, G. Thurmair. [A modular open-source focused crawler for mining monolingual and bilingual corpora from the web](http://www.aclweb.org/anthology/W13-2506.pdf). In the 6th Workshop on Building and Using Comparable Corpora. 2013.
2. P. Pecina, A. Toral, V. Papavassiliou, P. Prokopidis, A. Tamchyna, A. Way, J.V. Genabith. [Domain adaptation of statistical machine translation with domain-focused web crawling](http://link.springer.com/article/10.1007%2Fs10579-014-9282-3). Language Resources and Evaluation. Vol. 49:1. 2015.
3. M. Esplà-Gomis, F. Klubička, N. Ljubešić, S. Ortiz-Rojas, V. Papavassiliou, P. Prokopidis. [Comparing two acquisition systems for automatically building an English-Croatian parallel corpus from multilingual websites](www.lrec-conf.org/proceedings/lrec2014/pdf/529_Paper.pdf). In LREC 2014.
4. R. Rubino, T. Pirineny, M. Esplà-Gomis, N. Ljubešić, S. Ortiz-Rojas, V. Papavassiliou, P. Prokopidis, A. Toral. [Abu-MaTran at WMT 2015 Translation Task: Morphological Segmentation and Web Crawling](www.aclweb.org/anthology/W15-3022.pdf). In WMT2015